" -labels: ['status/0-triage', 'kind/bug'] -body: - - type: textarea - attributes: - label: Description - description: | - Briefly describe the problem you are having. - - Include both the current behavior (what you are seeing) as well as what you expected to happen. - validations: - required: true - - type: markdown - attributes: - value: | - [Docker Swarm](https://www.mirantis.com/software/swarm/) uses a distinct compose file parser and - as such doesn't support some of the recent features of Docker Compose. Please contact Mirantis - if you need assistance with compose file support in Docker Swarm. - - type: textarea - attributes: - label: Steps To Reproduce - description: Steps to reproduce the behavior. - placeholder: | - 1. In this environment... - 2. With this config... - 3. Run '...' - 4. See error... - validations: - required: false - - type: textarea - attributes: - label: Compose Version - description: | - Paste output of `docker compose version` and `docker-compose version`. - render: Text - validations: - required: false - - type: textarea - attributes: - label: Docker Environment - description: Paste output of `docker info`. - render: Text - validations: - required: false - - type: textarea - attributes: - label: Anything else? - description: | - Links? References? Anything that will give us more context about the issue you are encountering! - - Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in. - validations: - required: false diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml deleted file mode 100644 index cc4b65bf24d..00000000000 --- a/.github/ISSUE_TEMPLATE/config.yml +++ /dev/null @@ -1,11 +0,0 @@ -blank_issues_enabled: true -contact_links: - - name: Docker Community Slack - url: https://dockr.ly/slack - about: 'Use the #docker-compose channel' - - name: Docker Support Forums - url: https://forums.docker.com/c/open-source-projects/compose/15 - about: 'Use the "Open Source Projects > Compose" category' - - name: 'Ask on Stack Overflow' - url: https://stackoverflow.com/questions/tagged/docker-compose - about: 'Use the [docker-compose] tag when creating new questions' diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 00000000000..603d34c38ab --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,32 @@ +--- +name: Feature request +about: Suggest an idea to improve Compose +title: '' +labels: kind/feature +assignees: '' + +--- + + + +**Is your feature request related to a problem? Please describe.** +A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] + +**Describe the solution you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Additional context** +Add any other context or screenshots about the feature request here. diff --git a/.github/ISSUE_TEMPLATE/feature_request.yaml b/.github/ISSUE_TEMPLATE/feature_request.yaml deleted file mode 100644 index 677a1684fc0..00000000000 --- a/.github/ISSUE_TEMPLATE/feature_request.yaml +++ /dev/null @@ -1,13 +0,0 @@ -name: Feature request -description: Missing functionality? Come tell us about it! -labels: - - kind/feature - - status/0-triage -body: - - type: textarea - id: description - attributes: - label: Description - description: What is the feature you want to see? - validations: - required: true diff --git a/.github/ISSUE_TEMPLATE/question-about-using-compose.md b/.github/ISSUE_TEMPLATE/question-about-using-compose.md new file mode 100644 index 00000000000..ccb4e9b33bb --- /dev/null +++ b/.github/ISSUE_TEMPLATE/question-about-using-compose.md @@ -0,0 +1,12 @@ +--- +name: Question about using Compose +about: This is not the appropriate channel +title: '' +labels: kind/question +assignees: '' + +--- + +Please post on our forums: https://forums.docker.com for questions about using `docker-compose`. + +Posts that are not a bug report or a feature/enhancement request will not be addressed on this issue tracker. diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md deleted file mode 100644 index 00e87ff8eaa..00000000000 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ /dev/null @@ -1,6 +0,0 @@ -**What I did** - -**Related issue** - - -**(not mandatory) A picture of a cute animal, if possible in relation to what you did** diff --git a/.github/SECURITY.md b/.github/SECURITY.md deleted file mode 100644 index 88f7528d524..00000000000 --- a/.github/SECURITY.md +++ /dev/null @@ -1,44 +0,0 @@ -# Security Policy - -The maintainers of Docker Compose take security seriously. If you discover -a security issue, please bring it to their attention right away! - -## Reporting a Vulnerability - -Please **DO NOT** file a public issue, instead send your report privately -to [security@docker.com](mailto:security@docker.com). - -Reporter(s) can expect a response within 72 hours, acknowledging the issue was -received. - -## Review Process - -After receiving the report, an initial triage and technical analysis is -performed to confirm the report and determine its scope. We may request -additional information in this stage of the process. - -Once a reviewer has confirmed the relevance of the report, a draft security -advisory will be created on GitHub. The draft advisory will be used to discuss -the issue with maintainers, the reporter(s), and where applicable, other -affected parties under embargo. - -If the vulnerability is accepted, a timeline for developing a patch, public -disclosure, and patch release will be determined. If there is an embargo period -on public disclosure before the patch release, the reporter(s) are expected to -participate in the discussion of the timeline and abide by agreed upon dates -for public disclosure. - -## Accreditation - -Security reports are greatly appreciated and we will publicly thank you, -although we will keep your name confidential if you request it. We also like to -send gifts - if you're into swag, make sure to let us know. We do not currently -offer a paid security bounty program at this time. - -## Supported Versions - -This project docs not provide long-term supported versions, and only the current -release and `main` branch are actively maintained. Docker Compose v1, and the -corresponding [v1 branch](https://github.com/docker/compose/tree/v1) reached -EOL and are no longer supported. - diff --git a/.github/dependabot.yml b/.github/dependabot.yml deleted file mode 100644 index 3810add71fc..00000000000 --- a/.github/dependabot.yml +++ /dev/null @@ -1,23 +0,0 @@ -version: 2 -updates: - - package-ecosystem: gomod - directory: / - schedule: - interval: daily - ignore: - # docker + moby deps require coordination - - dependency-name: "github.com/docker/buildx" - # buildx is still 0.x - update-types: ["version-update:semver-minor"] - - dependency-name: "github.com/moby/buildkit" - # buildkit is still 0.x - update-types: [ "version-update:semver-minor" ] - - dependency-name: "github.com/docker/cli" - update-types: ["version-update:semver-major"] - - dependency-name: "github.com/docker/docker" - update-types: ["version-update:semver-major"] - - dependency-name: "github.com/containerd/containerd" - # containerd major/minor must be kept in sync with moby - update-types: [ "version-update:semver-major", "version-update:semver-minor" ] - # OTEL dependencies should be upgraded in sync with engine, cli, buildkit and buildx projects - - dependency-name: "go.opentelemetry.io/*" diff --git a/.github/stale.yml b/.github/stale.yml index c14cb12918a..6de76aef987 100644 --- a/.github/stale.yml +++ b/.github/stale.yml @@ -1,7 +1,7 @@ # Configuration for probot-stale - https://github.com/probot/stale # Number of days of inactivity before an Issue or Pull Request becomes stale -daysUntilStale: 90 +daysUntilStale: 180 # Number of days of inactivity before an Issue or Pull Request with the stale label is closed. # Set to false to disable. If disabled, issues still need to be closed manually, but will remain marked as stale. @@ -12,7 +12,7 @@ onlyLabels: [] # Issues or Pull Requests with these labels will never be considered stale. Set to `[]` to disable exemptLabels: - - "kind/feature" + - kind/feature # Set to true to ignore issues in a project (defaults to false) exemptProjects: false @@ -56,4 +56,4 @@ only: issues # issues: # exemptLabels: -# - confirmed \ No newline at end of file +# - confirmed diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml deleted file mode 100644 index 3c6ef149d23..00000000000 --- a/.github/workflows/ci.yml +++ /dev/null @@ -1,338 +0,0 @@ -name: ci - -concurrency: - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -on: - push: - branches: - - 'main' - tags: - - 'v*' - pull_request: - workflow_dispatch: - inputs: - debug_enabled: - description: 'To run with tmate enter "debug_enabled"' - required: false - default: "false" - -permissions: - contents: read # to fetch code (actions/checkout) - -jobs: - validate: - runs-on: ubuntu-latest - strategy: - fail-fast: false - matrix: - target: - - lint - - validate-go-mod - - validate-headers - - validate-docs - steps: - - - name: Checkout - uses: actions/checkout@v4 - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v3 - - - name: Run - run: | - make ${{ matrix.target }} - - binary: - uses: docker/github-builder/.github/workflows/bake.yml@v1 - permissions: - contents: read # same as global permission - id-token: write # for signing attestation(s) with GitHub OIDC Token - with: - runner: amd64 - artifact-name: compose - artifact-upload: true - cache: true - cache-scope: binary - target: release - output: local - sbom: true - sign: ${{ github.event_name != 'pull_request' }} - - binary-finalize: - runs-on: ubuntu-latest - needs: - - binary - steps: - - - name: Download artifacts - uses: actions/download-artifact@v7 - with: - path: /tmp/compose-output - name: ${{ needs.binary.outputs.artifact-name }} - - - name: Rename provenance and sbom - run: | - for pdir in /tmp/compose-output/*/; do - ( - cd "$pdir" - binname=$(find . -name 'docker-compose-*') - filename=$(basename "${binname%.exe}") - mv "provenance.json" "${filename}.provenance.json" - mv "sbom-binary.spdx.json" "${filename}.sbom.json" - find . -name 'sbom*.json' -exec rm {} \; - if [ -f "provenance.sigstore.json" ]; then - mv "provenance.sigstore.json" "${filename}.sigstore.json" - fi - ) - done - mkdir -p "./bin/release" - mv /tmp/compose-output/**/* "./bin/release/" - - - name: Create checksum file - working-directory: ./bin/release - run: | - find . -type f -print0 | sort -z | xargs -r0 shasum -a 256 -b | sed 's# \*\./# *#' > $RUNNER_TEMP/checksums.txt - shasum -a 256 -U -c $RUNNER_TEMP/checksums.txt - mv $RUNNER_TEMP/checksums.txt . - cat checksums.txt | while read sum file; do - if [[ "${file#\*}" == docker-compose-* && "${file#\*}" != *.provenance.json && "${file#\*}" != *.sbom.json && "${file#\*}" != *.sigstore.json ]]; then - echo "$sum $file" > ${file#\*}.sha256 - fi - done - - - name: Upload artifacts - uses: actions/upload-artifact@v6 - with: - name: release - path: ./bin/release/* - if-no-files-found: error - - bin-image-test: - if: github.event_name == 'pull_request' - uses: docker/github-builder/.github/workflows/bake.yml@v1 - with: - runner: amd64 - target: image-cross - cache: true - cache-scope: bin-image-test - output: image - push: false - sbom: true - set-meta-labels: true - meta-images: | - compose-bin - meta-tags: | - type=ref,event=pr - meta-bake-target: meta-helper - - test: - runs-on: ubuntu-latest - steps: - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v3 - - - name: Test - uses: docker/bake-action@v6 - with: - targets: test - set: | - *.cache-from=type=gha,scope=test - *.cache-to=type=gha,scope=test - - - name: Gather coverage data - uses: actions/upload-artifact@v4 - with: - name: coverage-data-unit - path: bin/coverage/unit/ - if-no-files-found: error - - - name: Unit Test Summary - uses: test-summary/action@v2 - with: - paths: bin/coverage/unit/report.xml - if: always() - - e2e: - runs-on: ubuntu-latest - name: e2e (${{ matrix.mode }}, ${{ matrix.channel }}) - strategy: - fail-fast: false - matrix: - include: - # current stable - - mode: plugin - engine: 29 - channel: stable - - mode: standalone - engine: 29 - channel: stable - - # old stable (latest major - 1) - - mode: plugin - engine: 28 - channel: oldstable - - mode: standalone - engine: 28 - channel: oldstable - steps: - - name: Prepare - run: | - mode=${{ matrix.mode }} - engine=${{ matrix.engine }} - echo "MODE_ENGINE_PAIR=${mode}-${engine}" >> $GITHUB_ENV - - - name: Checkout - uses: actions/checkout@v4 - - - name: Install Docker ${{ matrix.engine }} - run: | - sudo systemctl stop docker.service - sudo apt-get purge docker-ce docker-ce-cli containerd.io docker-compose-plugin docker-ce-rootless-extras docker-buildx-plugin - sudo apt-get install curl - curl -fsSL https://test.docker.com -o get-docker.sh - sudo sh ./get-docker.sh --version ${{ matrix.engine }} - - - name: Check Docker Version - run: docker --version - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v3 - - - name: Set up Docker Model - run: | - sudo apt-get install docker-model-plugin - docker model version - - - name: Set up Go - uses: actions/setup-go@v6 - with: - go-version-file: '.go-version' - check-latest: true - cache: true - - - name: Build example provider - run: make example-provider - - - name: Build - uses: docker/bake-action@v6 - with: - source: . - targets: binary-with-coverage - set: | - *.cache-from=type=gha,scope=binary-linux-amd64 - *.cache-from=type=gha,scope=binary-e2e-${{ matrix.mode }} - *.cache-to=type=gha,scope=binary-e2e-${{ matrix.mode }},mode=max - env: - BUILD_TAGS: e2e - - - name: Setup tmate session - if: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.debug_enabled }} - uses: mxschmitt/action-tmate@8b4e4ac71822ed7e0ad5fb3d1c33483e9e8fb270 # v3.11 - with: - limit-access-to-actor: true - github-token: ${{ secrets.GITHUB_TOKEN }} - - - name: Test plugin mode - if: ${{ matrix.mode == 'plugin' }} - run: | - rm -rf ./bin/coverage/e2e - mkdir -p ./bin/coverage/e2e - make e2e-compose GOCOVERDIR=bin/coverage/e2e TEST_FLAGS="-v" - - - name: Gather coverage data - if: ${{ matrix.mode == 'plugin' }} - uses: actions/upload-artifact@v4 - with: - name: coverage-data-e2e-${{ env.MODE_ENGINE_PAIR }} - path: bin/coverage/e2e/ - if-no-files-found: error - - - name: Test standalone mode - if: ${{ matrix.mode == 'standalone' }} - run: | - rm -f /usr/local/bin/docker-compose - cp bin/build/docker-compose /usr/local/bin - make e2e-compose-standalone - - - name: e2e Test Summary - uses: test-summary/action@v2 - with: - paths: /tmp/report/report.xml - if: always() - - coverage: - runs-on: ubuntu-latest - needs: - - test - - e2e - steps: - # codecov won't process the report without the source code available - - name: Checkout - uses: actions/checkout@v4 - - name: Set up Go - uses: actions/setup-go@v6 - with: - go-version-file: '.go-version' - check-latest: true - - name: Download unit test coverage - uses: actions/download-artifact@v4 - with: - name: coverage-data-unit - path: coverage/unit - merge-multiple: true - - name: Download E2E test coverage - uses: actions/download-artifact@v4 - with: - pattern: coverage-data-e2e-* - path: coverage/e2e - merge-multiple: true - - name: Merge coverage reports - run: | - go tool covdata textfmt -i=./coverage/unit,./coverage/e2e -o ./coverage.txt - - name: Store coverage report in GitHub Actions - uses: actions/upload-artifact@v4 - with: - name: go-covdata-txt - path: ./coverage.txt - if-no-files-found: error - - name: Upload coverage to Codecov - uses: codecov/codecov-action@v5 - with: - files: ./coverage.txt - - release: - permissions: - contents: write # to create a release (ncipollo/release-action) - runs-on: ubuntu-latest - needs: - - binary-finalize - steps: - - - name: Checkout - uses: actions/checkout@v4 - - - name: Download artifacts - uses: actions/download-artifact@v7 - with: - path: ./bin/release - name: release - - - name: List artifacts - run: | - tree -nh ./bin/release - - - name: Check artifacts - run: | - find bin/release -type f -exec file -e ascii -- {} + - - - name: GitHub Release - if: startsWith(github.ref, 'refs/tags/v') - uses: ncipollo/release-action@58ae73b360456532aafd58ee170c045abbeaee37 # v1.10.0 - with: - artifacts: ./bin/release/* - generateReleaseNotes: true - draft: true - token: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/docs-upstream.yml b/.github/workflows/docs-upstream.yml deleted file mode 100644 index 214c88381fd..00000000000 --- a/.github/workflows/docs-upstream.yml +++ /dev/null @@ -1,51 +0,0 @@ -# this workflow runs the remote validate bake target from docker/docs -# to check if yaml reference docs used in this repo are valid -name: docs-upstream - -# Default to 'contents: read', which grants actions to read commits. -# -# If any permission is set, any permission not included in the list is -# implicitly set to "none". -# -# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions -permissions: - contents: read - -concurrency: - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -on: - push: - branches: - - 'main' - - 'v[0-9]*' - paths: - - '.github/workflows/docs-upstream.yml' - - 'docs/**' - pull_request: - paths: - - '.github/workflows/docs-upstream.yml' - - 'docs/**' - -jobs: - docs-yaml: - runs-on: ubuntu-latest - steps: - - - name: Checkout - uses: actions/checkout@v4 - - - name: Upload reference YAML docs - uses: actions/upload-artifact@v4 - with: - name: docs-yaml - path: docs/reference - retention-days: 1 - - validate: - uses: docker/docs/.github/workflows/validate-upstream.yml@main - needs: - - docs-yaml - with: - module-name: docker/compose diff --git a/.github/workflows/merge.yml b/.github/workflows/merge.yml deleted file mode 100644 index 1e0d37fd24f..00000000000 --- a/.github/workflows/merge.yml +++ /dev/null @@ -1,141 +0,0 @@ -name: merge - -concurrency: - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -on: - push: - branches: - - 'main' - tags: - - 'v*' - -permissions: - contents: read # to fetch code (actions/checkout) - -env: - REPO_SLUG: "docker/compose-bin" - -jobs: - e2e: - name: Build and test - runs-on: ${{ matrix.os }} - timeout-minutes: 15 - strategy: - fail-fast: false - matrix: - os: [desktop-windows, desktop-macos, desktop-m1] - # mode: [plugin, standalone] - mode: [plugin] - env: - GO111MODULE: "on" - steps: - - uses: actions/checkout@v4 - - - uses: actions/setup-go@v6 - with: - go-version-file: '.go-version' - cache: true - check-latest: true - - - name: List Docker resources on machine - run: | - docker ps --all - docker volume ls - docker network ls - docker image ls - - name: Remove Docker resources on machine - continue-on-error: true - run: | - docker kill $(docker ps -q) - docker rm -f $(docker ps -aq) - docker volume rm -f $(docker volume ls -q) - docker ps --all - - - name: Unit tests - run: make test - - - name: Build binaries - run: | - make - - name: Check arch of go compose binary - run: | - file ./bin/build/docker-compose - if: ${{ !contains(matrix.os, 'desktop-windows') }} - - - name: Test plugin mode - if: ${{ matrix.mode == 'plugin' }} - run: | - make e2e-compose - - - name: Test standalone mode - if: ${{ matrix.mode == 'standalone' }} - run: | - make e2e-compose-standalone - - bin-image-prepare: - runs-on: ubuntu-24.04 - outputs: - repo-slug: ${{ env.REPO_SLUG }} - steps: - # FIXME: can't use env object in reusable workflow inputs: https://github.com/orgs/community/discussions/26671 - - run: echo "Exposing env vars for reusable workflow" - - bin-image: - uses: docker/github-builder/.github/workflows/bake.yml@v1 - needs: - - bin-image-prepare - permissions: - contents: read # same as global permission - id-token: write # for signing attestation(s) with GitHub OIDC Token - with: - runner: amd64 - target: image-cross - cache: true - cache-scope: bin-image - output: image - push: ${{ github.event_name != 'pull_request' }} - sbom: true - set-meta-labels: true - meta-images: | - ${{ needs.bin-image-prepare.outputs.repo-slug }} - meta-tags: | - type=ref,event=tag - type=edge - meta-bake-target: meta-helper - secrets: - registry-auths: | - - registry: docker.io - username: ${{ secrets.DOCKERPUBLICBOT_USERNAME }} - password: ${{ secrets.DOCKERPUBLICBOT_WRITE_PAT }} - - desktop-edge-test: - runs-on: ubuntu-latest - needs: bin-image - steps: - - - name: Generate Token - id: generate_token - uses: actions/create-github-app-token@v1 - with: - app-id: ${{ vars.DOCKERDESKTOP_APP_ID }} - private-key: ${{ secrets.DOCKERDESKTOP_APP_PRIVATEKEY }} - owner: docker - repositories: | - ${{ secrets.DOCKERDESKTOP_REPO }} - - - name: Trigger Docker Desktop e2e with edge version - uses: actions/github-script@v7 - with: - github-token: ${{ steps.generate_token.outputs.token }} - script: | - await github.rest.actions.createWorkflowDispatch({ - owner: 'docker', - repo: '${{ secrets.DOCKERDESKTOP_REPO }}', - workflow_id: 'compose-edge-integration.yml', - ref: 'main', - inputs: { - "image-tag": "${{ env.REPO_SLUG }}:edge" - } - }) diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml deleted file mode 100644 index b8f0e5e2500..00000000000 --- a/.github/workflows/scorecards.yml +++ /dev/null @@ -1,63 +0,0 @@ -name: Scorecards supply-chain security -on: - # Only the default branch is supported. - branch_protection_rule: - schedule: - - cron: '44 9 * * 4' - push: - branches: [ "main" ] - -jobs: - analysis: - name: Scorecards analysis - runs-on: ubuntu-latest - permissions: - # Needed to upload the results to code-scanning dashboard. - security-events: write - # Used to receive a badge. - id-token: write - # read permissions to all the other objects - actions: read - attestations: read - checks: read - contents: read - deployments: read - issues: read - discussions: read - packages: read - pages: read - pull-requests: read - statuses: read - - steps: - - name: "Checkout code" - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # tag=v4.4.2 - with: - persist-credentials: false - - - name: "Run analysis" - uses: ossf/scorecard-action@62b2cac7ed8198b15735ed49ab1e5cf35480ba46 # tag=v2.4.0 - with: - results_file: results.sarif - results_format: sarif - - # Publish the results for public repositories to enable scorecard badges. For more details, see - # https://github.com/ossf/scorecard-action#publishing-results. - # For private repositories, `publish_results` will automatically be set to `false`, regardless - # of the value entered here. - publish_results: true - - # Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF - # format to the repository Actions tab. - - name: "Upload artifact" - uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # tag=v4.5.0 - with: - name: SARIF file - path: results.sarif - retention-days: 5 - - # Upload the results to GitHub's code scanning dashboard. - - name: "Upload to code-scanning" - uses: github/codeql-action/upload-sarif@3096afedf9873361b2b2f65e1445b13272c83eb8 # tag=v2.20.00 - with: - sarif_file: results.sarif diff --git a/.github/workflows/stale.yml b/.github/workflows/stale.yml deleted file mode 100644 index 2a747ee1c25..00000000000 --- a/.github/workflows/stale.yml +++ /dev/null @@ -1,33 +0,0 @@ -name: 'Close stale issues' - -# Default to 'contents: read', which grants actions to read commits. -# -# If any permission is set, any permission not included in the list is -# implicitly set to "none". -# -# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions -permissions: - contents: read - -on: - schedule: - - cron: '0 0 * * 0,3' # at midnight UTC every Sunday and Wednesday -jobs: - stale: - runs-on: ubuntu-latest - permissions: - issues: write - pull-requests: write - steps: - - uses: actions/stale@v9 - with: - repo-token: ${{ secrets.GITHUB_TOKEN }} - stale-issue-message: > - This issue has been automatically marked as stale because it has not had - recent activity. It will be closed if no further activity occurs. Thank you - for your contributions. - days-before-issue-stale: 150 # marks stale after 5 months - days-before-issue-close: 30 # closes 1 month after being marked with no action - stale-issue-label: "stale" - exempt-issue-labels: "kind/feature,kind/enhancement" - \ No newline at end of file diff --git a/.gitignore b/.gitignore index f01544f105f..79888274847 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,18 @@ -bin/ -/.vscode/ -coverage.out -covdatafiles/ +*.egg-info +*.pyc +*.swo +*.swp +.cache +.coverage* .DS_Store -pkg/e2e/*.tar +.idea + +/.tox +/binaries +/build +/compose/GITSHA +/coverage-html +/dist +/docs/_site +/README.rst +/*venv diff --git a/.go-version b/.go-version deleted file mode 100644 index 5759850c03e..00000000000 --- a/.go-version +++ /dev/null @@ -1 +0,0 @@ -1.25.7 \ No newline at end of file diff --git a/.golangci.yml b/.golangci.yml deleted file mode 100644 index 7d75550fea0..00000000000 --- a/.golangci.yml +++ /dev/null @@ -1,110 +0,0 @@ -version: "2" -run: - concurrency: 2 -linters: - default: none - enable: - - copyloopvar - - depguard - - errcheck - - errorlint - - forbidigo - - gocritic - - gocyclo - - gomodguard - - govet - - ineffassign - - lll - - misspell - - nakedret - - nolintlint - - revive - - staticcheck - - testifylint - - unconvert - - unparam - - unused - settings: - depguard: - rules: - all: - deny: - - pkg: io/ioutil - desc: io/ioutil package has been deprecated - - pkg: github.com/docker/docker/errdefs - desc: use github.com/containerd/errdefs instead. - - pkg: golang.org/x/exp/maps - desc: use stdlib maps package - - pkg: golang.org/x/exp/slices - desc: use stdlib slices package - - pkg: gopkg.in/yaml.v2 - desc: compose-go uses yaml.v3 - forbidigo: - analyze-types: true - forbid: - - pattern: 'context\.Background' - pkg: '^context$' - msg: "in tests, use t.Context() instead of context.Background()" - - pattern: 'context\.TODO' - pkg: '^context$' - msg: "in tests, use t.Context() instead of context.TODO()" - gocritic: - disabled-checks: - - paramTypeCombine - - unnamedResult - - whyNoLint - enabled-tags: - - diagnostic - - opinionated - - style - gocyclo: - min-complexity: 16 - gomodguard: - blocked: - modules: - - github.com/pkg/errors: - recommendations: - - errors - - fmt - versions: - - github.com/distribution/distribution: - reason: use distribution/reference - - gotest.tools: - version: < 3.0.0 - reason: deprecated, pre-modules version - lll: - line-length: 200 - revive: - rules: - - name: package-comments - disabled: true - exclusions: - generated: lax - paths: - - third_party$ - - builtin$ - - examples$ - rules: - - path-except: '_test\.go' - linters: - - forbidigo -issues: - max-issues-per-linter: 0 - max-same-issues: 0 -formatters: - enable: - - gci - - gofumpt - exclusions: - generated: lax - paths: - - third_party$ - - builtin$ - - examples$ - settings: - gci: - sections: - - standard - - default - - localmodule - custom-order: true # make the section order the same as the order of "sections". diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 00000000000..45f6f6fcf1f --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,28 @@ +- repo: git://github.com/pre-commit/pre-commit-hooks + sha: 'v0.9.1' + hooks: + - id: check-added-large-files + - id: check-docstring-first + - id: check-merge-conflict + - id: check-yaml + - id: check-json + - id: debug-statements + - id: end-of-file-fixer + - id: flake8 + - id: name-tests-test + exclude: 'tests/(integration/testcases\.py|helpers\.py)' + - id: requirements-txt-fixer + - id: trailing-whitespace +- repo: git://github.com/asottile/reorder_python_imports + sha: v1.3.4 + hooks: + - id: reorder-python-imports + language_version: 'python3.9' + args: + - --py3-plus +- repo: https://github.com/asottile/pyupgrade + rev: v2.1.0 + hooks: + - id: pyupgrade + args: + - --py3-plus diff --git a/BUILDING.md b/BUILDING.md deleted file mode 100644 index e9861f08140..00000000000 --- a/BUILDING.md +++ /dev/null @@ -1,95 +0,0 @@ - -### Prerequisites - -* Windows: - * [Docker Desktop](https://docs.docker.com/desktop/setup/install/windows-install/) - * make - * go (see [go.mod](go.mod) for minimum version) -* macOS: - * [Docker Desktop](https://docs.docker.com/desktop/setup/install/mac-install/) - * make - * go (see [go.mod](go.mod) for minimum version) -* Linux: - * [Docker 20.10 or later](https://docs.docker.com/engine/install/) - * make - * go (see [go.mod](go.mod) for minimum version) - -### Building the CLI - -Once you have the prerequisites installed, you can build the CLI using: - -```console -make -``` - -This will output a `docker-compose` CLI plugin for your host machine in -`./bin/build`. - -You can statically cross compile the CLI for Windows, macOS, and Linux using the -`cross` target. - -### Unit tests - -To run all of the unit tests, run: - -```console -make test -``` - -If you need to update a golden file simply do `go test ./... -test.update-golden`. - -### End-to-end tests -To run e2e tests, the Compose CLI binary needs to be built. All the commands to run e2e tests propose a version -with the prefix `build-and-e2e` to first build the CLI before executing tests. - -Note that this requires a local Docker Engine to be running. - -#### Whole end-to-end tests suite - -To execute both CLI and standalone e2e tests, run : - -```console -make e2e -``` - -Or if you need to build the CLI, run: -```console -make build-and-e2e -``` - -#### Plugin end-to-end tests suite - -To execute CLI plugin e2e tests, run : - -```console -make e2e-compose -``` - -Or if you need to build the CLI, run: -```console -make build-and-e2e-compose -``` - -#### Standalone end-to-end tests suite - -To execute the standalone CLI e2e tests, run : - -```console -make e2e-compose-standalone -``` - -Or if you need to build the CLI, run: - -```console -make build-and-e2e-compose-standalone -``` - -## Releases - -To create a new release: -* Check that the CI is green on the main branch for the commit you want to release -* Run the release GitHub Actions workflow with a tag of form vx.y.z following existing tags. - -This will automatically create a new tag, release and make binaries for -Windows, macOS, and Linux available for download on the -[releases page](https://github.com/docker/compose/releases). diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 00000000000..3a9078d27d6 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,2515 @@ +Change log +========== + +1.27.4 (2020-09-24) +------------------- + +### Bugs + +- Remove path checks for bind mounts + +- Fix port rendering to output long form syntax for non-v1 + +- Add protocol to the docker socket address + +1.27.3 (2020-09-16) +------------------- + +### Bugs + +- Merge `max_replicas_per_node` on `docker-compose config` + +- Fix `depends_on` serialization on `docker-compose config` + +- Fix scaling when some containers are not running on `docker-compose up` + +- Enable relative paths for `driver_opts.device` for `local` driver + +- Allow strings for `cpus` fields + +1.27.2 (2020-09-10) +------------------- + +### Bugs + +- Fix bug on `docker-compose run` container attach + +1.27.1 (2020-09-10) +------------------- + +### Bugs + +- Fix `docker-compose run` when `service.scale` is specified + +- Allow `driver` property for external networks as temporary workaround for swarm network propagation issue + +- Pin new internal schema version to `3.9` as the default + +- Preserve the version when configured in the compose file + +1.27.0 (2020-09-07) +------------------- + +### Features + +- Merge 2.x and 3.x compose formats and align with COMPOSE_SPEC schema + +- Implement service mode for ipc + +- Pass `COMPOSE_PROJECT_NAME` environment variable in container mode + +- Make run behave in the same way as up + +- Use `docker build` on `docker-compose run` when `COMPOSE_DOCKER_CLI_BUILD` environment variable is set + +- Use docker-py default API version for engine queries (`auto`) + +- Parse `network_mode` on build + +### Bugs + +- Ignore build context path validation when building is not required + +- Fix float to bytes conversion via docker-py bump to 4.3.1 + +- Fix scale bug when deploy section is set + +- Fix `docker-py` bump in `setup.py` + +- Fix experimental build failure detection + +- Fix context propagation to docker cli + +### Miscellaneous + +- Drop support for Python 2.7 + +- Bump `docker-py` to 4.3.1 + +- Bump `tox` to 3.19.0 + +- Bump `virtualenv` to 20.0.30 + +- Add script for docs syncronization + +1.26.2 (2020-07-02) +------------------- + +### Bugs + +- Enforce `docker-py` 4.2.2 as minimum version when installing with pip + +1.26.1 (2020-06-30) +------------------- + +### Features + +- Bump `docker-py` from 4.2.1 to 4.2.2 + +### Bugs + +- Enforce `docker-py` 4.2.1 as minimum version when installing with pip + +- Fix context load for non-docker endpoints + +1.26.0 (2020-06-03) +------------------- + +### Features + +- Add `docker context` support + +- Add missing test dependency `ddt` to `setup.py` + +- Add `--attach-dependencies` to command `up` for attaching to dependencies + +- Allow compatibility option with `COMPOSE_COMPATIBILITY` environment variable + +- Bump `Pytest` to 5.3.4 and add refactor compatibility with new version + +- Bump `OpenSSL` from 1.1.1f to 1.1.1g + +- Bump `docker-py` from 4.2.0 to 4.2.1 + +### Bugs + +- Properly escape values coming from env_files + +- Sync compose-schemas with upstream (docker/cli) + +- Remove `None` entries on exec command + +- Add `python-dotenv` to delegate `.env` file processing + +- Don't adjust output on terminal width when piped into another command + +- Show an error message when `version` attribute is malformed + +- Fix HTTPS connection when DOCKER_HOST is remote + +1.25.5 (2020-02-04) +------------------- + +### Features + +- Bump OpenSSL from 1.1.1d to 1.1.1f + +- Add 3.8 compose version + +1.25.4 (2020-01-23) +------------------- + +### Bugfixes + +- Fix CI script to enforce the minimal MacOS version to 10.11 + +- Fix docker-compose exec for keys with no value + +1.25.3 (2020-01-23) +------------------- + +### Bugfixes + +- Fix CI script to enforce the compilation with Python3 + +- Fix binary's sha256 in the release page + +1.25.2 (2020-01-20) +------------------- + +### Features + +- Allow compatibility option with `COMPOSE_COMPATIBILITY` environment variable + +- Bump PyInstaller from 3.5 to 3.6 + +- Bump pysocks from 1.6.7 to 1.7.1 + +- Bump websocket-client from 0.32.0 to 0.57.0 + +- Bump urllib3 from 1.24.2 to 1.25.7 + +- Bump jsonschema from 3.0.1 to 3.2.0 + +- Bump PyYAML from 4.2b1 to 5.3 + +- Bump certifi from 2017.4.17 to 2019.11.28 + +- Bump coverage from 4.5.4 to 5.0.3 + +- Bump paramiko from 2.6.0 to 2.7.1 + +- Bump cached-property from 1.3.0 to 1.5.1 + +- Bump minor Linux and MacOSX dependencies + +### Bugfixes + +- Validate version format on formats 2+ + +- Assume infinite terminal width when not running in a terminal + +1.25.1 (2020-01-06) +------------------- + +### Features + +- Bump `pytest-cov` 2.8.1 + +- Bump `flake8` 3.7.9 + +- Bump `coverage` 4.5.4 + +### Bugfixes + +- Decode APIError explanation to unicode before usage on start and create of a container + +- Reports when images that cannot be pulled and must be built + +- Discard label `com.docker.compose.filepaths` having None as value. Typically, when coming from stdin + +- Added OSX binary as a directory to solve slow start up time caused by MacOS Catalina binary scan + +- Passed in HOME env-var in container mode (running with `script/run/run.sh`) + +- Reverted behavior of "only pull images that we can't build" and replace by a warning informing the image we can't pull and must be built + + +1.25.0 (2019-11-18) +------------------- + +### Features + +- Set no-colors to true if CLICOLOR env variable is set to 0 + +- Add working dir, config files and env file in service labels + +- Add dependencies for ARM build + +- Add BuildKit support, use `DOCKER_BUILDKIT=1` and `COMPOSE_DOCKER_CLI_BUILD=1` + +- Bump paramiko to 2.6.0 + +- Add working dir, config files and env file in service labels + +- Add tag `docker-compose:latest` + +- Add `docker-compose:<version>-alpine` image/tag + +- Add `docker-compose:<version>-debian` image/tag + +- Bumped `docker-py` 4.1.0 + +- Supports `requests` up to 2.22.0 version + +- Drops empty tag on `build:cache_from` + +- `Dockerfile` now generates `libmusl` binaries for alpine + +- Only pull images that can't be built + +- Attribute `scale` can now accept `0` as a value + +- Added `--quiet` build flag + +- Added `--no-interpolate` to `docker-compose config` + +- Bump OpenSSL for macOS build (`1.1.0j` to `1.1.1c`) + +- Added `--no-rm` to `build` command + +- Added support for `credential_spec` + +- Resolve digests without pulling image + +- Upgrade `pyyaml` to `4.2b1` + +- Lowered severity to `warning` if `down` tries to remove nonexisting image + +- Use improved API fields for project events when possible + +- Update `setup.py` for modern `pypi/setuptools` and remove `pandoc` dependencies + +- Removed `Dockerfile.armhf` which is no longer needed + +### Bugfixes + +- Make container service color deterministic, remove red from chosen colors + +- Fix non ascii chars error. Python2 only + +- Format image size as decimal to be align with Docker CLI + +- Use Python Posix support to get tty size + +- Fix same file 'extends' optimization + +- Use python POSIX support to get tty size + +- Format image size as decimal to be align with Docker CLI + +- Fixed stdin_open + +- Fixed `--remove-orphans` when used with `up --no-start` + +- Fixed `docker-compose ps --all` + +- Fixed `depends_on` dependency recreation behavior + +- Fixed bash completion for `build --memory` + +- Fixed misleading warning concerning env vars when performing an `exec` command + +- Fixed failure check in parallel_execute_watch + +- Fixed race condition after pulling image + +- Fixed error on duplicate mount points + +- Fixed merge on networks section + +- Always connect Compose container to `stdin` + +- Fixed the presentation of failed services on 'docker-compose start' when containers are not available + +1.24.1 (2019-06-24) +------------------- + +### Bugfixes + +- Fixed acceptance tests + +1.24.0 (2019-03-28) +------------------- + +### Features + +- Added support for connecting to the Docker Engine using the `ssh` protocol. + +- Added a `--all` flag to `docker-compose ps` to include stopped one-off containers + in the command's output. + +- Add bash completion for `ps --all|-a` + +- Support for credential_spec + +- Add `--parallel` to `docker build`'s options in `bash` and `zsh` completion + +### Bugfixes + +- Fixed a bug where some valid credential helpers weren't properly handled by Compose + when attempting to pull images from private registries. + +- Fixed an issue where the output of `docker-compose start` before containers were created + was misleading + +- To match the Docker CLI behavior and to avoid confusing issues, Compose will no longer + accept whitespace in variable names sourced from environment files. + +- Compose will now report a configuration error if a service attempts to declare + duplicate mount points in the volumes section. + +- Fixed an issue with the containerized version of Compose that prevented users from + writing to stdin during interactive sessions started by `run` or `exec`. + +- One-off containers started by `run` no longer adopt the restart policy of the service, + and are instead set to never restart. + +- Fixed an issue that caused some container events to not appear in the output of + the `docker-compose events` command. + +- Missing images will no longer stop the execution of `docker-compose down` commands + (a warning will be displayed instead). + +- Force `virtualenv` version for macOS CI + +- Fix merging of compose files when network has `None` config + +- Fix `CTRL+C` issues by enabling `bootloader_ignore_signals` in `pyinstaller` + +- Bump `docker-py` version to `3.7.2` to fix SSH and proxy config issues + +- Fix release script and some typos on release documentation + +1.23.2 (2018-11-28) +------------------- + +### Bugfixes + +- Reverted a 1.23.0 change that appended random strings to container names + created by `docker-compose up`, causing addressability issues. + Note: Containers created by `docker-compose run` will continue to use + randomly generated names to avoid collisions during parallel runs. + +- Fixed an issue where some `dockerfile` paths would fail unexpectedly when + attempting to build on Windows. + +- Fixed a bug where build context URLs would fail to build on Windows. + +- Fixed a bug that caused `run` and `exec` commands to fail for some otherwise + accepted values of the `--host` parameter. + +- Fixed an issue where overrides for the `storage_opt` and `isolation` keys in + service definitions weren't properly applied. + +- Fixed a bug where some invalid Compose files would raise an uncaught + exception during validation. + +1.23.1 (2018-11-01) +------------------- + +### Bugfixes + +- Fixed a bug where working with containers created with a previous (< 1.23.0) + version of Compose would cause unexpected crashes + +- Fixed an issue where the behavior of the `--project-directory` flag would + vary depending on which subcommand was being used. + +1.23.0 (2018-10-30) +------------------- + +### Important note + +The default naming scheme for containers created by Compose in this version +has changed from `<project>_<service>_<index>` to +`<project>_<service>_<index>_<slug>`, where `<slug>` is a randomly-generated +hexadecimal string. Please make sure to update scripts relying on the old +naming scheme accordingly before upgrading. + +### Features + +- Logs for containers restarting after a crash will now appear in the output + of the `up` and `logs` commands. + +- Added `--hash` option to the `docker-compose config` command, allowing users + to print a hash string for each service's configuration to facilitate rolling + updates. + +- Added `--parallel` flag to the `docker-compose build` command, allowing + Compose to build up to 5 images simultaneously. + +- Output for the `pull` command now reports status / progress even when pulling + multiple images in parallel. + +- For images with multiple names, Compose will now attempt to match the one + present in the service configuration in the output of the `images` command. + +### Bugfixes + +- Parallel `run` commands for the same service will no longer fail due to name + collisions. + +- Fixed an issue where paths longer than 260 characters on Windows clients would + cause `docker-compose build` to fail. + +- Fixed a bug where attempting to mount `/var/run/docker.sock` with + Docker Desktop for Windows would result in failure. + +- The `--project-directory` option is now used by Compose to determine where to + look for the `.env` file. + +- `docker-compose build` no longer fails when attempting to pull an image with + credentials provided by the gcloud credential helper. + +- Fixed the `--exit-code-from` option in `docker-compose up` to always report + the actual exit code even when the watched container isn't the cause of the + exit. + +- Fixed an issue that would prevent recreating a service in some cases where + a volume would be mapped to the same mountpoint as a volume declared inside + the image's Dockerfile. + +- Fixed a bug that caused hash configuration with multiple networks to be + inconsistent, causing some services to be unnecessarily restarted. + +- Fixed a bug that would cause failures with variable substitution for services + with a name containing one or more dot characters + +- Fixed a pipe handling issue when using the containerized version of Compose. + +- Fixed a bug causing `external: false` entries in the Compose file to be + printed as `external: true` in the output of `docker-compose config` + +- Fixed a bug where issuing a `docker-compose pull` command on services + without a defined image key would cause Compose to crash + +- Volumes and binds are now mounted in the order they're declared in the + service definition + +### Miscellaneous + +- The `zsh` completion script has been updated with new options, and no + longer suggests container names where service names are expected. + +1.22.0 (2018-07-17) +------------------- + +### Features + +#### Compose format version 3.7 + +- Introduced version 3.7 of the `docker-compose.yml` specification. + This version requires Docker Engine 18.06.0 or above. + +- Added support for `rollback_config` in the deploy configuration + +- Added support for the `init` parameter in service configurations + +- Added support for extension fields in service, network, volume, secret, + and config configurations + +#### Compose format version 2.4 + +- Added support for extension fields in service, network, + and volume configurations + +### Bugfixes + +- Fixed a bug that prevented deployment with some Compose files when + `DOCKER_DEFAULT_PLATFORM` was set + +- Compose will no longer try to create containers or volumes with + invalid starting characters + +- Fixed several bugs that prevented Compose commands from working properly + with containers created with an older version of Compose + +- Fixed an issue with the output of `docker-compose config` with the + `--compatibility-mode` flag enabled when the source file contains + attachable networks + +- Fixed a bug that prevented the `gcloud` credential store from working + properly when used with the Compose binary on UNIX + +- Fixed a bug that caused connection errors when trying to operate + over a non-HTTPS TCP connection on Windows + +- Fixed a bug that caused builds to fail on Windows if the Dockerfile + was located in a subdirectory of the build context + +- Fixed an issue that prevented proper parsing of UTF-8 BOM encoded + Compose files on Windows + +- Fixed an issue with handling of the double-wildcard (`**`) pattern in `.dockerignore` files when using `docker-compose build` + +- Fixed a bug that caused auth values in legacy `.dockercfg` files to be ignored +- `docker-compose build` will no longer attempt to create image names starting with an invalid character + +1.21.2 (2018-05-03) +------------------- + +### Bugfixes + +- Fixed a bug where the ip_range attribute in IPAM configs was prevented + from passing validation + +1.21.1 (2018-04-27) +------------------- + +### Bugfixes + +- In 1.21.0, we introduced a change to how project names are sanitized for + internal use in resource names. This caused issues when manipulating an + existing, deployed application whose name had changed as a result. + This release properly detects resources using "legacy" naming conventions. + +- Fixed an issue where specifying an in-context Dockerfile using an absolute + path would fail despite being valid. + +- Fixed a bug where IPAM option changes were incorrectly detected, preventing + redeployments. + +- Validation of v2 files now properly checks the structure of IPAM configs. + +- Improved support for credentials stores on Windows to include binaries using + extensions other than `.exe`. The list of valid extensions is determined by + the contents of the `PATHEXT` environment variable. + +- Fixed a bug where Compose would generate invalid binds containing duplicate + elements with some v3.2 files, triggering errors at the Engine level during + deployment. + +1.21.0 (2018-04-10) +------------------- + +### New features + +#### Compose file version 2.4 + +- Introduced version 2.4 of the `docker-compose.yml` specification. + This version requires Docker Engine 17.12.0 or above. + +- Added support for the `platform` parameter in service definitions. + If supplied, the parameter is also used when performing build for the + service. + +#### Compose file version 2.2 and up + +- Added support for the `cpu_rt_period` and `cpu_rt_runtime` parameters + in service definitions (2.x only). + +#### Compose file version 2.1 and up + +- Added support for the `cpu_period` parameter in service definitions + (2.x only). + +- Added support for the `isolation` parameter in service build configurations. + Additionally, the `isolation` parameter is used for builds as well if no + `build.isolation` parameter is defined. (2.x only) + +#### All formats + +- Added support for the `--workdir` flag in `docker-compose exec`. + +- Added support for the `--compress` flag in `docker-compose build`. + +- `docker-compose pull` is now performed in parallel by default. You can + opt out using the `--no-parallel` flag. The `--parallel` flag is now + deprecated and will be removed in a future version. + +- Dashes and underscores in project names are no longer stripped out. + +- `docker-compose build` now supports the use of Dockerfile from outside + the build context. + +### Bugfixes + +- Compose now checks that the volume's configuration matches the remote + volume, and errors out if a mismatch is detected. + +- Fixed a bug that caused Compose to raise unexpected errors when attempting + to create several one-off containers in parallel. + +- Fixed a bug with argument parsing when using `docker-machine config` to + generate TLS flags for `exec` and `run` commands. + +- Fixed a bug where variable substitution with an empty default value + (e.g. `${VAR:-}`) would print an incorrect warning. + +- Improved resilience when encoding of the Compose file doesn't match the + system's. Users are encouraged to use UTF-8 when possible. + +- Fixed a bug where external overlay networks in Swarm would be incorrectly + recognized as inexistent by Compose, interrupting otherwise valid + operations. + +1.20.1 (2018-03-21) +------------------- + +### Bugfixes + +- Fixed an issue where `docker-compose build` would error out if the + build context contained directory symlinks + +1.20.0 (2018-03-20) +------------------- + +### New features + +#### Compose file version 3.6 + +- Introduced version 3.6 of the `docker-compose.yml` specification. + This version requires Docker Engine 18.02.0 or above. + +- Added support for the `tmpfs.size` property in volume mappings + +#### Compose file version 3.2 and up + +- The `--build-arg` option can now be used without specifying a service + in `docker-compose build` + +#### Compose file version 2.3 + +- Added support for `device_cgroup_rules` in service definitions + +- Added support for the `tmpfs.size` property in long-form volume mappings + +- The `--build-arg` option can now be used without specifying a service + in `docker-compose build` + +#### All formats + +- Added a `--log-level` option to the top-level `docker-compose` command. + Accepted values are `debug`, `info`, `warning`, `error`, `critical`. + Default log level is `info` + +- `docker-compose run` now allows users to unset the container's entrypoint + +- Proxy configuration found in the `~/.docker/config.json` file now populates + environment and build args for containers created by Compose + +- Added the `--use-aliases` flag to `docker-compose run`, indicating that + network aliases declared in the service's config should be used for the + running container + +- Added the `--include-deps` flag to `docker-compose pull` + +- `docker-compose run` now kills and removes the running container upon + receiving `SIGHUP` + +- `docker-compose ps` now shows the containers' health status if available + +- Added the long-form `--detach` option to the `exec`, `run` and `up` + commands + +### Bugfixes + +- Fixed `.dockerignore` handling, notably with regard to absolute paths + and last-line precedence rules + +- Fixed an issue where Compose would make costly DNS lookups when connecting + to the Engine when using Docker For Mac + +- Fixed a bug introduced in 1.19.0 which caused the default certificate path + to not be honored by Compose + +- Fixed a bug where Compose would incorrectly check whether a symlink's + destination was accessible when part of a build context + +- Fixed a bug where `.dockerignore` files containing lines of whitespace + caused Compose to error out on Windows + +- Fixed a bug where `--tls*` and `--host` options wouldn't be properly honored + for interactive `run` and `exec` commands + +- A `seccomp:<filepath>` entry in the `security_opt` config now correctly + sends the contents of the file to the engine + +- ANSI output for `up` and `down` operations should no longer affect the wrong + lines + +- Improved support for non-unicode locales + +- Fixed a crash occurring on Windows when the user's home directory name + contained non-ASCII characters + +- Fixed a bug occurring during builds caused by files with a negative `mtime` + values in the build context + +- Fixed an encoding bug when streaming build progress + +1.19.0 (2018-02-07) +------------------- + +### Breaking changes + +- On UNIX platforms, interactive `run` and `exec` commands now require + the `docker` CLI to be installed on the client by default. To revert + to the previous behavior, users may set the `COMPOSE_INTERACTIVE_NO_CLI` + environment variable. + +### New features + +#### Compose file version 3.x + +- The output of the `config` command should now merge `deploy` options from + several Compose files in a more accurate manner + +#### Compose file version 2.3 + +- Added support for the `runtime` option in service definitions + +#### Compose file version 2.1 and up + +- Added support for the `${VAR:?err}` and `${VAR?err}` variable interpolation + syntax to indicate mandatory variables + +#### Compose file version 2.x + +- Added `priority` key to service network mappings, allowing the user to + define in which order the specified service will connect to each network + +#### All formats + +- Added `--renew-anon-volumes` (shorthand `-V`) to the `up` command, + preventing Compose from recovering volume data from previous containers for + anonymous volumes + +- Added limit for number of simultaneous parallel operations, which should + prevent accidental resource exhaustion of the server. Default is 64 and + can be configured using the `COMPOSE_PARALLEL_LIMIT` environment variable + +- Added `--always-recreate-deps` flag to the `up` command to force recreating + dependent services along with the dependency owner + +- Added `COMPOSE_IGNORE_ORPHANS` environment variable to forgo orphan + container detection and suppress warnings + +- Added `COMPOSE_FORCE_WINDOWS_HOST` environment variable to force Compose + to parse volume definitions as if the Docker host was a Windows system, + even if Compose itself is currently running on UNIX + +- Bash completion should now be able to better differentiate between running, + stopped and paused services + +### Bugfixes + +- Fixed a bug that would cause the `build` command to report a connection + error when the build context contained unreadable files or FIFO objects. + These file types will now be handled appropriately + +- Fixed various issues around interactive `run`/`exec` sessions. + +- Fixed a bug where setting TLS options with environment and CLI flags + simultaneously would result in part of the configuration being ignored + +- Fixed a bug where the DOCKER_TLS_VERIFY environment variable was being + ignored by Compose + +- Fixed a bug where the `-d` and `--timeout` flags in `up` were erroneously + marked as incompatible + +- Fixed a bug where the recreation of a service would break if the image + associated with the previous container had been removed + +- Fixed a bug where updating a mount's target would break Compose when + trying to recreate the associated service + +- Fixed a bug where `tmpfs` volumes declared using the extended syntax in + Compose files using version 3.2 would be erroneously created as anonymous + volumes instead + +- Fixed a bug where type conversion errors would print a stacktrace instead + of exiting gracefully + +- Fixed some errors related to unicode handling + +- Dependent services no longer get recreated along with the dependency owner + if their configuration hasn't changed + +- Added better validation of `labels` fields in Compose files. Label values + containing scalar types (number, boolean) now get automatically converted + to strings + +1.18.0 (2017-12-15) +------------------- + +### New features + +#### Compose file version 3.5 + +- Introduced version 3.5 of the `docker-compose.yml` specification. + This version requires Docker Engine 17.06.0 or above + +- Added support for the `shm_size` parameter in build configurations + +- Added support for the `isolation` parameter in service definitions + +- Added support for custom names for network, secret and config definitions + +#### Compose file version 2.3 + +- Added support for `extra_hosts` in build configuration + +- Added support for the [long syntax](https://docs.docker.com/compose/compose-file/#long-syntax-3) for volume entries, as previously introduced in the 3.2 format. + Note that using this syntax will create [mounts](https://docs.docker.com/engine/admin/volumes/bind-mounts/) instead of volumes. + +#### Compose file version 2.1 and up + +- Added support for the `oom_kill_disable` parameter in service definitions + (2.x only) + +- Added support for custom names for network definitions (2.x only) + + +#### All formats + +- Values interpolated from the environment will now be converted to the + proper type when used in non-string fields. + +- Added support for `--label` in `docker-compose run` + +- Added support for `--timeout` in `docker-compose down` + +- Added support for `--memory` in `docker-compose build` + +- Setting `stop_grace_period` in service definitions now also sets the + container's `stop_timeout` + +### Bugfixes + +- Fixed an issue where Compose was still handling service hostname according + to legacy engine behavior, causing hostnames containing dots to be cut up + +- Fixed a bug where the `X-Y:Z` syntax for ports was considered invalid + by Compose + +- Fixed an issue with CLI logging causing duplicate messages and inelegant + output to occur + +- Fixed an issue that caused `stop_grace_period` to be ignored when using + multiple Compose files + +- Fixed a bug that caused `docker-compose images` to crash when using + untagged images + +- Fixed a bug where the valid `${VAR:-}` syntax would cause Compose to + error out + +- Fixed a bug where `env_file` entries using an UTF-8 BOM were being read + incorrectly + +- Fixed a bug where missing secret files would generate an empty directory + in their place + +- Fixed character encoding issues in the CLI's error handlers + +- Added validation for the `test` field in healthchecks + +- Added validation for the `subnet` field in IPAM configurations + +- Added validation for `volumes` properties when using the long syntax in + service definitions + +- The CLI now explicit prevents using `-d` and `--timeout` together + in `docker-compose up` + +1.17.1 (2017-11-08) +------------------ + +### Bugfixes + +- Fixed a bug that would prevent creating new containers when using + container labels in the list format as part of the service's definition. + +1.17.0 (2017-11-02) +------------------- + +### New features + +#### Compose file version 3.4 + +- Introduced version 3.4 of the `docker-compose.yml` specification. + This version requires to be used with Docker Engine 17.06.0 or above. + +- Added support for `cache_from`, `network` and `target` options in build + configurations + +- Added support for the `order` parameter in the `update_config` section + +- Added support for setting a custom name in volume definitions using + the `name` parameter + +#### Compose file version 2.3 + +- Added support for `shm_size` option in build configuration + +#### Compose file version 2.x + +- Added support for extension fields (`x-*`). Also available for v3.4 files + +#### All formats + +- Added new `--no-start` to the `up` command, allowing users to create all + resources (networks, volumes, containers) without starting services. + The `create` command is deprecated in favor of this new option + +### Bugfixes + +- Fixed a bug where `extra_hosts` values would be overridden by extension + files instead of merging together + +- Fixed a bug where the validation for v3.2 files would prevent using the + `consistency` field in service volume definitions + +- Fixed a bug that would cause a crash when configuration fields expecting + unique items would contain duplicates + +- Fixed a bug where mount overrides with a different mode would create a + duplicate entry instead of overriding the original entry + +- Fixed a bug where build labels declared as a list wouldn't be properly + parsed + +- Fixed a bug where the output of `docker-compose config` would be invalid + for some versions if the file contained custom-named external volumes + +- Improved error handling when issuing a build command on Windows using an + unsupported file version + +- Fixed an issue where networks with identical names would sometimes be + created when running `up` commands concurrently. + +1.16.1 (2017-09-01) +------------------- + +### Bugfixes + +- Fixed bug that prevented using `extra_hosts` in several configuration files. + +1.16.0 (2017-08-31) +------------------- + +### New features + +#### Compose file version 2.3 + +- Introduced version 2.3 of the `docker-compose.yml` specification. + This version requires to be used with Docker Engine 17.06.0 or above. + +- Added support for the `target` parameter in build configurations + +- Added support for the `start_period` parameter in healthcheck + configurations + +#### Compose file version 2.x + +- Added support for the `blkio_config` parameter in service definitions + +- Added support for setting a custom name in volume definitions using + the `name` parameter (not available for version 2.0) + +#### All formats + +- Added new CLI flag `--no-ansi` to suppress ANSI control characters in + output + +### Bugfixes + +- Fixed a bug where nested `extends` instructions weren't resolved + properly, causing "file not found" errors + +- Fixed several issues with `.dockerignore` parsing + +- Fixed issues where logs of TTY-enabled services were being printed + incorrectly and causing `MemoryError` exceptions + +- Fixed a bug where printing application logs would sometimes be interrupted + by a `UnicodeEncodeError` exception on Python 3 + +- The `$` character in the output of `docker-compose config` is now + properly escaped + +- Fixed a bug where running `docker-compose top` would sometimes fail + with an uncaught exception + +- Fixed a bug where `docker-compose pull` with the `--parallel` flag + would return a `0` exit code when failing + +- Fixed an issue where keys in `deploy.resources` were not being validated + +- Fixed an issue where the `logging` options in the output of + `docker-compose config` would be set to `null`, an invalid value + +- Fixed the output of the `docker-compose images` command when an image + would come from a private repository using an explicit port number + +- Fixed the output of `docker-compose config` when a port definition used + `0` as the value for the published port + +1.15.0 (2017-07-26) +------------------- + +### New features + +#### Compose file version 2.2 + +- Added support for the `network` parameter in build configurations. + +#### Compose file version 2.1 and up + +- The `pid` option in a service's definition now supports a `service:<name>` + value. + +- Added support for the `storage_opt` parameter in in service definitions. + This option is not available for the v3 format + +#### All formats + +- Added `--quiet` flag to `docker-compose pull`, suppressing progress output + +- Some improvements to CLI output + +### Bugfixes + +- Volumes specified through the `--volume` flag of `docker-compose run` now + complement volumes declared in the service's definition instead of replacing + them + +- Fixed a bug where using multiple Compose files would unset the scale value + defined inside the Compose file. + +- Fixed an issue where the `credHelpers` entries in the `config.json` file + were not being honored by Compose + +- Fixed a bug where using multiple Compose files with port declarations + would cause failures in Python 3 environments + +- Fixed a bug where some proxy-related options present in the user's + environment would prevent Compose from running + +- Fixed an issue where the output of `docker-compose config` would be invalid + if the original file used `Y` or `N` values + +- Fixed an issue preventing `up` operations on a previously created stack on + Windows Engine. + +1.14.0 (2017-06-19) +------------------- + +### New features + +#### Compose file version 3.3 + +- Introduced version 3.3 of the `docker-compose.yml` specification. + This version requires to be used with Docker Engine 17.06.0 or above. + Note: the `credential_spec` and `configs` keys only apply to Swarm services + and will be ignored by Compose + +#### Compose file version 2.2 + +- Added the following parameters in service definitions: `cpu_count`, + `cpu_percent`, `cpus` + +#### Compose file version 2.1 + +- Added support for build labels. This feature is also available in the + 2.2 and 3.3 formats. + +#### All formats + +- Added shorthand `-u` for `--user` flag in `docker-compose exec` + +- Differences in labels between the Compose file and remote network + will now print a warning instead of preventing redeployment. + +### Bugfixes + +- Fixed a bug where service's dependencies were being rescaled to their + default scale when running a `docker-compose run` command + +- Fixed a bug where `docker-compose rm` with the `--stop` flag was not + behaving properly when provided with a list of services to remove + +- Fixed a bug where `cache_from` in the build section would be ignored when + using more than one Compose file. + +- Fixed a bug that prevented binding the same port to different IPs when + using more than one Compose file. + +- Fixed a bug where override files would not be picked up by Compose if they + had the `.yaml` extension + +- Fixed a bug on Windows Engine where networks would be incorrectly flagged + for recreation + +- Fixed a bug where services declaring ports would cause crashes on some + versions of Python 3 + +- Fixed a bug where the output of `docker-compose config` would sometimes + contain invalid port definitions + +1.13.0 (2017-05-02) +------------------- + +### Breaking changes + +- `docker-compose up` now resets a service's scaling to its default value. + You can use the newly introduced `--scale` option to specify a custom + scale value + +### New features + +#### Compose file version 2.2 + +- Introduced version 2.2 of the `docker-compose.yml` specification. This + version requires to be used with Docker Engine 1.13.0 or above + +- Added support for `init` in service definitions. + +- Added support for `scale` in service definitions. The configuration's value + can be overridden using the `--scale` flag in `docker-compose up`. + Please note that the `scale` command is disabled for this file format + +#### Compose file version 2.x + +- Added support for `options` in the `ipam` section of network definitions + +### Bugfixes + +- Fixed a bug where paths provided to compose via the `-f` option were not + being resolved properly + +- Fixed a bug where the `ext_ip::target_port` notation in the ports section + was incorrectly marked as invalid + +- Fixed an issue where the `exec` command would sometimes not return control + to the terminal when using the `-d` flag + +- Fixed a bug where secrets were missing from the output of the `config` + command for v3.2 files + +- Fixed an issue where `docker-compose` would hang if no internet connection + was available + +- Fixed an issue where paths containing unicode characters passed via the `-f` + flag were causing Compose to crash + +- Fixed an issue where the output of `docker-compose config` would be invalid + if the Compose file contained external secrets + +- Fixed a bug where using `--exit-code-from` with `up` would fail if Compose + was installed in a Python 3 environment + +- Fixed a bug where recreating containers using a combination of `tmpfs` and + `volumes` would result in an invalid config state + + +1.12.0 (2017-04-04) +------------------- + +### New features + +#### Compose file version 3.2 + +- Introduced version 3.2 of the `docker-compose.yml` specification + +- Added support for `cache_from` in the `build` section of services + +- Added support for the new expanded ports syntax in service definitions + +- Added support for the new expanded volumes syntax in service definitions + +#### Compose file version 2.1 + +- Added support for `pids_limit` in service definitions + +#### Compose file version 2.0 and up + +- Added `--volumes` option to `docker-compose config` that lists named + volumes declared for that project + +- Added support for `mem_reservation` in service definitions (2.x only) + +- Added support for `dns_opt` in service definitions (2.x only) + +#### All formats + +- Added a new `docker-compose images` command that lists images used by + the current project's containers + +- Added a `--stop` (shorthand `-s`) option to `docker-compose rm` that stops + the running containers before removing them + +- Added a `--resolve-image-digests` option to `docker-compose config` that + pins the image version for each service to a permanent digest + +- Added a `--exit-code-from SERVICE` option to `docker-compose up`. When + used, `docker-compose` will exit on any container's exit with the code + corresponding to the specified service's exit code + +- Added a `--parallel` option to `docker-compose pull` that enables images + for multiple services to be pulled simultaneously + +- Added a `--build-arg` option to `docker-compose build` + +- Added a `--volume <volume_mapping>` (shorthand `-v`) option to + `docker-compose run` to declare runtime volumes to be mounted + +- Added a `--project-directory PATH` option to `docker-compose` that will + affect path resolution for the project + +- When using `--abort-on-container-exit` in `docker-compose up`, the exit + code for the container that caused the abort will be the exit code of + the `docker-compose up` command + +- Users can now configure which path separator character they want to use + to separate the `COMPOSE_FILE` environment value using the + `COMPOSE_PATH_SEPARATOR` environment variable + +- Added support for port range to single port in port mappings + (e.g. `8000-8010:80`) + +### Bugfixes + +- `docker-compose run --rm` now removes anonymous volumes after execution, + matching the behavior of `docker run --rm`. + +- Fixed a bug where override files containing port lists would cause a + TypeError to be raised + +- Fixed a bug where the `deploy` key would be missing from the output of + `docker-compose config` + +- Fixed a bug where scaling services up or down would sometimes re-use + obsolete containers + +- Fixed a bug where the output of `docker-compose config` would be invalid + if the project declared anonymous volumes + +- Variable interpolation now properly occurs in the `secrets` section of + the Compose file + +- The `secrets` section now properly appears in the output of + `docker-compose config` + +- Fixed a bug where changes to some networks properties would not be + detected against previously created networks + +- Fixed a bug where `docker-compose` would crash when trying to write into + a closed pipe + +- Fixed an issue where Compose would not pick up on the value of + COMPOSE_TLS_VERSION when used in combination with command-line TLS flags + +1.11.2 (2017-02-17) +------------------- + +### Bugfixes + +- Fixed a bug that was preventing secrets configuration from being + loaded properly + +- Fixed a bug where the `docker-compose config` command would fail + if the config file contained secrets definitions + +- Fixed an issue where Compose on some linux distributions would + pick up and load an outdated version of the requests library + +- Fixed an issue where socket-type files inside a build folder + would cause `docker-compose` to crash when trying to build that + service + +- Fixed an issue where recursive wildcard patterns `**` were not being + recognized in `.dockerignore` files. + +1.11.1 (2017-02-09) +------------------- + +### Bugfixes + +- Fixed a bug where the 3.1 file format was not being recognized as valid + by the Compose parser + +1.11.0 (2017-02-08) +------------------- + +### New Features + +#### Compose file version 3.1 + +- Introduced version 3.1 of the `docker-compose.yml` specification. This + version requires Docker Engine 1.13.0 or above. It introduces support + for secrets. See the documentation for more information + +#### Compose file version 2.0 and up + +- Introduced the `docker-compose top` command that displays processes running + for the different services managed by Compose. + +### Bugfixes + +- Fixed a bug where extending a service defining a healthcheck dictionary + would cause `docker-compose` to error out. + +- Fixed an issue where the `pid` entry in a service definition was being + ignored when using multiple Compose files. + +1.10.1 (2017-02-01) +------------------ + +### Bugfixes + +- Fixed an issue where presence of older versions of the docker-py + package would cause unexpected crashes while running Compose + +- Fixed an issue where healthcheck dependencies would be lost when + using multiple compose files for a project + +- Fixed a few issues that made the output of the `config` command + invalid + +- Fixed an issue where adding volume labels to v3 Compose files would + result in an error + +- Fixed an issue on Windows where build context paths containing unicode + characters were being improperly encoded + +- Fixed a bug where Compose would occasionally crash while streaming logs + when containers would stop or restart + +1.10.0 (2017-01-18) +------------------- + +### New Features + +#### Compose file version 3.0 + +- Introduced version 3.0 of the `docker-compose.yml` specification. This + version requires to be used with Docker Engine 1.13 or above and is + specifically designed to work with the `docker stack` commands. + +#### Compose file version 2.1 and up + +- Healthcheck configuration can now be done in the service definition using + the `healthcheck` parameter + +- Containers dependencies can now be set up to wait on positive healthchecks + when declared using `depends_on`. See the documentation for the updated + syntax. + **Note:** This feature will not be ported to version 3 Compose files. + +- Added support for the `sysctls` parameter in service definitions + +- Added support for the `userns_mode` parameter in service definitions + +- Compose now adds identifying labels to networks and volumes it creates + +#### Compose file version 2.0 and up + +- Added support for the `stop_grace_period` option in service definitions. + +### Bugfixes + +- Colored output now works properly on Windows. + +- Fixed a bug where docker-compose run would fail to set up link aliases + in interactive mode on Windows. + +- Networks created by Compose are now always made attachable + (Compose files v2.1 and up). + +- Fixed a bug where falsy values of `COMPOSE_CONVERT_WINDOWS_PATHS` + (`0`, `false`, empty value) were being interpreted as true. + +- Fixed a bug where forward slashes in some .dockerignore patterns weren't + being parsed correctly on Windows + + +1.9.0 (2016-11-16) +----------------- + +**Breaking changes** + +- When using Compose with Docker Toolbox/Machine on Windows, volume paths are + no longer converted from `C:\Users` to `/c/Users`-style by default. To + re-enable this conversion so that your volumes keep working, set the + environment variable `COMPOSE_CONVERT_WINDOWS_PATHS=1`. Users of + Docker for Windows are not affected and do not need to set the variable. + +New Features + +- Interactive mode for `docker-compose run` and `docker-compose exec` is + now supported on Windows platforms. Please note that the `docker` binary + is required to be present on the system for this feature to work. + +- Introduced version 2.1 of the `docker-compose.yml` specification. This + version requires to be used with Docker Engine 1.12 or above. + - Added support for setting volume labels and network labels in + `docker-compose.yml`. + - Added support for the `isolation` parameter in service definitions. + - Added support for link-local IPs in the service networks definitions. + - Added support for shell-style inline defaults in variable interpolation. + The supported forms are `${FOO-default}` (fall back if FOO is unset) and + `${FOO:-default}` (fall back if FOO is unset or empty). + +- Added support for the `group_add` and `oom_score_adj` parameters in + service definitions. + +- Added support for the `internal` and `enable_ipv6` parameters in network + definitions. + +- Compose now defaults to using the `npipe` protocol on Windows. + +- Overriding a `logging` configuration will now properly merge the `options` + mappings if the `driver` values do not conflict. + +Bug Fixes + +- Fixed several bugs related to `npipe` protocol support on Windows. + +- Fixed an issue with Windows paths being incorrectly converted when + using Docker on Windows Server. + +- Fixed a bug where an empty `restart` value would sometimes result in an + exception being raised. + +- Fixed an issue where service logs containing unicode characters would + sometimes cause an error to occur. + +- Fixed a bug where unicode values in environment variables would sometimes + raise a unicode exception when retrieved. + +- Fixed an issue where Compose would incorrectly detect a configuration + mismatch for overlay networks. + + +1.8.1 (2016-09-22) +----------------- + +Bug Fixes + +- Fixed a bug where users using a credentials store were not able + to access their private images. + +- Fixed a bug where users using identity tokens to authenticate + were not able to access their private images. + +- Fixed a bug where an `HttpHeaders` entry in the docker configuration + file would cause Compose to crash when trying to build an image. + +- Fixed a few bugs related to the handling of Windows paths in volume + binding declarations. + +- Fixed a bug where Compose would sometimes crash while trying to + read a streaming response from the engine. + +- Fixed an issue where Compose would crash when encountering an API error + while streaming container logs. + +- Fixed an issue where Compose would erroneously try to output logs from + drivers not handled by the Engine's API. + +- Fixed a bug where options from the `docker-machine config` command would + not be properly interpreted by Compose. + +- Fixed a bug where the connection to the Docker Engine would + sometimes fail when running a large number of services simultaneously. + +- Fixed an issue where Compose would sometimes print a misleading + suggestion message when running the `bundle` command. + +- Fixed a bug where connection errors would not be handled properly by + Compose during the project initialization phase. + +- Fixed a bug where a misleading error would appear when encountering + a connection timeout. + + +1.8.0 (2016-06-14) +----------------- + +**Breaking Changes** + +- As announced in 1.7.0, `docker-compose rm` now removes containers + created by `docker-compose run` by default. + +- Setting `entrypoint` on a service now empties out any default + command that was set on the image (i.e. any `CMD` instruction in the + Dockerfile used to build it). This makes it consistent with + the `--entrypoint` flag to `docker run`. + +New Features + +- Added `docker-compose bundle`, a command that builds a bundle file + to be consumed by the new *Docker Stack* commands in Docker 1.12. + +- Added `docker-compose push`, a command that pushes service images + to a registry. + +- Compose now supports specifying a custom TLS version for + interaction with the Docker Engine using the `COMPOSE_TLS_VERSION` + environment variable. + +Bug Fixes + +- Fixed a bug where Compose would erroneously try to read `.env` + at the project's root when it is a directory. + +- `docker-compose run -e VAR` now passes `VAR` through from the shell + to the container, as with `docker run -e VAR`. + +- Improved config merging when multiple compose files are involved + for several service sub-keys. + +- Fixed a bug where volume mappings containing Windows drives would + sometimes be parsed incorrectly. + +- Fixed a bug in Windows environment where volume mappings of the + host's root directory would be parsed incorrectly. + +- Fixed a bug where `docker-compose config` would output an invalid + Compose file if external networks were specified. + +- Fixed an issue where unset buildargs would be assigned a string + containing `'None'` instead of the expected empty value. + +- Fixed a bug where yes/no prompts on Windows would not show before + receiving input. + +- Fixed a bug where trying to `docker-compose exec` on Windows + without the `-d` option would exit with a stacktrace. This will + still fail for the time being, but should do so gracefully. + +- Fixed a bug where errors during `docker-compose up` would show + an unrelated stacktrace at the end of the process. + +- `docker-compose create` and `docker-compose start` show more + descriptive error messages when something goes wrong. + + +1.7.1 (2016-05-04) +----------------- + +Bug Fixes + +- Fixed a bug where the output of `docker-compose config` for v1 files + would be an invalid configuration file. + +- Fixed a bug where `docker-compose config` would not check the validity + of links. + +- Fixed an issue where `docker-compose help` would not output a list of + available commands and generic options as expected. + +- Fixed an issue where filtering by service when using `docker-compose logs` + would not apply for newly created services. + +- Fixed a bug where unchanged services would sometimes be recreated in + in the up phase when using Compose with Python 3. + +- Fixed an issue where API errors encountered during the up phase would + not be recognized as a failure state by Compose. + +- Fixed a bug where Compose would raise a NameError because of an undefined + exception name on non-Windows platforms. + +- Fixed a bug where the wrong version of `docker-py` would sometimes be + installed alongside Compose. + +- Fixed a bug where the host value output by `docker-machine config default` + would not be recognized as valid options by the `docker-compose` + command line. + +- Fixed an issue where Compose would sometimes exit unexpectedly while + reading events broadcasted by a Swarm cluster. + +- Corrected a statement in the docs about the location of the `.env` file, + which is indeed read from the current directory, instead of in the same + location as the Compose file. + + +1.7.0 (2016-04-13) +------------------ + +**Breaking Changes** + +- `docker-compose logs` no longer follows log output by default. It now + matches the behaviour of `docker logs` and exits after the current logs + are printed. Use `-f` to get the old default behaviour. + +- Booleans are no longer allows as values for mappings in the Compose file + (for keys `environment`, `labels` and `extra_hosts`). Previously this + was a warning. Boolean values should be quoted so they become string values. + +New Features + +- Compose now looks for a `.env` file in the directory where it's run and + reads any environment variables defined inside, if they're not already + set in the shell environment. This lets you easily set defaults for + variables used in the Compose file, or for any of the `COMPOSE_*` or + `DOCKER_*` variables. + +- Added a `--remove-orphans` flag to both `docker-compose up` and + `docker-compose down` to remove containers for services that were removed + from the Compose file. + +- Added a `--all` flag to `docker-compose rm` to include containers created + by `docker-compose run`. This will become the default behavior in the next + version of Compose. + +- Added support for all the same TLS configuration flags used by the `docker` + client: `--tls`, `--tlscert`, `--tlskey`, etc. + +- Compose files now support the `tmpfs` and `shm_size` options. + +- Added the `--workdir` flag to `docker-compose run` + +- `docker-compose logs` now shows logs for new containers that are created + after it starts. + +- The `COMPOSE_FILE` environment variable can now contain multiple files, + separated by the host system's standard path separator (`:` on Mac/Linux, + `;` on Windows). + +- You can now specify a static IP address when connecting a service to a + network with the `ipv4_address` and `ipv6_address` options. + +- Added `--follow`, `--timestamp`, and `--tail` flags to the + `docker-compose logs` command. + +- `docker-compose up`, and `docker-compose start` will now start containers + in parallel where possible. + +- `docker-compose stop` now stops containers in reverse dependency order + instead of all at once. + +- Added the `--build` flag to `docker-compose up` to force it to build a new + image. It now shows a warning if an image is automatically built when the + flag is not used. + +- Added the `docker-compose exec` command for executing a process in a running + container. + + +Bug Fixes + +- `docker-compose down` now removes containers created by + `docker-compose run`. + +- A more appropriate error is shown when a timeout is hit during `up` when + using a tty. + +- Fixed a bug in `docker-compose down` where it would abort if some resources + had already been removed. + +- Fixed a bug where changes to network aliases would not trigger a service + to be recreated. + +- Fix a bug where a log message was printed about creating a new volume + when it already existed. + +- Fixed a bug where interrupting `up` would not always shut down containers. + +- Fixed a bug where `log_opt` and `log_driver` were not properly carried over + when extending services in the v1 Compose file format. + +- Fixed a bug where empty values for build args would cause file validation + to fail. + +1.6.2 (2016-02-23) +------------------ + +- Fixed a bug where connecting to a TLS-enabled Docker Engine would fail with + a certificate verification error. + +1.6.1 (2016-02-23) +------------------ + +Bug Fixes + +- Fixed a bug where recreating a container multiple times would cause the + new container to be started without the previous volumes. + +- Fixed a bug where Compose would set the value of unset environment variables + to an empty string, instead of a key without a value. + +- Provide a better error message when Compose requires a more recent version + of the Docker API. + +- Add a missing config field `network.aliases` which allows setting a network + scoped alias for a service. + +- Fixed a bug where `run` would not start services listed in `depends_on`. + +- Fixed a bug where `networks` and `network_mode` where not merged when using + extends or multiple Compose files. + +- Fixed a bug with service aliases where the short container id alias was + only contained 10 characters, instead of the 12 characters used in previous + versions. + +- Added a missing log message when creating a new named volume. + +- Fixed a bug where `build.args` was not merged when using `extends` or + multiple Compose files. + +- Fixed some bugs with config validation when null values or incorrect types + were used instead of a mapping. + +- Fixed a bug where a `build` section without a `context` would show a stack + trace instead of a helpful validation message. + +- Improved compatibility with swarm by only setting a container affinity to + the previous instance of a services' container when the service uses an + anonymous container volume. Previously the affinity was always set on all + containers. + +- Fixed the validation of some `driver_opts` would cause an error if a number + was used instead of a string. + +- Some improvements to the `run.sh` script used by the Compose container install + option. + +- Fixed a bug with `up --abort-on-container-exit` where Compose would exit, + but would not stop other containers. + +- Corrected the warning message that is printed when a boolean value is used + as a value in a mapping. + + +1.6.0 (2016-01-15) +------------------ + +Major Features: + +- Compose 1.6 introduces a new format for `docker-compose.yml` which lets + you define networks and volumes in the Compose file as well as services. It + also makes a few changes to the structure of some configuration options. + + You don't have to use it - your existing Compose files will run on Compose + 1.6 exactly as they do today. + + Check the upgrade guide for full details: + https://docs.docker.com/compose/compose-file#upgrading + +- Support for networking has exited experimental status and is the recommended + way to enable communication between containers. + + If you use the new file format, your app will use networking. If you aren't + ready yet, just leave your Compose file as it is and it'll continue to work + just the same. + + By default, you don't have to configure any networks. In fact, using + networking with Compose involves even less configuration than using links. + Consult the networking guide for how to use it: + https://docs.docker.com/compose/networking + + The experimental flags `--x-networking` and `--x-network-driver`, introduced + in Compose 1.5, have been removed. + +- You can now pass arguments to a build if you're using the new file format: + + build: + context: . + args: + buildno: 1 + +- You can now specify both a `build` and an `image` key if you're using the + new file format. `docker-compose build` will build the image and tag it with + the name you've specified, while `docker-compose pull` will attempt to pull + it. + +- There's a new `events` command for monitoring container events from + the application, much like `docker events`. This is a good primitive for + building tools on top of Compose for performing actions when particular + things happen, such as containers starting and stopping. + +- There's a new `depends_on` option for specifying dependencies between + services. This enforces the order of startup, and ensures that when you run + `docker-compose up SERVICE` on a service with dependencies, those are started + as well. + +New Features: + +- Added a new command `config` which validates and prints the Compose + configuration after interpolating variables, resolving relative paths, and + merging multiple files and `extends`. + +- Added a new command `create` for creating containers without starting them. + +- Added a new command `down` to stop and remove all the resources created by + `up` in a single command. + +- Added support for the `cpu_quota` configuration option. + +- Added support for the `stop_signal` configuration option. + +- Commands `start`, `restart`, `pause`, and `unpause` now exit with an + error status code if no containers were modified. + +- Added a new `--abort-on-container-exit` flag to `up` which causes `up` to + stop all container and exit once the first container exits. + +- Removed support for `FIG_FILE`, `FIG_PROJECT_NAME`, and no longer reads + `fig.yml` as a default Compose file location. + +- Removed the `migrate-to-labels` command. + +- Removed the `--allow-insecure-ssl` flag. + + +Bug Fixes: + +- Fixed a validation bug that prevented the use of a range of ports in + the `expose` field. + +- Fixed a validation bug that prevented the use of arrays in the `entrypoint` + field if they contained duplicate entries. + +- Fixed a bug that caused `ulimits` to be ignored when used with `extends`. + +- Fixed a bug that prevented ipv6 addresses in `extra_hosts`. + +- Fixed a bug that caused `extends` to be ignored when included from + multiple Compose files. + +- Fixed an incorrect warning when a container volume was defined in + the Compose file. + +- Fixed a bug that prevented the force shutdown behaviour of `up` and + `logs`. + +- Fixed a bug that caused `None` to be printed as the network driver name + when the default network driver was used. + +- Fixed a bug where using the string form of `dns` or `dns_search` would + cause an error. + +- Fixed a bug where a container would be reported as "Up" when it was + in the restarting state. + +- Fixed a confusing error message when DOCKER_CERT_PATH was not set properly. + +- Fixed a bug where attaching to a container would fail if it was using a + non-standard logging driver (or none at all). + + +1.5.2 (2015-12-03) +------------------ + +- Fixed a bug which broke the use of `environment` and `env_file` with + `extends`, and caused environment keys without values to have a `None` + value, instead of a value from the host environment. + +- Fixed a regression in 1.5.1 that caused a warning about volumes to be + raised incorrectly when containers were recreated. + +- Fixed a bug which prevented building a `Dockerfile` that used `ADD <url>` + +- Fixed a bug with `docker-compose restart` which prevented it from + starting stopped containers. + +- Fixed handling of SIGTERM and SIGINT to properly stop containers + +- Add support for using a url as the value of `build` + +- Improved the validation of the `expose` option + + +1.5.1 (2015-11-12) +------------------ + +- Add the `--force-rm` option to `build`. + +- Add the `ulimit` option for services in the Compose file. + +- Fixed a bug where `up` would error with "service needs to be built" if + a service changed from using `image` to using `build`. + +- Fixed a bug that would cause incorrect output of parallel operations + on some terminals. + +- Fixed a bug that prevented a container from being recreated when the + mode of a `volumes_from` was changed. + +- Fixed a regression in 1.5.0 where non-utf-8 unicode characters would cause + `up` or `logs` to crash. + +- Fixed a regression in 1.5.0 where Compose would use a success exit status + code when a command fails due to an HTTP timeout communicating with the + docker daemon. + +- Fixed a regression in 1.5.0 where `name` was being accepted as a valid + service option which would override the actual name of the service. + +- When using `--x-networking` Compose no longer sets the hostname to the + container name. + +- When using `--x-networking` Compose will only create the default network + if at least one container is using the network. + +- When printings logs during `up` or `logs`, flush the output buffer after + each line to prevent buffering issues from hiding logs. + +- Recreate a container if one of its dependencies is being created. + Previously a container was only recreated if it's dependencies already + existed, but were being recreated as well. + +- Add a warning when a `volume` in the Compose file is being ignored + and masked by a container volume from a previous container. + +- Improve the output of `pull` when run without a tty. + +- When using multiple Compose files, validate each before attempting to merge + them together. Previously invalid files would result in not helpful errors. + +- Allow dashes in keys in the `environment` service option. + +- Improve validation error messages by including the filename as part of the + error message. + + +1.5.0 (2015-11-03) +------------------ + +**Breaking changes:** + +With the introduction of variable substitution support in the Compose file, any +Compose file that uses an environment variable (`$VAR` or `${VAR}`) in the `command:` +or `entrypoint:` field will break. + +Previously these values were interpolated inside the container, with a value +from the container environment. In Compose 1.5.0, the values will be +interpolated on the host, with a value from the host environment. + +To migrate a Compose file to 1.5.0, escape the variables with an extra `$` +(ex: `$$VAR` or `$${VAR}`). See +https://github.com/docker/compose/blob/8cc8e61/docs/compose-file.md#variable-substitution + +Major features: + +- Compose is now available for Windows. + +- Environment variables can be used in the Compose file. See + https://github.com/docker/compose/blob/8cc8e61/docs/compose-file.md#variable-substitution + +- Multiple compose files can be specified, allowing you to override + settings in the default Compose file. See + https://github.com/docker/compose/blob/8cc8e61/docs/reference/docker-compose.md + for more details. + +- Compose now produces better error messages when a file contains + invalid configuration. + +- `up` now waits for all services to exit before shutting down, + rather than shutting down as soon as one container exits. + +- Experimental support for the new docker networking system can be + enabled with the `--x-networking` flag. Read more here: + https://github.com/docker/docker/blob/8fee1c20/docs/userguide/dockernetworks.md + +New features: + +- You can now optionally pass a mode to `volumes_from`, e.g. + `volumes_from: ["servicename:ro"]`. + +- Since Docker now lets you create volumes with names, you can refer to those + volumes by name in `docker-compose.yml`. For example, + `volumes: ["mydatavolume:/data"]` will mount the volume named + `mydatavolume` at the path `/data` inside the container. + + If the first component of an entry in `volumes` starts with a `.`, `/` or + `~`, it is treated as a path and expansion of relative paths is performed as + necessary. Otherwise, it is treated as a volume name and passed straight + through to Docker. + + Read more on named volumes and volume drivers here: + https://github.com/docker/docker/blob/244d9c33/docs/userguide/dockervolumes.md + +- `docker-compose build --pull` instructs Compose to pull the base image for + each Dockerfile before building. + +- `docker-compose pull --ignore-pull-failures` instructs Compose to continue + if it fails to pull a single service's image, rather than aborting. + +- You can now specify an IPC namespace in `docker-compose.yml` with the `ipc` + option. + +- Containers created by `docker-compose run` can now be named with the + `--name` flag. + +- If you install Compose with pip or use it as a library, it now works with + Python 3. + +- `image` now supports image digests (in addition to ids and tags), e.g. + `image: "busybox@sha256:38a203e1986cf79639cfb9b2e1d6e773de84002feea2d4eb006b52004ee8502d"` + +- `ports` now supports ranges of ports, e.g. + + ports: + - "3000-3005" + - "9000-9001:8000-8001" + +- `docker-compose run` now supports a `-p|--publish` parameter, much like + `docker run -p`, for publishing specific ports to the host. + +- `docker-compose pause` and `docker-compose unpause` have been implemented, + analogous to `docker pause` and `docker unpause`. + +- When using `extends` to copy configuration from another service in the same + Compose file, you can omit the `file` option. + +- Compose can be installed and run as a Docker image. This is an experimental + feature. + +Bug fixes: + +- All values for the `log_driver` option which are supported by the Docker + daemon are now supported by Compose. + +- `docker-compose build` can now be run successfully against a Swarm cluster. + + +1.4.2 (2015-09-22) +------------------ + +- Fixed a regression in the 1.4.1 release that would cause `docker-compose up` + without the `-d` option to exit immediately. + +1.4.1 (2015-09-10) +------------------ + +The following bugs have been fixed: + +- Some configuration changes (notably changes to `links`, `volumes_from`, and + `net`) were not properly triggering a container recreate as part of + `docker-compose up`. +- `docker-compose up <service>` was showing logs for all services instead of + just the specified services. +- Containers with custom container names were showing up in logs as + `service_number` instead of their custom container name. +- When scaling a service sometimes containers would be recreated even when + the configuration had not changed. + + +1.4.0 (2015-08-04) +------------------ + +- By default, `docker-compose up` now only recreates containers for services whose configuration has changed since they were created. This should result in a dramatic speed-up for many applications. + + The experimental `--x-smart-recreate` flag which introduced this feature in Compose 1.3.0 has been removed, and a `--force-recreate` flag has been added for when you want to recreate everything. + +- Several of Compose's commands - `scale`, `stop`, `kill` and `rm` - now perform actions on multiple containers in parallel, rather than in sequence, which will run much faster on larger applications. + +- You can now specify a custom name for a service's container with `container_name`. Because Docker container names must be unique, this means you can't scale the service beyond one container. + +- You no longer have to specify a `file` option when using `extends` - it will default to the current file. + +- Service names can now contain dots, dashes and underscores. + +- Compose can now read YAML configuration from standard input, rather than from a file, by specifying `-` as the filename. This makes it easier to generate configuration dynamically: + + $ echo 'redis: {"image": "redis"}' | docker-compose --file - up + +- There's a new `docker-compose version` command which prints extended information about Compose's bundled dependencies. + +- `docker-compose.yml` now supports `log_opt` as well as `log_driver`, allowing you to pass extra configuration to a service's logging driver. + +- `docker-compose.yml` now supports `memswap_limit`, similar to `docker run --memory-swap`. + +- When mounting volumes with the `volumes` option, you can now pass in any mode supported by the daemon, not just `:ro` or `:rw`. For example, SELinux users can pass `:z` or `:Z`. + +- You can now specify a custom volume driver with the `volume_driver` option in `docker-compose.yml`, much like `docker run --volume-driver`. + +- A bug has been fixed where Compose would fail to pull images from private registries serving plain (unsecured) HTTP. The `--allow-insecure-ssl` flag, which was previously used to work around this issue, has been deprecated and now has no effect. + +- A bug has been fixed where `docker-compose build` would fail if the build depended on a private Hub image or an image from a private registry. + +- A bug has been fixed where Compose would crash if there were containers which the Docker daemon had not finished removing. + +- Two bugs have been fixed where Compose would sometimes fail with a "Duplicate bind mount" error, or fail to attach volumes to a container, if there was a volume path specified in `docker-compose.yml` with a trailing slash. + +Thanks @mnowster, @dnephin, @ekristen, @funkyfuture, @jeffk and @lukemarsden! + +1.3.3 (2015-07-15) +------------------ + +Two regressions have been fixed: + +- When stopping containers gracefully, Compose was setting the timeout to 0, effectively forcing a SIGKILL every time. +- Compose would sometimes crash depending on the formatting of container data returned from the Docker API. + +1.3.2 (2015-07-14) +------------------ + +The following bugs have been fixed: + +- When there were one-off containers created by running `docker-compose run` on an older version of Compose, `docker-compose run` would fail with a name collision. Compose now shows an error if you have leftover containers of this type lying around, and tells you how to remove them. +- Compose was not reading Docker authentication config files created in the new location, `~/docker/config.json`, and authentication against private registries would therefore fail. +- When a container had a pseudo-TTY attached, its output in `docker-compose up` would be truncated. +- `docker-compose up --x-smart-recreate` would sometimes fail when an image tag was updated. +- `docker-compose up` would sometimes create two containers with the same numeric suffix. +- `docker-compose rm` and `docker-compose ps` would sometimes list services that aren't part of the current project (though no containers were erroneously removed). +- Some `docker-compose` commands would not show an error if invalid service names were passed in. + +Thanks @dano, @josephpage, @kevinsimper, @lieryan, @phemmer, @soulrebel and @sschepens! + +1.3.1 (2015-06-21) +------------------ + +The following bugs have been fixed: + +- `docker-compose build` would always attempt to pull the base image before building. +- `docker-compose help migrate-to-labels` failed with an error. +- If no network mode was specified, Compose would set it to "bridge", rather than allowing the Docker daemon to use its configured default network mode. + +1.3.0 (2015-06-18) +------------------ + +Firstly, two important notes: + +- **This release contains breaking changes, and you will need to either remove or migrate your existing containers before running your app** - see the [upgrading section of the install docs](https://github.com/docker/compose/blob/1.3.0rc1/docs/install.md#upgrading) for details. + +- Compose now requires Docker 1.6.0 or later. + +We've done a lot of work in this release to remove hacks and make Compose more stable: + +- Compose now uses container labels, rather than names, to keep track of containers. This makes Compose both faster and easier to integrate with your own tools. + +- Compose no longer uses "intermediate containers" when recreating containers for a service. This makes `docker-compose up` less complex and more resilient to failure. + +There are some new features: + +- `docker-compose up` has an **experimental** new behaviour: it will only recreate containers for services whose configuration has changed in `docker-compose.yml`. This will eventually become the default, but for now you can take it for a spin: + + $ docker-compose up --x-smart-recreate + +- When invoked in a subdirectory of a project, `docker-compose` will now climb up through parent directories until it finds a `docker-compose.yml`. + +Several new configuration keys have been added to `docker-compose.yml`: + +- `dockerfile`, like `docker build --file`, lets you specify an alternate Dockerfile to use with `build`. +- `labels`, like `docker run --labels`, lets you add custom metadata to containers. +- `extra_hosts`, like `docker run --add-host`, lets you add entries to a container's `/etc/hosts` file. +- `pid: host`, like `docker run --pid=host`, lets you reuse the same PID namespace as the host machine. +- `cpuset`, like `docker run --cpuset-cpus`, lets you specify which CPUs to allow execution in. +- `read_only`, like `docker run --read-only`, lets you mount a container's filesystem as read-only. +- `security_opt`, like `docker run --security-opt`, lets you specify [security options](https://docs.docker.com/engine/reference/run/#security-configuration). +- `log_driver`, like `docker run --log-driver`, lets you specify a [log driver](https://docs.docker.com/engine/reference/run/#logging-drivers-log-driver). + +Many bugs have been fixed, including the following: + +- The output of `docker-compose run` was sometimes truncated, especially when running under Jenkins. +- A service's volumes would sometimes not update after volume configuration was changed in `docker-compose.yml`. +- Authenticating against third-party registries would sometimes fail. +- `docker-compose run --rm` would fail to remove the container if the service had a `restart` policy in place. +- `docker-compose scale` would refuse to scale a service beyond 1 container if it exposed a specific port number on the host. +- Compose would refuse to create multiple volume entries with the same host path. + +Thanks @ahromis, @albers, @aleksandr-vin, @antoineco, @ccverak, @chernjie, @dnephin, @edmorley, @fordhurley, @josephpage, @KyleJamesWalker, @lsowen, @mchasal, @noironetworks, @sdake, @sdurrheimer, @sherter, @stephenlawrence, @thaJeztah, @thieman, @turtlemonvh, @twhiteman, @vdemeester, @xuxinkun and @zwily! + +1.2.0 (2015-04-16) +------------------ + +- `docker-compose.yml` now supports an `extends` option, which enables a service to inherit configuration from another service in another configuration file. This is really good for sharing common configuration between apps, or for configuring the same app for different environments. Here's the [documentation](https://github.com/docker/compose/blob/master/docs/yml.md#extends). + +- When using Compose with a Swarm cluster, containers that depend on one another will be co-scheduled on the same node. This means that most Compose apps will now work out of the box, as long as they don't use `build`. + +- Repeated invocations of `docker-compose up` when using Compose with a Swarm cluster now work reliably. + +- Directories passed to `build`, filenames passed to `env_file` and volume host paths passed to `volumes` are now treated as relative to the *directory of the configuration file*, not the directory that `docker-compose` is being run in. In the majority of cases, those are the same, but if you use the `-f|--file` argument to specify a configuration file in another directory, **this is a breaking change**. + +- A service can now share another service's network namespace with `net: container:<service>`. + +- `volumes_from` and `net: container:<service>` entries are taken into account when resolving dependencies, so `docker-compose up <service>` will correctly start all dependencies of `<service>`. + +- `docker-compose run` now accepts a `--user` argument to specify a user to run the command as, just like `docker run`. + +- The `up`, `stop` and `restart` commands now accept a `--timeout` (or `-t`) argument to specify how long to wait when attempting to gracefully stop containers, just like `docker stop`. + +- `docker-compose rm` now accepts `-f` as a shorthand for `--force`, just like `docker rm`. + +Thanks, @abesto, @albers, @alunduil, @dnephin, @funkyfuture, @gilclark, @IanVS, @KingsleyKelly, @knutwalker, @thaJeztah and @vmalloc! + +1.1.0 (2015-02-25) +------------------ + +Fig has been renamed to Docker Compose, or just Compose for short. This has several implications for you: + +- The command you type is now `docker-compose`, not `fig`. +- You should rename your fig.yml to docker-compose.yml. +- If you’re installing via PyPI, the package is now `docker-compose`, so install it with `pip install docker-compose`. + +Besides that, there’s a lot of new stuff in this release: + +- We’ve made a few small changes to ensure that Compose will work with Swarm, Docker’s new clustering tool (https://github.com/docker/swarm). Eventually you'll be able to point Compose at a Swarm cluster instead of a standalone Docker host and it’ll run your containers on the cluster with no extra work from you. As Swarm is still developing, integration is rough and lots of Compose features don't work yet. + +- `docker-compose run` now has a `--service-ports` flag for exposing ports on the given service. This is useful for e.g. running your webapp with an interactive debugger. + +- You can now link to containers outside your app with the `external_links` option in docker-compose.yml. + +- You can now prevent `docker-compose up` from automatically building images with the `--no-build` option. This will make fewer API calls and run faster. + +- If you don’t specify a tag when using the `image` key, Compose will default to the `latest` tag, rather than pulling all tags. + +- `docker-compose kill` now supports the `-s` flag, allowing you to specify the exact signal you want to send to a service’s containers. + +- docker-compose.yml now has an `env_file` key, analogous to `docker run --env-file`, letting you specify multiple environment variables in a separate file. This is great if you have a lot of them, or if you want to keep sensitive information out of version control. + +- docker-compose.yml now supports the `dns_search`, `cap_add`, `cap_drop`, `cpu_shares` and `restart` options, analogous to `docker run`’s `--dns-search`, `--cap-add`, `--cap-drop`, `--cpu-shares` and `--restart` options. + +- Compose now ships with Bash tab completion - see the installation and usage docs at https://github.com/docker/compose/blob/1.1.0/docs/completion.md + +- A number of bugs have been fixed - see the milestone for details: https://github.com/docker/compose/issues?q=milestone%3A1.1.0+ + +Thanks @dnephin, @squebe, @jbalonso, @raulcd, @benlangfield, @albers, @ggtools, @bersace, @dtenenba, @petercv, @drewkett, @TFenby, @paulRbr, @Aigeruth and @salehe! + +1.0.1 (2014-11-04) +------------------ + + - Added an `--allow-insecure-ssl` option to allow `fig up`, `fig run` and `fig pull` to pull from insecure registries. + - Fixed `fig run` not showing output in Jenkins. + - Fixed a bug where Fig couldn't build Dockerfiles with ADD statements pointing at URLs. + +1.0.0 (2014-10-16) +------------------ + +The highlights: + + - [Fig has joined Docker.](https://www.orchardup.com/blog/orchard-is-joining-docker) Fig will continue to be maintained, but we'll also be incorporating the best bits of Fig into Docker itself. + + This means the GitHub repository has moved to [https://github.com/docker/fig](https://github.com/docker/fig) and our IRC channel is now #docker-fig on Freenode. + + - Fig can be used with the [official Docker OS X installer](https://docs.docker.com/installation/mac/). Boot2Docker will mount the home directory from your host machine so volumes work as expected. + + - Fig supports Docker 1.3. + + - It is now possible to connect to the Docker daemon using TLS by using the `DOCKER_CERT_PATH` and `DOCKER_TLS_VERIFY` environment variables. + + - There is a new `fig port` command which outputs the host port binding of a service, in a similar way to `docker port`. + + - There is a new `fig pull` command which pulls the latest images for a service. + + - There is a new `fig restart` command which restarts a service's containers. + + - Fig creates multiple containers in service by appending a number to the service name (e.g. `db_1`, `db_2`, etc). As a convenience, Fig will now give the first container an alias of the service name (e.g. `db`). + + This link alias is also a valid hostname and added to `/etc/hosts` so you can connect to linked services using their hostname. For example, instead of resolving the environment variables `DB_PORT_5432_TCP_ADDR` and `DB_PORT_5432_TCP_PORT`, you could just use the hostname `db` and port `5432` directly. + + - Volume definitions now support `ro` mode, expanding `~` and expanding environment variables. + + - `.dockerignore` is supported when building. + + - The project name can be set with the `FIG_PROJECT_NAME` environment variable. + + - The `--env` and `--entrypoint` options have been added to `fig run`. + + - The Fig binary for Linux is now linked against an older version of glibc so it works on CentOS 6 and Debian Wheezy. + +Other things: + + - `fig ps` now works on Jenkins and makes fewer API calls to the Docker daemon. + - `--verbose` displays more useful debugging output. + - When starting a service where `volumes_from` points to a service without any containers running, that service will now be started. + - Lots of docs improvements. Notably, environment variables are documented and official repositories are used throughout. + +Thanks @dnephin, @d11wtq, @marksteve, @rubbish, @jbalonso, @timfreund, @alunduil, @mieciu, @shuron, @moss, @suzaku and @chmouel! Whew. + +0.5.2 (2014-07-28) +------------------ + + - Added a `--no-cache` option to `fig build`, which bypasses the cache just like `docker build --no-cache`. + - Fixed the `dns:` fig.yml option, which was causing fig to error out. + - Fixed a bug where fig couldn't start under Python 2.6. + - Fixed a log-streaming bug that occasionally caused fig to exit. + +Thanks @dnephin and @marksteve! + + +0.5.1 (2014-07-11) +------------------ + + - If a service has a command defined, `fig run [service]` with no further arguments will run it. + - The project name now defaults to the directory containing fig.yml, not the current working directory (if they're different) + - `volumes_from` now works properly with containers as well as services + - Fixed a race condition when recreating containers in `fig up` + +Thanks @ryanbrainard and @d11wtq! + + +0.5.0 (2014-07-11) +------------------ + + - Fig now starts links when you run `fig run` or `fig up`. + + For example, if you have a `web` service which depends on a `db` service, `fig run web ...` will start the `db` service. + + - Environment variables can now be resolved from the environment that Fig is running in. Just specify it as a blank variable in your `fig.yml` and, if set, it'll be resolved: + ``` + environment: + RACK_ENV: development + SESSION_SECRET: + ``` + + - `volumes_from` is now supported in `fig.yml`. All of the volumes from the specified services and containers will be mounted: + + ``` + volumes_from: + - service_name + - container_name + ``` + + - A host address can now be specified in `ports`: + + ``` + ports: + - "0.0.0.0:8000:8000" + - "127.0.0.1:8001:8001" + ``` + + - The `net` and `workdir` options are now supported in `fig.yml`. + - The `hostname` option now works in the same way as the Docker CLI, splitting out into a `domainname` option. + - TTY behaviour is far more robust, and resizes are supported correctly. + - Load YAML files safely. + +Thanks to @d11wtq, @ryanbrainard, @rail44, @j0hnsmith, @binarin, @Elemecca, @mozz100 and @marksteve for their help with this release! + + +0.4.2 (2014-06-18) +------------------ + + - Fix various encoding errors when using `fig run`, `fig up` and `fig build`. + +0.4.1 (2014-05-08) +------------------ + + - Add support for Docker 0.11.0. (Thanks @marksteve!) + - Make project name configurable. (Thanks @jefmathiot!) + - Return correct exit code from `fig run`. + +0.4.0 (2014-04-29) +------------------ + + - Support Docker 0.9 and 0.10 + - Display progress bars correctly when pulling images (no more ski slopes) + - `fig up` now stops all services when any container exits + - Added support for the `privileged` config option in fig.yml (thanks @kvz!) + - Shortened and aligned log prefixes in `fig up` output + - Only containers started with `fig run` link back to their own service + - Handle UTF-8 correctly when streaming `fig build/run/up` output (thanks @mauvm and @shanejonas!) + - Error message improvements + +0.3.2 (2014-03-05) +------------------ + + - Added an `--rm` option to `fig run`. (Thanks @marksteve!) + - Added an `expose` option to `fig.yml`. + +0.3.1 (2014-03-04) +------------------ + + - Added contribution instructions. (Thanks @kvz!) + - Fixed `fig rm` throwing an error. + - Fixed a bug in `fig ps` on Docker 0.8.1 when there is a container with no command. + +0.3.0 (2014-03-03) +------------------ + + - We now ship binaries for OS X and Linux. No more having to install with Pip! + - Add `-f` flag to specify alternate `fig.yml` files + - Add support for custom link names + - Fix a bug where recreating would sometimes hang + - Update docker-py to support Docker 0.8.0. + - Various documentation improvements + - Various error message improvements + +Thanks @marksteve, @Gazler and @teozkr! + +0.2.2 (2014-02-17) +------------------ + + - Resolve dependencies using Cormen/Tarjan topological sort + - Fix `fig up` not printing log output + - Stop containers in reverse order to starting + - Fix scale command not binding ports + +Thanks to @barnybug and @dustinlacewell for their work on this release. + +0.2.1 (2014-02-04) +------------------ + + - General improvements to error reporting (#77, #79) + +0.2.0 (2014-01-31) +------------------ + + - Link services to themselves so run commands can access the running service. (#67) + - Much better documentation. + - Make service dependency resolution more reliable. (#48) + - Load Fig configurations with a `.yaml` extension. (#58) + +Big thanks to @cameronmaske, @mrchrisadams and @damianmoore for their help with this release. + +0.1.4 (2014-01-27) +------------------ + + - Add a link alias without the project name. This makes the environment variables a little shorter: `REDIS_1_PORT_6379_TCP_ADDR`. (#54) + +0.1.3 (2014-01-23) +------------------ + + - Fix ports sometimes being configured incorrectly. (#46) + - Fix log output sometimes not displaying. (#47) + +0.1.2 (2014-01-22) +------------------ + + - Add `-T` option to `fig run` to disable pseudo-TTY. (#34) + - Fix `fig up` requiring the ubuntu image to be pulled to recreate containers. (#33) Thanks @cameronmaske! + - Improve reliability, fix arrow keys and fix a race condition in `fig run`. (#34, #39, #40) + +0.1.1 (2014-01-17) +------------------ + + - Fix bug where ports were not exposed correctly (#29). Thanks @dustinlacewell! + +0.1.0 (2014-01-16) +------------------ + + - Containers are recreated on each `fig up`, ensuring config is up-to-date with `fig.yml` (#2) + - Add `fig scale` command (#9) + - Use `DOCKER_HOST` environment variable to find Docker daemon, for consistency with the official Docker client (was previously `DOCKER_URL`) (#19) + - Truncate long commands in `fig ps` (#18) + - Fill out CLI help banners for commands (#15, #16) + - Show a friendlier error when `fig.yml` is missing (#4) + - Fix bug with `fig build` logging (#3) + - Fix bug where builds would time out if a step took a long time without generating output (#6) + - Fix bug where streaming container output over the Unix socket raised an error (#7) + +Big thanks to @tomstuart, @EnTeQuAk, @schickling, @aronasorman and @GeoffreyPlitt. + +0.0.2 (2014-01-02) +------------------ + + - Improve documentation + - Try to connect to Docker on `tcp://localdocker:4243` and a UNIX socket in addition to `localhost`. + - Improve `fig up` behaviour + - Add confirmation prompt to `fig rm` + - Add `fig build` command + +0.0.1 (2013-12-20) +------------------ + +Initial release. diff --git a/CHANGES.md b/CHANGES.md new file mode 120000 index 00000000000..83b694704ba --- /dev/null +++ b/CHANGES.md @@ -0,0 +1 @@ +CHANGELOG.md \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9b80b8abc45..5bf7cb1318c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,339 +1,76 @@ -# Contributing to Docker +# Contributing to Compose -Want to hack on Docker? Awesome! We have a contributor's guide that explains -[setting up a Docker development environment and the contribution -process](https://docs.docker.com/contribute/). +Compose is a part of the Docker project, and follows the same rules and +principles. Take a read of [Docker's contributing guidelines](https://github.com/docker/docker/blob/master/CONTRIBUTING.md) +to get an overview. -This page contains information about reporting issues as well as some tips and -guidelines useful to experienced open source contributors. Finally, make sure -you read our [community guidelines](#docker-community-guidelines) before you -start participating. +## TL;DR -## Topics +Pull requests will need: -- [Contributing to Docker](#contributing-to-docker) - - [Topics](#topics) - - [Reporting security issues](#reporting-security-issues) - - [Reporting other issues](#reporting-other-issues) - - [Quick contribution tips and guidelines](#quick-contribution-tips-and-guidelines) - - [Pull requests are always welcome](#pull-requests-are-always-welcome) - - [Talking to other Docker users and contributors](#talking-to-other-docker-users-and-contributors) - - [Conventions](#conventions) - - [Merge approval](#merge-approval) - - [Sign your work](#sign-your-work) - - [How can I become a maintainer?](#how-can-i-become-a-maintainer) - - [Docker community guidelines](#docker-community-guidelines) - - [Coding Style](#coding-style) + - Tests + - Documentation + - [To be signed off](https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work) + - A logical series of [well written commits](https://github.com/alphagov/styleguides/blob/master/git.md) -## Reporting security issues +## Development environment -The Docker maintainers take security seriously. If you discover a security -issue, please bring it to their attention right away! +If you're looking contribute to Compose +but you're new to the project or maybe even to Python, here are the steps +that should get you started. -Please **DO NOT** file a public issue, instead, send your report privately to -[security@docker.com](mailto:security@docker.com). +1. Fork [https://github.com/docker/compose](https://github.com/docker/compose) + to your username. +2. Clone your forked repository locally `git clone git@github.com:yourusername/compose.git`. +3. You must [configure a remote](https://help.github.com/articles/configuring-a-remote-for-a-fork/) for your fork so that you can [sync changes you make](https://help.github.com/articles/syncing-a-fork/) with the original repository. +4. Enter the local directory `cd compose`. +5. Set up a development environment by running `python setup.py develop`. This + will install the dependencies and set up a symlink from your `docker-compose` + executable to the checkout of the repository. When you now run + `docker-compose` from anywhere on your machine, it will run your development + version of Compose. -Security reports are greatly appreciated and we will publicly thank you for them. -We also like to send gifts—if you're into Docker swag, make sure to let -us know. We currently do not offer a paid security bounty program but are not -ruling it out in the future. +## Install pre-commit hooks +This step is optional, but recommended. Pre-commit hooks will run style checks +and in some cases fix style issues for you, when you commit code. -## Reporting other issues +Install the git pre-commit hooks using [tox](https://tox.readthedocs.io) by +running `tox -e pre-commit` or by following the +[pre-commit install guide](http://pre-commit.com/#install). -A great way to contribute to the project is to send a detailed report when you -encounter an issue. We always appreciate a well-written, thorough bug report, -and will thank you for it! +To run the style checks at any time run `tox -e pre-commit`. -Check that [our issue database](https://github.com/docker/compose/labels/Docker%20Compose%20V2) -doesn't already include that problem or suggestion before submitting an issue. -If you find a match, you can use the "subscribe" button to get notified of -updates. Do *not* leave random "+1" or "I have this too" comments, as they -only clutter the discussion, and don't help to resolve it. However, if you -have ways to reproduce the issue or have additional information that may help -resolve the issue, please leave a comment. +## Submitting a pull request -When reporting issues, always include: +See Docker's [basic contribution workflow](https://docs.docker.com/v17.06/opensource/code/#code-contribution-workflow) for a guide on how to submit a pull request for code. -* The output of `docker version`. -* The output of `docker context show`. -* The output of `docker info`. +## Documentation changes -Also, include the steps required to reproduce the problem if possible and -applicable. This information will help us review and fix your issue faster. -When sending lengthy log files, consider posting them as a gist -(https://gist.github.com). -Don't forget to remove sensitive data from your log files before posting (you -can replace those parts with "REDACTED"). +Issues and pull requests to update the documentation should be submitted to the [docs repo](https://github.com/docker/docker.github.io). You can learn more about contributing to the documentation [here](https://docs.docker.com/opensource/#how-to-contribute-to-the-docs). -_Note:_ -Maintainers might request additional information to diagnose an issue, -if initial reporter doesn't answer within a reasonable delay (a few weeks), -issue will be closed. +## Running the test suite -## Quick contribution tips and guidelines +Use the test script to run linting checks and then the full test suite against +different Python interpreters: -This section gives the experienced contributor some tips and guidelines. + $ script/test/default -### Pull requests are always welcome +Tests are run against a Docker daemon inside a container, so that we can test +against multiple Docker versions. By default they'll run against only the latest +Docker version - set the `DOCKER_VERSIONS` environment variable to "all" to run +against all supported versions: -Not sure if that typo is worth a pull request? Found a bug and know how to fix -it? Do it! We will appreciate it. Any significant change, like adding a backend, -should be documented as -[a GitHub issue](https://github.com/docker/compose/issues) -before anybody starts working on it. + $ DOCKER_VERSIONS=all script/test/default -We are always thrilled to receive pull requests. We do our best to process them -quickly. If your pull request is not accepted on the first try, -don't get discouraged! +Arguments to `script/test/default` are passed through to the `tox` executable, so +you can specify a test directory, file, module, class or method: -### Talking to other Docker users and contributors + $ script/test/default tests/unit + $ script/test/default tests/unit/cli_test.py + $ script/test/default tests/unit/config/config_test.py::ConfigTest + $ script/test/default tests/unit/config/config_test.py::ConfigTest::test_load -<table class="tg"> - <col width="45%"> - <col width="65%"> - <tr> - <td>Community Slack</td> - <td> - The Docker Community has a dedicated Slack chat to discuss features and issues. You can sign-up <a href="https://www.docker.com/community/" target="_blank">with this link</a>. - </td> - </tr> - <tr> - <td>Forums</td> - <td> - A public forum for users to discuss questions and explore current design patterns and - best practices about Docker and related projects in the Docker Ecosystem. To participate, - just log in with your Docker Hub account on <a href="https://forums.docker.com" target="_blank">https://forums.docker.com</a>. - </td> - </tr> - <tr> - <td>Twitter</td> - <td> - You can follow <a href="https://twitter.com/docker/" target="_blank">Docker's Twitter feed</a> - to get updates on our products. You can also tweet us questions or just - share blogs or stories. - </td> - </tr> - <tr> - <td>Stack Overflow</td> - <td> - Stack Overflow has over 17000 Docker questions listed. We regularly - monitor <a href="https://stackoverflow.com/questions/tagged/docker" target="_blank">Docker questions</a> - and so do many other knowledgeable Docker users. - </td> - </tr> -</table> +## Finding things to work on - -### Conventions - -Fork the repository and make changes on your fork in a feature branch: - -- If it's a bug fix branch, name it XXXX-something where XXXX is the number of - the issue. -- If it's a feature branch, create an enhancement issue to announce - your intentions, and name it XXXX-something where XXXX is the number of the - issue. - -Submit unit tests for your changes. Go has a great test framework built in; use -it! Take a look at existing tests for inspiration. Also, end-to-end tests are -available. Run the full test suite, both unit tests and e2e tests on your -branch before submitting a pull request. See [BUILDING.md](BUILDING.md) for -instructions to build and run tests. - -Write clean code. Universally formatted code promotes ease of writing, reading, -and maintenance. Always run `gofmt -s -w file.go` on each changed file before -committing your changes. Most editors have plug-ins that do this automatically. - -Pull request descriptions should be as clear as possible and include a reference -to all the issues that they address. - -Commit messages must start with a capitalized and short summary (max. 50 chars) -written in the imperative, followed by an optional, more detailed explanatory -text which is separated from the summary by an empty line. - -Code review comments may be added to your pull request. Discuss, then make the -suggested modifications and push additional commits to your feature branch. Post -a comment after pushing. New commits show up in the pull request automatically, -but the reviewers are notified only when you comment. - -Pull requests must be cleanly rebased on top of the base branch without multiple branches -mixed into the PR. - -**Git tip**: If your PR no longer merges cleanly, use `rebase master` in your -feature branch to update your pull request rather than `merge master`. - -Before you make a pull request, squash your commits into logical units of work -using `git rebase -i` and `git push -f`. A logical unit of work is a consistent -set of patches that should be reviewed together: for example, upgrading the -version of a vendored dependency and taking advantage of its now available new -feature constitute two separate units of work. Implementing a new function and -calling it in another file constitute a single logical unit of work. The very -high majority of submissions should have a single commit, so if in doubt: squash -down to one. - -After every commit, make sure the test suite passes. Include documentation -changes in the same pull request so that a revert would remove all traces of -the feature or fix. - -Include an issue reference like `Closes #XXXX` or `Fixes #XXXX` in the pull -request description that closes an issue. Including references automatically -closes the issue on a merge. - -Please do not add yourself to the `AUTHORS` file, as it is regenerated regularly -from the Git history. - -Please see the [Coding Style](#coding-style) for further guidelines. - -### Merge approval - -Docker maintainers use LGTM (Looks Good To Me) in comments on the code review to -indicate acceptance. - -A change requires at least 2 LGTMs from the maintainers of each -component affected. - -For more details, see the [MAINTAINERS](MAINTAINERS) page. - -### Sign your work - -The sign-off is a simple line at the end of the explanation for the patch. Your -signature certifies that you wrote the patch or otherwise have the right to pass -it on as an open-source patch. The rules are pretty simple: if you can certify -the below (from [developercertificate.org](https://developercertificate.org/)): - -``` -Developer Certificate of Origin -Version 1.1 - -Copyright (C) 2004, 2006 The Linux Foundation and its contributors. -660 York Street, Suite 102, -San Francisco, CA 94110 USA - -Everyone is permitted to copy and distribute verbatim copies of this -license document, but changing it is not allowed. - -Developer's Certificate of Origin 1.1 - -By making a contribution to this project, I certify that: - -(a) The contribution was created in whole or in part by me and I - have the right to submit it under the open source license - indicated in the file; or - -(b) The contribution is based upon previous work that, to the best - of my knowledge, is covered under an appropriate open source - license and I have the right under that license to submit that - work with modifications, whether created in whole or in part - by me, under the same open source license (unless I am - permitted to submit under a different license), as indicated - in the file; or - -(c) The contribution was provided directly to me by some other - person who certified (a), (b) or (c) and I have not modified - it. - -(d) I understand and agree that this project and the contribution - are public and that a record of the contribution (including all - personal information I submit with it, including my sign-off) is - maintained indefinitely and may be redistributed consistent with - this project or the open source license(s) involved. -``` - -Then you just add a line to every git commit message: - - Signed-off-by: Joe Smith <joe.smith@email.com> - -Use your real name (sorry, no pseudonyms or anonymous contributions.) - -If you set your `user.name` and `user.email` git configs, you can sign your -commit automatically with `git commit -s`. - -### How can I become a maintainer? - -The procedures for adding new maintainers are explained in the global -[MAINTAINERS](https://github.com/docker/opensource/blob/main/MAINTAINERS) -file in the -[https://github.com/docker/opensource/](https://github.com/docker/opensource/) -repository. - -Don't forget: being a maintainer is a time investment. Make sure you -will have time to make yourself available. You don't have to be a -maintainer to make a difference on the project! - -## Docker community guidelines - -We want to keep the Docker community awesome, growing and collaborative. We need -your help to keep it that way. To help with this we've come up with some general -guidelines for the community as a whole: - -* Be nice: Be courteous, respectful and polite to fellow community members: - no regional, racial, gender or other abuse will be tolerated. We like - nice people way better than mean ones! - -* Encourage diversity and participation: Make everyone in our community feel - welcome, regardless of their background and the extent of their - contributions, and do everything possible to encourage participation in - our community. - -* Keep it legal: Basically, don't get us in trouble. Share only content that - you own, do not share private or sensitive information, and don't break - the law. - -* Stay on topic: Make sure that you are posting to the correct channel and - avoid off-topic discussions. Remember when you update an issue or respond - to an email you are potentially sending it to a large number of people. Please - consider this before you update. Also, remember that nobody likes spam. - -* Don't send emails to the maintainers: There's no need to send emails to the - maintainers to ask them to investigate an issue or to take a look at a - pull request. Instead of sending an email, GitHub mentions should be - used to ping maintainers to review a pull request, a proposal or an - issue. - -## Coding Style - -Unless explicitly stated, we follow all coding guidelines from the Go -community. While some of these standards may seem arbitrary, they somehow seem -to result in a solid, consistent codebase. - -It is possible that the code base does not currently comply with these -guidelines. We are not looking for a massive PR that fixes this, since that -goes against the spirit of the guidelines. All new contributors should make their -best effort to clean up and make the code base better than they left it. -Obviously, apply your best judgement. Remember, the goal here is to make the -code base easier for humans to navigate and understand. Always keep that in -mind when nudging others to comply. - -The rules: - -1. All code should be formatted with `gofmt -s`. -2. All code should pass the default levels of - [`golint`](https://github.com/golang/lint). -3. All code should follow the guidelines covered in [Effective - Go](https://go.dev/doc/effective_go) and [Go Code Review - Comments](https://go.dev/wiki/CodeReviewComments). -4. Include code comments. Tell us the why, the history and the context. -5. Document _all_ declarations and methods, even private ones. Declare - expectations, caveats and anything else that may be important. If a type - gets exported, having the comments already there will ensure it's ready. -6. Variable name length should be proportional to its context and no longer. - `noCommaALongVariableNameLikeThisIsNotMoreClearWhenASimpleCommentWouldDo`. - In practice, short methods will have short variable names and globals will - have longer names. -7. No underscores in package names. If you need a compound name, step back, - and re-examine why you need a compound name. If you still think you need a - compound name, lose the underscore. -8. No utils or helpers packages. If a function is not general enough to - warrant its own package, it has not been written generally enough to be a - part of a util package. Just leave it unexported and well-documented. -9. All tests should run with `go test` and outside tooling should not be - required. No, we don't need another unit testing framework. Assertion - packages are acceptable if they provide _real_ incremental value. -10. Even though we call these "rules" above, they are actually just - guidelines. Since you've read all the rules, you now know that. - -If you are having trouble getting into the mood of idiomatic Go, we recommend -reading through [Effective Go](https://go.dev/doc/effective_go). The -[Go Blog](https://go.dev/blog/) is also a great resource. Drinking the -kool-aid is a lot easier than going thirsty. +[Issues marked with the `exp/beginner` label](https://github.com/docker/compose/issues?q=is%3Aopen+is%3Aissue+label%3Aexp%2Fbeginner) are a good starting point for people looking to make their first contribution to the project. diff --git a/Dockerfile b/Dockerfile index 7ab1c8a18dd..a4fc34a92c1 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,197 +1,100 @@ -# syntax=docker/dockerfile:1 +ARG DOCKER_VERSION=19.03 +ARG PYTHON_VERSION=3.9.0 +ARG BUILD_ALPINE_VERSION=3.12 +ARG BUILD_CENTOS_VERSION=7 +ARG BUILD_DEBIAN_VERSION=slim-buster -# Copyright 2020 Docker Compose CLI authors +ARG RUNTIME_ALPINE_VERSION=3.12 +ARG RUNTIME_CENTOS_VERSION=7 +ARG RUNTIME_DEBIAN_VERSION=buster-slim -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at +ARG DISTRO=alpine -# http://www.apache.org/licenses/LICENSE-2.0 +FROM docker:${DOCKER_VERSION} AS docker-cli -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -ARG GO_VERSION=1.25.7 -ARG XX_VERSION=1.9.0 -ARG GOLANGCI_LINT_VERSION=v2.8.0 -ARG ADDLICENSE_VERSION=v1.0.0 - -ARG BUILD_TAGS="e2e" -ARG DOCS_FORMATS="md,yaml" -ARG LICENSE_FILES=".*$Dockerfile\|Makefile\|\.go\|\.hcl\|\.sh$" - -# xx is a helper for cross-compilation -FROM --platform=${BUILDPLATFORM} tonistiigi/xx:${XX_VERSION} AS xx - -# osxcross contains the MacOSX cross toolchain for xx -FROM crazymax/osxcross:15.5-alpine AS osxcross - -FROM golangci/golangci-lint:${GOLANGCI_LINT_VERSION}-alpine AS golangci-lint -FROM ghcr.io/google/addlicense:${ADDLICENSE_VERSION} AS addlicense - -FROM --platform=${BUILDPLATFORM} golang:${GO_VERSION}-alpine3.22 AS base -COPY --from=xx / / +FROM python:${PYTHON_VERSION}-alpine${BUILD_ALPINE_VERSION} AS build-alpine RUN apk add --no-cache \ - clang \ - docker \ - file \ - findutils \ - git \ - make \ - protoc \ - protobuf-dev -WORKDIR /src -ENV CGO_ENABLED=0 - -FROM base AS build-base -COPY go.* . -RUN --mount=type=cache,target=/go/pkg/mod \ - --mount=type=cache,target=/root/.cache/go-build \ - go mod download - -FROM build-base AS vendored -RUN --mount=type=bind,target=.,rw \ - --mount=type=cache,target=/go/pkg/mod \ - go mod tidy && mkdir /out && cp go.mod go.sum /out - -FROM scratch AS vendor-update -COPY --from=vendored /out / - -FROM vendored AS vendor-validate -RUN --mount=type=bind,target=.,rw <<EOT - set -e - git add -A - cp -rf /out/* . - diff=$(git status --porcelain -- go.mod go.sum) - if [ -n "$diff" ]; then - echo >&2 'ERROR: Vendor result differs. Please vendor your package with "make go-mod-tidy"' - echo "$diff" - exit 1 - fi -EOT - -FROM build-base AS build -ARG BUILD_TAGS -ARG BUILD_FLAGS -ARG TARGETPLATFORM -RUN --mount=type=bind,target=. \ - --mount=type=cache,target=/root/.cache \ - --mount=type=cache,target=/go/pkg/mod \ - --mount=type=bind,from=osxcross,src=/osxsdk,target=/xx-sdk \ - xx-go --wrap && \ - if [ "$(xx-info os)" == "darwin" ]; then export CGO_ENABLED=1; export BUILD_TAGS=fsnotify,$BUILD_TAGS; fi && \ - make build GO_BUILDTAGS="$BUILD_TAGS" DESTDIR=/out && \ - xx-verify --static /out/docker-compose - -FROM build-base AS lint -ARG BUILD_TAGS -ENV GOLANGCI_LINT_CACHE=/cache/golangci-lint -RUN --mount=type=bind,target=. \ - --mount=type=cache,target=/root/.cache \ - --mount=type=cache,target=/go/pkg/mod \ - --mount=type=cache,target=/cache/golangci-lint \ - --mount=from=golangci-lint,source=/usr/bin/golangci-lint,target=/usr/bin/golangci-lint \ - golangci-lint cache status && \ - golangci-lint run --build-tags "$BUILD_TAGS" ./... - -FROM build-base AS test -ARG CGO_ENABLED=0 -ARG BUILD_TAGS -RUN --mount=type=bind,target=. \ - --mount=type=cache,target=/root/.cache \ - --mount=type=cache,target=/go/pkg/mod \ - rm -rf /tmp/coverage && \ - mkdir -p /tmp/coverage && \ - rm -rf /tmp/report && \ - mkdir -p /tmp/report && \ - go run gotest.tools/gotestsum@latest --format testname --junitfile "/tmp/report/report.xml" -- -tags "$BUILD_TAGS" -v -cover -covermode=atomic $(go list $(TAGS) ./... | grep -vE 'e2e') -args -test.gocoverdir="/tmp/coverage" && \ - go tool covdata percent -i=/tmp/coverage - -FROM scratch AS test-coverage -COPY --from=test --link /tmp/coverage / -COPY --from=test --link /tmp/report / - -FROM base AS license-set -ARG LICENSE_FILES -RUN --mount=type=bind,target=.,rw \ - --mount=from=addlicense,source=/app/addlicense,target=/usr/bin/addlicense \ - find . -regex "${LICENSE_FILES}" | xargs addlicense -c 'Docker Compose CLI' -l apache && \ - mkdir /out && \ - find . -regex "${LICENSE_FILES}" | cpio -pdm /out - -FROM scratch AS license-update -COPY --from=set /out / - -FROM base AS license-validate -ARG LICENSE_FILES -RUN --mount=type=bind,target=. \ - --mount=from=addlicense,source=/app/addlicense,target=/usr/bin/addlicense \ - find . -regex "${LICENSE_FILES}" | xargs addlicense -check -c 'Docker Compose CLI' -l apache -ignore validate -ignore testdata -ignore resolvepath -v - -FROM base AS docsgen -WORKDIR /src -RUN --mount=target=. \ - --mount=target=/root/.cache,type=cache \ - --mount=type=cache,target=/go/pkg/mod \ - go build -o /out/docsgen ./docs/yaml/main/generate.go - -FROM --platform=${BUILDPLATFORM} alpine AS docs-build -RUN apk add --no-cache rsync git -WORKDIR /src -COPY --from=docsgen /out/docsgen /usr/bin -ARG DOCS_FORMATS -RUN --mount=target=/context \ - --mount=target=.,type=tmpfs <<EOT - set -e - rsync -a /context/. . - docsgen --formats "$DOCS_FORMATS" --source "docs/reference" - mkdir /out - cp -r docs/reference /out -EOT - -FROM scratch AS docs-update -COPY --from=docs-build /out /out - -FROM docs-build AS docs-validate -RUN --mount=target=/context \ - --mount=target=.,type=tmpfs <<EOT - set -e - rsync -a /context/. . - git add -A - rm -rf docs/reference/* - cp -rf /out/* ./docs/ - if [ -n "$(git status --porcelain -- docs/reference)" ]; then - echo >&2 'ERROR: Docs result differs. Please update with "make docs"' - git status --porcelain -- docs/reference - exit 1 - fi -EOT - -FROM scratch AS binary-unix -COPY --link --from=build /out/docker-compose / -FROM binary-unix AS binary-darwin -FROM binary-unix AS binary-linux -FROM scratch AS binary-windows -COPY --link --from=build /out/docker-compose /docker-compose.exe -FROM binary-$TARGETOS AS binary -# enable scanning for this stage -ARG BUILDKIT_SBOM_SCAN_STAGE=true - -FROM --platform=$BUILDPLATFORM alpine AS releaser -WORKDIR /work -ARG TARGETOS + bash \ + build-base \ + ca-certificates \ + curl \ + gcc \ + git \ + libc-dev \ + libffi-dev \ + libgcc \ + make \ + musl-dev \ + openssl \ + openssl-dev \ + zlib-dev +ENV BUILD_BOOTLOADER=1 + +FROM python:${PYTHON_VERSION}-${BUILD_DEBIAN_VERSION} AS build-debian +RUN apt-get update && apt-get install --no-install-recommends -y \ + curl \ + gcc \ + git \ + libc-dev \ + libffi-dev \ + libgcc-8-dev \ + libssl-dev \ + make \ + openssl \ + zlib1g-dev + +FROM centos:${BUILD_CENTOS_VERSION} AS build-centos +RUN yum install -y \ + gcc \ + git \ + libffi-devel \ + make \ + openssl \ + openssl-devel +WORKDIR /tmp/python3/ +ARG PYTHON_VERSION +RUN curl -L https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz | tar xzf - \ + && cd Python-${PYTHON_VERSION} \ + && ./configure --enable-optimizations --enable-shared --prefix=/usr LDFLAGS="-Wl,-rpath /usr/lib" \ + && make altinstall +RUN alternatives --install /usr/bin/python python /usr/bin/python2.7 50 +RUN alternatives --install /usr/bin/python python /usr/bin/python$(echo "${PYTHON_VERSION%.*}") 60 +RUN curl https://bootstrap.pypa.io/get-pip.py | python - + +FROM build-${DISTRO} AS build +ENTRYPOINT ["sh", "/usr/local/bin/docker-compose-entrypoint.sh"] +WORKDIR /code/ +COPY docker-compose-entrypoint.sh /usr/local/bin/ +COPY --from=docker-cli /usr/local/bin/docker /usr/local/bin/docker +RUN pip install \ + virtualenv==20.2.2 \ + tox==3.20.1 +COPY requirements-dev.txt . +COPY requirements-indirect.txt . +COPY requirements.txt . +RUN pip install -r requirements.txt -r requirements-indirect.txt -r requirements-dev.txt +COPY .pre-commit-config.yaml . +COPY tox.ini . +COPY setup.py . +COPY README.md . +COPY compose compose/ +RUN tox --notest +COPY . . +ARG GIT_COMMIT=unknown +ENV DOCKER_COMPOSE_GITSHA=$GIT_COMMIT +RUN script/build/linux-entrypoint + +FROM scratch AS bin ARG TARGETARCH -ARG TARGETVARIANT -RUN --mount=from=binary \ - mkdir -p /out && \ - # TODO: should just use standard arch - TARGETARCH=$([ "$TARGETARCH" = "amd64" ] && echo "x86_64" || echo "$TARGETARCH"); \ - TARGETARCH=$([ "$TARGETARCH" = "arm64" ] && echo "aarch64" || echo "$TARGETARCH"); \ - cp docker-compose* "/out/docker-compose-${TARGETOS}-${TARGETARCH}${TARGETVARIANT}$(ls docker-compose* | sed -e 's/^docker-compose//')" - -FROM scratch AS release -COPY --from=releaser /out/ / +ARG TARGETOS +COPY --from=build /usr/local/bin/docker-compose /docker-compose-${TARGETOS}-${TARGETARCH} + +FROM alpine:${RUNTIME_ALPINE_VERSION} AS runtime-alpine +FROM debian:${RUNTIME_DEBIAN_VERSION} AS runtime-debian +FROM centos:${RUNTIME_CENTOS_VERSION} AS runtime-centos +FROM runtime-${DISTRO} AS runtime +COPY docker-compose-entrypoint.sh /usr/local/bin/ +ENTRYPOINT ["sh", "/usr/local/bin/docker-compose-entrypoint.sh"] +COPY --from=docker-cli /usr/local/bin/docker /usr/local/bin/docker +COPY --from=build /usr/local/bin/docker-compose /usr/local/bin/docker-compose diff --git a/Jenkinsfile b/Jenkinsfile new file mode 100644 index 00000000000..40629216a38 --- /dev/null +++ b/Jenkinsfile @@ -0,0 +1,115 @@ +#!groovy + +def dockerVersions = ['19.03.13'] +def baseImages = ['alpine', 'debian'] +def pythonVersions = ['py39'] + +pipeline { + agent none + + options { + skipDefaultCheckout(true) + buildDiscarder(logRotator(daysToKeepStr: '30')) + timeout(time: 2, unit: 'HOURS') + timestamps() + } + environment { + DOCKER_BUILDKIT="1" + } + + stages { + stage('Build test images') { + // TODO use declarative 1.5.0 `matrix` once available on CI + parallel { + stage('alpine') { + agent { + label 'ubuntu && amd64 && !zfs' + } + steps { + buildImage('alpine') + } + } + stage('debian') { + agent { + label 'ubuntu && amd64 && !zfs' + } + steps { + buildImage('debian') + } + } + } + } + stage('Test') { + steps { + // TODO use declarative 1.5.0 `matrix` once available on CI + script { + def testMatrix = [:] + baseImages.each { baseImage -> + dockerVersions.each { dockerVersion -> + pythonVersions.each { pythonVersion -> + testMatrix["${baseImage}_${dockerVersion}_${pythonVersion}"] = runTests(dockerVersion, pythonVersion, baseImage) + } + } + } + + parallel testMatrix + } + } + } + } +} + + +def buildImage(baseImage) { + def scmvar = checkout(scm) + def imageName = "dockerbuildbot/compose:${baseImage}-${scmvar.GIT_COMMIT}" + image = docker.image(imageName) + + withDockerRegistry(credentialsId:'dockerbuildbot-index.docker.io') { + try { + image.pull() + } catch (Exception exc) { + ansiColor('xterm') { + sh """docker build -t ${imageName} \\ + --target build \\ + --build-arg DISTRO="${baseImage}" \\ + --build-arg GIT_COMMIT="${scmvar.GIT_COMMIT}" \\ + .\\ + """ + sh "docker push ${imageName}" + } + echo "${imageName}" + return imageName + } + } +} + +def runTests(dockerVersion, pythonVersion, baseImage) { + return { + stage("python=${pythonVersion} docker=${dockerVersion} ${baseImage}") { + node("ubuntu && amd64 && !zfs") { + def scmvar = checkout(scm) + def imageName = "dockerbuildbot/compose:${baseImage}-${scmvar.GIT_COMMIT}" + def storageDriver = sh(script: "docker info -f \'{{.Driver}}\'", returnStdout: true).trim() + echo "Using local system's storage driver: ${storageDriver}" + withDockerRegistry(credentialsId:'dockerbuildbot-index.docker.io') { + sh """docker run \\ + -t \\ + --rm \\ + --privileged \\ + --volume="\$(pwd)/.git:/code/.git" \\ + --volume="/var/run/docker.sock:/var/run/docker.sock" \\ + -e "TAG=${imageName}" \\ + -e "STORAGE_DRIVER=${storageDriver}" \\ + -e "DOCKER_VERSIONS=${dockerVersion}" \\ + -e "BUILD_NUMBER=${env.BUILD_NUMBER}" \\ + -e "PY_TEST_VERSIONS=${pythonVersion}" \\ + --entrypoint="script/test/ci" \\ + ${imageName} \\ + --verbose + """ + } + } + } + } +} diff --git a/LICENSE b/LICENSE index 7a4a3ea2424..27448585ad4 100644 --- a/LICENSE +++ b/LICENSE @@ -176,18 +176,7 @@ END OF TERMS AND CONDITIONS - APPENDIX: How to apply the Apache License to your work. - - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "[]" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. - - Copyright [yyyy] [name of copyright owner] + Copyright 2014 Docker, Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -199,4 +188,4 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and - limitations under the License. \ No newline at end of file + limitations under the License. diff --git a/MAINTAINERS b/MAINTAINERS new file mode 100644 index 00000000000..7e178147e84 --- /dev/null +++ b/MAINTAINERS @@ -0,0 +1,105 @@ +# Compose maintainers file +# +# This file describes who runs the docker/compose project and how. +# This is a living document - if you see something out of date or missing, speak up! +# +# It is structured to be consumable by both humans and programs. +# To extract its contents programmatically, use any TOML-compliant parser. +# +# This file is compiled into the MAINTAINERS file in docker/opensource. +# +[Org] + [Org."Core maintainers"] + people = [ + "aiordache", + "ndeloof", + "rumpl", + "ulyssessouza", + ] + [Org.Alumni] + people = [ + # Aanand Prasad is one of the two creators of the fig project + # which later went on to become docker-compose, and a longtime + # maintainer responsible for several keystone features + "aanand", + # Ben Firshman is also one of the fig creators and contributed + # heavily to the project's design and UX as well as the + # day-to-day maintenance + "bfirsh", + # Mazz Mosley made significant contributions to the project + # in 2015 with solid bugfixes and improved error handling + # among them + "mnowster", + # Daniel Nephin is one of the longest-running maintainers on + # the Compose project, and has contributed several major features + # including muti-file support, variable interpolation, secrets + # emulation and many more + "dnephin", + + "shin-", + "mefyl", + "mnottale", + ] + +[people] + +# A reference list of all people associated with the project. +# All other sections should refer to people by their canonical key +# in the people section. + + # ADD YOURSELF HERE IN ALPHABETICAL ORDER + + [people.aanand] + Name = "Aanand Prasad" + Email = "aanand.prasad@gmail.com" + GitHub = "aanand" + + [people.aiordache] + Name = "Anca Iordache" + Email = "anca.iordache@docker.com" + GitHub = "aiordache" + + [people.bfirsh] + Name = "Ben Firshman" + Email = "ben@firshman.co.uk" + GitHub = "bfirsh" + + [people.dnephin] + Name = "Daniel Nephin" + Email = "dnephin@gmail.com" + GitHub = "dnephin" + + [people.mefyl] + Name = "Quentin Hocquet" + Email = "quentin.hocquet@docker.com" + GitHub = "mefyl" + + [people.mnottale] + Name = "Matthieu Nottale" + Email = "matthieu.nottale@docker.com" + GitHub = "mnottale" + + [people.mnowster] + Name = "Mazz Mosley" + Email = "mazz@houseofmnowster.com" + GitHub = "mnowster" + + [people.ndeloof] + Name = "Nicolas De Loof" + Email = "nicolas.deloof@gmail.com" + GitHub = "ndeloof" + + [people.rumpl] + Name = "Djordje Lukic" + Email = "djordje.lukic@docker.com" + GitHub = "rumpl" + + [people.shin-] + Name = "Joffrey F" + Email = "f.joffrey@gmail.com" + GitHub = "shin-" + + [people.ulyssessouza] + Name = "Ulysses Domiciano Souza" + Email = "ulysses.souza@docker.com" + GitHub = "ulyssessouza" diff --git a/MANIFEST.in b/MANIFEST.in new file mode 100644 index 00000000000..313b4e00814 --- /dev/null +++ b/MANIFEST.in @@ -0,0 +1,15 @@ +include Dockerfile +include LICENSE +include requirements-indirect.txt +include requirements.txt +include requirements-dev.txt +include tox.ini +include *.md +include README.md +include compose/config/*.json +include compose/GITSHA +recursive-include contrib/completion * +recursive-include tests * +global-exclude *.pyc +global-exclude *.pyo +global-exclude *.un~ diff --git a/Makefile b/Makefile index a29eebe7739..0a7a5c366b4 100644 --- a/Makefile +++ b/Makefile @@ -1,162 +1,57 @@ -# Copyright 2020 Docker Compose CLI authors +TAG = "docker-compose:alpine-$(shell git rev-parse --short HEAD)" +GIT_VOLUME = "--volume=$(shell pwd)/.git:/code/.git" -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at +DOCKERFILE ?="Dockerfile" +DOCKER_BUILD_TARGET ?="build" -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -PKG := github.com/docker/compose/v5 -VERSION ?= $(shell git describe --match 'v[0-9]*' --dirty='.m' --always --tags) - -GO_LDFLAGS ?= -w -X ${PKG}/internal.Version=${VERSION} -GO_BUILDTAGS ?= e2e -DRIVE_PREFIX?= -ifeq ($(OS),Windows_NT) - DETECTED_OS = Windows - DRIVE_PREFIX=C: -else - DETECTED_OS = $(shell uname -s) -endif - -ifeq ($(DETECTED_OS),Windows) - BINARY_EXT=.exe -endif - -BUILD_FLAGS?= -TEST_FLAGS?= -E2E_TEST?= -ifneq ($(E2E_TEST),) - TEST_FLAGS:=$(TEST_FLAGS) -run '$(E2E_TEST)' +UNAME_S := $(shell uname -s) +ifeq ($(UNAME_S),Linux) + BUILD_SCRIPT = linux endif - -EXCLUDE_E2E_TESTS?= -ifneq ($(EXCLUDE_E2E_TESTS),) - TEST_FLAGS:=$(TEST_FLAGS) -skip '$(EXCLUDE_E2E_TESTS)' +ifeq ($(UNAME_S),Darwin) + BUILD_SCRIPT = osx endif -BUILDX_CMD ?= docker buildx - -# DESTDIR overrides the output path for binaries and other artifacts -# this is used by docker/docker-ce-packaging for the apt/rpm builds, -# so it's important that the resulting binary ends up EXACTLY at the -# path $DESTDIR/docker-compose when specified. -# -# See https://github.com/docker/docker-ce-packaging/blob/e43fbd37e48fde49d907b9195f23b13537521b94/rpm/SPECS/docker-compose-plugin.spec#L47 -# -# By default, all artifacts go to subdirectories under ./bin/ in the -# repo root, e.g. ./bin/build, ./bin/coverage, ./bin/release. -DESTDIR ?= - -all: build - -.PHONY: build ## Build the compose cli-plugin -build: - GO111MODULE=on go build $(BUILD_FLAGS) -trimpath -tags "$(GO_BUILDTAGS)" -ldflags "$(GO_LDFLAGS)" -o "$(or $(DESTDIR),./bin/build)/docker-compose$(BINARY_EXT)" ./cmd - -.PHONY: binary -binary: - BUILD_TAGS="$(GO_BUILDTAGS)" $(BUILDX_CMD) bake binary - -.PHONY: binary-with-coverage -binary-with-coverage: - BUILD_TAGS="$(GO_BUILDTAGS)" $(BUILDX_CMD) bake binary-with-coverage - -.PHONY: install -install: binary - mkdir -p ~/.docker/cli-plugins - install $(or $(DESTDIR),./bin/build)/docker-compose ~/.docker/cli-plugins/docker-compose +COMPOSE_SPEC_SCHEMA_PATH = "compose/config/compose_spec.json" +COMPOSE_SPEC_RAW_URL = "https://raw.githubusercontent.com/compose-spec/compose-spec/master/schema/compose-spec.json" -.PHONY: e2e-compose -e2e-compose: example-provider ## Run end to end local tests in plugin mode. Set E2E_TEST=TestName to run a single test - go run gotest.tools/gotestsum@latest --format testname --junitfile "/tmp/report/report.xml" -- -v $(TEST_FLAGS) -count=1 ./pkg/e2e +all: cli -.PHONY: e2e-compose-standalone -e2e-compose-standalone: ## Run End to end local tests in standalone mode. Set E2E_TEST=TestName to run a single test - go run gotest.tools/gotestsum@latest --format testname --junitfile "/tmp/report/report.xml" -- $(TEST_FLAGS) -v -count=1 -parallel=1 --tags=standalone ./pkg/e2e +cli: download-compose-spec ## Compile the cli + ./script/build/$(BUILD_SCRIPT) -.PHONY: build-and-e2e-compose -build-and-e2e-compose: build e2e-compose ## Compile the compose cli-plugin and run end to end local tests in plugin mode. Set E2E_TEST=TestName to run a single test +download-compose-spec: ## Download the compose-spec schema from it's repo + curl -so $(COMPOSE_SPEC_SCHEMA_PATH) $(COMPOSE_SPEC_RAW_URL) -.PHONY: build-and-e2e-compose-standalone -build-and-e2e-compose-standalone: build e2e-compose-standalone ## Compile the compose cli-plugin and run End to end local tests in standalone mode. Set E2E_TEST=TestName to run a single test - -.PHONY: example-provider -example-provider: ## build example provider for e2e tests - go build -o bin/build/example-provider docs/examples/provider.go - -.PHONY: mocks -mocks: - mockgen --version >/dev/null 2>&1 || go install go.uber.org/mock/mockgen@v0.4.0 - mockgen -destination pkg/mocks/mock_docker_cli.go -package mocks github.com/docker/cli/cli/command Cli - mockgen -destination pkg/mocks/mock_docker_api.go -package mocks github.com/docker/docker/client APIClient - mockgen -destination pkg/mocks/mock_docker_compose_api.go -package mocks -source=./pkg/api/api.go Service - -.PHONY: e2e -e2e: e2e-compose e2e-compose-standalone ## Run end to end local tests in both modes. Set E2E_TEST=TestName to run a single test - -.PHONY: build-and-e2e -build-and-e2e: build e2e-compose e2e-compose-standalone ## Compile the compose cli-plugin and run end to end local tests in both modes. Set E2E_TEST=TestName to run a single test - -.PHONY: cross -cross: ## Compile the CLI for linux, darwin and windows - $(BUILDX_CMD) bake binary-cross - -.PHONY: test -test: ## Run unit tests - $(BUILDX_CMD) bake test - -.PHONY: cache-clear cache-clear: ## Clear the builder cache - $(BUILDX_CMD) prune --force --filter type=exec.cachemount --filter=unused-for=24h + @docker builder prune --force --filter type=exec.cachemount --filter=unused-for=24h -.PHONY: lint -lint: ## run linter(s) - $(BUILDX_CMD) bake lint +base-image: ## Builds base image + docker build -f $(DOCKERFILE) -t $(TAG) --target $(DOCKER_BUILD_TARGET) . -.PHONY: fmt -fmt: - gofumpt --version >/dev/null 2>&1 || go install mvdan.cc/gofumpt@latest - gofumpt -w . +lint: base-image ## Run linter + docker run --rm \ + --tty \ + $(GIT_VOLUME) \ + $(TAG) \ + tox -e pre-commit -.PHONY: docs -docs: ## generate documentation - $(eval $@_TMP_OUT := $(shell mktemp -d -t compose-output.XXXXXXXXXX)) - $(BUILDX_CMD) bake --set "*.output=type=local,dest=$($@_TMP_OUT)" docs-update - rm -rf ./docs/internal - cp -R "$(DRIVE_PREFIX)$($@_TMP_OUT)"/out/* ./docs/ - rm -rf "$(DRIVE_PREFIX)$($@_TMP_OUT)"/* +test-unit: base-image ## Run tests + docker run --rm \ + --tty \ + $(GIT_VOLUME) \ + $(TAG) \ + pytest -v tests/unit/ -.PHONY: validate-docs -validate-docs: ## validate the doc does not change - $(BUILDX_CMD) bake docs-validate +test: ## Run all tests + ./script/test/default -.PHONY: check-dependencies -check-dependencies: ## check dependency updates - go list -u -m -f '{{if not .Indirect}}{{if .Update}}{{.}}{{end}}{{end}}' all - -.PHONY: validate-headers -validate-headers: ## Check license header for all files - $(BUILDX_CMD) bake license-validate - -.PHONY: go-mod-tidy -go-mod-tidy: ## Run go mod tidy in a container and output resulting go.mod and go.sum - $(BUILDX_CMD) bake vendor-update - -.PHONY: validate-go-mod -validate-go-mod: ## Validate go.mod and go.sum are up-to-date - $(BUILDX_CMD) bake vendor-validate - -validate: validate-go-mod validate-headers validate-docs ## Validate sources - -pre-commit: validate check-dependencies lint build test e2e-compose +pre-commit: lint test-unit cli help: ## Show help @echo Please specify a build target. The choices are: @grep -E '^[0-9a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' + +FORCE: + +.PHONY: all cli download-compose-spec cache-clear base-image lint test-unit test pre-commit help diff --git a/NOTICE b/NOTICE deleted file mode 100644 index dffae91878a..00000000000 --- a/NOTICE +++ /dev/null @@ -1,4 +0,0 @@ -Docker Compose V2 -Copyright 2020 Docker Compose authors - -This product includes software developed at Docker, Inc. (https://www.docker.com). diff --git a/README.md b/README.md index 254376f9e4b..d0d23d8af60 100644 --- a/README.md +++ b/README.md @@ -1,41 +1,30 @@ -# Table of Contents -- [Docker Compose](#docker-compose) -- [Where to get Docker Compose](#where-to-get-docker-compose) - + [Windows and macOS](#windows-and-macos) - + [Linux](#linux) -- [Quick Start](#quick-start) -- [Contributing](#contributing) -- [Legacy](#legacy) - -# Docker Compose - -[![GitHub release](https://img.shields.io/github/v/release/docker/compose.svg?style=flat-square)](https://github.com/docker/compose/releases/latest) -[![PkgGoDev](https://img.shields.io/badge/go.dev-docs-007d9c?style=flat-square&logo=go&logoColor=white)](https://pkg.go.dev/github.com/docker/compose/v5) -[![Build Status](https://img.shields.io/github/actions/workflow/status/docker/compose/ci.yml?label=ci&logo=github&style=flat-square)](https://github.com/docker/compose/actions?query=workflow%3Aci) -[![Go Report Card](https://goreportcard.com/badge/github.com/docker/compose/v5?style=flat-square)](https://goreportcard.com/report/github.com/docker/compose/v5) -[![Codecov](https://codecov.io/gh/docker/compose/branch/main/graph/badge.svg?token=HP3K4Y4ctu)](https://codecov.io/gh/docker/compose) -[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/docker/compose/badge)](https://api.securityscorecards.dev/projects/github.com/docker/compose) +Docker Compose +============== +[![Build Status](https://ci-next.docker.com/public/buildStatus/icon?job=compose/master)](https://ci-next.docker.com/public/job/compose/job/master/) + ![Docker Compose](logo.png?raw=true "Docker Compose Logo") Docker Compose is a tool for running multi-container applications on Docker defined using the [Compose file format](https://compose-spec.io). -A Compose file is used to define how one or more containers that make up +A Compose file is used to define how the one or more containers that make up your application are configured. Once you have a Compose file, you can create and start your application with a -single command: `docker compose up`. +single command: `docker-compose up`. -> **Note**: About Docker Swarm -> Docker Swarm used to rely on the legacy compose file format but did not adopt the compose specification -> so is missing some of the recent enhancements in the compose syntax. After -> [acquisition by Mirantis](https://www.mirantis.com/software/swarm/) swarm isn't maintained by Docker Inc, and -> as such some Docker Compose features aren't accessible to swarm users. +Compose files can be used to deploy applications locally, or to the cloud on +[Amazon ECS](https://aws.amazon.com/ecs) or +[Microsoft ACI](https://azure.microsoft.com/services/container-instances/) using +the Docker CLI. You can read more about how to do this: +- [Compose for Amazon ECS](https://docs.docker.com/engine/context/ecs-integration/) +- [Compose for Microsoft ACI](https://docs.docker.com/engine/context/aci-integration/) -# Where to get Docker Compose +Where to get Docker Compose +---------------------------- ### Windows and macOS Docker Compose is included in -[Docker Desktop](https://www.docker.com/products/docker-desktop/) +[Docker Desktop](https://www.docker.com/products/docker-desktop) for Windows and macOS. ### Linux @@ -43,25 +32,25 @@ for Windows and macOS. You can download Docker Compose binaries from the [release page](https://github.com/docker/compose/releases) on this repository. -Rename the relevant binary for your OS to `docker-compose` and copy it to `$HOME/.docker/cli-plugins` - -Or copy it into one of these folders to install it system-wide: +### Using pip -* `/usr/local/lib/docker/cli-plugins` OR `/usr/local/libexec/docker/cli-plugins` -* `/usr/lib/docker/cli-plugins` OR `/usr/libexec/docker/cli-plugins` +If your platform is not supported, you can download Docker Compose using `pip`: -(might require making the downloaded file executable with `chmod +x`) +```console +pip install docker-compose +``` +> **Note:** Docker Compose requires Python 3.6 or later. Quick Start ----------- -Using Docker Compose is a three-step process: +Using Docker Compose is basically a three-step process: 1. Define your app's environment with a `Dockerfile` so it can be reproduced anywhere. -2. Define the services that make up your app in `compose.yaml` so +2. Define the services that make up your app in `docker-compose.yml` so they can be run together in an isolated environment. -3. Lastly, run `docker compose up` and Compose will start and run your entire +3. Lastly, run `docker-compose up` and Compose will start and run your entire app. A Compose file looks like this: @@ -78,16 +67,22 @@ services: image: redis ``` +You can find examples of Compose applications in our +[Awesome Compose repository](https://github.com/docker/awesome-compose). + +For more information about the Compose format, see the +[Compose file reference](https://docs.docker.com/compose/compose-file/). + Contributing ------------ Want to help develop Docker Compose? Check out our -[contributing documentation](CONTRIBUTING.md). +[contributing documentation](https://github.com/docker/compose/blob/master/CONTRIBUTING.md). If you find an issue, please report it on the [issue tracker](https://github.com/docker/compose/issues/new/choose). -Legacy -------------- +Releasing +--------- -The Python version of Compose is available under the `v1` [branch](https://github.com/docker/compose/tree/v1). +Releases are built by maintainers, following an outline of the [release process](https://github.com/docker/compose/blob/master/project/RELEASE-PROCESS.md). diff --git a/Release.Jenkinsfile b/Release.Jenkinsfile new file mode 100644 index 00000000000..38cc53433fc --- /dev/null +++ b/Release.Jenkinsfile @@ -0,0 +1,310 @@ +#!groovy + +def dockerVersions = ['19.03.13', '18.09.9'] +def baseImages = ['alpine', 'debian'] +def pythonVersions = ['py39'] + +pipeline { + agent none + + options { + skipDefaultCheckout(true) + buildDiscarder(logRotator(daysToKeepStr: '30')) + timeout(time: 2, unit: 'HOURS') + timestamps() + } + environment { + DOCKER_BUILDKIT="1" + } + + stages { + stage('Build test images') { + // TODO use declarative 1.5.0 `matrix` once available on CI + parallel { + stage('alpine') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + buildImage('alpine') + } + } + stage('debian') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + buildImage('debian') + } + } + } + } + stage('Test') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + // TODO use declarative 1.5.0 `matrix` once available on CI + script { + def testMatrix = [:] + baseImages.each { baseImage -> + dockerVersions.each { dockerVersion -> + pythonVersions.each { pythonVersion -> + testMatrix["${baseImage}_${dockerVersion}_${pythonVersion}"] = runTests(dockerVersion, pythonVersion, baseImage) + } + } + } + + parallel testMatrix + } + } + } + stage('Generate Changelog') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + checkout scm + withCredentials([string(credentialsId: 'github-compose-release-test-token', variable: 'GITHUB_TOKEN')]) { + sh "./script/release/generate_changelog.sh" + } + archiveArtifacts artifacts: 'CHANGELOG.md' + stash( name: "changelog", includes: 'CHANGELOG.md' ) + } + } + stage('Package') { + parallel { + stage('macosx binary') { + agent { + label 'mac-python' + } + environment { + DEPLOYMENT_TARGET="10.11" + } + steps { + checkout scm + sh './script/setup/osx' + sh 'tox -e py39 -- tests/unit' + sh './script/build/osx' + dir ('dist') { + checksum('docker-compose-Darwin-x86_64') + checksum('docker-compose-Darwin-x86_64.tgz') + } + archiveArtifacts artifacts: 'dist/*', fingerprint: true + dir("dist") { + stash name: "bin-darwin" + } + } + } + stage('linux binary') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + checkout scm + sh ' ./script/build/linux' + dir ('dist') { + checksum('docker-compose-Linux-x86_64') + } + archiveArtifacts artifacts: 'dist/*', fingerprint: true + dir("dist") { + stash name: "bin-linux" + } + } + } + stage('windows binary') { + agent { + label 'windows-python' + } + environment { + PATH = "C:\\Python39;C:\\Python39\\Scripts;$PATH" + } + steps { + checkout scm + bat 'tox.exe -e py39 -- tests/unit' + powershell '.\\script\\build\\windows.ps1' + dir ('dist') { + checksum('docker-compose-Windows-x86_64.exe') + } + archiveArtifacts artifacts: 'dist/*', fingerprint: true + dir("dist") { + stash name: "bin-win" + } + } + } + stage('alpine image') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + buildRuntimeImage('alpine') + } + } + stage('debian image') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + buildRuntimeImage('debian') + } + } + } + } + stage('Release') { + when { + buildingTag() + } + parallel { + stage('Pushing images') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + pushRuntimeImage('alpine') + pushRuntimeImage('debian') + } + } + stage('Creating Github Release') { + agent { + label 'linux && docker && ubuntu-2004' + } + environment { + GITHUB_TOKEN = credentials('github-release-token') + } + steps { + checkout scm + sh 'mkdir -p dist' + dir("dist") { + unstash "bin-darwin" + unstash "bin-linux" + unstash "bin-win" + unstash "changelog" + sh(""" + curl -SfL https://github.com/github/hub/releases/download/v2.13.0/hub-linux-amd64-2.13.0.tgz | tar xzv --wildcards 'hub-*/bin/hub' --strip=2 + ./hub release create --draft --prerelease=${env.TAG_NAME !=~ /v[0-9\.]+/} \\ + -a docker-compose-Darwin-x86_64 \\ + -a docker-compose-Darwin-x86_64.sha256 \\ + -a docker-compose-Darwin-x86_64.tgz \\ + -a docker-compose-Darwin-x86_64.tgz.sha256 \\ + -a docker-compose-Linux-x86_64 \\ + -a docker-compose-Linux-x86_64.sha256 \\ + -a docker-compose-Windows-x86_64.exe \\ + -a docker-compose-Windows-x86_64.exe.sha256 \\ + -a ../script/run/run.sh \\ + -F CHANGELOG.md \${TAG_NAME} + """) + } + } + } + stage('Publishing Python packages') { + agent { + label 'linux && docker && ubuntu-2004' + } + environment { + PYPIRC = credentials('pypirc-docker-dsg-cibot') + } + steps { + checkout scm + sh """ + rm -rf build/ dist/ + pip3 install wheel + python3 setup.py sdist bdist_wheel + pip3 install twine + ~/.local/bin/twine upload --config-file ${PYPIRC} ./dist/docker-compose-*.tar.gz ./dist/docker_compose-*-py2.py3-none-any.whl + """ + } + } + } + } + } +} + + +def buildImage(baseImage) { + def scmvar = checkout(scm) + def imageName = "dockerbuildbot/compose:${baseImage}-${scmvar.GIT_COMMIT}" + image = docker.image(imageName) + + withDockerRegistry(credentialsId:'dockerbuildbot-index.docker.io') { + try { + image.pull() + } catch (Exception exc) { + ansiColor('xterm') { + sh """docker build -t ${imageName} \\ + --target build \\ + --build-arg DISTRO="${baseImage}" \\ + --build-arg GIT_COMMIT="${scmvar.GIT_COMMIT}" \\ + .\\ + """ + sh "docker push ${imageName}" + } + echo "${imageName}" + return imageName + } + } +} + +def runTests(dockerVersion, pythonVersion, baseImage) { + return { + stage("python=${pythonVersion} docker=${dockerVersion} ${baseImage}") { + node("linux && docker && ubuntu-2004") { + def scmvar = checkout(scm) + def imageName = "dockerbuildbot/compose:${baseImage}-${scmvar.GIT_COMMIT}" + def storageDriver = sh(script: "docker info -f \'{{.Driver}}\'", returnStdout: true).trim() + echo "Using local system's storage driver: ${storageDriver}" + withDockerRegistry(credentialsId:'dockerbuildbot-index.docker.io') { + sh """docker run \\ + -t \\ + --rm \\ + --privileged \\ + --volume="\$(pwd)/.git:/code/.git" \\ + --volume="/var/run/docker.sock:/var/run/docker.sock" \\ + -e "TAG=${imageName}" \\ + -e "STORAGE_DRIVER=${storageDriver}" \\ + -e "DOCKER_VERSIONS=${dockerVersion}" \\ + -e "BUILD_NUMBER=${env.BUILD_NUMBER}" \\ + -e "PY_TEST_VERSIONS=${pythonVersion}" \\ + --entrypoint="script/test/ci" \\ + ${imageName} \\ + --verbose + """ + } + } + } + } +} + +def buildRuntimeImage(baseImage) { + scmvar = checkout scm + def imageName = "docker/compose:${baseImage}-${env.BRANCH_NAME}" + ansiColor('xterm') { + sh """docker build -t ${imageName} \\ + --build-arg DISTRO="${baseImage}" \\ + --build-arg GIT_COMMIT="${scmvar.GIT_COMMIT.take(7)}" \\ + . + """ + } + sh "mkdir -p dist" + sh "docker save ${imageName} -o dist/docker-compose-${baseImage}.tar" + stash name: "compose-${baseImage}", includes: "dist/docker-compose-${baseImage}.tar" +} + +def pushRuntimeImage(baseImage) { + unstash "compose-${baseImage}" + sh "docker load -i dist/docker-compose-${baseImage}.tar" + withDockerRegistry(credentialsId: 'dockerhub-dockerdsgcibot') { + sh "docker push docker/compose:${baseImage}-${env.TAG_NAME}" + if (baseImage == "alpine" && env.TAG_NAME != null) { + sh "docker tag docker/compose:alpine-${env.TAG_NAME} docker/compose:${env.TAG_NAME}" + sh "docker push docker/compose:${env.TAG_NAME}" + } + } +} + +def checksum(filepath) { + if (isUnix()) { + sh "openssl sha256 -r -out ${filepath}.sha256 ${filepath}" + } else { + powershell "(Get-FileHash -Path ${filepath} -Algorithm SHA256 | % hash).ToLower() + ' *${filepath}' | Out-File -encoding ascii ${filepath}.sha256" + } +} diff --git a/SWARM.md b/SWARM.md new file mode 100644 index 00000000000..c6f378a9a34 --- /dev/null +++ b/SWARM.md @@ -0,0 +1 @@ +This file has moved to: https://docs.docker.com/compose/swarm/ diff --git a/bin/docker-compose b/bin/docker-compose new file mode 100755 index 00000000000..5976e1d4aa5 --- /dev/null +++ b/bin/docker-compose @@ -0,0 +1,3 @@ +#!/usr/bin/env python +from compose.cli.main import main +main() diff --git a/cmd/cmdtrace/cmd_span.go b/cmd/cmdtrace/cmd_span.go deleted file mode 100644 index cce3e5db3f3..00000000000 --- a/cmd/cmdtrace/cmd_span.go +++ /dev/null @@ -1,148 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package cmdtrace - -import ( - "context" - "errors" - "fmt" - "sort" - "strings" - "time" - - dockercli "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - flag "github.com/spf13/pflag" - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/codes" - "go.opentelemetry.io/otel/trace" - - commands "github.com/docker/compose/v5/cmd/compose" - "github.com/docker/compose/v5/internal/tracing" -) - -// Setup should be called as part of the command's PersistentPreRunE -// as soon as possible after initializing the dockerCli. -// -// It initializes the tracer for the CLI using both auto-detection -// from the Docker context metadata as well as standard OTEL_ env -// vars, creates a root span for the command, and wraps the actual -// command invocation to ensure the span is properly finalized and -// exported before exit. -func Setup(cmd *cobra.Command, dockerCli command.Cli, args []string) error { - tracingShutdown, err := tracing.InitTracing(dockerCli) - if err != nil { - return fmt.Errorf("initializing tracing: %w", err) - } - - ctx := cmd.Context() - ctx, cmdSpan := otel.Tracer("").Start( - ctx, - "cli/"+strings.Join(commandName(cmd), "-"), - ) - cmdSpan.SetAttributes( - attribute.StringSlice("cli.flags", getFlags(cmd.Flags())), - attribute.Bool("cli.isatty", dockerCli.In().IsTerminal()), - ) - - cmd.SetContext(ctx) - wrapRunE(cmd, cmdSpan, tracingShutdown) - return nil -} - -// wrapRunE injects a wrapper function around the command's actual RunE (or Run) -// method. This is necessary to capture the command result for reporting as well -// as flushing any spans before exit. -// -// Unfortunately, PersistentPostRun(E) can't be used for this purpose because it -// only runs if RunE does _not_ return an error, but this should run unconditionally. -func wrapRunE(c *cobra.Command, cmdSpan trace.Span, tracingShutdown tracing.ShutdownFunc) { - origRunE := c.RunE - if origRunE == nil { - origRun := c.Run - //nolint:unparam // wrapper function for RunE, always returns nil by design - origRunE = func(cmd *cobra.Command, args []string) error { - origRun(cmd, args) - return nil - } - c.Run = nil - } - - c.RunE = func(cmd *cobra.Command, args []string) error { - cmdErr := origRunE(cmd, args) - if cmdSpan != nil { - if cmdErr != nil && !errors.Is(cmdErr, context.Canceled) { - // default exit code is 1 if a more descriptive error - // wasn't returned - exitCode := 1 - var statusErr dockercli.StatusError - if errors.As(cmdErr, &statusErr) { - exitCode = statusErr.StatusCode - } - cmdSpan.SetStatus(codes.Error, "CLI command returned error") - cmdSpan.RecordError(cmdErr, trace.WithAttributes( - attribute.Int("exit_code", exitCode), - )) - - } else { - cmdSpan.SetStatus(codes.Ok, "") - } - cmdSpan.End() - } - if tracingShutdown != nil { - // use background for root context because the cmd's context might have - // been canceled already - ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) - defer cancel() - // TODO(milas): add an env var to enable logging from the - // OTel components for debugging purposes - _ = tracingShutdown(ctx) - } - return cmdErr - } -} - -// commandName returns the path components for a given command, -// in reverse alphabetical order for consistent usage metrics. -// -// The root Compose command and anything before (i.e. "docker") -// are not included. -// -// For example: -// - docker compose alpha watch -> [watch, alpha] -// - docker-compose up -> [up] -func commandName(cmd *cobra.Command) []string { - var name []string - for c := cmd; c != nil; c = c.Parent() { - if c.Name() == commands.PluginName { - break - } - name = append(name, c.Name()) - } - sort.Sort(sort.Reverse(sort.StringSlice(name))) - return name -} - -func getFlags(fs *flag.FlagSet) []string { - var result []string - fs.Visit(func(flag *flag.Flag) { - result = append(result, flag.Name) - }) - return result -} diff --git a/cmd/cmdtrace/cmd_span_test.go b/cmd/cmdtrace/cmd_span_test.go deleted file mode 100644 index 27becd3c363..00000000000 --- a/cmd/cmdtrace/cmd_span_test.go +++ /dev/null @@ -1,113 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package cmdtrace - -import ( - "reflect" - "testing" - - "github.com/spf13/cobra" - flag "github.com/spf13/pflag" - - commands "github.com/docker/compose/v5/cmd/compose" -) - -func TestGetFlags(t *testing.T) { - // Initialize flagSet with flags - fs := flag.NewFlagSet("up", flag.ContinueOnError) - var ( - detach string - timeout string - ) - fs.StringVar(&detach, "detach", "d", "") - fs.StringVar(&timeout, "timeout", "t", "") - _ = fs.Set("detach", "detach") - _ = fs.Set("timeout", "timeout") - - tests := []struct { - name string - input *flag.FlagSet - expected []string - }{ - { - name: "NoFlags", - input: flag.NewFlagSet("NoFlags", flag.ContinueOnError), - expected: nil, - }, - { - name: "Flags", - input: fs, - expected: []string{"detach", "timeout"}, - }, - } - - for _, test := range tests { - t.Run(test.name, func(t *testing.T) { - result := getFlags(test.input) - if !reflect.DeepEqual(result, test.expected) { - t.Errorf("Expected %v, but got %v", test.expected, result) - } - }) - } -} - -func TestCommandName(t *testing.T) { - tests := []struct { - name string - setupCmd func() *cobra.Command - want []string - }{ - { - name: "docker compose alpha watch -> [watch, alpha]", - setupCmd: func() *cobra.Command { - dockerCmd := &cobra.Command{Use: "docker"} - composeCmd := &cobra.Command{Use: commands.PluginName} - alphaCmd := &cobra.Command{Use: "alpha"} - watchCmd := &cobra.Command{Use: "watch"} - - dockerCmd.AddCommand(composeCmd) - composeCmd.AddCommand(alphaCmd) - alphaCmd.AddCommand(watchCmd) - - return watchCmd - }, - want: []string{"watch", "alpha"}, - }, - { - name: "docker-compose up -> [up]", - setupCmd: func() *cobra.Command { - dockerComposeCmd := &cobra.Command{Use: commands.PluginName} - upCmd := &cobra.Command{Use: "up"} - - dockerComposeCmd.AddCommand(upCmd) - - return upCmd - }, - want: []string{"up"}, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - cmd := tt.setupCmd() - got := commandName(cmd) - if !reflect.DeepEqual(got, tt.want) { - t.Errorf("commandName() = %v, want %v", got, tt.want) - } - }) - } -} diff --git a/cmd/compatibility/convert.go b/cmd/compatibility/convert.go deleted file mode 100644 index 78d9b8303c5..00000000000 --- a/cmd/compatibility/convert.go +++ /dev/null @@ -1,116 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compatibility - -import ( - "fmt" - "os" - "strings" - - "github.com/docker/compose/v5/cmd/compose" -) - -func getCompletionCommands() []string { - return []string{ - "__complete", - "__completeNoDesc", - } -} - -func getBoolFlags() []string { - return []string{ - "--debug", "-D", - "--verbose", - "--tls", - "--tlsverify", - } -} - -func getStringFlags() []string { - return []string{ - "--tlscacert", - "--tlscert", - "--tlskey", - "--host", "-H", - "--context", - "--log-level", - } -} - -// Convert transforms standalone docker-compose args into CLI plugin compliant ones -func Convert(args []string) []string { - var rootFlags []string - command := []string{compose.PluginName} - l := len(args) -ARGS: - for i := 0; i < l; i++ { - arg := args[i] - if contains(getCompletionCommands(), arg) { - command = append([]string{arg}, command...) - continue - } - if arg != "" && arg[0] != '-' { - command = append(command, args[i:]...) - break - } - - switch arg { - case "--verbose": - arg = "--debug" - case "-h": - // docker cli has deprecated -h to avoid ambiguity with -H, while docker-compose still support it - arg = "--help" - case "--version", "-v": - // redirect --version pseudo-command to actual command - arg = "version" - } - - if contains(getBoolFlags(), arg) { - rootFlags = append(rootFlags, arg) - continue - } - for _, flag := range getStringFlags() { - if arg == flag { - i++ - if i >= l { - fmt.Fprintf(os.Stderr, "flag needs an argument: '%s'\n", arg) - os.Exit(1) - } - rootFlags = append(rootFlags, arg, args[i]) - continue ARGS - } - if strings.HasPrefix(arg, flag) { - _, val, found := strings.Cut(arg, "=") - if found { - rootFlags = append(rootFlags, flag, val) - continue ARGS - } - } - } - command = append(command, arg) - } - return append(rootFlags, command...) -} - -func contains(array []string, needle string) bool { - for _, val := range array { - if val == needle { - return true - } - } - return false -} diff --git a/cmd/compatibility/convert_test.go b/cmd/compatibility/convert_test.go deleted file mode 100644 index ae01665e92a..00000000000 --- a/cmd/compatibility/convert_test.go +++ /dev/null @@ -1,132 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compatibility - -import ( - "errors" - "os" - "os/exec" - "testing" - - "gotest.tools/v3/assert" -) - -func Test_convert(t *testing.T) { - tests := []struct { - name string - args []string - want []string - wantErr bool - }{ - { - name: "compose only", - args: []string{"up"}, - want: []string{"compose", "up"}, - }, - { - name: "with context", - args: []string{"--context", "foo", "-f", "compose.yaml", "up"}, - want: []string{"--context", "foo", "compose", "-f", "compose.yaml", "up"}, - }, - { - name: "with context arg", - args: []string{"--context=foo", "-f", "compose.yaml", "up"}, - want: []string{"--context", "foo", "compose", "-f", "compose.yaml", "up"}, - }, - { - name: "with host", - args: []string{"--host", "tcp://1.2.3.4", "up"}, - want: []string{"--host", "tcp://1.2.3.4", "compose", "up"}, - }, - { - name: "compose --verbose", - args: []string{"--verbose"}, - want: []string{"--debug", "compose"}, - }, - { - name: "compose --version", - args: []string{"--version"}, - want: []string{"compose", "version"}, - }, - { - name: "compose -v", - args: []string{"-v"}, - want: []string{"compose", "version"}, - }, - { - name: "help", - args: []string{"-h"}, - want: []string{"compose", "--help"}, - }, - { - name: "issues/1962", - args: []string{"psql", "-h", "postgres"}, - want: []string{"compose", "psql", "-h", "postgres"}, // -h should not be converted to --help - }, - { - name: "issues/8648", - args: []string{"exec", "mongo", "mongo", "--host", "mongo"}, - want: []string{"compose", "exec", "mongo", "mongo", "--host", "mongo"}, // --host is passed to exec - }, - { - name: "issues/12", - args: []string{"--log-level", "INFO", "up"}, - want: []string{"--log-level", "INFO", "compose", "up"}, - }, - { - name: "empty string argument", - args: []string{"--project-directory", "", "ps"}, - want: []string{"compose", "--project-directory", "", "ps"}, - }, - { - name: "compose as project name", - args: []string{"--project-name", "compose", "down", "--remove-orphans"}, - want: []string{"compose", "--project-name", "compose", "down", "--remove-orphans"}, - }, - { - name: "completion command", - args: []string{"__complete", "up"}, - want: []string{"__complete", "compose", "up"}, - }, - { - name: "string flag without argument", - args: []string{"--log-level"}, - wantErr: true, - }, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - if tt.wantErr { - if os.Getenv("BE_CRASHER") == "1" { - Convert(tt.args) - return - } - cmd := exec.Command(os.Args[0], "-test.run=^"+t.Name()+"$") - cmd.Env = append(os.Environ(), "BE_CRASHER=1") - err := cmd.Run() - var e *exec.ExitError - if errors.As(err, &e) && !e.Success() { - return - } - t.Fatalf("process ran with err %v, want exit status 1", err) - } else { - got := Convert(tt.args) - assert.DeepEqual(t, tt.want, got) - } - }) - } -} diff --git a/cmd/compose/alpha.go b/cmd/compose/alpha.go deleted file mode 100644 index 8acc969ca30..00000000000 --- a/cmd/compose/alpha.go +++ /dev/null @@ -1,37 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" -) - -// alphaCommand groups all experimental subcommands -func alphaCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - cmd := &cobra.Command{ - Short: "Experimental commands", - Use: "alpha [COMMAND]", - Hidden: true, - Annotations: map[string]string{ - "experimentalCLI": "true", - }, - } - cmd.AddCommand( - vizCommand(p, dockerCli, backendOptions), - publishCommand(p, dockerCli, backendOptions), - generateCommand(p, dockerCli, backendOptions), - ) - return cmd -} diff --git a/cmd/compose/attach.go b/cmd/compose/attach.go deleted file mode 100644 index 20f23fca20f..00000000000 --- a/cmd/compose/attach.go +++ /dev/null @@ -1,86 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type attachOpts struct { - *composeOptions - - service string - index int - - detachKeys string - noStdin bool - proxy bool -} - -func attachCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := attachOpts{ - composeOptions: &composeOptions{ - ProjectOptions: p, - }, - } - runCmd := &cobra.Command{ - Use: "attach [OPTIONS] SERVICE", - Short: "Attach local standard input, output, and error streams to a service's running container", - Args: cobra.MinimumNArgs(1), - PreRunE: Adapt(func(ctx context.Context, args []string) error { - opts.service = args[0] - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runAttach(ctx, dockerCli, backendOptions, opts) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - runCmd.Flags().IntVar(&opts.index, "index", 0, "index of the container if service has multiple replicas.") - runCmd.Flags().StringVarP(&opts.detachKeys, "detach-keys", "", "", "Override the key sequence for detaching from a container.") - - runCmd.Flags().BoolVar(&opts.noStdin, "no-stdin", false, "Do not attach STDIN") - runCmd.Flags().BoolVar(&opts.proxy, "sig-proxy", true, "Proxy all received signals to the process") - return runCmd -} - -func runAttach(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts attachOpts) error { - projectName, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - attachOpts := api.AttachOptions{ - Service: opts.service, - Index: opts.index, - DetachKeys: opts.detachKeys, - NoStdin: opts.noStdin, - Proxy: opts.proxy, - } - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Attach(ctx, projectName, attachOpts) -} diff --git a/cmd/compose/bridge.go b/cmd/compose/bridge.go deleted file mode 100644 index fcc780844bd..00000000000 --- a/cmd/compose/bridge.go +++ /dev/null @@ -1,155 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - - "github.com/distribution/reference" - "github.com/docker/cli/cli/command" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/pkg/stringid" - "github.com/docker/go-units" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/bridge" - "github.com/docker/compose/v5/pkg/compose" -) - -func bridgeCommand(p *ProjectOptions, dockerCli command.Cli) *cobra.Command { - cmd := &cobra.Command{ - Use: "bridge CMD [OPTIONS]", - Short: "Convert compose files into another model", - TraverseChildren: true, - } - cmd.AddCommand( - convertCommand(p, dockerCli), - transformersCommand(dockerCli), - ) - return cmd -} - -func convertCommand(p *ProjectOptions, dockerCli command.Cli) *cobra.Command { - convertOpts := bridge.ConvertOptions{} - cmd := &cobra.Command{ - Use: "convert", - Short: "Convert compose files to Kubernetes manifests, Helm charts, or another model", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runConvert(ctx, dockerCli, p, convertOpts) - }), - } - flags := cmd.Flags() - flags.StringVarP(&convertOpts.Output, "output", "o", "out", "The output directory for the Kubernetes resources") - flags.StringArrayVarP(&convertOpts.Transformations, "transformation", "t", nil, "Transformation to apply to compose model (default: docker/compose-bridge-kubernetes)") - flags.StringVar(&convertOpts.Templates, "templates", "", "Directory containing transformation templates") - return cmd -} - -func runConvert(ctx context.Context, dockerCli command.Cli, p *ProjectOptions, opts bridge.ConvertOptions) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, _, err := p.ToProject(ctx, dockerCli, backend, nil) - if err != nil { - return err - } - return bridge.Convert(ctx, dockerCli, project, opts) -} - -func transformersCommand(dockerCli command.Cli) *cobra.Command { - cmd := &cobra.Command{ - Use: "transformations CMD [OPTIONS]", - Short: "Manage transformation images", - } - cmd.AddCommand( - listTransformersCommand(dockerCli), - createTransformerCommand(dockerCli), - ) - return cmd -} - -func listTransformersCommand(dockerCli command.Cli) *cobra.Command { - options := lsOptions{} - cmd := &cobra.Command{ - Use: "list", - Aliases: []string{"ls"}, - Short: "List available transformations", - RunE: Adapt(func(ctx context.Context, args []string) error { - transformers, err := bridge.ListTransformers(ctx, dockerCli) - if err != nil { - return err - } - return displayTransformer(dockerCli, transformers, options) - }), - } - cmd.Flags().StringVar(&options.Format, "format", "table", "Format the output. Values: [table | json]") - cmd.Flags().BoolVarP(&options.Quiet, "quiet", "q", false, "Only display transformer names") - return cmd -} - -func displayTransformer(dockerCli command.Cli, transformers []image.Summary, options lsOptions) error { - if options.Quiet { - for _, t := range transformers { - if len(t.RepoTags) > 0 { - _, _ = fmt.Fprintln(dockerCli.Out(), t.RepoTags[0]) - } else { - _, _ = fmt.Fprintln(dockerCli.Out(), t.ID) - } - } - return nil - } - return formatter.Print(transformers, options.Format, dockerCli.Out(), - func(w io.Writer) { - for _, img := range transformers { - id := stringid.TruncateID(img.ID) - size := units.HumanSizeWithPrecision(float64(img.Size), 3) - repo, tag := "<none>", "<none>" - if len(img.RepoTags) > 0 { - ref, err := reference.ParseDockerRef(img.RepoTags[0]) - if err == nil { - // ParseDockerRef will reject a local image ID - repo = reference.FamiliarName(ref) - if tagged, ok := ref.(reference.Tagged); ok { - tag = tagged.Tag() - } - } - } - - _, _ = fmt.Fprintf(w, "%s\t%s\t%s\t%s\n", id, repo, tag, size) - } - }, - "IMAGE ID", "REPO", "TAGS", "SIZE") -} - -func createTransformerCommand(dockerCli command.Cli) *cobra.Command { - var opts bridge.CreateTransformerOptions - cmd := &cobra.Command{ - Use: "create [OPTION] PATH", - Short: "Create a new transformation", - RunE: Adapt(func(ctx context.Context, args []string) error { - opts.Dest = args[0] - return bridge.CreateTransformer(ctx, dockerCli, opts) - }), - } - cmd.Flags().StringVarP(&opts.From, "from", "f", "", "Existing transformation to copy (default: docker/compose-bridge-kubernetes)") - return cmd -} diff --git a/cmd/compose/build.go b/cmd/compose/build.go deleted file mode 100644 index 996cf4d9ef0..00000000000 --- a/cmd/compose/build.go +++ /dev/null @@ -1,178 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - "strings" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - cliopts "github.com/docker/cli/opts" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/display" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type buildOptions struct { - *ProjectOptions - quiet bool - pull bool - push bool - args []string - noCache bool - memory cliopts.MemBytes - ssh string - builder string - deps bool - print bool - check bool - sbom string - provenance string -} - -func (opts buildOptions) toAPIBuildOptions(services []string) (api.BuildOptions, error) { - var SSHKeys []types.SSHKey - if opts.ssh != "" { - id, path, found := strings.Cut(opts.ssh, "=") - if !found && id != "default" { - return api.BuildOptions{}, fmt.Errorf("invalid ssh key %q", opts.ssh) - } - SSHKeys = append(SSHKeys, types.SSHKey{ - ID: id, - Path: path, - }) - } - builderName := opts.builder - if builderName == "" { - builderName = os.Getenv("BUILDX_BUILDER") - } - - uiMode := display.Mode - if uiMode == display.ModeJSON { - uiMode = "rawjson" - } - - return api.BuildOptions{ - Pull: opts.pull, - Push: opts.push, - Progress: uiMode, - Args: types.NewMappingWithEquals(opts.args), - NoCache: opts.noCache, - Quiet: opts.quiet, - Services: services, - Deps: opts.deps, - Memory: int64(opts.memory), - Print: opts.print, - Check: opts.check, - SSHs: SSHKeys, - Builder: builderName, - SBOM: opts.sbom, - Provenance: opts.provenance, - }, nil -} - -func buildCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := buildOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "build [OPTIONS] [SERVICE...]", - Short: "Build or rebuild services", - PreRunE: Adapt(func(ctx context.Context, args []string) error { - if opts.quiet { - display.Mode = display.ModeQuiet - devnull, err := os.Open(os.DevNull) - if err != nil { - return err - } - os.Stdout = devnull - } - return nil - }), - RunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - if cmd.Flags().Changed("ssh") && opts.ssh == "" { - opts.ssh = "default" - } - if cmd.Flags().Changed("progress") && opts.ssh == "" { - fmt.Fprint(os.Stderr, "--progress is a global compose flag, better use `docker compose --progress xx build ...\n") - } - return runBuild(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.BoolVar(&opts.push, "push", false, "Push service images") - flags.BoolVarP(&opts.quiet, "quiet", "q", false, "Suppress the build output") - flags.BoolVar(&opts.pull, "pull", false, "Always attempt to pull a newer version of the image") - flags.StringArrayVar(&opts.args, "build-arg", []string{}, "Set build-time variables for services") - flags.StringVar(&opts.ssh, "ssh", "", "Set SSH authentications used when building service images. (use 'default' for using your default SSH Agent)") - flags.StringVar(&opts.builder, "builder", "", "Set builder to use") - flags.BoolVar(&opts.deps, "with-dependencies", false, "Also build dependencies (transitively)") - flags.StringVar(&opts.provenance, "provenance", "", `Add a provenance attestation`) - flags.StringVar(&opts.sbom, "sbom", "", `Add a SBOM attestation`) - - flags.Bool("parallel", true, "Build images in parallel. DEPRECATED") - flags.MarkHidden("parallel") //nolint:errcheck - flags.Bool("compress", true, "Compress the build context using gzip. DEPRECATED") - flags.MarkHidden("compress") //nolint:errcheck - flags.Bool("force-rm", true, "Always remove intermediate containers. DEPRECATED") - flags.MarkHidden("force-rm") //nolint:errcheck - flags.BoolVar(&opts.noCache, "no-cache", false, "Do not use cache when building the image") - flags.Bool("no-rm", false, "Do not remove intermediate containers after a successful build. DEPRECATED") - flags.MarkHidden("no-rm") //nolint:errcheck - flags.VarP(&opts.memory, "memory", "m", "Set memory limit for the build container. Not supported by BuildKit.") - flags.StringVar(&p.Progress, "progress", "", fmt.Sprintf(`Set type of ui output (%s)`, strings.Join(printerModes, ", "))) - flags.MarkHidden("progress") //nolint:errcheck - flags.BoolVar(&opts.print, "print", false, "Print equivalent bake file") - flags.BoolVar(&opts.check, "check", false, "Check build configuration") - - return cmd -} - -func runBuild(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts buildOptions, services []string) error { - if opts.print { - backendOptions.Add(compose.WithEventProcessor(display.Quiet())) - } - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - opts.All = true // do not drop resources as build may involve some dependencies by additional_contexts - project, _, err := opts.ToProject(ctx, dockerCli, backend, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - if err := applyPlatforms(project, false); err != nil { - return err - } - - apiBuildOptions, err := opts.toAPIBuildOptions(services) - if err != nil { - return err - } - apiBuildOptions.Attestations = true - - return backend.Build(ctx, project, apiBuildOptions) -} diff --git a/cmd/compose/commit.go b/cmd/compose/commit.go deleted file mode 100644 index 730deb2e561..00000000000 --- a/cmd/compose/commit.go +++ /dev/null @@ -1,97 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/opts" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type commitOptions struct { - *ProjectOptions - - service string - reference string - - pause bool - comment string - author string - changes opts.ListOpts - - index int -} - -func commitCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - options := commitOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "commit [OPTIONS] SERVICE [REPOSITORY[:TAG]]", - Short: "Create a new image from a service container's changes", - Args: cobra.RangeArgs(1, 2), - PreRunE: Adapt(func(ctx context.Context, args []string) error { - options.service = args[0] - if len(args) > 1 { - options.reference = args[1] - } - - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runCommit(ctx, dockerCli, backendOptions, options) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - flags := cmd.Flags() - flags.IntVar(&options.index, "index", 0, "index of the container if service has multiple replicas.") - - flags.BoolVarP(&options.pause, "pause", "p", true, "Pause container during commit") - flags.StringVarP(&options.comment, "message", "m", "", "Commit message") - flags.StringVarP(&options.author, "author", "a", "", `Author (e.g., "John Hannibal Smith <hannibal@a-team.com>")`) - options.changes = opts.NewListOpts(nil) - flags.VarP(&options.changes, "change", "c", "Apply Dockerfile instruction to the created image") - - return cmd -} - -func runCommit(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, options commitOptions) error { - projectName, err := options.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Commit(ctx, projectName, api.CommitOptions{ - Service: options.service, - Reference: options.reference, - Pause: options.pause, - Comment: options.comment, - Author: options.author, - Changes: options.changes, - Index: options.index, - }) -} diff --git a/cmd/compose/completion.go b/cmd/compose/completion.go deleted file mode 100644 index 0d6c9fe4d40..00000000000 --- a/cmd/compose/completion.go +++ /dev/null @@ -1,119 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "sort" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -// validArgsFn defines a completion func to be returned to fetch completion options -type validArgsFn func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) - -func noCompletion() validArgsFn { - return func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - return []string{}, cobra.ShellCompDirectiveNoSpace - } -} - -func completeServiceNames(dockerCli command.Cli, p *ProjectOptions) validArgsFn { - return func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - p.Offline = true - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return nil, cobra.ShellCompDirectiveNoFileComp - } - - project, _, err := p.ToProject(cmd.Context(), dockerCli, backend, nil) - if err != nil { - return nil, cobra.ShellCompDirectiveNoFileComp - } - var values []string - serviceNames := append(project.ServiceNames(), project.DisabledServiceNames()...) - for _, s := range serviceNames { - if toComplete == "" || strings.HasPrefix(s, toComplete) { - values = append(values, s) - } - } - return values, cobra.ShellCompDirectiveNoFileComp - } -} - -func completeProjectNames(dockerCli command.Cli, backendOptions *BackendOptions) func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - return func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return nil, cobra.ShellCompDirectiveError - } - - list, err := backend.List(cmd.Context(), api.ListOptions{ - All: true, - }) - if err != nil { - return nil, cobra.ShellCompDirectiveError - } - var values []string - for _, stack := range list { - if strings.HasPrefix(stack.Name, toComplete) { - values = append(values, stack.Name) - } - } - return values, cobra.ShellCompDirectiveNoFileComp - } -} - -func completeProfileNames(dockerCli command.Cli, p *ProjectOptions) validArgsFn { - return func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - p.Offline = true - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return nil, cobra.ShellCompDirectiveNoFileComp - } - - project, _, err := p.ToProject(cmd.Context(), dockerCli, backend, nil) - if err != nil { - return nil, cobra.ShellCompDirectiveNoFileComp - } - - allProfileNames := project.AllServices().GetProfiles() - sort.Strings(allProfileNames) - - var values []string - for _, profileName := range allProfileNames { - if strings.HasPrefix(profileName, toComplete) { - values = append(values, profileName) - } - } - return values, cobra.ShellCompDirectiveNoFileComp - } -} - -func completeScaleArgs(cli command.Cli, p *ProjectOptions) cobra.CompletionFunc { - return func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - completions, directive := completeServiceNames(cli, p)(cmd, args, toComplete) - for i, completion := range completions { - completions[i] = completion + "=" - } - return completions, directive - } -} diff --git a/cmd/compose/compose.go b/cmd/compose/compose.go deleted file mode 100644 index 4adc460de19..00000000000 --- a/cmd/compose/compose.go +++ /dev/null @@ -1,724 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "encoding/json" - "errors" - "fmt" - "io" - "os" - "os/signal" - "path/filepath" - "strconv" - "strings" - "syscall" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/dotenv" - "github.com/compose-spec/compose-go/v2/loader" - composepaths "github.com/compose-spec/compose-go/v2/paths" - "github.com/compose-spec/compose-go/v2/types" - composegoutils "github.com/compose-spec/compose-go/v2/utils" - "github.com/docker/buildx/util/logutil" - dockercli "github.com/docker/cli/cli" - "github.com/docker/cli/cli-plugins/metadata" - "github.com/docker/cli/cli/command" - "github.com/docker/cli/pkg/kvfile" - "github.com/morikuni/aec" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/cmd/display" - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" - "github.com/docker/compose/v5/pkg/remote" - "github.com/docker/compose/v5/pkg/utils" -) - -const ( - // ComposeParallelLimit set the limit running concurrent operation on docker engine - ComposeParallelLimit = "COMPOSE_PARALLEL_LIMIT" - // ComposeProjectName define the project name to be used, instead of guessing from parent directory - ComposeProjectName = "COMPOSE_PROJECT_NAME" - // ComposeCompatibility try to mimic compose v1 as much as possible - ComposeCompatibility = api.ComposeCompatibility - // ComposeRemoveOrphans remove "orphaned" containers, i.e. containers tagged for current project but not declared as service - ComposeRemoveOrphans = "COMPOSE_REMOVE_ORPHANS" - // ComposeIgnoreOrphans ignore "orphaned" containers - ComposeIgnoreOrphans = "COMPOSE_IGNORE_ORPHANS" - // ComposeEnvFiles defines the env files to use if --env-file isn't used - ComposeEnvFiles = "COMPOSE_ENV_FILES" - // ComposeMenu defines if the navigation menu should be rendered. Can be also set via --menu - ComposeMenu = "COMPOSE_MENU" - // ComposeProgress defines type of progress output, if --progress isn't used - ComposeProgress = "COMPOSE_PROGRESS" -) - -// rawEnv load a dot env file using docker/cli key=value parser, without attempt to interpolate or evaluate values -func rawEnv(r io.Reader, filename string, vars map[string]string, lookup func(key string) (string, bool)) error { - lines, err := kvfile.ParseFromReader(r, lookup) - if err != nil { - return fmt.Errorf("failed to parse env_file %s: %w", filename, err) - } - for _, line := range lines { - key, value, _ := strings.Cut(line, "=") - vars[key] = value - } - return nil -} - -var stdioToStdout bool - -func init() { - // compose evaluates env file values for interpolation - // `raw` format allows to load env_file with the same parser used by docker run --env-file - dotenv.RegisterFormat("raw", rawEnv) - - if v, ok := os.LookupEnv("COMPOSE_STATUS_STDOUT"); ok { - stdioToStdout, _ = strconv.ParseBool(v) - } -} - -// Command defines a compose CLI command as a func with args -type Command func(context.Context, []string) error - -// CobraCommand defines a cobra command function -type CobraCommand func(context.Context, *cobra.Command, []string) error - -// AdaptCmd adapt a CobraCommand func to cobra library -func AdaptCmd(fn CobraCommand) func(cmd *cobra.Command, args []string) error { - return func(cmd *cobra.Command, args []string) error { - ctx, cancel := context.WithCancel(cmd.Context()) - - s := make(chan os.Signal, 1) - signal.Notify(s, syscall.SIGTERM, syscall.SIGINT) - go func() { - <-s - cancel() - signal.Stop(s) - close(s) - }() - - err := fn(ctx, cmd, args) - if api.IsErrCanceled(err) || errors.Is(ctx.Err(), context.Canceled) { - err = dockercli.StatusError{ - StatusCode: 130, - } - } - if display.Mode == display.ModeJSON { - err = makeJSONError(err) - } - return err - } -} - -// Adapt a Command func to cobra library -func Adapt(fn Command) func(cmd *cobra.Command, args []string) error { - return AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - return fn(ctx, args) - }) -} - -type ProjectOptions struct { - ProjectName string - Profiles []string - ConfigPaths []string - WorkDir string - ProjectDir string - EnvFiles []string - Compatibility bool - Progress string - Offline bool - All bool - insecureRegistries []string -} - -// ProjectFunc does stuff within a types.Project -type ProjectFunc func(ctx context.Context, project *types.Project) error - -// ProjectServicesFunc does stuff within a types.Project and a selection of services -type ProjectServicesFunc func(ctx context.Context, project *types.Project, services []string) error - -// WithProject creates a cobra run command from a ProjectFunc based on configured project options and selected services -func (o *ProjectOptions) WithProject(fn ProjectFunc, dockerCli command.Cli) func(cmd *cobra.Command, args []string) error { - return o.WithServices(dockerCli, func(ctx context.Context, project *types.Project, services []string) error { - return fn(ctx, project) - }) -} - -// WithServices creates a cobra run command from a ProjectFunc based on configured project options and selected services -func (o *ProjectOptions) WithServices(dockerCli command.Cli, fn ProjectServicesFunc) func(cmd *cobra.Command, args []string) error { - return Adapt(func(ctx context.Context, services []string) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, metrics, err := o.ToProject(ctx, dockerCli, backend, services, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - ctx = context.WithValue(ctx, tracing.MetricsKey{}, metrics) - - project, err = project.WithServicesEnvironmentResolved(true) - if err != nil { - return err - } - - return fn(ctx, project, services) - }) -} - -type jsonErrorData struct { - Error bool `json:"error,omitempty"` - Message string `json:"message,omitempty"` -} - -func errorAsJSON(message string) string { - errorMessage := &jsonErrorData{ - Error: true, - Message: message, - } - marshal, err := json.Marshal(errorMessage) - if err == nil { - return string(marshal) - } else { - return message - } -} - -func makeJSONError(err error) error { - if err == nil { - return nil - } - var statusErr dockercli.StatusError - if errors.As(err, &statusErr) { - return dockercli.StatusError{ - StatusCode: statusErr.StatusCode, - Status: errorAsJSON(statusErr.Status), - } - } - return fmt.Errorf("%s", errorAsJSON(err.Error())) -} - -func (o *ProjectOptions) addProjectFlags(f *pflag.FlagSet) { - f.StringArrayVar(&o.Profiles, "profile", []string{}, "Specify a profile to enable") - f.StringVarP(&o.ProjectName, "project-name", "p", "", "Project name") - f.StringArrayVarP(&o.ConfigPaths, "file", "f", []string{}, "Compose configuration files") - f.StringArrayVar(&o.insecureRegistries, "insecure-registry", []string{}, "Use insecure registry to pull Compose OCI artifacts. Doesn't apply to images") - _ = f.MarkHidden("insecure-registry") - f.StringArrayVar(&o.EnvFiles, "env-file", defaultStringArrayVar(ComposeEnvFiles), "Specify an alternate environment file") - f.StringVar(&o.ProjectDir, "project-directory", "", "Specify an alternate working directory\n(default: the path of the, first specified, Compose file)") - f.StringVar(&o.WorkDir, "workdir", "", "DEPRECATED! USE --project-directory INSTEAD.\nSpecify an alternate working directory\n(default: the path of the, first specified, Compose file)") - f.BoolVar(&o.Compatibility, "compatibility", false, "Run compose in backward compatibility mode") - f.StringVar(&o.Progress, "progress", os.Getenv(ComposeProgress), fmt.Sprintf(`Set type of progress output (%s)`, strings.Join(printerModes, ", "))) - f.BoolVar(&o.All, "all-resources", false, "Include all resources, even those not used by services") - _ = f.MarkHidden("workdir") -} - -// get default value for a command line flag that is set by a coma-separated value in environment variable -func defaultStringArrayVar(env string) []string { - return strings.FieldsFunc(os.Getenv(env), func(c rune) bool { - return c == ',' - }) -} - -func (o *ProjectOptions) projectOrName(ctx context.Context, dockerCli command.Cli, services ...string) (*types.Project, string, error) { - name := o.ProjectName - var project *types.Project - if len(o.ConfigPaths) > 0 || o.ProjectName == "" { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return nil, "", err - } - - p, _, err := o.ToProject(ctx, dockerCli, backend, services, cli.WithDiscardEnvFile, cli.WithoutEnvironmentResolution) - if err != nil { - envProjectName := os.Getenv(ComposeProjectName) - if envProjectName != "" { - return nil, envProjectName, nil - } - return nil, "", err - } - project = p - name = p.Name - } - return project, name, nil -} - -func (o *ProjectOptions) toProjectName(ctx context.Context, dockerCli command.Cli) (string, error) { - if o.ProjectName != "" { - return o.ProjectName, nil - } - - envProjectName := os.Getenv(ComposeProjectName) - if envProjectName != "" { - return envProjectName, nil - } - - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return "", err - } - - project, _, err := o.ToProject(ctx, dockerCli, backend, nil) - if err != nil { - return "", err - } - return project.Name, nil -} - -func (o *ProjectOptions) ToModel(ctx context.Context, dockerCli command.Cli, services []string, po ...cli.ProjectOptionsFn) (map[string]any, error) { - remotes := o.remoteLoaders(dockerCli) - for _, r := range remotes { - po = append(po, cli.WithResourceLoader(r)) - } - - options, err := o.toProjectOptions(po...) - if err != nil { - return nil, err - } - - if o.Compatibility || utils.StringToBool(options.Environment[ComposeCompatibility]) { - api.Separator = "_" - } - - return options.LoadModel(ctx) -} - -// ToProject loads a Compose project using the LoadProject API. -// Accepts optional cli.ProjectOptionsFn to control loader behavior. -func (o *ProjectOptions) ToProject(ctx context.Context, dockerCli command.Cli, backend api.Compose, services []string, po ...cli.ProjectOptionsFn) (*types.Project, tracing.Metrics, error) { - var metrics tracing.Metrics - remotes := o.remoteLoaders(dockerCli) - - // Setup metrics listener to collect project data - metricsListener := func(event string, metadata map[string]any) { - switch event { - case "extends": - metrics.CountExtends++ - case "include": - paths := metadata["path"].(types.StringList) - for _, path := range paths { - var isRemote bool - for _, r := range remotes { - if r.Accept(path) { - isRemote = true - break - } - } - if isRemote { - metrics.CountIncludesRemote++ - } else { - metrics.CountIncludesLocal++ - } - } - } - } - - loadOpts := api.ProjectLoadOptions{ - ProjectName: o.ProjectName, - ConfigPaths: o.ConfigPaths, - WorkingDir: o.ProjectDir, - EnvFiles: o.EnvFiles, - Profiles: o.Profiles, - Services: services, - Offline: o.Offline, - All: o.All, - Compatibility: o.Compatibility, - ProjectOptionsFns: po, - LoadListeners: []api.LoadListener{metricsListener}, - OCI: api.OCIOptions{ - InsecureRegistries: o.insecureRegistries, - }, - } - - project, err := backend.LoadProject(ctx, loadOpts) - if err != nil { - return nil, metrics, err - } - - return project, metrics, nil -} - -func (o *ProjectOptions) remoteLoaders(dockerCli command.Cli) []loader.ResourceLoader { - if o.Offline { - return nil - } - git := remote.NewGitRemoteLoader(dockerCli, o.Offline) - oci := remote.NewOCIRemoteLoader(dockerCli, o.Offline, api.OCIOptions{}) - return []loader.ResourceLoader{git, oci} -} - -func (o *ProjectOptions) toProjectOptions(po ...cli.ProjectOptionsFn) (*cli.ProjectOptions, error) { - opts := []cli.ProjectOptionsFn{ - cli.WithWorkingDirectory(o.ProjectDir), - // First apply os.Environment, always win - cli.WithOsEnv, - } - - if _, present := os.LookupEnv("PWD"); !present { - if pwd, err := os.Getwd(); err != nil { - return nil, err - } else { - opts = append(opts, cli.WithEnv([]string{"PWD=" + pwd})) - } - } - - opts = append(opts, - // Load PWD/.env if present and no explicit --env-file has been set - cli.WithEnvFiles(o.EnvFiles...), - // read dot env file to populate project environment - cli.WithDotEnv, - // get compose file path set by COMPOSE_FILE - cli.WithConfigFileEnv, - // if none was selected, get default compose.yaml file from current dir or parent folder - cli.WithDefaultConfigPath, - // .. and then, a project directory != PWD maybe has been set so let's load .env file - cli.WithEnvFiles(o.EnvFiles...), //nolint:gocritic // intentionally applying cli.WithEnvFiles twice. - cli.WithDotEnv, //nolint:gocritic // intentionally applying cli.WithDotEnv twice. - // eventually COMPOSE_PROFILES should have been set - cli.WithDefaultProfiles(o.Profiles...), - cli.WithName(o.ProjectName), - ) - - return cli.NewProjectOptions(o.ConfigPaths, append(po, opts...)...) -} - -// PluginName is the name of the plugin -const PluginName = "compose" - -// RunningAsStandalone detects when running as a standalone program -func RunningAsStandalone() bool { - return len(os.Args) < 2 || os.Args[1] != metadata.MetadataSubcommandName && os.Args[1] != PluginName -} - -type BackendOptions struct { - Options []compose.Option -} - -func (o *BackendOptions) Add(option compose.Option) { - o.Options = append(o.Options, option) -} - -// RootCommand returns the compose command with its child commands -func RootCommand(dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { //nolint:gocyclo - // filter out useless commandConn.CloseWrite warning message that can occur - // when using a remote context that is unreachable: "commandConn.CloseWrite: commandconn: failed to wait: signal: killed" - // https://github.com/docker/cli/blob/e1f24d3c93df6752d3c27c8d61d18260f141310c/cli/connhelper/commandconn/commandconn.go#L203-L215 - logrus.AddHook(logutil.NewFilter([]logrus.Level{ - logrus.WarnLevel, - }, - "commandConn.CloseWrite:", - "commandConn.CloseRead:", - )) - - opts := ProjectOptions{} - var ( - ansi string - noAnsi bool - verbose bool - version bool - parallel int - dryRun bool - ) - c := &cobra.Command{ - Short: "Docker Compose", - Long: "Define and run multi-container applications with Docker", - Use: PluginName, - TraverseChildren: true, - // By default (no Run/RunE in parent c) for typos in subcommands, cobra displays the help of parent c but exit(0) ! - RunE: func(cmd *cobra.Command, args []string) error { - if len(args) == 0 { - return cmd.Help() - } - if version { - return versionCommand(dockerCli).Execute() - } - _ = cmd.Help() - return dockercli.StatusError{ - StatusCode: 1, - Status: fmt.Sprintf("unknown docker command: %q", "compose "+args[0]), - } - }, - PersistentPreRunE: func(cmd *cobra.Command, args []string) error { - parent := cmd.Root() - if parent != nil { - parentPrerun := parent.PersistentPreRunE - if parentPrerun != nil { - err := parentPrerun(cmd, args) - if err != nil { - return err - } - } - } - - if verbose { - logrus.SetLevel(logrus.TraceLevel) - } - - err := setEnvWithDotEnv(opts, dockerCli) - if err != nil { - return err - } - if noAnsi { - if ansi != "auto" { - return errors.New(`cannot specify DEPRECATED "--no-ansi" and "--ansi". Please use only "--ansi"`) - } - ansi = "never" - fmt.Fprint(os.Stderr, "option '--no-ansi' is DEPRECATED ! Please use '--ansi' instead.\n") - } - if v, ok := os.LookupEnv("COMPOSE_ANSI"); ok && !cmd.Flags().Changed("ansi") { - ansi = v - } - formatter.SetANSIMode(dockerCli, ansi) - - if noColor, ok := os.LookupEnv("NO_COLOR"); ok && noColor != "" { - display.NoColor() - formatter.SetANSIMode(dockerCli, formatter.Never) - } - - switch ansi { - case "never": - display.Mode = display.ModePlain - case "always": - display.Mode = display.ModeTTY - } - - detached, _ := cmd.Flags().GetBool("detach") - var ep api.EventProcessor - switch opts.Progress { - case "", display.ModeAuto: - switch { - case ansi == "never": - display.Mode = display.ModePlain - ep = display.Plain(dockerCli.Err()) - case dockerCli.Out().IsTerminal(): - ep = display.Full(dockerCli.Err(), stdinfo(dockerCli), detached) - default: - ep = display.Plain(dockerCli.Err()) - } - case display.ModeTTY: - if ansi == "never" { - return fmt.Errorf("can't use --progress tty while ANSI support is disabled") - } - display.Mode = display.ModeTTY - ep = display.Full(dockerCli.Err(), stdinfo(dockerCli), detached) - - case display.ModePlain: - if ansi == "always" { - return fmt.Errorf("can't use --progress plain while ANSI support is forced") - } - display.Mode = display.ModePlain - ep = display.Plain(dockerCli.Err()) - case display.ModeQuiet, "none": - display.Mode = display.ModeQuiet - ep = display.Quiet() - case display.ModeJSON: - display.Mode = display.ModeJSON - logrus.SetFormatter(&logrus.JSONFormatter{}) - ep = display.JSON(dockerCli.Err()) - default: - return fmt.Errorf("unsupported --progress value %q", opts.Progress) - } - backendOptions.Add(compose.WithEventProcessor(ep)) - - // (4) options validation / normalization - if opts.WorkDir != "" { - if opts.ProjectDir != "" { - return errors.New(`cannot specify DEPRECATED "--workdir" and "--project-directory". Please use only "--project-directory" instead`) - } - opts.ProjectDir = opts.WorkDir - fmt.Fprint(os.Stderr, aec.Apply("option '--workdir' is DEPRECATED at root level! Please use '--project-directory' instead.\n", aec.RedF)) - } - for i, file := range opts.EnvFiles { - file = composepaths.ExpandUser(file) - if !filepath.IsAbs(file) { - file, err := filepath.Abs(file) - if err != nil { - return err - } - opts.EnvFiles[i] = file - } else { - opts.EnvFiles[i] = file - } - } - - composeCmd := cmd - for composeCmd.Name() != PluginName { - if !composeCmd.HasParent() { - return fmt.Errorf("error parsing command line, expected %q", PluginName) - } - composeCmd = composeCmd.Parent() - } - - if v, ok := os.LookupEnv(ComposeParallelLimit); ok && !composeCmd.Flags().Changed("parallel") { - i, err := strconv.Atoi(v) - if err != nil { - return fmt.Errorf("%s must be an integer (found: %q)", ComposeParallelLimit, v) - } - parallel = i - } - if parallel > 0 { - logrus.Debugf("Limiting max concurrency to %d jobs", parallel) - backendOptions.Add(compose.WithMaxConcurrency(parallel)) - } - - // dry run detection - if dryRun { - backendOptions.Add(compose.WithDryRun) - } - return nil - }, - } - - c.AddCommand( - upCommand(&opts, dockerCli, backendOptions), - downCommand(&opts, dockerCli, backendOptions), - startCommand(&opts, dockerCli, backendOptions), - restartCommand(&opts, dockerCli, backendOptions), - stopCommand(&opts, dockerCli, backendOptions), - psCommand(&opts, dockerCli, backendOptions), - listCommand(dockerCli, backendOptions), - logsCommand(&opts, dockerCli, backendOptions), - configCommand(&opts, dockerCli), - killCommand(&opts, dockerCli, backendOptions), - runCommand(&opts, dockerCli, backendOptions), - removeCommand(&opts, dockerCli, backendOptions), - execCommand(&opts, dockerCli, backendOptions), - attachCommand(&opts, dockerCli, backendOptions), - exportCommand(&opts, dockerCli, backendOptions), - commitCommand(&opts, dockerCli, backendOptions), - pauseCommand(&opts, dockerCli, backendOptions), - unpauseCommand(&opts, dockerCli, backendOptions), - topCommand(&opts, dockerCli, backendOptions), - eventsCommand(&opts, dockerCli, backendOptions), - portCommand(&opts, dockerCli, backendOptions), - imagesCommand(&opts, dockerCli, backendOptions), - versionCommand(dockerCli), - buildCommand(&opts, dockerCli, backendOptions), - pushCommand(&opts, dockerCli, backendOptions), - pullCommand(&opts, dockerCli, backendOptions), - createCommand(&opts, dockerCli, backendOptions), - copyCommand(&opts, dockerCli, backendOptions), - waitCommand(&opts, dockerCli, backendOptions), - scaleCommand(&opts, dockerCli, backendOptions), - statsCommand(&opts, dockerCli), - watchCommand(&opts, dockerCli, backendOptions), - publishCommand(&opts, dockerCli, backendOptions), - alphaCommand(&opts, dockerCli, backendOptions), - bridgeCommand(&opts, dockerCli), - volumesCommand(&opts, dockerCli, backendOptions), - ) - - c.Flags().SetInterspersed(false) - opts.addProjectFlags(c.Flags()) - c.RegisterFlagCompletionFunc( //nolint:errcheck - "project-name", - completeProjectNames(dockerCli, backendOptions), - ) - c.RegisterFlagCompletionFunc( //nolint:errcheck - "project-directory", - func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - return []string{}, cobra.ShellCompDirectiveFilterDirs - }, - ) - c.RegisterFlagCompletionFunc( //nolint:errcheck - "file", - func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - return []string{"yaml", "yml"}, cobra.ShellCompDirectiveFilterFileExt - }, - ) - c.RegisterFlagCompletionFunc( //nolint:errcheck - "profile", - completeProfileNames(dockerCli, &opts), - ) - c.RegisterFlagCompletionFunc( //nolint:errcheck - "progress", - cobra.FixedCompletions(printerModes, cobra.ShellCompDirectiveNoFileComp), - ) - - c.Flags().StringVar(&ansi, "ansi", "auto", `Control when to print ANSI control characters ("never"|"always"|"auto")`) - c.Flags().IntVar(¶llel, "parallel", -1, `Control max parallelism, -1 for unlimited`) - c.Flags().BoolVarP(&version, "version", "v", false, "Show the Docker Compose version information") - c.PersistentFlags().BoolVar(&dryRun, "dry-run", false, "Execute command in dry run mode") - c.Flags().MarkHidden("version") //nolint:errcheck - c.Flags().BoolVar(&noAnsi, "no-ansi", false, `Do not print ANSI control characters (DEPRECATED)`) - c.Flags().MarkHidden("no-ansi") //nolint:errcheck - c.Flags().BoolVar(&verbose, "verbose", false, "Show more output") - c.Flags().MarkHidden("verbose") //nolint:errcheck - return c -} - -func stdinfo(dockerCli command.Cli) io.Writer { - if stdioToStdout { - return dockerCli.Out() - } - return dockerCli.Err() -} - -func setEnvWithDotEnv(opts ProjectOptions, dockerCli command.Cli) error { - // Check if we're using a remote config (OCI or Git) - // If so, skip env loading as remote loaders haven't been initialized yet - // and trying to process the path would fail - remoteLoaders := opts.remoteLoaders(dockerCli) - for _, path := range opts.ConfigPaths { - for _, loader := range remoteLoaders { - if loader.Accept(path) { - // Remote config - skip env loading for now - // It will be loaded later when the project is fully initialized - return nil - } - } - } - - options, err := cli.NewProjectOptions(opts.ConfigPaths, - cli.WithWorkingDirectory(opts.ProjectDir), - cli.WithOsEnv, - cli.WithEnvFiles(opts.EnvFiles...), - cli.WithDotEnv, - ) - if err != nil { - return err - } - envFromFile, err := dotenv.GetEnvFromFile(composegoutils.GetAsEqualsMap(os.Environ()), options.EnvFiles) - if err != nil { - return err - } - for k, v := range envFromFile { - if _, ok := os.LookupEnv(k); !ok && strings.HasPrefix(k, "COMPOSE_") { - if err := os.Setenv(k, v); err != nil { - return err - } - } - } - return nil -} - -var printerModes = []string{ - display.ModeAuto, - display.ModeTTY, - display.ModePlain, - display.ModeJSON, - display.ModeQuiet, -} diff --git a/cmd/compose/compose_oci_test.go b/cmd/compose/compose_oci_test.go deleted file mode 100644 index 7450f7633cf..00000000000 --- a/cmd/compose/compose_oci_test.go +++ /dev/null @@ -1,76 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/mocks" -) - -func TestSetEnvWithDotEnv_WithOCIArtifact(t *testing.T) { - // Test that setEnvWithDotEnv doesn't fail when using OCI artifacts - ctrl := gomock.NewController(t) - defer ctrl.Finish() - cli := mocks.NewMockCli(ctrl) - - opts := ProjectOptions{ - ConfigPaths: []string{"oci://docker.io/dockersamples/welcome-to-docker"}, - ProjectDir: "", - EnvFiles: []string{}, - } - - err := setEnvWithDotEnv(opts, cli) - assert.NilError(t, err, "setEnvWithDotEnv should not fail with OCI artifact path") -} - -func TestSetEnvWithDotEnv_WithGitRemote(t *testing.T) { - // Test that setEnvWithDotEnv doesn't fail when using Git remotes - ctrl := gomock.NewController(t) - defer ctrl.Finish() - cli := mocks.NewMockCli(ctrl) - - opts := ProjectOptions{ - ConfigPaths: []string{"https://github.com/docker/compose.git"}, - ProjectDir: "", - EnvFiles: []string{}, - } - - err := setEnvWithDotEnv(opts, cli) - assert.NilError(t, err, "setEnvWithDotEnv should not fail with Git remote path") -} - -func TestSetEnvWithDotEnv_WithLocalPath(t *testing.T) { - // Test that setEnvWithDotEnv still works with local paths - // This will fail if the file doesn't exist, but it should not panic - // or produce invalid paths - ctrl := gomock.NewController(t) - defer ctrl.Finish() - cli := mocks.NewMockCli(ctrl) - - opts := ProjectOptions{ - ConfigPaths: []string{"compose.yaml"}, - ProjectDir: "", - EnvFiles: []string{}, - } - - // This may error if files don't exist, but should not panic - _ = setEnvWithDotEnv(opts, cli) -} diff --git a/cmd/compose/compose_test.go b/cmd/compose/compose_test.go deleted file mode 100644 index 708929ff8cd..00000000000 --- a/cmd/compose/compose_test.go +++ /dev/null @@ -1,55 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "gotest.tools/v3/assert" -) - -func TestFilterServices(t *testing.T) { - p := &types.Project{ - Services: types.Services{ - "foo": { - Name: "foo", - Links: []string{"bar"}, - }, - "bar": { - Name: "bar", - DependsOn: map[string]types.ServiceDependency{ - "zot": {}, - }, - }, - "zot": { - Name: "zot", - }, - "qix": { - Name: "qix", - }, - }, - } - p, err := p.WithSelectedServices([]string{"bar"}) - assert.NilError(t, err) - - assert.Equal(t, len(p.Services), 2) - _, err = p.GetService("bar") - assert.NilError(t, err) - _, err = p.GetService("zot") - assert.NilError(t, err) -} diff --git a/cmd/compose/config.go b/cmd/compose/config.go deleted file mode 100644 index 4b5f1892730..00000000000 --- a/cmd/compose/config.go +++ /dev/null @@ -1,567 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "context" - "encoding/json" - "fmt" - "io" - "os" - "sort" - "strings" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/template" - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - "go.yaml.in/yaml/v4" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type configOptions struct { - *ProjectOptions - Format string - Output string - quiet bool - resolveImageDigests bool - noInterpolate bool - noNormalize bool - noResolvePath bool - noResolveEnv bool - services bool - volumes bool - networks bool - models bool - profiles bool - images bool - hash string - noConsistency bool - variables bool - environment bool - lockImageDigests bool -} - -func (o *configOptions) ToProject(ctx context.Context, dockerCli command.Cli, backend api.Compose, services []string) (*types.Project, error) { - project, _, err := o.ProjectOptions.ToProject(ctx, dockerCli, backend, services, o.toProjectOptionsFns()...) - return project, err -} - -func (o *configOptions) ToModel(ctx context.Context, dockerCli command.Cli, services []string, po ...cli.ProjectOptionsFn) (map[string]any, error) { - po = append(po, o.toProjectOptionsFns()...) - return o.ProjectOptions.ToModel(ctx, dockerCli, services, po...) -} - -// toProjectOptionsFns converts config options to cli.ProjectOptionsFn -func (o *configOptions) toProjectOptionsFns() []cli.ProjectOptionsFn { - fns := []cli.ProjectOptionsFn{ - cli.WithInterpolation(!o.noInterpolate), - cli.WithResolvedPaths(!o.noResolvePath), - cli.WithNormalization(!o.noNormalize), - cli.WithConsistency(!o.noConsistency), - cli.WithDefaultProfiles(o.Profiles...), - cli.WithDiscardEnvFile, - } - if o.noResolveEnv { - fns = append(fns, cli.WithoutEnvironmentResolution) - } - return fns -} - -func configCommand(p *ProjectOptions, dockerCli command.Cli) *cobra.Command { - opts := configOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "config [OPTIONS] [SERVICE...]", - Short: "Parse, resolve and render compose file in canonical format", - PreRunE: Adapt(func(ctx context.Context, args []string) error { - if opts.quiet { - devnull, err := os.Open(os.DevNull) - if err != nil { - return err - } - os.Stdout = devnull - } - if p.Compatibility { - opts.noNormalize = true - } - if opts.lockImageDigests { - opts.resolveImageDigests = true - } - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - if opts.services { - return runServices(ctx, dockerCli, opts) - } - if opts.volumes { - return runVolumes(ctx, dockerCli, opts) - } - if opts.networks { - return runNetworks(ctx, dockerCli, opts) - } - if opts.models { - return runModels(ctx, dockerCli, opts) - } - if opts.hash != "" { - return runHash(ctx, dockerCli, opts) - } - if opts.profiles { - return runProfiles(ctx, dockerCli, opts, args) - } - if opts.images { - return runConfigImages(ctx, dockerCli, opts, args) - } - if opts.variables { - return runVariables(ctx, dockerCli, opts, args) - } - if opts.environment { - return runEnvironment(ctx, dockerCli, opts, args) - } - - if opts.Format == "" { - opts.Format = "yaml" - } - return runConfig(ctx, dockerCli, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.StringVar(&opts.Format, "format", "", "Format the output. Values: [yaml | json]") - flags.BoolVar(&opts.resolveImageDigests, "resolve-image-digests", false, "Pin image tags to digests") - flags.BoolVar(&opts.lockImageDigests, "lock-image-digests", false, "Produces an override file with image digests") - flags.BoolVarP(&opts.quiet, "quiet", "q", false, "Only validate the configuration, don't print anything") - flags.BoolVar(&opts.noInterpolate, "no-interpolate", false, "Don't interpolate environment variables") - flags.BoolVar(&opts.noNormalize, "no-normalize", false, "Don't normalize compose model") - flags.BoolVar(&opts.noResolvePath, "no-path-resolution", false, "Don't resolve file paths") - flags.BoolVar(&opts.noConsistency, "no-consistency", false, "Don't check model consistency - warning: may produce invalid Compose output") - flags.BoolVar(&opts.noResolveEnv, "no-env-resolution", false, "Don't resolve service env files") - - flags.BoolVar(&opts.services, "services", false, "Print the service names, one per line.") - flags.BoolVar(&opts.volumes, "volumes", false, "Print the volume names, one per line.") - flags.BoolVar(&opts.networks, "networks", false, "Print the network names, one per line.") - flags.BoolVar(&opts.models, "models", false, "Print the model names, one per line.") - flags.BoolVar(&opts.profiles, "profiles", false, "Print the profile names, one per line.") - flags.BoolVar(&opts.images, "images", false, "Print the image names, one per line.") - flags.StringVar(&opts.hash, "hash", "", "Print the service config hash, one per line.") - flags.BoolVar(&opts.variables, "variables", false, "Print model variables and default values.") - flags.BoolVar(&opts.environment, "environment", false, "Print environment used for interpolation.") - flags.StringVarP(&opts.Output, "output", "o", "", "Save to file (default to stdout)") - - return cmd -} - -func runConfig(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) (err error) { - var content []byte - if opts.noInterpolate { - content, err = runConfigNoInterpolate(ctx, dockerCli, opts, services) - if err != nil { - return err - } - } else { - content, err = runConfigInterpolate(ctx, dockerCli, opts, services) - if err != nil { - return err - } - } - - if !opts.noInterpolate { - content = escapeDollarSign(content) - } - - if opts.quiet { - return nil - } - - if opts.Output != "" && len(content) > 0 { - return os.WriteFile(opts.Output, content, 0o666) - } - _, err = fmt.Fprint(dockerCli.Out(), string(content)) - return err -} - -func runConfigInterpolate(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) ([]byte, error) { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return nil, err - } - - project, err := opts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return nil, err - } - - if opts.resolveImageDigests { - project, err = project.WithImagesResolved(compose.ImageDigestResolver(ctx, dockerCli.ConfigFile(), dockerCli.Client())) - if err != nil { - return nil, err - } - } - - if !opts.noResolveEnv { - project, err = project.WithServicesEnvironmentResolved(true) - if err != nil { - return nil, err - } - } - - if !opts.noConsistency { - err := project.CheckContainerNameUnicity() - if err != nil { - return nil, err - } - } - - if opts.lockImageDigests { - project = imagesOnly(project) - } - - var content []byte - switch opts.Format { - case "json": - content, err = project.MarshalJSON() - case "yaml": - content, err = project.MarshalYAML() - default: - return nil, fmt.Errorf("unsupported format %q", opts.Format) - } - if err != nil { - return nil, err - } - return content, nil -} - -// imagesOnly return project with all attributes removed but service.images -func imagesOnly(project *types.Project) *types.Project { - digests := types.Services{} - for name, config := range project.Services { - digests[name] = types.ServiceConfig{ - Image: config.Image, - } - } - project = &types.Project{Services: digests} - return project -} - -func runConfigNoInterpolate(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) ([]byte, error) { - // we can't use ToProject, so the model we render here is only partially resolved - model, err := opts.ToModel(ctx, dockerCli, services) - if err != nil { - return nil, err - } - - if opts.resolveImageDigests { - err = resolveImageDigests(ctx, dockerCli, model) - if err != nil { - return nil, err - } - } - - if opts.lockImageDigests { - for key, e := range model { - if key != "services" { - delete(model, key) - } else { - for _, s := range e.(map[string]any) { - service := s.(map[string]any) - for key := range service { - if key != "image" { - delete(service, key) - } - } - } - } - } - } - - return formatModel(model, opts.Format) -} - -func resolveImageDigests(ctx context.Context, dockerCli command.Cli, model map[string]any) (err error) { - // create a pseudo-project so we can rely on WithImagesResolved to resolve images - p := &types.Project{ - Services: types.Services{}, - } - services := model["services"].(map[string]any) - for name, s := range services { - service := s.(map[string]any) - if image, ok := service["image"]; ok { - p.Services[name] = types.ServiceConfig{ - Image: image.(string), - } - } - } - - p, err = p.WithImagesResolved(compose.ImageDigestResolver(ctx, dockerCli.ConfigFile(), dockerCli.Client())) - if err != nil { - return err - } - - // Collect image resolved with digest and update model accordingly - for name, s := range services { - service := s.(map[string]any) - config := p.Services[name] - if config.Image != "" { - service["image"] = config.Image - } - services[name] = service - } - model["services"] = services - return nil -} - -func formatModel(model map[string]any, format string) (content []byte, err error) { - switch format { - case "json": - return json.MarshalIndent(model, "", " ") - case "yaml": - buf := bytes.NewBuffer([]byte{}) - encoder := yaml.NewEncoder(buf) - encoder.SetIndent(2) - err = encoder.Encode(model) - return buf.Bytes(), err - default: - return nil, fmt.Errorf("unsupported format %q", format) - } -} - -func runServices(ctx context.Context, dockerCli command.Cli, opts configOptions) error { - if opts.noInterpolate { - // we can't use ToProject, so the model we render here is only partially resolved - data, err := opts.ToModel(ctx, dockerCli, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - if _, ok := data["services"]; ok { - for serviceName := range data["services"].(map[string]any) { - _, _ = fmt.Fprintln(dockerCli.Out(), serviceName) - } - } - - return nil - } - - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, _, err := opts.ProjectOptions.ToProject(ctx, dockerCli, backend, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - err = project.ForEachService(project.ServiceNames(), func(serviceName string, _ *types.ServiceConfig) error { - _, _ = fmt.Fprintln(dockerCli.Out(), serviceName) - return nil - }) - - return err -} - -func runVolumes(ctx context.Context, dockerCli command.Cli, opts configOptions) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, _, err := opts.ProjectOptions.ToProject(ctx, dockerCli, backend, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - for n := range project.Volumes { - _, _ = fmt.Fprintln(dockerCli.Out(), n) - } - return nil -} - -func runNetworks(ctx context.Context, dockerCli command.Cli, opts configOptions) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, _, err := opts.ProjectOptions.ToProject(ctx, dockerCli, backend, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - for n := range project.Networks { - _, _ = fmt.Fprintln(dockerCli.Out(), n) - } - return nil -} - -func runModels(ctx context.Context, dockerCli command.Cli, opts configOptions) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, _, err := opts.ProjectOptions.ToProject(ctx, dockerCli, backend, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - for _, model := range project.Models { - if model.Model != "" { - _, _ = fmt.Fprintln(dockerCli.Out(), model.Model) - } - } - return nil -} - -func runHash(ctx context.Context, dockerCli command.Cli, opts configOptions) error { - var services []string - if opts.hash != "*" { - services = append(services, strings.Split(opts.hash, ",")...) - } - - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, _, err := opts.ProjectOptions.ToProject(ctx, dockerCli, backend, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - if err := applyPlatforms(project, true); err != nil { - return err - } - - if len(services) == 0 { - services = project.ServiceNames() - } - - sorted := services - sort.Slice(sorted, func(i, j int) bool { - return sorted[i] < sorted[j] - }) - - for _, name := range sorted { - s, err := project.GetService(name) - if err != nil { - return err - } - - hash, err := compose.ServiceHash(s) - if err != nil { - return err - } - _, _ = fmt.Fprintf(dockerCli.Out(), "%s %s\n", name, hash) - } - return nil -} - -func runProfiles(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) error { - set := map[string]struct{}{} - - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, err := opts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return err - } - for _, s := range project.AllServices() { - for _, p := range s.Profiles { - set[p] = struct{}{} - } - } - profiles := make([]string, 0, len(set)) - for p := range set { - profiles = append(profiles, p) - } - sort.Strings(profiles) - for _, p := range profiles { - _, _ = fmt.Fprintln(dockerCli.Out(), p) - } - return nil -} - -func runConfigImages(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, err := opts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return err - } - - for _, s := range project.Services { - _, _ = fmt.Fprintln(dockerCli.Out(), api.GetImageNameOrDefault(s, project.Name)) - } - return nil -} - -func runVariables(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) error { - opts.noInterpolate = true - model, err := opts.ToModel(ctx, dockerCli, services, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - variables := template.ExtractVariables(model, template.DefaultPattern) - - if opts.Format == "yaml" { - result, err := yaml.Marshal(variables) - if err != nil { - return err - } - fmt.Print(string(result)) - return nil - } - - return formatter.Print(variables, opts.Format, dockerCli.Out(), func(w io.Writer) { - for name, variable := range variables { - _, _ = fmt.Fprintf(w, "%s\t%t\t%s\t%s\n", name, variable.Required, variable.DefaultValue, variable.PresenceValue) - } - }, "NAME", "REQUIRED", "DEFAULT VALUE", "ALTERNATE VALUE") -} - -func runEnvironment(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, err := opts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return err - } - - for _, v := range project.Environment.Values() { - fmt.Println(v) - } - return nil -} - -func escapeDollarSign(marshal []byte) []byte { - dollar := []byte{'$'} - escDollar := []byte{'$', '$'} - return bytes.ReplaceAll(marshal, dollar, escDollar) -} diff --git a/cmd/compose/cp.go b/cmd/compose/cp.go deleted file mode 100644 index 17f7411309d..00000000000 --- a/cmd/compose/cp.go +++ /dev/null @@ -1,95 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type copyOptions struct { - *ProjectOptions - - source string - destination string - index int - all bool - followLink bool - copyUIDGID bool -} - -func copyCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := copyOptions{ - ProjectOptions: p, - } - copyCmd := &cobra.Command{ - Use: `cp [OPTIONS] SERVICE:SRC_PATH DEST_PATH|- - docker compose cp [OPTIONS] SRC_PATH|- SERVICE:DEST_PATH`, - Short: "Copy files/folders between a service container and the local filesystem", - Args: cli.ExactArgs(2), - PreRunE: Adapt(func(ctx context.Context, args []string) error { - if args[0] == "" { - return errors.New("source can not be empty") - } - if args[1] == "" { - return errors.New("destination can not be empty") - } - return nil - }), - RunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - opts.source = args[0] - opts.destination = args[1] - return runCopy(ctx, dockerCli, backendOptions, opts) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - flags := copyCmd.Flags() - flags.IntVar(&opts.index, "index", 0, "Index of the container if service has multiple replicas") - flags.BoolVar(&opts.all, "all", false, "Include containers created by the run command") - flags.BoolVarP(&opts.followLink, "follow-link", "L", false, "Always follow symbol link in SRC_PATH") - flags.BoolVarP(&opts.copyUIDGID, "archive", "a", false, "Archive mode (copy all uid/gid information)") - - return copyCmd -} - -func runCopy(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts copyOptions) error { - name, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Copy(ctx, name, api.CopyOptions{ - Source: opts.source, - Destination: opts.destination, - All: opts.all, - Index: opts.index, - FollowLink: opts.followLink, - CopyUIDGID: opts.copyUIDGID, - }) -} diff --git a/cmd/compose/create.go b/cmd/compose/create.go deleted file mode 100644 index 5f9f7908315..00000000000 --- a/cmd/compose/create.go +++ /dev/null @@ -1,223 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "slices" - "strconv" - "strings" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type createOptions struct { - Build bool - noBuild bool - Pull string - pullChanged bool - removeOrphans bool - ignoreOrphans bool - forceRecreate bool - noRecreate bool - recreateDeps bool - noInherit bool - timeChanged bool - timeout int - quietPull bool - scale []string - AssumeYes bool -} - -func createCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := createOptions{} - buildOpts := buildOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "create [OPTIONS] [SERVICE...]", - Short: "Creates containers for a service", - PreRunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - opts.pullChanged = cmd.Flags().Changed("pull") - if opts.Build && opts.noBuild { - return fmt.Errorf("--build and --no-build are incompatible") - } - if opts.forceRecreate && opts.noRecreate { - return fmt.Errorf("--force-recreate and --no-recreate are incompatible") - } - return nil - }), - RunE: p.WithServices(dockerCli, func(ctx context.Context, project *types.Project, services []string) error { - return runCreate(ctx, dockerCli, backendOptions, opts, buildOpts, project, services) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.BoolVar(&opts.Build, "build", false, "Build images before starting containers") - flags.BoolVar(&opts.noBuild, "no-build", false, "Don't build an image, even if it's policy") - flags.StringVar(&opts.Pull, "pull", "policy", `Pull image before running ("always"|"missing"|"never"|"build")`) - flags.BoolVar(&opts.quietPull, "quiet-pull", false, "Pull without printing progress information") - flags.BoolVar(&opts.forceRecreate, "force-recreate", false, "Recreate containers even if their configuration and image haven't changed") - flags.BoolVar(&opts.noRecreate, "no-recreate", false, "If containers already exist, don't recreate them. Incompatible with --force-recreate.") - flags.BoolVar(&opts.removeOrphans, "remove-orphans", false, "Remove containers for services not defined in the Compose file") - flags.StringArrayVar(&opts.scale, "scale", []string{}, "Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present.") - flags.BoolVarP(&opts.AssumeYes, "yes", "y", false, `Assume "yes" as answer to all prompts and run non-interactively`) - flags.SetNormalizeFunc(func(f *pflag.FlagSet, name string) pflag.NormalizedName { - // assumeYes was introduced by mistake as `--y` - if name == "y" { - logrus.Warn("--y is deprecated, please use --yes instead") - name = "yes" - } - return pflag.NormalizedName(name) - }) - return cmd -} - -func runCreate(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, createOpts createOptions, buildOpts buildOptions, project *types.Project, services []string) error { - if err := createOpts.Apply(project); err != nil { - return err - } - - var build *api.BuildOptions - if !createOpts.noBuild { - bo, err := buildOpts.toAPIBuildOptions(services) - if err != nil { - return err - } - build = &bo - } - - if createOpts.AssumeYes { - backendOptions.Options = append(backendOptions.Options, compose.WithPrompt(compose.AlwaysOkPrompt())) - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Create(ctx, project, api.CreateOptions{ - Build: build, - Services: services, - RemoveOrphans: createOpts.removeOrphans, - IgnoreOrphans: createOpts.ignoreOrphans, - Recreate: createOpts.recreateStrategy(), - RecreateDependencies: createOpts.dependenciesRecreateStrategy(), - Inherit: !createOpts.noInherit, - Timeout: createOpts.GetTimeout(), - QuietPull: createOpts.quietPull, - }) -} - -func (opts createOptions) recreateStrategy() string { - if opts.noRecreate { - return api.RecreateNever - } - if opts.forceRecreate { - return api.RecreateForce - } - if opts.noInherit { - return api.RecreateForce - } - return api.RecreateDiverged -} - -func (opts createOptions) dependenciesRecreateStrategy() string { - if opts.noRecreate { - return api.RecreateNever - } - if opts.recreateDeps { - return api.RecreateForce - } - return api.RecreateDiverged -} - -func (opts createOptions) GetTimeout() *time.Duration { - if opts.timeChanged { - t := time.Duration(opts.timeout) * time.Second - return &t - } - return nil -} - -func (opts createOptions) Apply(project *types.Project) error { - if opts.pullChanged { - if !opts.isPullPolicyValid() { - return fmt.Errorf("invalid --pull option %q", opts.Pull) - } - for i, service := range project.Services { - service.PullPolicy = opts.Pull - project.Services[i] = service - } - } - // N.B. opts.Build means "force build all", but images can still be built - // when this is false - // e.g. if a service has pull_policy: build or its local image is policy - if opts.Build { - for i, service := range project.Services { - if service.Build == nil { - continue - } - service.PullPolicy = types.PullPolicyBuild - project.Services[i] = service - } - } - - if err := applyPlatforms(project, true); err != nil { - return err - } - - err := applyScaleOpts(project, opts.scale) - if err != nil { - return err - } - return nil -} - -func applyScaleOpts(project *types.Project, opts []string) error { - for _, scale := range opts { - name, val, ok := strings.Cut(scale, "=") - if !ok || val == "" { - return fmt.Errorf("invalid --scale option %q. Should be SERVICE=NUM", scale) - } - replicas, err := strconv.Atoi(val) - if err != nil { - return err - } - err = setServiceScale(project, name, replicas) - if err != nil { - return err - } - } - return nil -} - -func (opts createOptions) isPullPolicyValid() bool { - pullPolicies := []string{ - types.PullPolicyAlways, types.PullPolicyNever, types.PullPolicyBuild, - types.PullPolicyMissing, types.PullPolicyIfNotPresent, - } - return slices.Contains(pullPolicies, opts.Pull) -} diff --git a/cmd/compose/down.go b/cmd/compose/down.go deleted file mode 100644 index d74c8175292..00000000000 --- a/cmd/compose/down.go +++ /dev/null @@ -1,104 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - "time" - - "github.com/docker/cli/cli/command" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" - "github.com/docker/compose/v5/pkg/utils" -) - -type downOptions struct { - *ProjectOptions - removeOrphans bool - timeChanged bool - timeout int - volumes bool - images string -} - -func downCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := downOptions{ - ProjectOptions: p, - } - downCmd := &cobra.Command{ - Use: "down [OPTIONS] [SERVICES]", - Short: "Stop and remove containers, networks", - PreRunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - opts.timeChanged = cmd.Flags().Changed("timeout") - if opts.images != "" { - if opts.images != "all" && opts.images != "local" { - return fmt.Errorf("invalid value for --rmi: %q", opts.images) - } - } - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runDown(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := downCmd.Flags() - removeOrphans := utils.StringToBool(os.Getenv(ComposeRemoveOrphans)) - flags.BoolVar(&opts.removeOrphans, "remove-orphans", removeOrphans, "Remove containers for services not defined in the Compose file") - flags.IntVarP(&opts.timeout, "timeout", "t", 0, "Specify a shutdown timeout in seconds") - flags.BoolVarP(&opts.volumes, "volumes", "v", false, `Remove named volumes declared in the "volumes" section of the Compose file and anonymous volumes attached to containers`) - flags.StringVar(&opts.images, "rmi", "", `Remove images used by services. "local" remove only images that don't have a custom tag ("local"|"all")`) - flags.SetNormalizeFunc(func(f *pflag.FlagSet, name string) pflag.NormalizedName { - if name == "volume" { - name = "volumes" - logrus.Warn("--volume is deprecated, please use --volumes") - } - return pflag.NormalizedName(name) - }) - return downCmd -} - -func runDown(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts downOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - var timeout *time.Duration - if opts.timeChanged { - timeoutValue := time.Duration(opts.timeout) * time.Second - timeout = &timeoutValue - } - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Down(ctx, name, api.DownOptions{ - RemoveOrphans: opts.removeOrphans, - Project: project, - Timeout: timeout, - Images: opts.images, - Volumes: opts.volumes, - Services: services, - }) -} diff --git a/cmd/compose/events.go b/cmd/compose/events.go deleted file mode 100644 index 7f8a4a77df4..00000000000 --- a/cmd/compose/events.go +++ /dev/null @@ -1,93 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "encoding/json" - "fmt" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type eventsOpts struct { - *composeOptions - json bool - since string - until string -} - -func eventsCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := eventsOpts{ - composeOptions: &composeOptions{ - ProjectOptions: p, - }, - } - cmd := &cobra.Command{ - Use: "events [OPTIONS] [SERVICE...]", - Short: "Receive real time events from containers", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runEvents(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - cmd.Flags().BoolVar(&opts.json, "json", false, "Output events as a stream of json objects") - cmd.Flags().StringVar(&opts.since, "since", "", "Show all events created since timestamp") - cmd.Flags().StringVar(&opts.until, "until", "", "Stream events until this timestamp") - return cmd -} - -func runEvents(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts eventsOpts, services []string) error { - name, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Events(ctx, name, api.EventsOptions{ - Services: services, - Since: opts.since, - Until: opts.until, - Consumer: func(event api.Event) error { - if opts.json { - marshal, err := json.Marshal(map[string]any{ - "time": event.Timestamp, - "type": "container", - "service": event.Service, - "id": event.Container, - "action": event.Status, - "attributes": event.Attributes, - }) - if err != nil { - return err - } - _, _ = fmt.Fprintln(dockerCli.Out(), string(marshal)) - } else { - _, _ = fmt.Fprintln(dockerCli.Out(), event) - } - return nil - }, - }) -} diff --git a/cmd/compose/exec.go b/cmd/compose/exec.go deleted file mode 100644 index f548730dc06..00000000000 --- a/cmd/compose/exec.go +++ /dev/null @@ -1,143 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "os" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type execOpts struct { - *composeOptions - - service string - command []string - environment []string - workingDir string - - noTty bool - user string - detach bool - index int - privileged bool - interactive bool -} - -func execCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := execOpts{ - composeOptions: &composeOptions{ - ProjectOptions: p, - }, - } - runCmd := &cobra.Command{ - Use: "exec [OPTIONS] SERVICE COMMAND [ARGS...]", - Short: "Execute a command in a running container", - Args: cobra.MinimumNArgs(2), - PreRunE: Adapt(func(ctx context.Context, args []string) error { - opts.service = args[0] - opts.command = args[1:] - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - err := runExec(ctx, dockerCli, backendOptions, opts) - if err != nil { - logrus.Debugf("%v", err) - var cliError cli.StatusError - if ok := errors.As(err, &cliError); ok { - os.Exit(err.(cli.StatusError).StatusCode) //nolint: errorlint - } - } - return err - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - runCmd.Flags().BoolVarP(&opts.detach, "detach", "d", false, "Detached mode: Run command in the background") - runCmd.Flags().StringArrayVarP(&opts.environment, "env", "e", []string{}, "Set environment variables") - runCmd.Flags().IntVar(&opts.index, "index", 0, "Index of the container if service has multiple replicas") - runCmd.Flags().BoolVarP(&opts.privileged, "privileged", "", false, "Give extended privileges to the process") - runCmd.Flags().StringVarP(&opts.user, "user", "u", "", "Run the command as this user") - runCmd.Flags().BoolVarP(&opts.noTty, "no-tty", "T", !dockerCli.Out().IsTerminal(), "Disable pseudo-TTY allocation. By default 'docker compose exec' allocates a TTY.") - runCmd.Flags().StringVarP(&opts.workingDir, "workdir", "w", "", "Path to workdir directory for this command") - - runCmd.Flags().BoolVarP(&opts.interactive, "interactive", "i", true, "Keep STDIN open even if not attached") - runCmd.Flags().MarkHidden("interactive") //nolint:errcheck - runCmd.Flags().BoolP("tty", "t", true, "Allocate a pseudo-TTY") - runCmd.Flags().MarkHidden("tty") //nolint:errcheck - - runCmd.Flags().SetInterspersed(false) - runCmd.Flags().SetNormalizeFunc(func(f *pflag.FlagSet, name string) pflag.NormalizedName { - if name == "no-TTY" { // legacy - name = "no-tty" - } - return pflag.NormalizedName(name) - }) - return runCmd -} - -func runExec(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts execOpts) error { - projectName, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - projectOptions, err := opts.composeOptions.toProjectOptions() //nolint:staticcheck - if err != nil { - return err - } - lookupFn := func(k string) (string, bool) { - v, ok := projectOptions.Environment[k] - return v, ok - } - execOpts := api.RunOptions{ - Service: opts.service, - Command: opts.command, - Environment: compose.ToMobyEnv(types.NewMappingWithEquals(opts.environment).Resolve(lookupFn)), - Tty: !opts.noTty, - User: opts.user, - Privileged: opts.privileged, - Index: opts.index, - Detach: opts.detach, - WorkingDir: opts.workingDir, - Interactive: opts.interactive, - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - exitCode, err := backend.Exec(ctx, projectName, execOpts) - if exitCode != 0 { - errMsg := fmt.Sprintf("exit status %d", exitCode) - if err != nil && err.Error() != "" { - errMsg = err.Error() - } - return cli.StatusError{StatusCode: exitCode, Status: errMsg} - } - return err -} diff --git a/cmd/compose/export.go b/cmd/compose/export.go deleted file mode 100644 index 4c7eaf7ef5d..00000000000 --- a/cmd/compose/export.go +++ /dev/null @@ -1,79 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type exportOptions struct { - *ProjectOptions - - service string - output string - index int -} - -func exportCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - options := exportOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "export [OPTIONS] SERVICE", - Short: "Export a service container's filesystem as a tar archive", - Args: cobra.MinimumNArgs(1), - PreRunE: Adapt(func(ctx context.Context, args []string) error { - options.service = args[0] - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runExport(ctx, dockerCli, backendOptions, options) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - flags := cmd.Flags() - flags.IntVar(&options.index, "index", 0, "index of the container if service has multiple replicas.") - flags.StringVarP(&options.output, "output", "o", "", "Write to a file, instead of STDOUT") - - return cmd -} - -func runExport(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, options exportOptions) error { - projectName, err := options.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - exportOptions := api.ExportOptions{ - Service: options.service, - Index: options.index, - Output: options.output, - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Export(ctx, projectName, exportOptions) -} diff --git a/cmd/compose/generate.go b/cmd/compose/generate.go deleted file mode 100644 index 3fe5b2389e5..00000000000 --- a/cmd/compose/generate.go +++ /dev/null @@ -1,91 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type generateOptions struct { - *ProjectOptions - Format string -} - -func generateCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := generateOptions{ - ProjectOptions: p, - } - - cmd := &cobra.Command{ - Use: "generate [OPTIONS] [CONTAINERS...]", - Short: "EXPERIMENTAL - Generate a Compose file from existing containers", - PreRunE: Adapt(func(ctx context.Context, args []string) error { - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runGenerate(ctx, dockerCli, backendOptions, opts, args) - }), - } - - cmd.Flags().StringVar(&opts.ProjectName, "name", "", "Project name to set in the Compose file") - cmd.Flags().StringVar(&opts.ProjectDir, "project-dir", "", "Directory to use for the project") - cmd.Flags().StringVar(&opts.Format, "format", "yaml", "Format the output. Values: [yaml | json]") - return cmd -} - -func runGenerate(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts generateOptions, containers []string) error { - _, _ = fmt.Fprintln(os.Stderr, "generate command is EXPERIMENTAL") - if len(containers) == 0 { - return fmt.Errorf("at least one container must be specified") - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - project, err := backend.Generate(ctx, api.GenerateOptions{ - Containers: containers, - ProjectName: opts.ProjectName, - }) - if err != nil { - return err - } - - var content []byte - switch opts.Format { - case "json": - content, err = project.MarshalJSON() - case "yaml": - content, err = project.MarshalYAML() - default: - return fmt.Errorf("unsupported format %q", opts.Format) - } - if err != nil { - return err - } - fmt.Println(string(content)) - - return nil -} diff --git a/cmd/compose/images.go b/cmd/compose/images.go deleted file mode 100644 index dbb7a56471d..00000000000 --- a/cmd/compose/images.go +++ /dev/null @@ -1,153 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - "maps" - "slices" - "strings" - "time" - - "github.com/containerd/platforms" - "github.com/docker/cli/cli/command" - "github.com/docker/docker/pkg/stringid" - "github.com/docker/go-units" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type imageOptions struct { - *ProjectOptions - Quiet bool - Format string -} - -func imagesCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := imageOptions{ - ProjectOptions: p, - } - imgCmd := &cobra.Command{ - Use: "images [OPTIONS] [SERVICE...]", - Short: "List images used by the created containers", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runImages(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - imgCmd.Flags().StringVar(&opts.Format, "format", "table", "Format the output. Values: [table | json]") - imgCmd.Flags().BoolVarP(&opts.Quiet, "quiet", "q", false, "Only display IDs") - return imgCmd -} - -func runImages(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts imageOptions, services []string) error { - projectName, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - images, err := backend.Images(ctx, projectName, api.ImagesOptions{ - Services: services, - }) - if err != nil { - return err - } - - if opts.Quiet { - ids := []string{} - for _, img := range images { - id := img.ID - if i := strings.IndexRune(img.ID, ':'); i >= 0 { - id = id[i+1:] - } - if !slices.Contains(ids, id) { - ids = append(ids, id) - } - } - for _, img := range ids { - _, _ = fmt.Fprintln(dockerCli.Out(), img) - } - return nil - } - if opts.Format == "json" { - - type img struct { - ID string `json:"ID"` - ContainerName string `json:"ContainerName"` - Repository string `json:"Repository"` - Tag string `json:"Tag"` - Platform string `json:"Platform"` - Size int64 `json:"Size"` - Created *time.Time `json:"Created,omitempty"` - LastTagTime time.Time `json:"LastTagTime,omitzero"` - } - // Convert map to slice - var imageList []img - for ctr, i := range images { - lastTagTime := i.LastTagTime - imageList = append(imageList, img{ - ContainerName: ctr, - ID: i.ID, - Repository: i.Repository, - Tag: i.Tag, - Platform: platforms.Format(i.Platform), - Size: i.Size, - Created: i.Created, - LastTagTime: lastTagTime, - }) - } - json, err := formatter.ToJSON(imageList, "", "") - if err != nil { - return err - } - _, err = fmt.Fprintln(dockerCli.Out(), json) - return err - } - - return formatter.Print(images, opts.Format, dockerCli.Out(), - func(w io.Writer) { - for _, container := range slices.Sorted(maps.Keys(images)) { - img := images[container] - id := stringid.TruncateID(img.ID) - size := units.HumanSizeWithPrecision(float64(img.Size), 3) - repo := img.Repository - if repo == "" { - repo = "<none>" - } - tag := img.Tag - if tag == "" { - tag = "<none>" - } - created := "N/A" - if img.Created != nil { - created = units.HumanDuration(time.Now().UTC().Sub(*img.Created)) + " ago" - } - _, _ = fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\t%s\n", - container, repo, tag, platforms.Format(img.Platform), id, size, created) - } - }, - "CONTAINER", "REPOSITORY", "TAG", "PLATFORM", "IMAGE ID", "SIZE", "CREATED") -} diff --git a/cmd/compose/kill.go b/cmd/compose/kill.go deleted file mode 100644 index ee488d2ec6f..00000000000 --- a/cmd/compose/kill.go +++ /dev/null @@ -1,81 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "os" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" - "github.com/docker/compose/v5/pkg/utils" -) - -type killOptions struct { - *ProjectOptions - removeOrphans bool - signal string -} - -func killCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := killOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "kill [OPTIONS] [SERVICE...]", - Short: "Force stop service containers", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runKill(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - flags := cmd.Flags() - removeOrphans := utils.StringToBool(os.Getenv(ComposeRemoveOrphans)) - flags.BoolVar(&opts.removeOrphans, "remove-orphans", removeOrphans, "Remove containers for services not defined in the Compose file") - flags.StringVarP(&opts.signal, "signal", "s", "SIGKILL", "SIGNAL to send to the container") - - return cmd -} - -func runKill(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts killOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - err = backend.Kill(ctx, name, api.KillOptions{ - RemoveOrphans: opts.removeOrphans, - Project: project, - Services: services, - Signal: opts.signal, - }) - if errors.Is(err, api.ErrNoResources) { - _, _ = fmt.Fprintln(stdinfo(dockerCli), "No container to kill") - return nil - } - return err -} diff --git a/cmd/compose/list.go b/cmd/compose/list.go deleted file mode 100644 index eaa843ac372..00000000000 --- a/cmd/compose/list.go +++ /dev/null @@ -1,122 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/opts" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type lsOptions struct { - Format string - Quiet bool - All bool - Filter opts.FilterOpt -} - -func listCommand(dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - lsOpts := lsOptions{Filter: opts.NewFilterOpt()} - lsCmd := &cobra.Command{ - Use: "ls [OPTIONS]", - Short: "List running compose projects", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runList(ctx, dockerCli, backendOptions, lsOpts) - }), - Args: cobra.NoArgs, - ValidArgsFunction: noCompletion(), - } - lsCmd.Flags().StringVar(&lsOpts.Format, "format", "table", "Format the output. Values: [table | json]") - lsCmd.Flags().BoolVarP(&lsOpts.Quiet, "quiet", "q", false, "Only display project names") - lsCmd.Flags().Var(&lsOpts.Filter, "filter", "Filter output based on conditions provided") - lsCmd.Flags().BoolVarP(&lsOpts.All, "all", "a", false, "Show all stopped Compose projects") - - return lsCmd -} - -var acceptedListFilters = map[string]bool{ - "name": true, -} - -func runList(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, lsOpts lsOptions) error { - filters := lsOpts.Filter.Value() - err := filters.Validate(acceptedListFilters) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - stackList, err := backend.List(ctx, api.ListOptions{All: lsOpts.All}) - if err != nil { - return err - } - - if filters.Len() > 0 { - var filtered []api.Stack - for _, s := range stackList { - if filters.Contains("name") && !filters.Match("name", s.Name) { - continue - } - filtered = append(filtered, s) - } - stackList = filtered - } - - if lsOpts.Quiet { - for _, s := range stackList { - _, _ = fmt.Fprintln(dockerCli.Out(), s.Name) - } - return nil - } - - view := viewFromStackList(stackList) - return formatter.Print(view, lsOpts.Format, dockerCli.Out(), func(w io.Writer) { - for _, stack := range view { - _, _ = fmt.Fprintf(w, "%s\t%s\t%s\n", stack.Name, stack.Status, stack.ConfigFiles) - } - }, "NAME", "STATUS", "CONFIG FILES") -} - -type stackView struct { - Name string - Status string - ConfigFiles string -} - -func viewFromStackList(stackList []api.Stack) []stackView { - retList := make([]stackView, len(stackList)) - for i, s := range stackList { - retList[i] = stackView{ - Name: s.Name, - Status: strings.TrimSpace(fmt.Sprintf("%s %s", s.Status, s.Reason)), - ConfigFiles: s.ConfigFiles, - } - } - return retList -} diff --git a/cmd/compose/logs.go b/cmd/compose/logs.go deleted file mode 100644 index de1452220b3..00000000000 --- a/cmd/compose/logs.go +++ /dev/null @@ -1,133 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type logsOptions struct { - *ProjectOptions - composeOptions - follow bool - index int - tail string - since string - until string - noColor bool - noPrefix bool - timestamps bool -} - -func logsCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := logsOptions{ - ProjectOptions: p, - } - logsCmd := &cobra.Command{ - Use: "logs [OPTIONS] [SERVICE...]", - Short: "View output from containers", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runLogs(ctx, dockerCli, backendOptions, opts, args) - }), - PreRunE: func(cmd *cobra.Command, args []string) error { - if opts.index > 0 && len(args) != 1 { - return errors.New("--index requires one service to be selected") - } - return nil - }, - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := logsCmd.Flags() - flags.BoolVarP(&opts.follow, "follow", "f", false, "Follow log output") - flags.IntVar(&opts.index, "index", 0, "index of the container if service has multiple replicas") - flags.StringVar(&opts.since, "since", "", "Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)") - flags.StringVar(&opts.until, "until", "", "Show logs before a timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)") - flags.BoolVar(&opts.noColor, "no-color", false, "Produce monochrome output") - flags.BoolVar(&opts.noPrefix, "no-log-prefix", false, "Don't print prefix in logs") - flags.BoolVarP(&opts.timestamps, "timestamps", "t", false, "Show timestamps") - flags.StringVarP(&opts.tail, "tail", "n", "all", "Number of lines to show from the end of the logs for each container") - return logsCmd -} - -func runLogs(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts logsOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - // exclude services configured to ignore output (attach: false), until explicitly selected - if project != nil && len(services) == 0 { - for n, service := range project.Services { - if service.Attach == nil || *service.Attach { - services = append(services, n) - } - } - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - consumer := formatter.NewLogConsumer(ctx, dockerCli.Out(), dockerCli.Err(), !opts.noColor, !opts.noPrefix, false) - return backend.Logs(ctx, name, consumer, api.LogOptions{ - Project: project, - Services: services, - Follow: opts.follow, - Index: opts.index, - Tail: opts.tail, - Since: opts.since, - Until: opts.until, - Timestamps: opts.timestamps, - }) -} - -var _ api.LogConsumer = &logConsumer{} - -type logConsumer struct { - events api.EventProcessor -} - -func (l logConsumer) Log(containerName, message string) { - l.events.On(api.Resource{ - ID: containerName, - Text: message, - }) -} - -func (l logConsumer) Err(containerName, message string) { - l.events.On(api.Resource{ - ID: containerName, - Status: api.Error, - Text: message, - }) -} - -func (l logConsumer) Status(containerName, message string) { - l.events.On(api.Resource{ - ID: containerName, - Status: api.Error, - Text: message, - }) -} diff --git a/cmd/compose/options.go b/cmd/compose/options.go deleted file mode 100644 index 6454f6aed9b..00000000000 --- a/cmd/compose/options.go +++ /dev/null @@ -1,292 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - "os" - "slices" - "sort" - "strings" - "text/tabwriter" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/template" - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - - "github.com/docker/compose/v5/cmd/display" - "github.com/docker/compose/v5/cmd/prompt" - "github.com/docker/compose/v5/internal/tracing" -) - -func applyPlatforms(project *types.Project, buildForSinglePlatform bool) error { - defaultPlatform := project.Environment["DOCKER_DEFAULT_PLATFORM"] - for name, service := range project.Services { - if service.Build == nil { - continue - } - - // default platform only applies if the service doesn't specify - if defaultPlatform != "" && service.Platform == "" { - if len(service.Build.Platforms) > 0 && !slices.Contains(service.Build.Platforms, defaultPlatform) { - return fmt.Errorf("service %q build.platforms does not support value set by DOCKER_DEFAULT_PLATFORM: %s", name, defaultPlatform) - } - service.Platform = defaultPlatform - } - - if service.Platform != "" { - if len(service.Build.Platforms) > 0 { - if !slices.Contains(service.Build.Platforms, service.Platform) { - return fmt.Errorf("service %q build configuration does not support platform: %s", name, service.Platform) - } - } - - if buildForSinglePlatform || len(service.Build.Platforms) == 0 { - // if we're building for a single platform, we want to build for the platform we'll use to run the image - // similarly, if no build platforms were explicitly specified, it makes sense to build for the platform - // the image is designed for rather than allowing the builder to infer the platform - service.Build.Platforms = []string{service.Platform} - } - } - - // services can specify that they should be built for multiple platforms, which can be used - // with `docker compose build` to produce a multi-arch image - // other cases, such as `up` and `run`, need a single architecture to actually run - // if there is only a single platform present (which might have been inferred - // from service.Platform above), it will be used, even if it requires emulation. - // if there's more than one platform, then the list is cleared so that the builder - // can decide. - // TODO(milas): there's no validation that the platform the builder will pick is actually one - // of the supported platforms from the build definition - // e.g. `build.platforms: [linux/arm64, linux/amd64]` on a `linux/ppc64` machine would build - // for `linux/ppc64` instead of returning an error that it's not a valid platform for the service. - if buildForSinglePlatform && len(service.Build.Platforms) > 1 { - // empty indicates that the builder gets to decide - service.Build.Platforms = nil - } - project.Services[name] = service - } - return nil -} - -// isRemoteConfig checks if the main compose file is from a remote source (OCI or Git) -func isRemoteConfig(dockerCli command.Cli, options buildOptions) bool { - if len(options.ConfigPaths) == 0 { - return false - } - remoteLoaders := options.remoteLoaders(dockerCli) - for _, loader := range remoteLoaders { - if loader.Accept(options.ConfigPaths[0]) { - return true - } - } - return false -} - -// checksForRemoteStack handles environment variable prompts for remote configurations -func checksForRemoteStack(ctx context.Context, dockerCli command.Cli, project *types.Project, options buildOptions, assumeYes bool, cmdEnvs []string) error { - if !isRemoteConfig(dockerCli, options) { - return nil - } - if metrics, ok := ctx.Value(tracing.MetricsKey{}).(tracing.Metrics); ok && metrics.CountIncludesRemote > 0 { - if err := confirmRemoteIncludes(dockerCli, options, assumeYes); err != nil { - return err - } - } - displayLocationRemoteStack(dockerCli, project, options) - return promptForInterpolatedVariables(ctx, dockerCli, options.ProjectOptions, assumeYes, cmdEnvs) -} - -// Prepare the values map and collect all variables info -type varInfo struct { - name string - value string - source string - required bool - defaultValue string -} - -// promptForInterpolatedVariables displays all variables and their values at once, -// then prompts for confirmation -func promptForInterpolatedVariables(ctx context.Context, dockerCli command.Cli, projectOptions *ProjectOptions, assumeYes bool, cmdEnvs []string) error { - if assumeYes { - return nil - } - - varsInfo, noVariables, err := extractInterpolationVariablesFromModel(ctx, dockerCli, projectOptions, cmdEnvs) - if err != nil { - return err - } - - if noVariables { - return nil - } - - displayInterpolationVariables(dockerCli.Out(), varsInfo) - - // Prompt for confirmation - userInput := prompt.NewPrompt(dockerCli.In(), dockerCli.Out()) - msg := "\nDo you want to proceed with these variables? [Y/n]: " - confirmed, err := userInput.Confirm(msg, true) - if err != nil { - return err - } - - if !confirmed { - return fmt.Errorf("operation cancelled by user") - } - - return nil -} - -func extractInterpolationVariablesFromModel(ctx context.Context, dockerCli command.Cli, projectOptions *ProjectOptions, cmdEnvs []string) ([]varInfo, bool, error) { - cmdEnvMap := extractEnvCLIDefined(cmdEnvs) - - // Create a model without interpolation to extract variables - opts := configOptions{ - noInterpolate: true, - ProjectOptions: projectOptions, - } - - model, err := opts.ToModel(ctx, dockerCli, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return nil, false, err - } - - // Extract variables that need interpolation - variables := template.ExtractVariables(model, template.DefaultPattern) - if len(variables) == 0 { - return nil, true, nil - } - - var varsInfo []varInfo - proposedValues := make(map[string]string) - - for name, variable := range variables { - info := varInfo{ - name: name, - required: variable.Required, - defaultValue: variable.DefaultValue, - } - - // Determine value and source based on priority - if value, exists := cmdEnvMap[name]; exists { - info.value = value - info.source = "command-line" - proposedValues[name] = value - } else if value, exists := os.LookupEnv(name); exists { - info.value = value - info.source = "environment" - proposedValues[name] = value - } else if variable.DefaultValue != "" { - info.value = variable.DefaultValue - info.source = "compose file" - proposedValues[name] = variable.DefaultValue - } else { - info.value = "<unset>" - info.source = "none" - } - - varsInfo = append(varsInfo, info) - } - return varsInfo, false, nil -} - -func extractEnvCLIDefined(cmdEnvs []string) map[string]string { - // Parse command-line environment variables - cmdEnvMap := make(map[string]string) - for _, env := range cmdEnvs { - key, val, ok := strings.Cut(env, "=") - if ok { - cmdEnvMap[key] = val - } - } - return cmdEnvMap -} - -func displayInterpolationVariables(writer io.Writer, varsInfo []varInfo) { - // Display all variables in a table format - _, _ = fmt.Fprintln(writer, "\nFound the following variables in configuration:") - - w := tabwriter.NewWriter(writer, 0, 0, 3, ' ', 0) - _, _ = fmt.Fprintln(w, "VARIABLE\tVALUE\tSOURCE\tREQUIRED\tDEFAULT") - sort.Slice(varsInfo, func(a, b int) bool { - return varsInfo[a].name < varsInfo[b].name - }) - for _, info := range varsInfo { - required := "no" - if info.required { - required = "yes" - } - _, _ = fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", - info.name, - info.value, - info.source, - required, - info.defaultValue, - ) - } - _ = w.Flush() -} - -func displayLocationRemoteStack(dockerCli command.Cli, project *types.Project, options buildOptions) { - mainComposeFile := options.ProjectOptions.ConfigPaths[0] //nolint:staticcheck - if display.Mode != display.ModeQuiet && display.Mode != display.ModeJSON { - _, _ = fmt.Fprintf(dockerCli.Out(), "Your compose stack %q is stored in %q\n", mainComposeFile, project.WorkingDir) - } -} - -func confirmRemoteIncludes(dockerCli command.Cli, options buildOptions, assumeYes bool) error { - if assumeYes { - return nil - } - - var remoteIncludes []string - remoteLoaders := options.ProjectOptions.remoteLoaders(dockerCli) //nolint:staticcheck - for _, cf := range options.ProjectOptions.ConfigPaths { //nolint:staticcheck - for _, loader := range remoteLoaders { - if loader.Accept(cf) { - remoteIncludes = append(remoteIncludes, cf) - break - } - } - } - - if len(remoteIncludes) == 0 { - return nil - } - - _, _ = fmt.Fprintln(dockerCli.Out(), "\nWarning: This Compose project includes files from remote sources:") - for _, include := range remoteIncludes { - _, _ = fmt.Fprintf(dockerCli.Out(), " - %s\n", include) - } - _, _ = fmt.Fprintln(dockerCli.Out(), "\nRemote includes could potentially be malicious. Make sure you trust the source.") - - msg := "Do you want to continue? [y/N]: " - confirmed, err := prompt.NewPrompt(dockerCli.In(), dockerCli.Out()).Confirm(msg, false) - if err != nil { - return err - } - if !confirmed { - return fmt.Errorf("operation cancelled by user") - } - - return nil -} diff --git a/cmd/compose/options_test.go b/cmd/compose/options_test.go deleted file mode 100644 index 50f82752130..00000000000 --- a/cmd/compose/options_test.go +++ /dev/null @@ -1,385 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "fmt" - "io" - "os" - "path/filepath" - "strings" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/streams" - "github.com/stretchr/testify/require" - "go.uber.org/mock/gomock" - - "github.com/docker/compose/v5/pkg/mocks" -) - -func TestApplyPlatforms_InferFromRuntime(t *testing.T) { - makeProject := func() *types.Project { - return &types.Project{ - Services: types.Services{ - "test": { - Name: "test", - Image: "foo", - Build: &types.BuildConfig{ - Context: ".", - Platforms: []string{ - "linux/amd64", - "linux/arm64", - "alice/32", - }, - }, - Platform: "alice/32", - }, - }, - } - } - - t.Run("SinglePlatform", func(t *testing.T) { - project := makeProject() - require.NoError(t, applyPlatforms(project, true)) - require.EqualValues(t, []string{"alice/32"}, project.Services["test"].Build.Platforms) - }) - - t.Run("MultiPlatform", func(t *testing.T) { - project := makeProject() - require.NoError(t, applyPlatforms(project, false)) - require.EqualValues(t, []string{"linux/amd64", "linux/arm64", "alice/32"}, - project.Services["test"].Build.Platforms) - }) -} - -func TestApplyPlatforms_DockerDefaultPlatform(t *testing.T) { - makeProject := func() *types.Project { - return &types.Project{ - Environment: map[string]string{ - "DOCKER_DEFAULT_PLATFORM": "linux/amd64", - }, - Services: types.Services{ - "test": { - Name: "test", - Image: "foo", - Build: &types.BuildConfig{ - Context: ".", - Platforms: []string{ - "linux/amd64", - "linux/arm64", - }, - }, - }, - }, - } - } - - t.Run("SinglePlatform", func(t *testing.T) { - project := makeProject() - require.NoError(t, applyPlatforms(project, true)) - require.EqualValues(t, []string{"linux/amd64"}, project.Services["test"].Build.Platforms) - }) - - t.Run("MultiPlatform", func(t *testing.T) { - project := makeProject() - require.NoError(t, applyPlatforms(project, false)) - require.EqualValues(t, []string{"linux/amd64", "linux/arm64"}, - project.Services["test"].Build.Platforms) - }) -} - -func TestApplyPlatforms_UnsupportedPlatform(t *testing.T) { - makeProject := func() *types.Project { - return &types.Project{ - Environment: map[string]string{ - "DOCKER_DEFAULT_PLATFORM": "commodore/64", - }, - Services: types.Services{ - "test": { - Name: "test", - Image: "foo", - Build: &types.BuildConfig{ - Context: ".", - Platforms: []string{ - "linux/amd64", - "linux/arm64", - }, - }, - }, - }, - } - } - - t.Run("SinglePlatform", func(t *testing.T) { - project := makeProject() - require.EqualError(t, applyPlatforms(project, true), - `service "test" build.platforms does not support value set by DOCKER_DEFAULT_PLATFORM: commodore/64`) - }) - - t.Run("MultiPlatform", func(t *testing.T) { - project := makeProject() - require.EqualError(t, applyPlatforms(project, false), - `service "test" build.platforms does not support value set by DOCKER_DEFAULT_PLATFORM: commodore/64`) - }) -} - -func TestIsRemoteConfig(t *testing.T) { - ctrl := gomock.NewController(t) - defer ctrl.Finish() - cli := mocks.NewMockCli(ctrl) - - tests := []struct { - name string - configPaths []string - want bool - }{ - { - name: "empty config paths", - configPaths: []string{}, - want: false, - }, - { - name: "local file", - configPaths: []string{"docker-compose.yaml"}, - want: false, - }, - { - name: "OCI reference", - configPaths: []string{"oci://registry.example.com/stack:latest"}, - want: true, - }, - { - name: "GIT reference", - configPaths: []string{"git://github.com/user/repo.git"}, - want: true, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - opts := buildOptions{ - ProjectOptions: &ProjectOptions{ - ConfigPaths: tt.configPaths, - }, - } - got := isRemoteConfig(cli, opts) - require.Equal(t, tt.want, got) - }) - } -} - -func TestDisplayLocationRemoteStack(t *testing.T) { - ctrl := gomock.NewController(t) - defer ctrl.Finish() - cli := mocks.NewMockCli(ctrl) - - buf := new(bytes.Buffer) - cli.EXPECT().Out().Return(streams.NewOut(buf)).AnyTimes() - - project := &types.Project{ - Name: "test-project", - WorkingDir: "/tmp/test", - } - - options := buildOptions{ - ProjectOptions: &ProjectOptions{ - ConfigPaths: []string{"oci://registry.example.com/stack:latest"}, - }, - } - - displayLocationRemoteStack(cli, project, options) - - output := buf.String() - require.Equal(t, output, fmt.Sprintf("Your compose stack %q is stored in %q\n", "oci://registry.example.com/stack:latest", "/tmp/test")) -} - -func TestDisplayInterpolationVariables(t *testing.T) { - ctrl := gomock.NewController(t) - defer ctrl.Finish() - - tmpDir := t.TempDir() - - // Create a temporary compose file - composeContent := ` -services: - app: - image: nginx - environment: - - TEST_VAR=${TEST_VAR:?required} # required with default - - API_KEY=${API_KEY:?} # required without default - - DEBUG=${DEBUG:-true} # optional with default - - UNSET_VAR # optional without default -` - composePath := filepath.Join(tmpDir, "docker-compose.yml") - require.NoError(t, os.WriteFile(composePath, []byte(composeContent), 0o644)) - - buf := new(bytes.Buffer) - cli := mocks.NewMockCli(ctrl) - cli.EXPECT().Out().Return(streams.NewOut(buf)).AnyTimes() - - // Create ProjectOptions with the temporary compose file - projectOptions := &ProjectOptions{ - ConfigPaths: []string{composePath}, - } - - // Set up the context with necessary environment variables - t.Setenv("TEST_VAR", "test-value") - t.Setenv("API_KEY", "123456") - - // Extract variables from the model - info, noVariables, err := extractInterpolationVariablesFromModel(t.Context(), cli, projectOptions, []string{}) - require.NoError(t, err) - require.False(t, noVariables) - - // Display the variables - displayInterpolationVariables(cli.Out(), info) - - // Expected output format with proper spacing - expected := "\nFound the following variables in configuration:\n" + - "VARIABLE VALUE SOURCE REQUIRED DEFAULT\n" + - "API_KEY 123456 environment yes \n" + - "DEBUG true compose file no true\n" + - "TEST_VAR test-value environment yes \n" - - // Normalize spaces and newlines for comparison - normalizeSpaces := func(s string) string { - // Replace multiple spaces with a single space - s = strings.Join(strings.Fields(strings.TrimSpace(s)), " ") - return s - } - - actualOutput := buf.String() - - // Compare normalized strings - require.Equal(t, - normalizeSpaces(expected), - normalizeSpaces(actualOutput), - "\nExpected:\n%s\nGot:\n%s", expected, actualOutput) -} - -func TestConfirmRemoteIncludes(t *testing.T) { - ctrl := gomock.NewController(t) - defer ctrl.Finish() - cli := mocks.NewMockCli(ctrl) - - tests := []struct { - name string - opts buildOptions - assumeYes bool - userInput string - wantErr bool - errMessage string - wantPrompt bool - wantOutput string - }{ - { - name: "no remote includes", - opts: buildOptions{ - ProjectOptions: &ProjectOptions{ - ConfigPaths: []string{ - "docker-compose.yaml", - "./local/path/compose.yaml", - }, - }, - }, - assumeYes: false, - wantErr: false, - wantPrompt: false, - }, - { - name: "assume yes with remote includes", - opts: buildOptions{ - ProjectOptions: &ProjectOptions{ - ConfigPaths: []string{ - "oci://registry.example.com/stack:latest", - "git://github.com/user/repo.git", - }, - }, - }, - assumeYes: true, - wantErr: false, - wantPrompt: false, - }, - { - name: "user confirms remote includes", - opts: buildOptions{ - ProjectOptions: &ProjectOptions{ - ConfigPaths: []string{ - "oci://registry.example.com/stack:latest", - "git://github.com/user/repo.git", - }, - }, - }, - assumeYes: false, - userInput: "y\n", - wantErr: false, - wantPrompt: true, - wantOutput: "\nWarning: This Compose project includes files from remote sources:\n" + - " - oci://registry.example.com/stack:latest\n" + - " - git://github.com/user/repo.git\n" + - "\nRemote includes could potentially be malicious. Make sure you trust the source.\n" + - "Do you want to continue? [y/N]: ", - }, - { - name: "user rejects remote includes", - opts: buildOptions{ - ProjectOptions: &ProjectOptions{ - ConfigPaths: []string{ - "oci://registry.example.com/stack:latest", - }, - }, - }, - assumeYes: false, - userInput: "n\n", - wantErr: true, - errMessage: "operation cancelled by user", - wantPrompt: true, - wantOutput: "\nWarning: This Compose project includes files from remote sources:\n" + - " - oci://registry.example.com/stack:latest\n" + - "\nRemote includes could potentially be malicious. Make sure you trust the source.\n" + - "Do you want to continue? [y/N]: ", - }, - } - - buf := new(bytes.Buffer) - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - cli.EXPECT().Out().Return(streams.NewOut(buf)).AnyTimes() - - if tt.wantPrompt { - inbuf := io.NopCloser(bytes.NewBufferString(tt.userInput)) - cli.EXPECT().In().Return(streams.NewIn(inbuf)).AnyTimes() - } - - err := confirmRemoteIncludes(cli, tt.opts, tt.assumeYes) - - if tt.wantErr { - require.Error(t, err) - require.Equal(t, tt.errMessage, err.Error()) - } else { - require.NoError(t, err) - } - - if tt.wantOutput != "" { - require.Equal(t, tt.wantOutput, buf.String()) - } - buf.Reset() - }) - } -} diff --git a/cmd/compose/pause.go b/cmd/compose/pause.go deleted file mode 100644 index bb4cedba2ad..00000000000 --- a/cmd/compose/pause.go +++ /dev/null @@ -1,97 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type pauseOptions struct { - *ProjectOptions -} - -func pauseCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := pauseOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "pause [SERVICE...]", - Short: "Pause services", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runPause(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - return cmd -} - -func runPause(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts pauseOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Pause(ctx, name, api.PauseOptions{ - Services: services, - Project: project, - }) -} - -type unpauseOptions struct { - *ProjectOptions -} - -func unpauseCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := unpauseOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "unpause [SERVICE...]", - Short: "Unpause services", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runUnPause(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - return cmd -} - -func runUnPause(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts unpauseOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.UnPause(ctx, name, api.PauseOptions{ - Services: services, - Project: project, - }) -} diff --git a/cmd/compose/port.go b/cmd/compose/port.go deleted file mode 100644 index 862e3b5d68d..00000000000 --- a/cmd/compose/port.go +++ /dev/null @@ -1,86 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "strconv" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type portOptions struct { - *ProjectOptions - port uint16 - protocol string - index int -} - -func portCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := portOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "port [OPTIONS] SERVICE PRIVATE_PORT", - Short: "Print the public port for a port binding", - Args: cobra.MinimumNArgs(2), - PreRunE: Adapt(func(ctx context.Context, args []string) error { - port, err := strconv.ParseUint(args[1], 10, 16) - if err != nil { - return err - } - opts.port = uint16(port) - opts.protocol = strings.ToLower(opts.protocol) - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runPort(ctx, dockerCli, backendOptions, opts, args[0]) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - cmd.Flags().StringVar(&opts.protocol, "protocol", "tcp", "tcp or udp") - cmd.Flags().IntVar(&opts.index, "index", 0, "Index of the container if service has multiple replicas") - return cmd -} - -func runPort(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts portOptions, service string) error { - projectName, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - ip, port, err := backend.Port(ctx, projectName, service, opts.port, api.PortOptions{ - Protocol: opts.protocol, - Index: opts.index, - }) - if err != nil { - return err - } - - _, _ = fmt.Fprintf(dockerCli.Out(), "%s:%d\n", ip, port) - return nil -} diff --git a/cmd/compose/ps.go b/cmd/compose/ps.go deleted file mode 100644 index 878bb3186e1..00000000000 --- a/cmd/compose/ps.go +++ /dev/null @@ -1,184 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "slices" - "sort" - "strings" - - "github.com/docker/cli/cli/command" - cliformatter "github.com/docker/cli/cli/command/formatter" - cliflags "github.com/docker/cli/cli/flags" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type psOptions struct { - *ProjectOptions - Format string - All bool - Quiet bool - Services bool - Filter string - Status []string - noTrunc bool - Orphans bool -} - -func (p *psOptions) parseFilter() error { - if p.Filter == "" { - return nil - } - key, val, ok := strings.Cut(p.Filter, "=") - if !ok { - return errors.New("arguments to --filter should be in form KEY=VAL") - } - switch key { - case "status": - p.Status = append(p.Status, val) - return nil - case "source": - return api.ErrNotImplemented - default: - return fmt.Errorf("unknown filter %s", key) - } -} - -func psCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := psOptions{ - ProjectOptions: p, - } - psCmd := &cobra.Command{ - Use: "ps [OPTIONS] [SERVICE...]", - Short: "List containers", - PreRunE: func(cmd *cobra.Command, args []string) error { - return opts.parseFilter() - }, - RunE: Adapt(func(ctx context.Context, args []string) error { - return runPs(ctx, dockerCli, backendOptions, args, opts) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := psCmd.Flags() - flags.StringVar(&opts.Format, "format", "table", cliflags.FormatHelp) - flags.StringVar(&opts.Filter, "filter", "", "Filter services by a property (supported filters: status)") - flags.StringArrayVar(&opts.Status, "status", []string{}, "Filter services by status. Values: [paused | restarting | removing | running | dead | created | exited]") - flags.BoolVarP(&opts.Quiet, "quiet", "q", false, "Only display IDs") - flags.BoolVar(&opts.Services, "services", false, "Display services") - flags.BoolVar(&opts.Orphans, "orphans", true, "Include orphaned services (not declared by project)") - flags.BoolVarP(&opts.All, "all", "a", false, "Show all stopped containers (including those created by the run command)") - flags.BoolVar(&opts.noTrunc, "no-trunc", false, "Don't truncate output") - return psCmd -} - -func runPs(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, services []string, opts psOptions) error { //nolint:gocyclo - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - if project != nil { - names := project.ServiceNames() - if len(services) > 0 { - for _, service := range services { - if !slices.Contains(names, service) { - return fmt.Errorf("no such service: %s", service) - } - } - } else if !opts.Orphans { - // until user asks to list orphaned services, we only include those declared in project - services = names - } - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - containers, err := backend.Ps(ctx, name, api.PsOptions{ - Project: project, - All: opts.All || len(opts.Status) != 0, - Services: services, - }) - if err != nil { - return err - } - - if len(opts.Status) != 0 { - containers = filterByStatus(containers, opts.Status) - } - - sort.Slice(containers, func(i, j int) bool { - return containers[i].Name < containers[j].Name - }) - - if opts.Quiet { - for _, c := range containers { - _, _ = fmt.Fprintln(dockerCli.Out(), c.ID) - } - return nil - } - - if opts.Services { - services := []string{} - for _, c := range containers { - s := c.Service - if !slices.Contains(services, s) { - services = append(services, s) - } - } - _, _ = fmt.Fprintln(dockerCli.Out(), strings.Join(services, "\n")) - return nil - } - - if opts.Format == "" { - opts.Format = dockerCli.ConfigFile().PsFormat - } - - containerCtx := cliformatter.Context{ - Output: dockerCli.Out(), - Format: formatter.NewContainerFormat(opts.Format, opts.Quiet, false), - Trunc: !opts.noTrunc, - } - return formatter.ContainerWrite(containerCtx, containers) -} - -func filterByStatus(containers []api.ContainerSummary, statuses []string) []api.ContainerSummary { - var filtered []api.ContainerSummary - for _, c := range containers { - if hasStatus(c, statuses) { - filtered = append(filtered, c) - } - } - return filtered -} - -func hasStatus(c api.ContainerSummary, statuses []string) bool { - for _, status := range statuses { - if c.State == status { - return true - } - } - return false -} diff --git a/cmd/compose/publish.go b/cmd/compose/publish.go deleted file mode 100644 index 6dba282ebc4..00000000000 --- a/cmd/compose/publish.go +++ /dev/null @@ -1,102 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type publishOptions struct { - *ProjectOptions - resolveImageDigests bool - ociVersion string - withEnvironment bool - assumeYes bool - app bool - insecureRegistry bool -} - -func publishCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := publishOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "publish [OPTIONS] REPOSITORY[:TAG]", - Short: "Publish compose application", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runPublish(ctx, dockerCli, backendOptions, opts, args[0]) - }), - Args: cli.ExactArgs(1), - } - flags := cmd.Flags() - flags.BoolVar(&opts.resolveImageDigests, "resolve-image-digests", false, "Pin image tags to digests") - flags.StringVar(&opts.ociVersion, "oci-version", "", "OCI image/artifact specification version (automatically determined by default)") - flags.BoolVar(&opts.withEnvironment, "with-env", false, "Include environment variables in the published OCI artifact") - flags.BoolVarP(&opts.assumeYes, "yes", "y", false, `Assume "yes" as answer to all prompts`) - flags.BoolVar(&opts.app, "app", false, "Published compose application (includes referenced images)") - flags.BoolVar(&opts.insecureRegistry, "insecure-registry", false, "Use insecure registry") - flags.SetNormalizeFunc(func(f *pflag.FlagSet, name string) pflag.NormalizedName { - // assumeYes was introduced by mistake as `--y` - if name == "y" { - logrus.Warn("--y is deprecated, please use --yes instead") - name = "yes" - } - return pflag.NormalizedName(name) - }) - // Should **only** be used for testing purpose, we don't want to promote use of insecure registries - _ = flags.MarkHidden("insecure-registry") - - return cmd -} - -func runPublish(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts publishOptions, repository string) error { - if opts.assumeYes { - backendOptions.Options = append(backendOptions.Options, compose.WithPrompt(compose.AlwaysOkPrompt())) - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - project, metrics, err := opts.ToProject(ctx, dockerCli, backend, nil) - if err != nil { - return err - } - - if metrics.CountIncludesLocal > 0 { - return errors.New("cannot publish compose file with local includes") - } - - return backend.Publish(ctx, project, repository, api.PublishOptions{ - ResolveImageDigests: opts.resolveImageDigests || opts.app, - Application: opts.app, - OCIVersion: api.OCIVersion(opts.ociVersion), - WithEnvironment: opts.withEnvironment, - InsecureRegistry: opts.insecureRegistry, - }) -} diff --git a/cmd/compose/pull.go b/cmd/compose/pull.go deleted file mode 100644 index 694731155f6..00000000000 --- a/cmd/compose/pull.go +++ /dev/null @@ -1,122 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - "github.com/morikuni/aec" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type pullOptions struct { - *ProjectOptions - composeOptions - quiet bool - parallel bool - noParallel bool - includeDeps bool - ignorePullFailures bool - noBuildable bool - policy string -} - -func pullCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := pullOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "pull [OPTIONS] [SERVICE...]", - Short: "Pull service images", - PreRunE: func(cmd *cobra.Command, args []string) error { - if cmd.Flags().Changed("no-parallel") { - fmt.Fprint(os.Stderr, aec.Apply("option '--no-parallel' is DEPRECATED and will be ignored.\n", aec.RedF)) - } - if cmd.Flags().Changed("parallel") { - fmt.Fprint(os.Stderr, aec.Apply("option '--parallel' is DEPRECATED and will be ignored.\n", aec.RedF)) - } - return nil - }, - RunE: Adapt(func(ctx context.Context, args []string) error { - return runPull(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.BoolVarP(&opts.quiet, "quiet", "q", false, "Pull without printing progress information") - cmd.Flags().BoolVar(&opts.includeDeps, "include-deps", false, "Also pull services declared as dependencies") - cmd.Flags().BoolVar(&opts.parallel, "parallel", true, "DEPRECATED pull multiple images in parallel") - flags.MarkHidden("parallel") //nolint:errcheck - cmd.Flags().BoolVar(&opts.noParallel, "no-parallel", true, "DEPRECATED disable parallel pulling") - flags.MarkHidden("no-parallel") //nolint:errcheck - cmd.Flags().BoolVar(&opts.ignorePullFailures, "ignore-pull-failures", false, "Pull what it can and ignores images with pull failures") - cmd.Flags().BoolVar(&opts.noBuildable, "ignore-buildable", false, "Ignore images that can be built") - cmd.Flags().StringVar(&opts.policy, "policy", "", `Apply pull policy ("missing"|"always")`) - return cmd -} - -func (opts pullOptions) apply(project *types.Project, services []string) (*types.Project, error) { - if !opts.includeDeps { - var err error - project, err = project.WithSelectedServices(services, types.IgnoreDependencies) - if err != nil { - return nil, err - } - } - - if opts.policy != "" { - for i, service := range project.Services { - if service.Image == "" { - continue - } - service.PullPolicy = opts.policy - project.Services[i] = service - } - } - return project, nil -} - -func runPull(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts pullOptions, services []string) error { - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - project, _, err := opts.ToProject(ctx, dockerCli, backend, services, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - project, err = opts.apply(project, services) - if err != nil { - return err - } - - return backend.Pull(ctx, project, api.PullOptions{ - Quiet: opts.quiet, - IgnoreFailures: opts.ignorePullFailures, - IgnoreBuildable: opts.noBuildable, - }) -} diff --git a/cmd/compose/pullOptions_test.go b/cmd/compose/pullOptions_test.go deleted file mode 100644 index 05dd868edf7..00000000000 --- a/cmd/compose/pullOptions_test.go +++ /dev/null @@ -1,57 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "gotest.tools/v3/assert" -) - -func TestApplyPullOptions(t *testing.T) { - project := &types.Project{ - Services: types.Services{ - "must-build": { - Name: "must-build", - // No image, local build only - Build: &types.BuildConfig{ - Context: ".", - }, - }, - "has-build": { - Name: "has-build", - Image: "registry.example.com/myservice", - Build: &types.BuildConfig{ - Context: ".", - }, - }, - "must-pull": { - Name: "must-pull", - Image: "registry.example.com/another-service", - }, - }, - } - project, err := pullOptions{ - policy: types.PullPolicyMissing, - }.apply(project, nil) - assert.NilError(t, err) - - assert.Equal(t, project.Services["must-build"].PullPolicy, "") // still default - assert.Equal(t, project.Services["has-build"].PullPolicy, types.PullPolicyMissing) - assert.Equal(t, project.Services["must-pull"].PullPolicy, types.PullPolicyMissing) -} diff --git a/cmd/compose/push.go b/cmd/compose/push.go deleted file mode 100644 index 4dd23aedb6f..00000000000 --- a/cmd/compose/push.go +++ /dev/null @@ -1,79 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type pushOptions struct { - *ProjectOptions - composeOptions - IncludeDeps bool - Ignorefailures bool - Quiet bool -} - -func pushCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := pushOptions{ - ProjectOptions: p, - } - pushCmd := &cobra.Command{ - Use: "push [OPTIONS] [SERVICE...]", - Short: "Push service images", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runPush(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - pushCmd.Flags().BoolVar(&opts.Ignorefailures, "ignore-push-failures", false, "Push what it can and ignores images with push failures") - pushCmd.Flags().BoolVar(&opts.IncludeDeps, "include-deps", false, "Also push images of services declared as dependencies") - pushCmd.Flags().BoolVarP(&opts.Quiet, "quiet", "q", false, "Push without printing progress information") - - return pushCmd -} - -func runPush(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts pushOptions, services []string) error { - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - project, _, err := opts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return err - } - - if !opts.IncludeDeps { - project, err = project.WithSelectedServices(services, types.IgnoreDependencies) - if err != nil { - return err - } - } - - return backend.Push(ctx, project, api.PushOptions{ - IgnoreFailures: opts.Ignorefailures, - Quiet: opts.Quiet, - }) -} diff --git a/cmd/compose/remove.go b/cmd/compose/remove.go deleted file mode 100644 index d0765b20a53..00000000000 --- a/cmd/compose/remove.go +++ /dev/null @@ -1,88 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type removeOptions struct { - *ProjectOptions - force bool - stop bool - volumes bool -} - -func removeCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := removeOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "rm [OPTIONS] [SERVICE...]", - Short: "Removes stopped service containers", - Long: `Removes stopped service containers - -By default, anonymous volumes attached to containers will not be removed. You -can override this with -v. To list all volumes, use "docker volume ls". - -Any data which is not in a volume will be lost.`, - RunE: Adapt(func(ctx context.Context, args []string) error { - return runRemove(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - f := cmd.Flags() - f.BoolVarP(&opts.force, "force", "f", false, "Don't ask to confirm removal") - f.BoolVarP(&opts.stop, "stop", "s", false, "Stop the containers, if required, before removing") - f.BoolVarP(&opts.volumes, "volumes", "v", false, "Remove any anonymous volumes attached to containers") - f.BoolP("all", "a", false, "Deprecated - no effect") - f.MarkHidden("all") //nolint:errcheck - - return cmd -} - -func runRemove(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts removeOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - err = backend.Remove(ctx, name, api.RemoveOptions{ - Services: services, - Force: opts.force, - Volumes: opts.volumes, - Project: project, - Stop: opts.stop, - }) - if errors.Is(err, api.ErrNoResources) { - _, _ = fmt.Fprintln(stdinfo(dockerCli), "No stopped containers") - return nil - } - return err -} diff --git a/cmd/compose/restart.go b/cmd/compose/restart.go deleted file mode 100644 index e014b2a8efe..00000000000 --- a/cmd/compose/restart.go +++ /dev/null @@ -1,88 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "time" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type restartOptions struct { - *ProjectOptions - timeChanged bool - timeout int - noDeps bool -} - -func restartCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := restartOptions{ - ProjectOptions: p, - } - restartCmd := &cobra.Command{ - Use: "restart [OPTIONS] [SERVICE...]", - Short: "Restart service containers", - PreRun: func(cmd *cobra.Command, args []string) { - opts.timeChanged = cmd.Flags().Changed("timeout") - }, - RunE: Adapt(func(ctx context.Context, args []string) error { - return runRestart(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := restartCmd.Flags() - flags.IntVarP(&opts.timeout, "timeout", "t", 0, "Specify a shutdown timeout in seconds") - flags.BoolVar(&opts.noDeps, "no-deps", false, "Don't restart dependent services") - - return restartCmd -} - -func runRestart(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts restartOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli) - if err != nil { - return err - } - - if project != nil && len(services) > 0 { - project, err = project.WithServicesEnabled(services...) - if err != nil { - return err - } - } - - var timeout *time.Duration - if opts.timeChanged { - timeoutValue := time.Duration(opts.timeout) * time.Second - timeout = &timeoutValue - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Restart(ctx, name, api.RestartOptions{ - Timeout: timeout, - Services: services, - Project: project, - NoDeps: opts.noDeps, - }) -} diff --git a/cmd/compose/run.go b/cmd/compose/run.go deleted file mode 100644 index 573c0e14d4f..00000000000 --- a/cmd/compose/run.go +++ /dev/null @@ -1,351 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - "strings" - - composecli "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/dotenv" - "github.com/compose-spec/compose-go/v2/format" - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command" - "github.com/docker/cli/opts" - "github.com/mattn/go-shellwords" - xprogress "github.com/moby/buildkit/util/progress/progressui" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/cmd/display" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" - "github.com/docker/compose/v5/pkg/utils" -) - -type runOptions struct { - *composeOptions - Service string - Command []string - environment []string - envFiles []string - Detach bool - Remove bool - noTty bool - interactive bool - user string - workdir string - entrypoint string - entrypointCmd []string - capAdd opts.ListOpts - capDrop opts.ListOpts - labels []string - volumes []string - publish []string - useAliases bool - servicePorts bool - name string - noDeps bool - ignoreOrphans bool - removeOrphans bool - quiet bool - quietPull bool -} - -func (options runOptions) apply(project *types.Project) (*types.Project, error) { - if options.noDeps { - var err error - project, err = project.WithSelectedServices([]string{options.Service}, types.IgnoreDependencies) - if err != nil { - return nil, err - } - } - - target, err := project.GetService(options.Service) - if err != nil { - return nil, err - } - - target.Tty = !options.noTty - target.StdinOpen = options.interactive - - // --service-ports and --publish are incompatible - if !options.servicePorts { - if len(target.Ports) > 0 { - logrus.Debug("Running service without ports exposed as --service-ports=false") - } - target.Ports = []types.ServicePortConfig{} - for _, p := range options.publish { - config, err := types.ParsePortConfig(p) - if err != nil { - return nil, err - } - target.Ports = append(target.Ports, config...) - } - } - - for _, v := range options.volumes { - volume, err := format.ParseVolume(v) - if err != nil { - return nil, err - } - target.Volumes = append(target.Volumes, volume) - } - - for name := range project.Services { - if name == options.Service { - project.Services[name] = target - break - } - } - return project, nil -} - -func (options runOptions) getEnvironment(resolve func(string) (string, bool)) (types.Mapping, error) { - environment := types.NewMappingWithEquals(options.environment).Resolve(resolve).ToMapping() - for _, file := range options.envFiles { - f, err := os.Open(file) - if err != nil { - return nil, err - } - vars, err := dotenv.ParseWithLookup(f, func(k string) (string, bool) { - value, ok := environment[k] - return value, ok - }) - if err != nil { - return nil, nil - } - for k, v := range vars { - if _, ok := environment[k]; !ok { - environment[k] = v - } - } - } - return environment, nil -} - -func runCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - options := runOptions{ - composeOptions: &composeOptions{ - ProjectOptions: p, - }, - capAdd: opts.NewListOpts(nil), - capDrop: opts.NewListOpts(nil), - } - createOpts := createOptions{} - buildOpts := buildOptions{ - ProjectOptions: p, - } - // We remove the attribute from the option struct and use a dedicated var, to limit confusion and avoid anyone to use options.tty. - // The tty flag is here for convenience and let user do "docker compose run -it" the same way as they use the "docker run" command. - var ttyFlag bool - - cmd := &cobra.Command{ - Use: "run [OPTIONS] SERVICE [COMMAND] [ARGS...]", - Short: "Run a one-off command on a service", - Args: cobra.MinimumNArgs(1), - PreRunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - options.Service = args[0] - if len(args) > 1 { - options.Command = args[1:] - } - if len(options.publish) > 0 && options.servicePorts { - return fmt.Errorf("--service-ports and --publish are incompatible") - } - if cmd.Flags().Changed("entrypoint") { - command, err := shellwords.Parse(options.entrypoint) - if err != nil { - return err - } - options.entrypointCmd = command - } - if cmd.Flags().Changed("tty") { - if cmd.Flags().Changed("no-TTY") { - return fmt.Errorf("--tty and --no-TTY can't be used together") - } else { - options.noTty = !ttyFlag - } - } else if !cmd.Flags().Changed("no-TTY") && !cmd.Flags().Changed("interactive") && !dockerCli.In().IsTerminal() { - // while `docker run` requires explicit `-it` flags, Compose enables interactive mode and TTY by default - // but when compose is used from a script that has stdin piped from another command, we just can't - // Here, we detect we run "by default" (user didn't passed explicit flags) and disable TTY allocation if - // we don't have an actual terminal to attach to for interactive mode - options.noTty = true - } - - if options.quiet { - display.Mode = display.ModeQuiet - backendOptions.Add(compose.WithEventProcessor(display.Quiet())) - } - createOpts.pullChanged = cmd.Flags().Changed("pull") - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - project, _, err := p.ToProject(ctx, dockerCli, backend, []string{options.Service}, composecli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - project, err = project.WithServicesEnvironmentResolved(true) - if err != nil { - return err - } - - if createOpts.quietPull { - buildOpts.Progress = string(xprogress.QuietMode) - } - - options.ignoreOrphans = utils.StringToBool(project.Environment[ComposeIgnoreOrphans]) - return runRun(ctx, backend, project, options, createOpts, buildOpts, dockerCli) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.BoolVarP(&options.Detach, "detach", "d", false, "Run container in background and print container ID") - flags.StringArrayVarP(&options.environment, "env", "e", []string{}, "Set environment variables") - flags.StringArrayVar(&options.envFiles, "env-from-file", []string{}, "Set environment variables from file") - flags.StringArrayVarP(&options.labels, "label", "l", []string{}, "Add or override a label") - flags.BoolVar(&options.Remove, "rm", false, "Automatically remove the container when it exits") - flags.BoolVarP(&options.noTty, "no-TTY", "T", !dockerCli.Out().IsTerminal(), "Disable pseudo-TTY allocation (default: auto-detected)") - flags.StringVar(&options.name, "name", "", "Assign a name to the container") - flags.StringVarP(&options.user, "user", "u", "", "Run as specified username or uid") - flags.StringVarP(&options.workdir, "workdir", "w", "", "Working directory inside the container") - flags.StringVar(&options.entrypoint, "entrypoint", "", "Override the entrypoint of the image") - flags.Var(&options.capAdd, "cap-add", "Add Linux capabilities") - flags.Var(&options.capDrop, "cap-drop", "Drop Linux capabilities") - flags.BoolVar(&options.noDeps, "no-deps", false, "Don't start linked services") - flags.StringArrayVarP(&options.volumes, "volume", "v", []string{}, "Bind mount a volume") - flags.StringArrayVarP(&options.publish, "publish", "p", []string{}, "Publish a container's port(s) to the host") - flags.BoolVar(&options.useAliases, "use-aliases", false, "Use the service's network useAliases in the network(s) the container connects to") - flags.BoolVarP(&options.servicePorts, "service-ports", "P", false, "Run command with all service's ports enabled and mapped to the host") - flags.StringVar(&createOpts.Pull, "pull", "policy", `Pull image before running ("always"|"missing"|"never")`) - flags.BoolVarP(&options.quiet, "quiet", "q", false, "Don't print anything to STDOUT") - flags.BoolVar(&buildOpts.quiet, "quiet-build", false, "Suppress progress output from the build process") - flags.BoolVar(&options.quietPull, "quiet-pull", false, "Pull without printing progress information") - flags.BoolVar(&createOpts.Build, "build", false, "Build image before starting container") - flags.BoolVar(&options.removeOrphans, "remove-orphans", false, "Remove containers for services not defined in the Compose file") - - cmd.Flags().BoolVarP(&options.interactive, "interactive", "i", true, "Keep STDIN open even if not attached") - cmd.Flags().BoolVarP(&ttyFlag, "tty", "t", true, "Allocate a pseudo-TTY") - cmd.Flags().MarkHidden("tty") //nolint:errcheck - - flags.SetNormalizeFunc(normalizeRunFlags) - flags.SetInterspersed(false) - return cmd -} - -func normalizeRunFlags(f *pflag.FlagSet, name string) pflag.NormalizedName { - switch name { - case "volumes": - name = "volume" - case "labels": - name = "label" - } - return pflag.NormalizedName(name) -} - -func runRun(ctx context.Context, backend api.Compose, project *types.Project, options runOptions, createOpts createOptions, buildOpts buildOptions, dockerCli command.Cli) error { - project, err := options.apply(project) - if err != nil { - return err - } - - err = createOpts.Apply(project) - if err != nil { - return err - } - - if err := checksForRemoteStack(ctx, dockerCli, project, buildOpts, createOpts.AssumeYes, []string{}); err != nil { - return err - } - - labels := types.Labels{} - for _, s := range options.labels { - key, val, ok := strings.Cut(s, "=") - if !ok { - return fmt.Errorf("label must be set as KEY=VALUE") - } - labels[key] = val - } - - var buildForRun *api.BuildOptions - if !createOpts.noBuild { - bo, err := buildOpts.toAPIBuildOptions(nil) - if err != nil { - return err - } - buildForRun = &bo - } - - environment, err := options.getEnvironment(project.Environment.Resolve) - if err != nil { - return err - } - - // start container and attach to container streams - runOpts := api.RunOptions{ - CreateOptions: api.CreateOptions{ - Build: buildForRun, - RemoveOrphans: options.removeOrphans, - IgnoreOrphans: options.ignoreOrphans, - QuietPull: options.quietPull, - }, - Name: options.name, - Service: options.Service, - Command: options.Command, - Detach: options.Detach, - AutoRemove: options.Remove, - Tty: !options.noTty, - Interactive: options.interactive, - WorkingDir: options.workdir, - User: options.user, - CapAdd: options.capAdd.GetSlice(), - CapDrop: options.capDrop.GetSlice(), - Environment: environment.Values(), - Entrypoint: options.entrypointCmd, - Labels: labels, - UseNetworkAliases: options.useAliases, - NoDeps: options.noDeps, - Index: 0, - } - - for name, service := range project.Services { - if name == options.Service { - service.StdinOpen = options.interactive - project.Services[name] = service - } - } - - exitCode, err := backend.RunOneOffContainer(ctx, project, runOpts) - if exitCode != 0 { - errMsg := "" - if err != nil { - errMsg = err.Error() - } - return cli.StatusError{StatusCode: exitCode, Status: errMsg} - } - return err -} diff --git a/cmd/compose/scale.go b/cmd/compose/scale.go deleted file mode 100644 index 8070858e9a1..00000000000 --- a/cmd/compose/scale.go +++ /dev/null @@ -1,107 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "maps" - "slices" - "strconv" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type scaleOptions struct { - *ProjectOptions - noDeps bool -} - -func scaleCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := scaleOptions{ - ProjectOptions: p, - } - scaleCmd := &cobra.Command{ - Use: "scale [SERVICE=REPLICAS...]", - Short: "Scale services ", - Args: cobra.MinimumNArgs(1), - RunE: Adapt(func(ctx context.Context, args []string) error { - serviceTuples, err := parseServicesReplicasArgs(args) - if err != nil { - return err - } - return runScale(ctx, dockerCli, backendOptions, opts, serviceTuples) - }), - ValidArgsFunction: completeScaleArgs(dockerCli, p), - } - flags := scaleCmd.Flags() - flags.BoolVar(&opts.noDeps, "no-deps", false, "Don't start linked services") - - return scaleCmd -} - -func runScale(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts scaleOptions, serviceReplicaTuples map[string]int) error { - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - services := slices.Sorted(maps.Keys(serviceReplicaTuples)) - project, _, err := opts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return err - } - - if opts.noDeps { - if project, err = project.WithSelectedServices(services, types.IgnoreDependencies); err != nil { - return err - } - } - - for key, value := range serviceReplicaTuples { - service, err := project.GetService(key) - if err != nil { - return err - } - service.SetScale(value) - project.Services[key] = service - } - - return backend.Scale(ctx, project, api.ScaleOptions{Services: services}) -} - -func parseServicesReplicasArgs(args []string) (map[string]int, error) { - serviceReplicaTuples := map[string]int{} - for _, arg := range args { - key, val, ok := strings.Cut(arg, "=") - if !ok || key == "" || val == "" { - return nil, fmt.Errorf("invalid scale specifier: %s", arg) - } - intValue, err := strconv.Atoi(val) - if err != nil { - return nil, fmt.Errorf("invalid scale specifier: can't parse replica value as int: %v", arg) - } - serviceReplicaTuples[key] = intValue - } - return serviceReplicaTuples, nil -} diff --git a/cmd/compose/start.go b/cmd/compose/start.go deleted file mode 100644 index bd5f10c4634..00000000000 --- a/cmd/compose/start.go +++ /dev/null @@ -1,77 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "time" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type startOptions struct { - *ProjectOptions - wait bool - waitTimeout int -} - -func startCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := startOptions{ - ProjectOptions: p, - } - startCmd := &cobra.Command{ - Use: "start [SERVICE...]", - Short: "Start services", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runStart(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := startCmd.Flags() - flags.BoolVar(&opts.wait, "wait", false, "Wait for services to be running|healthy. Implies detached mode.") - flags.IntVar(&opts.waitTimeout, "wait-timeout", 0, "Maximum duration in seconds to wait for the project to be running|healthy") - - return startCmd -} - -func runStart(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts startOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - var timeout time.Duration - if opts.waitTimeout > 0 { - timeout = time.Duration(opts.waitTimeout) * time.Second - } - return backend.Start(ctx, name, api.StartOptions{ - AttachTo: services, - Project: project, - Services: services, - Wait: opts.wait, - WaitTimeout: timeout, - }) -} diff --git a/cmd/compose/stats.go b/cmd/compose/stats.go deleted file mode 100644 index cef2daf275d..00000000000 --- a/cmd/compose/stats.go +++ /dev/null @@ -1,84 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/command/container" - "github.com/docker/docker/api/types/filters" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" -) - -type statsOptions struct { - ProjectOptions *ProjectOptions - all bool - format string - noStream bool - noTrunc bool -} - -func statsCommand(p *ProjectOptions, dockerCli command.Cli) *cobra.Command { - opts := statsOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "stats [OPTIONS] [SERVICE]", - Short: "Display a live stream of container(s) resource usage statistics", - Args: cobra.MaximumNArgs(1), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runStats(ctx, dockerCli, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.BoolVarP(&opts.all, "all", "a", false, "Show all containers (default shows just running)") - flags.StringVar(&opts.format, "format", "", `Format output using a custom template: -'table': Print output in table format with column headers (default) -'table TEMPLATE': Print output in table format using the given Go template -'json': Print in JSON format -'TEMPLATE': Print output using the given Go template. -Refer to https://docs.docker.com/engine/cli/formatting/ for more information about formatting output with templates`) - flags.BoolVar(&opts.noStream, "no-stream", false, "Disable streaming stats and only pull the first result") - flags.BoolVar(&opts.noTrunc, "no-trunc", false, "Do not truncate output") - return cmd -} - -func runStats(ctx context.Context, dockerCli command.Cli, opts statsOptions, service []string) error { - name, err := opts.ProjectOptions.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - filter := []filters.KeyValuePair{ - filters.Arg("label", fmt.Sprintf("%s=%s", api.ProjectLabel, name)), - } - if len(service) > 0 { - filter = append(filter, filters.Arg("label", fmt.Sprintf("%s=%s", api.ServiceLabel, service[0]))) - } - args := filters.NewArgs(filter...) - return container.RunStats(ctx, dockerCli, &container.StatsOptions{ - All: opts.all, - NoStream: opts.noStream, - NoTrunc: opts.noTrunc, - Format: opts.format, - Filters: &args, - }) -} diff --git a/cmd/compose/stop.go b/cmd/compose/stop.go deleted file mode 100644 index 6bc3faaa96b..00000000000 --- a/cmd/compose/stop.go +++ /dev/null @@ -1,77 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "time" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type stopOptions struct { - *ProjectOptions - timeChanged bool - timeout int -} - -func stopCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := stopOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "stop [OPTIONS] [SERVICE...]", - Short: "Stop services", - PreRun: func(cmd *cobra.Command, args []string) { - opts.timeChanged = cmd.Flags().Changed("timeout") - }, - RunE: Adapt(func(ctx context.Context, args []string) error { - return runStop(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.IntVarP(&opts.timeout, "timeout", "t", 0, "Specify a shutdown timeout in seconds") - - return cmd -} - -func runStop(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts stopOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - var timeout *time.Duration - if opts.timeChanged { - timeoutValue := time.Duration(opts.timeout) * time.Second - timeout = &timeoutValue - } - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Stop(ctx, name, api.StopOptions{ - Timeout: timeout, - Services: services, - Project: project, - }) -} diff --git a/cmd/compose/top.go b/cmd/compose/top.go deleted file mode 100644 index 0d7f969c841..00000000000 --- a/cmd/compose/top.go +++ /dev/null @@ -1,149 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - "sort" - "strings" - "text/tabwriter" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type topOptions struct { - *ProjectOptions -} - -func topCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := topOptions{ - ProjectOptions: p, - } - topCmd := &cobra.Command{ - Use: "top [SERVICES...]", - Short: "Display the running processes", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runTop(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - return topCmd -} - -type ( - topHeader map[string]int // maps a proc title to its output index - topEntries map[string]string -) - -func runTop(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts topOptions, services []string) error { - projectName, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - containers, err := backend.Top(ctx, projectName, services) - if err != nil { - return err - } - - sort.Slice(containers, func(i, j int) bool { - return containers[i].Name < containers[j].Name - }) - - header, entries := collectTop(containers) - return topPrint(dockerCli.Out(), header, entries) -} - -func collectTop(containers []api.ContainerProcSummary) (topHeader, []topEntries) { - // map column name to its header (should keep working if backend.Top returns - // varying columns for different containers) - header := topHeader{"SERVICE": 0, "#": 1} - - // assume one process per container and grow if needed - entries := make([]topEntries, 0, len(containers)) - - for _, container := range containers { - for _, proc := range container.Processes { - entry := topEntries{ - "SERVICE": container.Service, - "#": container.Replica, - } - for i, title := range container.Titles { - if _, exists := header[title]; !exists { - header[title] = len(header) - } - entry[title] = proc[i] - } - entries = append(entries, entry) - } - } - - // ensure CMD is the right-most column - if pos, ok := header["CMD"]; ok { - maxPos := pos - for h, i := range header { - if i > maxPos { - maxPos = i - } - if i > pos { - header[h] = i - 1 - } - } - header["CMD"] = maxPos - } - - return header, entries -} - -func topPrint(out io.Writer, headers topHeader, rows []topEntries) error { - if len(rows) == 0 { - return nil - } - - w := tabwriter.NewWriter(out, 4, 1, 2, ' ', 0) - - // write headers in the order we've encountered them - h := make([]string, len(headers)) - for title, index := range headers { - h[index] = title - } - _, _ = fmt.Fprintln(w, strings.Join(h, "\t")) - - for _, row := range rows { - // write proc data in header order - r := make([]string, len(headers)) - for title, index := range headers { - if v, ok := row[title]; ok { - r[index] = v - } else { - r[index] = "-" - } - } - _, _ = fmt.Fprintln(w, strings.Join(r, "\t")) - } - return w.Flush() -} diff --git a/cmd/compose/top_test.go b/cmd/compose/top_test.go deleted file mode 100644 index 3e212ed85b0..00000000000 --- a/cmd/compose/top_test.go +++ /dev/null @@ -1,330 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "strings" - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - - "github.com/docker/compose/v5/pkg/api" -) - -var topTestCases = []struct { - name string - titles []string - procs [][]string - - header topHeader - entries []topEntries - output string -}{ - { - name: "noprocs", - titles: []string{"UID", "PID", "PPID", "C", "STIME", "TTY", "TIME", "CMD"}, - procs: [][]string{}, - header: topHeader{"SERVICE": 0, "#": 1}, - entries: []topEntries{}, - output: "", - }, - { - name: "simple", - titles: []string{"UID", "PID", "PPID", "C", "STIME", "TTY", "TIME", "CMD"}, - procs: [][]string{{"root", "1", "1", "0", "12:00", "?", "00:00:01", "/entrypoint"}}, - header: topHeader{ - "SERVICE": 0, - "#": 1, - "UID": 2, - "PID": 3, - "PPID": 4, - "C": 5, - "STIME": 6, - "TTY": 7, - "TIME": 8, - "CMD": 9, - }, - entries: []topEntries{ - { - "SERVICE": "simple", - "#": "1", - "UID": "root", - "PID": "1", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:01", - "CMD": "/entrypoint", - }, - }, - output: trim(` - SERVICE # UID PID PPID C STIME TTY TIME CMD - simple 1 root 1 1 0 12:00 ? 00:00:01 /entrypoint - `), - }, - { - name: "noppid", - titles: []string{"UID", "PID", "C", "STIME", "TTY", "TIME", "CMD"}, - procs: [][]string{{"root", "1", "0", "12:00", "?", "00:00:02", "/entrypoint"}}, - header: topHeader{ - "SERVICE": 0, - "#": 1, - "UID": 2, - "PID": 3, - "C": 4, - "STIME": 5, - "TTY": 6, - "TIME": 7, - "CMD": 8, - }, - entries: []topEntries{ - { - "SERVICE": "noppid", - "#": "1", - "UID": "root", - "PID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:02", - "CMD": "/entrypoint", - }, - }, - output: trim(` - SERVICE # UID PID C STIME TTY TIME CMD - noppid 1 root 1 0 12:00 ? 00:00:02 /entrypoint - `), - }, - { - name: "extra-hdr", - titles: []string{"UID", "GID", "PID", "PPID", "C", "STIME", "TTY", "TIME", "CMD"}, - procs: [][]string{{"root", "1", "1", "1", "0", "12:00", "?", "00:00:03", "/entrypoint"}}, - header: topHeader{ - "SERVICE": 0, - "#": 1, - "UID": 2, - "GID": 3, - "PID": 4, - "PPID": 5, - "C": 6, - "STIME": 7, - "TTY": 8, - "TIME": 9, - "CMD": 10, - }, - entries: []topEntries{ - { - "SERVICE": "extra-hdr", - "#": "1", - "UID": "root", - "GID": "1", - "PID": "1", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:03", - "CMD": "/entrypoint", - }, - }, - output: trim(` - SERVICE # UID GID PID PPID C STIME TTY TIME CMD - extra-hdr 1 root 1 1 1 0 12:00 ? 00:00:03 /entrypoint - `), - }, - { - name: "multiple", - titles: []string{"UID", "PID", "PPID", "C", "STIME", "TTY", "TIME", "CMD"}, - procs: [][]string{ - {"root", "1", "1", "0", "12:00", "?", "00:00:04", "/entrypoint"}, - {"root", "123", "1", "0", "12:00", "?", "00:00:42", "sleep infinity"}, - }, - header: topHeader{ - "SERVICE": 0, - "#": 1, - "UID": 2, - "PID": 3, - "PPID": 4, - "C": 5, - "STIME": 6, - "TTY": 7, - "TIME": 8, - "CMD": 9, - }, - entries: []topEntries{ - { - "SERVICE": "multiple", - "#": "1", - "UID": "root", - "PID": "1", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:04", - "CMD": "/entrypoint", - }, - { - "SERVICE": "multiple", - "#": "1", - "UID": "root", - "PID": "123", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:42", - "CMD": "sleep infinity", - }, - }, - output: trim(` - SERVICE # UID PID PPID C STIME TTY TIME CMD - multiple 1 root 1 1 0 12:00 ? 00:00:04 /entrypoint - multiple 1 root 123 1 0 12:00 ? 00:00:42 sleep infinity - `), - }, -} - -// TestRunTopCore only tests the core functionality of runTop: formatting -// and printing of the output of (api.Compose).Top(). -func TestRunTopCore(t *testing.T) { - t.Parallel() - - all := []api.ContainerProcSummary{} - - for _, tc := range topTestCases { - summary := api.ContainerProcSummary{ - Name: "not used", - Titles: tc.titles, - Processes: tc.procs, - Service: tc.name, - Replica: "1", - } - all = append(all, summary) - - t.Run(tc.name, func(t *testing.T) { - header, entries := collectTop([]api.ContainerProcSummary{summary}) - assert.Equal(t, tc.header, header) - assert.Equal(t, tc.entries, entries) - - var buf bytes.Buffer - err := topPrint(&buf, header, entries) - - require.NoError(t, err) - assert.Equal(t, tc.output, buf.String()) - }) - } - - t.Run("all", func(t *testing.T) { - header, entries := collectTop(all) - assert.Equal(t, topHeader{ - "SERVICE": 0, - "#": 1, - "UID": 2, - "PID": 3, - "PPID": 4, - "C": 5, - "STIME": 6, - "TTY": 7, - "TIME": 8, - "GID": 9, - "CMD": 10, - }, header) - assert.Equal(t, []topEntries{ - { - "SERVICE": "simple", - "#": "1", - "UID": "root", - "PID": "1", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:01", - "CMD": "/entrypoint", - }, { - "SERVICE": "noppid", - "#": "1", - "UID": "root", - "PID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:02", - "CMD": "/entrypoint", - }, { - "SERVICE": "extra-hdr", - "#": "1", - "UID": "root", - "GID": "1", - "PID": "1", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:03", - "CMD": "/entrypoint", - }, { - "SERVICE": "multiple", - "#": "1", - "UID": "root", - "PID": "1", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:04", - "CMD": "/entrypoint", - }, { - "SERVICE": "multiple", - "#": "1", - "UID": "root", - "PID": "123", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:42", - "CMD": "sleep infinity", - }, - }, entries) - - var buf bytes.Buffer - err := topPrint(&buf, header, entries) - require.NoError(t, err) - assert.Equal(t, trim(` - SERVICE # UID PID PPID C STIME TTY TIME GID CMD - simple 1 root 1 1 0 12:00 ? 00:00:01 - /entrypoint - noppid 1 root 1 - 0 12:00 ? 00:00:02 - /entrypoint - extra-hdr 1 root 1 1 0 12:00 ? 00:00:03 1 /entrypoint - multiple 1 root 1 1 0 12:00 ? 00:00:04 - /entrypoint - multiple 1 root 123 1 0 12:00 ? 00:00:42 - sleep infinity - `), buf.String()) - }) -} - -func trim(s string) string { - var out bytes.Buffer - for line := range strings.SplitSeq(strings.TrimSpace(s), "\n") { - out.WriteString(strings.TrimSpace(line)) - out.WriteRune('\n') - } - return out.String() -} diff --git a/cmd/compose/up.go b/cmd/compose/up.go deleted file mode 100644 index cda2678bbb2..00000000000 --- a/cmd/compose/up.go +++ /dev/null @@ -1,363 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "os" - "strings" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - xprogress "github.com/moby/buildkit/util/progress/progressui" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/cmd/display" - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" - "github.com/docker/compose/v5/pkg/utils" -) - -// composeOptions hold options common to `up` and `run` to run compose project -type composeOptions struct { - *ProjectOptions -} - -type upOptions struct { - *composeOptions - Detach bool - noStart bool - noDeps bool - cascadeStop bool - cascadeFail bool - exitCodeFrom string - noColor bool - noPrefix bool - attachDependencies bool - attach []string - noAttach []string - timestamp bool - wait bool - waitTimeout int - watch bool - navigationMenu bool - navigationMenuChanged bool -} - -func (opts upOptions) apply(project *types.Project, services []string) (*types.Project, error) { - if opts.noDeps { - var err error - project, err = project.WithSelectedServices(services, types.IgnoreDependencies) - if err != nil { - return nil, err - } - } - - if opts.exitCodeFrom != "" { - _, err := project.GetService(opts.exitCodeFrom) - if err != nil { - return nil, err - } - } - - return project, nil -} - -func (opts *upOptions) validateNavigationMenu(dockerCli command.Cli) { - if !dockerCli.Out().IsTerminal() { - opts.navigationMenu = false - return - } - // If --menu flag was not set - if !opts.navigationMenuChanged { - if envVar, ok := os.LookupEnv(ComposeMenu); ok { - opts.navigationMenu = utils.StringToBool(envVar) - return - } - // ...and COMPOSE_MENU env var is not defined we want the default value to be true - opts.navigationMenu = true - } -} - -func (opts upOptions) OnExit() api.Cascade { - switch { - case opts.cascadeStop: - return api.CascadeStop - case opts.cascadeFail: - return api.CascadeFail - default: - return api.CascadeIgnore - } -} - -func upCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - up := upOptions{} - create := createOptions{} - build := buildOptions{ProjectOptions: p} - upCmd := &cobra.Command{ - Use: "up [OPTIONS] [SERVICE...]", - Short: "Create and start containers", - PreRunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - create.pullChanged = cmd.Flags().Changed("pull") - create.timeChanged = cmd.Flags().Changed("timeout") - up.navigationMenuChanged = cmd.Flags().Changed("menu") - if !cmd.Flags().Changed("remove-orphans") { - create.removeOrphans = utils.StringToBool(os.Getenv(ComposeRemoveOrphans)) - } - return validateFlags(&up, &create) - }), - RunE: p.WithServices(dockerCli, func(ctx context.Context, project *types.Project, services []string) error { - create.ignoreOrphans = utils.StringToBool(project.Environment[ComposeIgnoreOrphans]) - if create.ignoreOrphans && create.removeOrphans { - return fmt.Errorf("cannot combine %s and --remove-orphans", ComposeIgnoreOrphans) - } - if len(up.attach) != 0 && up.attachDependencies { - return errors.New("cannot combine --attach and --attach-dependencies") - } - - up.validateNavigationMenu(dockerCli) - - if !p.All && len(project.Services) == 0 { - return fmt.Errorf("no service selected") - } - - return runUp(ctx, dockerCli, backendOptions, create, up, build, project, services) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := upCmd.Flags() - flags.BoolVarP(&up.Detach, "detach", "d", false, "Detached mode: Run containers in the background") - flags.BoolVar(&create.Build, "build", false, "Build images before starting containers") - flags.BoolVar(&create.noBuild, "no-build", false, "Don't build an image, even if it's policy") - flags.StringVar(&create.Pull, "pull", "policy", `Pull image before running ("always"|"missing"|"never")`) - flags.BoolVar(&create.removeOrphans, "remove-orphans", false, "Remove containers for services not defined in the Compose file") - flags.StringArrayVar(&create.scale, "scale", []string{}, "Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present.") - flags.BoolVar(&up.noColor, "no-color", false, "Produce monochrome output") - flags.BoolVar(&up.noPrefix, "no-log-prefix", false, "Don't print prefix in logs") - flags.BoolVar(&create.forceRecreate, "force-recreate", false, "Recreate containers even if their configuration and image haven't changed") - flags.BoolVar(&create.noRecreate, "no-recreate", false, "If containers already exist, don't recreate them. Incompatible with --force-recreate.") - flags.BoolVar(&up.noStart, "no-start", false, "Don't start the services after creating them") - flags.BoolVar(&up.cascadeStop, "abort-on-container-exit", false, "Stops all containers if any container was stopped. Incompatible with -d") - flags.BoolVar(&up.cascadeFail, "abort-on-container-failure", false, "Stops all containers if any container exited with failure. Incompatible with -d") - flags.StringVar(&up.exitCodeFrom, "exit-code-from", "", "Return the exit code of the selected service container. Implies --abort-on-container-exit") - flags.IntVarP(&create.timeout, "timeout", "t", 0, "Use this timeout in seconds for container shutdown when attached or when containers are already running") - flags.BoolVar(&up.timestamp, "timestamps", false, "Show timestamps") - flags.BoolVar(&up.noDeps, "no-deps", false, "Don't start linked services") - flags.BoolVar(&create.recreateDeps, "always-recreate-deps", false, "Recreate dependent containers. Incompatible with --no-recreate.") - flags.BoolVarP(&create.noInherit, "renew-anon-volumes", "V", false, "Recreate anonymous volumes instead of retrieving data from the previous containers") - flags.BoolVar(&create.quietPull, "quiet-pull", false, "Pull without printing progress information") - flags.BoolVar(&build.quiet, "quiet-build", false, "Suppress the build output") - flags.StringArrayVar(&up.attach, "attach", []string{}, "Restrict attaching to the specified services. Incompatible with --attach-dependencies.") - flags.StringArrayVar(&up.noAttach, "no-attach", []string{}, "Do not attach (stream logs) to the specified services") - flags.BoolVar(&up.attachDependencies, "attach-dependencies", false, "Automatically attach to log output of dependent services") - flags.BoolVar(&up.wait, "wait", false, "Wait for services to be running|healthy. Implies detached mode.") - flags.IntVar(&up.waitTimeout, "wait-timeout", 0, "Maximum duration in seconds to wait for the project to be running|healthy") - flags.BoolVarP(&up.watch, "watch", "w", false, "Watch source code and rebuild/refresh containers when files are updated.") - flags.BoolVar(&up.navigationMenu, "menu", false, "Enable interactive shortcuts when running attached. Incompatible with --detach. Can also be enable/disable by setting COMPOSE_MENU environment var.") - flags.BoolVarP(&create.AssumeYes, "yes", "y", false, `Assume "yes" as answer to all prompts and run non-interactively`) - flags.SetNormalizeFunc(func(f *pflag.FlagSet, name string) pflag.NormalizedName { - // assumeYes was introduced by mistake as `--y` - if name == "y" { - logrus.Warn("--y is deprecated, please use --yes instead") - name = "yes" - } - return pflag.NormalizedName(name) - }) - return upCmd -} - -//nolint:gocyclo -func validateFlags(up *upOptions, create *createOptions) error { - if up.waitTimeout < 0 { - return fmt.Errorf("--wait-timeout must be a non-negative integer") - } - if up.exitCodeFrom != "" && !up.cascadeFail { - up.cascadeStop = true - } - if up.cascadeStop && up.cascadeFail { - return fmt.Errorf("--abort-on-container-failure cannot be combined with --abort-on-container-exit") - } - if up.wait { - if up.attachDependencies || up.cascadeStop || len(up.attach) > 0 { - return fmt.Errorf("--wait cannot be combined with --abort-on-container-exit, --attach or --attach-dependencies") - } - up.Detach = true - } - if create.Build && create.noBuild { - return fmt.Errorf("--build and --no-build are incompatible") - } - if up.Detach && (up.attachDependencies || up.cascadeStop || up.cascadeFail || len(up.attach) > 0 || up.watch) { - if up.wait { - return fmt.Errorf("--wait cannot be combined with --abort-on-container-exit, --abort-on-container-failure, --attach, --attach-dependencies or --watch") - } else { - return fmt.Errorf("--detach cannot be combined with --abort-on-container-exit, --abort-on-container-failure, --attach, --attach-dependencies or --watch") - } - } - if create.noInherit && create.noRecreate { - return fmt.Errorf("--no-recreate and --renew-anon-volumes are incompatible") - } - if create.forceRecreate && create.noRecreate { - return fmt.Errorf("--force-recreate and --no-recreate are incompatible") - } - if create.recreateDeps && create.noRecreate { - return fmt.Errorf("--always-recreate-deps and --no-recreate are incompatible") - } - if create.noBuild && up.watch { - return fmt.Errorf("--no-build and --watch are incompatible") - } - return nil -} - -//nolint:gocyclo -func runUp( - ctx context.Context, - dockerCli command.Cli, - backendOptions *BackendOptions, - createOptions createOptions, - upOptions upOptions, - buildOptions buildOptions, - project *types.Project, - services []string, -) error { - if err := checksForRemoteStack(ctx, dockerCli, project, buildOptions, createOptions.AssumeYes, []string{}); err != nil { - return err - } - - err := createOptions.Apply(project) - if err != nil { - return err - } - - project, err = upOptions.apply(project, services) - if err != nil { - return err - } - - var build *api.BuildOptions - if !createOptions.noBuild { - if createOptions.quietPull { - buildOptions.Progress = string(xprogress.QuietMode) - } - // BuildOptions here is nested inside CreateOptions, so - // no service list is passed, it will implicitly pick all - // services being created, which includes any explicitly - // specified via "services" arg here as well as deps - bo, err := buildOptions.toAPIBuildOptions(nil) - if err != nil { - return err - } - bo.Services = project.ServiceNames() - bo.Deps = !upOptions.noDeps - build = &bo - } - - create := api.CreateOptions{ - Build: build, - Services: services, - RemoveOrphans: createOptions.removeOrphans, - IgnoreOrphans: createOptions.ignoreOrphans, - Recreate: createOptions.recreateStrategy(), - RecreateDependencies: createOptions.dependenciesRecreateStrategy(), - Inherit: !createOptions.noInherit, - Timeout: createOptions.GetTimeout(), - QuietPull: createOptions.quietPull, - } - - if createOptions.AssumeYes { - backendOptions.Options = append(backendOptions.Options, compose.WithPrompt(compose.AlwaysOkPrompt())) - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - if upOptions.noStart { - return backend.Create(ctx, project, create) - } - - var consumer api.LogConsumer - var attach []string - if !upOptions.Detach { - consumer = formatter.NewLogConsumer(ctx, dockerCli.Out(), dockerCli.Err(), !upOptions.noColor, !upOptions.noPrefix, upOptions.timestamp) - - var attachSet utils.Set[string] - if len(upOptions.attach) != 0 { - // services are passed explicitly with --attach, verify they're valid and then use them as-is - attachSet = utils.NewSet(upOptions.attach...) - unexpectedSvcs := attachSet.Diff(utils.NewSet(project.ServiceNames()...)) - if len(unexpectedSvcs) != 0 { - return fmt.Errorf("cannot attach to services not included in up: %s", strings.Join(unexpectedSvcs.Elements(), ", ")) - } - } else { - // mark services being launched (and potentially their deps) for attach - // if they didn't opt-out via Compose YAML - attachSet = utils.NewSet[string]() - var dependencyOpt types.DependencyOption = types.IgnoreDependencies - if upOptions.attachDependencies { - dependencyOpt = types.IncludeDependencies - } - if err := project.ForEachService(services, func(serviceName string, s *types.ServiceConfig) error { - if s.Attach == nil || *s.Attach { - attachSet.Add(serviceName) - } - return nil - }, dependencyOpt); err != nil { - return err - } - } - // filter out any services that have been explicitly marked for ignore with `--no-attach` - attachSet.RemoveAll(upOptions.noAttach...) - attach = attachSet.Elements() - } - - var timeout time.Duration - if upOptions.waitTimeout > 0 { - timeout = time.Duration(upOptions.waitTimeout) * time.Second - } - return backend.Up(ctx, project, api.UpOptions{ - Create: create, - Start: api.StartOptions{ - Project: project, - Attach: consumer, - AttachTo: attach, - ExitCodeFrom: upOptions.exitCodeFrom, - OnExit: upOptions.OnExit(), - Wait: upOptions.wait, - WaitTimeout: timeout, - Watch: upOptions.watch, - Services: services, - NavigationMenu: upOptions.navigationMenu && display.Mode != "plain" && dockerCli.In().IsTerminal(), - }, - }) -} - -func setServiceScale(project *types.Project, name string, replicas int) error { - service, err := project.GetService(name) - if err != nil { - return err - } - service.SetScale(replicas) - project.Services[name] = service - return nil -} diff --git a/cmd/compose/up_test.go b/cmd/compose/up_test.go deleted file mode 100644 index f567e97ad44..00000000000 --- a/cmd/compose/up_test.go +++ /dev/null @@ -1,91 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestApplyScaleOpt(t *testing.T) { - p := types.Project{ - Services: types.Services{ - "foo": { - Name: "foo", - }, - "bar": { - Name: "bar", - Deploy: &types.DeployConfig{ - Mode: "test", - }, - }, - }, - } - err := applyScaleOpts(&p, []string{"foo=2", "bar=3"}) - assert.NilError(t, err) - foo, err := p.GetService("foo") - assert.NilError(t, err) - assert.Equal(t, *foo.Scale, 2) - - bar, err := p.GetService("bar") - assert.NilError(t, err) - assert.Equal(t, *bar.Scale, 3) - assert.Equal(t, *bar.Deploy.Replicas, 3) -} - -func TestUpOptions_OnExit(t *testing.T) { - tests := []struct { - name string - args upOptions - want api.Cascade - }{ - { - name: "no cascade", - args: upOptions{}, - want: api.CascadeIgnore, - }, - { - name: "cascade stop", - args: upOptions{cascadeStop: true}, - want: api.CascadeStop, - }, - { - name: "cascade fail", - args: upOptions{cascadeFail: true}, - want: api.CascadeFail, - }, - { - name: "both set - stop takes precedence", - args: upOptions{ - cascadeStop: true, - cascadeFail: true, - }, - want: api.CascadeStop, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - got := tt.args.OnExit() - assert.Equal(t, got, tt.want) - }) - } -} diff --git a/cmd/compose/version.go b/cmd/compose/version.go deleted file mode 100644 index 4302c15697b..00000000000 --- a/cmd/compose/version.go +++ /dev/null @@ -1,69 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "fmt" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/internal" -) - -type versionOptions struct { - format string - short bool -} - -func versionCommand(dockerCli command.Cli) *cobra.Command { - opts := versionOptions{} - cmd := &cobra.Command{ - Use: "version [OPTIONS]", - Short: "Show the Docker Compose version information", - Args: cobra.NoArgs, - RunE: func(cmd *cobra.Command, _ []string) error { - runVersion(opts, dockerCli) - return nil - }, - PersistentPreRunE: func(cmd *cobra.Command, args []string) error { - // overwrite parent PersistentPreRunE to avoid trying to load - // compose file on version command if COMPOSE_FILE is set - return nil - }, - } - // define flags for backward compatibility with com.docker.cli - flags := cmd.Flags() - flags.StringVarP(&opts.format, "format", "f", "", "Format the output. Values: [pretty | json]. (Default: pretty)") - flags.BoolVar(&opts.short, "short", false, "Shows only Compose's version number") - - return cmd -} - -func runVersion(opts versionOptions, dockerCli command.Cli) { - if opts.short { - _, _ = fmt.Fprintln(dockerCli.Out(), strings.TrimPrefix(internal.Version, "v")) - return - } - if opts.format == formatter.JSON { - _, _ = fmt.Fprintf(dockerCli.Out(), "{\"version\":%q}\n", internal.Version) - return - } - _, _ = fmt.Fprintln(dockerCli.Out(), "Docker Compose version", internal.Version) -} diff --git a/cmd/compose/version_test.go b/cmd/compose/version_test.go deleted file mode 100644 index c9bf9b74eae..00000000000 --- a/cmd/compose/version_test.go +++ /dev/null @@ -1,77 +0,0 @@ -/* - Copyright 2025 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "testing" - - "github.com/docker/cli/cli/streams" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/internal" - "github.com/docker/compose/v5/pkg/mocks" -) - -func TestVersionCommand(t *testing.T) { - originalVersion := internal.Version - defer func() { - internal.Version = originalVersion - }() - internal.Version = "v9.9.9-test" - - tests := []struct { - name string - args []string - want string - }{ - { - name: "default", - args: []string{}, - want: "Docker Compose version v9.9.9-test\n", - }, - { - name: "short flag", - args: []string{"--short"}, - want: "9.9.9-test\n", - }, - { - name: "json flag", - args: []string{"--format", "json"}, - want: `{"version":"v9.9.9-test"}` + "\n", - }, - } - - for _, test := range tests { - t.Run(test.name, func(t *testing.T) { - ctrl := gomock.NewController(t) - defer ctrl.Finish() - - buf := new(bytes.Buffer) - cli := mocks.NewMockCli(ctrl) - cli.EXPECT().Out().Return(streams.NewOut(buf)).AnyTimes() - - cmd := versionCommand(cli) - cmd.SetArgs(test.args) - err := cmd.Execute() - assert.NilError(t, err) - - assert.Equal(t, test.want, buf.String()) - }) - } -} diff --git a/cmd/compose/viz.go b/cmd/compose/viz.go deleted file mode 100644 index 443d78c6261..00000000000 --- a/cmd/compose/viz.go +++ /dev/null @@ -1,105 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type vizOptions struct { - *ProjectOptions - includeNetworks bool - includePorts bool - includeImageName bool - indentationStr string -} - -func vizCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := vizOptions{ - ProjectOptions: p, - } - var indentationSize int - var useSpaces bool - - cmd := &cobra.Command{ - Use: "viz [OPTIONS]", - Short: "EXPERIMENTAL - Generate a graphviz graph from your compose file", - PreRunE: Adapt(func(ctx context.Context, args []string) error { - var err error - opts.indentationStr, err = preferredIndentationStr(indentationSize, useSpaces) - return err - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runViz(ctx, dockerCli, backendOptions, &opts) - }), - } - - cmd.Flags().BoolVar(&opts.includePorts, "ports", false, "Include service's exposed ports in output graph") - cmd.Flags().BoolVar(&opts.includeNetworks, "networks", false, "Include service's attached networks in output graph") - cmd.Flags().BoolVar(&opts.includeImageName, "image", false, "Include service's image name in output graph") - cmd.Flags().IntVar(&indentationSize, "indentation-size", 1, "Number of tabs or spaces to use for indentation") - cmd.Flags().BoolVar(&useSpaces, "spaces", false, "If given, space character ' ' will be used to indent,\notherwise tab character '\\t' will be used") - return cmd -} - -func runViz(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts *vizOptions) error { - _, _ = fmt.Fprintln(os.Stderr, "viz command is EXPERIMENTAL") - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - project, _, err := opts.ToProject(ctx, dockerCli, backend, nil) - if err != nil { - return err - } - - // build graph - graphStr, _ := backend.Viz(ctx, project, api.VizOptions{ - IncludeNetworks: opts.includeNetworks, - IncludePorts: opts.includePorts, - IncludeImageName: opts.includeImageName, - Indentation: opts.indentationStr, - }) - - fmt.Println(graphStr) - - return nil -} - -// preferredIndentationStr returns a single string given the indentation preference -func preferredIndentationStr(size int, useSpace bool) (string, error) { - if size < 0 { - return "", fmt.Errorf("invalid indentation size: %d", size) - } - - indentationStr := "\t" - if useSpace { - indentationStr = " " - } - return strings.Repeat(indentationStr, size), nil -} diff --git a/cmd/compose/viz_test.go b/cmd/compose/viz_test.go deleted file mode 100644 index f4a90501e33..00000000000 --- a/cmd/compose/viz_test.go +++ /dev/null @@ -1,94 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -func TestPreferredIndentationStr(t *testing.T) { - type args struct { - size int - useSpace bool - } - tests := []struct { - name string - args args - want string - wantErr bool - }{ - { - name: "should return '\\t\\t'", - args: args{ - size: 2, - useSpace: false, - }, - want: "\t\t", - wantErr: false, - }, - { - name: "should return ' '", - args: args{ - size: 4, - useSpace: true, - }, - want: " ", - wantErr: false, - }, - { - name: "should return ''", - args: args{ - size: 0, - useSpace: false, - }, - want: "", - wantErr: false, - }, - { - name: "should return ''", - args: args{ - size: 0, - useSpace: true, - }, - want: "", - wantErr: false, - }, - { - name: "should throw error because indentation size < 0", - args: args{ - size: -1, - useSpace: false, - }, - want: "", - wantErr: true, - }, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - got, err := preferredIndentationStr(tt.args.size, tt.args.useSpace) - if tt.wantErr { - require.Errorf(t, err, "preferredIndentationStr(%v, %v)", tt.args.size, tt.args.useSpace) - } else { - require.NoError(t, err) - assert.Equalf(t, tt.want, got, "preferredIndentationStr(%v, %v)", tt.args.size, tt.args.useSpace) - } - }) - } -} diff --git a/cmd/compose/volumes.go b/cmd/compose/volumes.go deleted file mode 100644 index e0da4f82e3b..00000000000 --- a/cmd/compose/volumes.go +++ /dev/null @@ -1,98 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "slices" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/command/formatter" - "github.com/docker/cli/cli/flags" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type volumesOptions struct { - *ProjectOptions - Quiet bool - Format string -} - -func volumesCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - options := volumesOptions{ - ProjectOptions: p, - } - - cmd := &cobra.Command{ - Use: "volumes [OPTIONS] [SERVICE...]", - Short: "List volumes", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runVol(ctx, dockerCli, backendOptions, args, options) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - cmd.Flags().BoolVarP(&options.Quiet, "quiet", "q", false, "Only display volume names") - cmd.Flags().StringVar(&options.Format, "format", "table", flags.FormatHelp) - - return cmd -} - -func runVol(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, services []string, options volumesOptions) error { - project, name, err := options.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - if project != nil { - names := project.ServiceNames() - for _, service := range services { - if !slices.Contains(names, service) { - return fmt.Errorf("no such service: %s", service) - } - } - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - volumes, err := backend.Volumes(ctx, name, api.VolumesOptions{ - Services: services, - }) - if err != nil { - return err - } - - if options.Quiet { - for _, v := range volumes { - _, _ = fmt.Fprintln(dockerCli.Out(), v.Name) - } - return nil - } - - volumeCtx := formatter.Context{ - Output: dockerCli.Out(), - Format: formatter.NewVolumeFormat(options.Format, options.Quiet), - } - - return formatter.VolumeWrite(volumeCtx, volumes) -} diff --git a/cmd/compose/wait.go b/cmd/compose/wait.go deleted file mode 100644 index 9d86fd314cf..00000000000 --- a/cmd/compose/wait.go +++ /dev/null @@ -1,79 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "os" - - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type waitOptions struct { - *ProjectOptions - - services []string - - downProject bool -} - -func waitCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := waitOptions{ - ProjectOptions: p, - } - - var statusCode int64 - var err error - cmd := &cobra.Command{ - Use: "wait SERVICE [SERVICE...] [OPTIONS]", - Short: "Block until containers of all (or specified) services stop.", - Args: cli.RequiresMinArgs(1), - RunE: Adapt(func(ctx context.Context, services []string) error { - opts.services = services - statusCode, err = runWait(ctx, dockerCli, backendOptions, &opts) - return err - }), - PostRun: func(cmd *cobra.Command, args []string) { - os.Exit(int(statusCode)) - }, - } - - cmd.Flags().BoolVar(&opts.downProject, "down-project", false, "Drops project when the first container stops") - - return cmd -} - -func runWait(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts *waitOptions) (int64, error) { - _, name, err := opts.projectOrName(ctx, dockerCli) - if err != nil { - return 0, err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return 0, err - } - return backend.Wait(ctx, name, api.WaitOptions{ - Services: opts.services, - DownProjectOnContainerExit: opts.downProject, - }) -} diff --git a/cmd/compose/watch.go b/cmd/compose/watch.go deleted file mode 100644 index c60b243860e..00000000000 --- a/cmd/compose/watch.go +++ /dev/null @@ -1,132 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/internal/locker" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type watchOptions struct { - *ProjectOptions - prune bool - noUp bool -} - -func watchCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - watchOpts := watchOptions{ - ProjectOptions: p, - } - buildOpts := buildOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "watch [SERVICE...]", - Short: "Watch build context for service and rebuild/refresh containers when files are updated", - PreRunE: Adapt(func(ctx context.Context, args []string) error { - return nil - }), - RunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - if cmd.Parent().Name() == "alpha" { - logrus.Warn("watch command is now available as a top level command") - } - return runWatch(ctx, dockerCli, backendOptions, watchOpts, buildOpts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - cmd.Flags().BoolVar(&buildOpts.quiet, "quiet", false, "hide build output") - cmd.Flags().BoolVar(&watchOpts.prune, "prune", true, "Prune dangling images on rebuild") - cmd.Flags().BoolVar(&watchOpts.noUp, "no-up", false, "Do not build & start services before watching") - return cmd -} - -func runWatch(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, watchOpts watchOptions, buildOpts buildOptions, services []string) error { - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - project, _, err := watchOpts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return err - } - - if err := applyPlatforms(project, true); err != nil { - return err - } - - build, err := buildOpts.toAPIBuildOptions(nil) - if err != nil { - return err - } - - // validation done -- ensure we have the lockfile for this project before doing work - l, err := locker.NewPidfile(project.Name) - if err != nil { - return fmt.Errorf("cannot take exclusive lock for project %q: %w", project.Name, err) - } - if err := l.Lock(); err != nil { - return fmt.Errorf("cannot take exclusive lock for project %q: %w", project.Name, err) - } - - if !watchOpts.noUp { - for index, service := range project.Services { - if service.Build != nil && service.Develop != nil { - service.PullPolicy = types.PullPolicyBuild - } - project.Services[index] = service - } - upOpts := api.UpOptions{ - Create: api.CreateOptions{ - Build: &build, - Services: services, - RemoveOrphans: false, - Recreate: api.RecreateDiverged, - RecreateDependencies: api.RecreateNever, - Inherit: true, - QuietPull: buildOpts.quiet, - }, - Start: api.StartOptions{ - Project: project, - Attach: nil, - Services: services, - }, - } - if err := backend.Up(ctx, project, upOpts); err != nil { - return err - } - } - - consumer := formatter.NewLogConsumer(ctx, dockerCli.Out(), dockerCli.Err(), false, false, false) - return backend.Watch(ctx, project, api.WatchOptions{ - Build: &build, - LogTo: consumer, - Prune: watchOpts.prune, - Services: services, - }) -} diff --git a/cmd/display/colors.go b/cmd/display/colors.go deleted file mode 100644 index a00b4ed6ad5..00000000000 --- a/cmd/display/colors.go +++ /dev/null @@ -1,47 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "github.com/morikuni/aec" -) - -type colorFunc func(string) string - -var ( - nocolor colorFunc = func(s string) string { - return s - } - - DoneColor colorFunc = aec.BlueF.Apply - TimerColor colorFunc = aec.BlueF.Apply - CountColor colorFunc = aec.YellowF.Apply - WarningColor colorFunc = aec.YellowF.With(aec.Bold).Apply - SuccessColor colorFunc = aec.GreenF.Apply - ErrorColor colorFunc = aec.RedF.With(aec.Bold).Apply - PrefixColor colorFunc = aec.CyanF.Apply -) - -func NoColor() { - DoneColor = nocolor - TimerColor = nocolor - CountColor = nocolor - WarningColor = nocolor - SuccessColor = nocolor - ErrorColor = nocolor - PrefixColor = nocolor -} diff --git a/cmd/display/dryrun.go b/cmd/display/dryrun.go deleted file mode 100644 index 2ab542e5b05..00000000000 --- a/cmd/display/dryrun.go +++ /dev/null @@ -1,21 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -const ( - DRYRUN_PREFIX = " DRY-RUN MODE - " -) diff --git a/cmd/display/json.go b/cmd/display/json.go deleted file mode 100644 index b8873596374..00000000000 --- a/cmd/display/json.go +++ /dev/null @@ -1,81 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "context" - "encoding/json" - "fmt" - "io" - - "github.com/docker/compose/v5/pkg/api" -) - -func JSON(out io.Writer) api.EventProcessor { - return &jsonWriter{ - out: out, - } -} - -type jsonWriter struct { - out io.Writer - dryRun bool -} - -type jsonMessage struct { - DryRun bool `json:"dry-run,omitempty"` - Tail bool `json:"tail,omitempty"` - ID string `json:"id,omitempty"` - ParentID string `json:"parent_id,omitempty"` - Status string `json:"status,omitempty"` - Text string `json:"text,omitempty"` - Details string `json:"details,omitempty"` - Current int64 `json:"current,omitempty"` - Total int64 `json:"total,omitempty"` - Percent int `json:"percent,omitempty"` -} - -func (p *jsonWriter) Start(ctx context.Context, operation string) { -} - -func (p *jsonWriter) Event(e api.Resource) { - message := &jsonMessage{ - DryRun: p.dryRun, - Tail: false, - ID: e.ID, - Status: e.StatusText(), - Text: e.Text, - Details: e.Details, - ParentID: e.ParentID, - Current: e.Current, - Total: e.Total, - Percent: e.Percent, - } - marshal, err := json.Marshal(message) - if err == nil { - _, _ = fmt.Fprintln(p.out, string(marshal)) - } -} - -func (p *jsonWriter) On(events ...api.Resource) { - for _, e := range events { - p.Event(e) - } -} - -func (p *jsonWriter) Done(_ string, _ bool) { -} diff --git a/cmd/display/json_test.go b/cmd/display/json_test.go deleted file mode 100644 index 0f0dff23a61..00000000000 --- a/cmd/display/json_test.go +++ /dev/null @@ -1,62 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "bytes" - "encoding/json" - "testing" - - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestJsonWriter_Event(t *testing.T) { - var out bytes.Buffer - w := &jsonWriter{ - out: &out, - dryRun: true, - } - - event := api.Resource{ - ID: "service1", - ParentID: "project", - Status: api.Working, - Text: api.StatusCreating, - Current: 50, - Total: 100, - Percent: 50, - } - w.Event(event) - - var actual jsonMessage - err := json.Unmarshal(out.Bytes(), &actual) - assert.NilError(t, err) - - expected := jsonMessage{ - DryRun: true, - ID: event.ID, - ParentID: event.ParentID, - Text: api.StatusCreating, - Status: "Working", - Current: event.Current, - Total: event.Total, - Percent: event.Percent, - } - assert.DeepEqual(t, expected, actual) -} diff --git a/cmd/display/mode.go b/cmd/display/mode.go deleted file mode 100644 index d66777b472c..00000000000 --- a/cmd/display/mode.go +++ /dev/null @@ -1,33 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -// Mode define how progress should be rendered, either as ModePlain or ModeTTY -var Mode = ModeAuto - -const ( - // ModeAuto detect console capabilities - ModeAuto = "auto" - // ModeTTY use terminal capability for advanced rendering - ModeTTY = "tty" - // ModePlain dump raw events to output - ModePlain = "plain" - // ModeQuiet don't display events - ModeQuiet = "quiet" - // ModeJSON outputs a machine-readable JSON stream - ModeJSON = "json" -) diff --git a/cmd/display/plain.go b/cmd/display/plain.go deleted file mode 100644 index 16f2816c011..00000000000 --- a/cmd/display/plain.go +++ /dev/null @@ -1,56 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "context" - "fmt" - "io" - - "github.com/docker/compose/v5/pkg/api" -) - -func Plain(out io.Writer) api.EventProcessor { - return &plainWriter{ - out: out, - } -} - -type plainWriter struct { - out io.Writer - dryRun bool -} - -func (p *plainWriter) Start(ctx context.Context, operation string) { -} - -func (p *plainWriter) Event(e api.Resource) { - prefix := "" - if p.dryRun { - prefix = DRYRUN_PREFIX - } - _, _ = fmt.Fprintln(p.out, prefix, e.ID, e.Text, e.Details) -} - -func (p *plainWriter) On(events ...api.Resource) { - for _, e := range events { - p.Event(e) - } -} - -func (p *plainWriter) Done(_ string, _ bool) { -} diff --git a/cmd/display/quiet.go b/cmd/display/quiet.go deleted file mode 100644 index 8e1537d8061..00000000000 --- a/cmd/display/quiet.go +++ /dev/null @@ -1,38 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "context" - - "github.com/docker/compose/v5/pkg/api" -) - -func Quiet() api.EventProcessor { - return &quiet{} -} - -type quiet struct{} - -func (q *quiet) Start(_ context.Context, _ string) { -} - -func (q *quiet) Done(_ string, _ bool) { -} - -func (q *quiet) On(_ ...api.Resource) { -} diff --git a/cmd/display/spinner.go b/cmd/display/spinner.go deleted file mode 100644 index e476deae80f..00000000000 --- a/cmd/display/spinner.go +++ /dev/null @@ -1,70 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "runtime" - "time" -) - -type Spinner struct { - time time.Time - index int - chars []string - stop bool - done string -} - -func NewSpinner() *Spinner { - chars := []string{ - "⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏", - } - done := "⠿" - - if runtime.GOOS == "windows" { - chars = []string{"-"} - done = "-" - } - - return &Spinner{ - index: 0, - time: time.Now(), - chars: chars, - done: done, - } -} - -func (s *Spinner) String() string { - if s.stop { - return s.done - } - - d := time.Since(s.time) - if d.Milliseconds() > 100 { - s.index = (s.index + 1) % len(s.chars) - } - - return s.chars[s.index] -} - -func (s *Spinner) Stop() { - s.stop = true -} - -func (s *Spinner) Restart() { - s.stop = false -} diff --git a/cmd/display/tty.go b/cmd/display/tty.go deleted file mode 100644 index 3a69ebaadc7..00000000000 --- a/cmd/display/tty.go +++ /dev/null @@ -1,664 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "context" - "fmt" - "io" - "iter" - "slices" - "strings" - "sync" - "time" - "unicode/utf8" - - "github.com/buger/goterm" - "github.com/docker/go-units" - "github.com/morikuni/aec" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -// Full creates an EventProcessor that render advanced UI within a terminal. -// On Start, TUI lists task with a progress timer -func Full(out io.Writer, info io.Writer, detached bool) api.EventProcessor { - return &ttyWriter{ - out: out, - info: info, - tasks: map[string]*task{}, - done: make(chan bool), - mtx: &sync.Mutex{}, - detached: detached, - } -} - -type ttyWriter struct { - out io.Writer - ids []string // tasks ids ordered as first event appeared - tasks map[string]*task - repeated bool - numLines int - done chan bool - mtx *sync.Mutex - dryRun bool // FIXME(ndeloof) (re)implement support for dry-run - operation string - ticker *time.Ticker - suspended bool - info io.Writer - detached bool -} - -type task struct { - ID string - parent string // the resource this task receives updates from - other parents will be ignored - parents utils.Set[string] // all resources to depend on this task - startTime time.Time - endTime time.Time - text string - details string - status api.EventStatus - current int64 - percent int - total int64 - spinner *Spinner -} - -func newTask(e api.Resource) task { - t := task{ - ID: e.ID, - parents: utils.NewSet[string](), - startTime: time.Now(), - text: e.Text, - details: e.Details, - status: e.Status, - current: e.Current, - percent: e.Percent, - total: e.Total, - spinner: NewSpinner(), - } - if e.ParentID != "" { - t.parent = e.ParentID - t.parents.Add(e.ParentID) - } - if e.Status == api.Done || e.Status == api.Error { - t.stop() - } - return t -} - -// update adjusts task state based on last received event -func (t *task) update(e api.Resource) { - if e.ParentID != "" { - t.parents.Add(e.ParentID) - // we may receive same event from distinct parents (typically: images sharing layers) - // to avoid status to flicker, only accept updates from our first declared parent - if t.parent != e.ParentID { - return - } - } - - // update task based on received event - switch e.Status { - case api.Done, api.Error, api.Warning: - if t.status != e.Status { - t.stop() - } - case api.Working: - t.hasMore() - } - t.status = e.Status - t.text = e.Text - t.details = e.Details - // progress can only go up - if e.Total > t.total { - t.total = e.Total - } - if e.Current > t.current { - t.current = e.Current - } - if e.Percent > t.percent { - t.percent = e.Percent - } -} - -func (t *task) stop() { - t.endTime = time.Now() - t.spinner.Stop() -} - -func (t *task) hasMore() { - t.spinner.Restart() -} - -func (t *task) Completed() bool { - switch t.status { - case api.Done, api.Error, api.Warning: - return true - default: - return false - } -} - -func (w *ttyWriter) Start(ctx context.Context, operation string) { - w.ticker = time.NewTicker(100 * time.Millisecond) - w.operation = operation - go func() { - for { - select { - case <-ctx.Done(): - // interrupted - w.ticker.Stop() - return - case <-w.done: - return - case <-w.ticker.C: - w.print() - } - } - }() -} - -func (w *ttyWriter) Done(operation string, success bool) { - w.print() - w.mtx.Lock() - defer w.mtx.Unlock() - w.ticker.Stop() - w.operation = "" - w.done <- true -} - -func (w *ttyWriter) On(events ...api.Resource) { - w.mtx.Lock() - defer w.mtx.Unlock() - for _, e := range events { - if e.ID == "Compose" { - _, _ = fmt.Fprintln(w.info, ErrorColor(e.Details)) - continue - } - - if w.operation != "start" && (e.Text == api.StatusStarted || e.Text == api.StatusStarting) && !w.detached { - // skip those events to avoid mix with container logs - continue - } - w.event(e) - } -} - -func (w *ttyWriter) event(e api.Resource) { - // Suspend print while a build is in progress, to avoid collision with buildkit Display - if e.Text == api.StatusBuilding { - w.ticker.Stop() - w.suspended = true - } else if w.suspended { - w.ticker.Reset(100 * time.Millisecond) - w.suspended = false - } - - if last, ok := w.tasks[e.ID]; ok { - last.update(e) - } else { - t := newTask(e) - w.tasks[e.ID] = &t - w.ids = append(w.ids, e.ID) - } - w.printEvent(e) -} - -func (w *ttyWriter) printEvent(e api.Resource) { - if w.operation != "" { - // event will be displayed by progress UI on ticker's ticks - return - } - - var color colorFunc - switch e.Status { - case api.Working: - color = SuccessColor - case api.Done: - color = SuccessColor - case api.Warning: - color = WarningColor - case api.Error: - color = ErrorColor - } - _, _ = fmt.Fprintf(w.out, "%s %s %s\n", e.ID, color(e.Text), e.Details) -} - -func (w *ttyWriter) parentTasks() iter.Seq[*task] { - return func(yield func(*task) bool) { - for _, id := range w.ids { // iterate on ids to enforce a consistent order - t := w.tasks[id] - if len(t.parents) == 0 { - yield(t) - } - } - } -} - -func (w *ttyWriter) childrenTasks(parent string) iter.Seq[*task] { - return func(yield func(*task) bool) { - for _, id := range w.ids { // iterate on ids to enforce a consistent order - t := w.tasks[id] - if t.parents.Has(parent) { - yield(t) - } - } - } -} - -// lineData holds pre-computed formatting for a task line -type lineData struct { - spinner string // rendered spinner with color - prefix string // dry-run prefix if any - taskID string // possibly abbreviated - progress string // progress bar and size info - status string // rendered status with color - details string // possibly abbreviated - timer string // rendered timer with color - statusPad int // padding before status to align - timerPad int // padding before timer to align - statusColor colorFunc -} - -func (w *ttyWriter) print() { - terminalWidth := goterm.Width() - terminalHeight := goterm.Height() - if terminalWidth <= 0 { - terminalWidth = 80 - } - if terminalHeight <= 0 { - terminalHeight = 24 - } - w.printWithDimensions(terminalWidth, terminalHeight) -} - -func (w *ttyWriter) printWithDimensions(terminalWidth, terminalHeight int) { - w.mtx.Lock() - defer w.mtx.Unlock() - if len(w.tasks) == 0 { - return - } - - up := w.numLines + 1 - if !w.repeated { - up-- - w.repeated = true - } - b := aec.NewBuilder( - aec.Hide, // Hide the cursor while we are printing - aec.Up(uint(up)), - aec.Column(0), - ) - _, _ = fmt.Fprint(w.out, b.ANSI) - defer func() { - _, _ = fmt.Fprint(w.out, aec.Show) - }() - - firstLine := fmt.Sprintf("[+] %s %d/%d", w.operation, numDone(w.tasks), len(w.tasks)) - _, _ = fmt.Fprintln(w.out, firstLine) - - // Collect parent tasks in original order - allTasks := slices.Collect(w.parentTasks()) - - // Available lines: terminal height - 2 (header line + potential "more" line) - maxLines := terminalHeight - 2 - if maxLines < 1 { - maxLines = 1 - } - - showMore := len(allTasks) > maxLines - tasksToShow := allTasks - if showMore { - tasksToShow = allTasks[:maxLines-1] // Reserve one line for "more" message - } - - // collect line data and compute timerLen - lines := make([]lineData, len(tasksToShow)) - var timerLen int - for i, t := range tasksToShow { - lines[i] = w.prepareLineData(t) - if len(lines[i].timer) > timerLen { - timerLen = len(lines[i].timer) - } - } - - // shorten details/taskID to fit terminal width - w.adjustLineWidth(lines, timerLen, terminalWidth) - - // compute padding - w.applyPadding(lines, terminalWidth, timerLen) - - // Render lines - numLines := 0 - for _, l := range lines { - _, _ = fmt.Fprint(w.out, lineText(l)) - numLines++ - } - - if showMore { - moreCount := len(allTasks) - len(tasksToShow) - moreText := fmt.Sprintf(" ... %d more", moreCount) - pad := terminalWidth - len(moreText) - if pad < 0 { - pad = 0 - } - _, _ = fmt.Fprintf(w.out, "%s%s\n", moreText, strings.Repeat(" ", pad)) - numLines++ - } - - // Clear any remaining lines from previous render - for i := numLines; i < w.numLines; i++ { - _, _ = fmt.Fprintln(w.out, strings.Repeat(" ", terminalWidth)) - numLines++ - } - w.numLines = numLines -} - -func (w *ttyWriter) applyPadding(lines []lineData, terminalWidth int, timerLen int) { - var maxBeforeStatus int - for i := range lines { - l := &lines[i] - // Width before statusPad: space(1) + spinner(1) + prefix + space(1) + taskID + progress - beforeStatus := 3 + lenAnsi(l.prefix) + utf8.RuneCountInString(l.taskID) + lenAnsi(l.progress) - if beforeStatus > maxBeforeStatus { - maxBeforeStatus = beforeStatus - } - } - - for i, l := range lines { - // Position before statusPad: space(1) + spinner(1) + prefix + space(1) + taskID + progress - beforeStatus := 3 + lenAnsi(l.prefix) + utf8.RuneCountInString(l.taskID) + lenAnsi(l.progress) - // statusPad aligns status; lineText adds 1 more space after statusPad - l.statusPad = maxBeforeStatus - beforeStatus - - // Format: beforeStatus + statusPad + space(1) + status - lineLen := beforeStatus + l.statusPad + 1 + utf8.RuneCountInString(l.status) - if l.details != "" { - lineLen += 1 + utf8.RuneCountInString(l.details) - } - l.timerPad = terminalWidth - lineLen - timerLen - if l.timerPad < 1 { - l.timerPad = 1 - } - lines[i] = l - - } -} - -func (w *ttyWriter) adjustLineWidth(lines []lineData, timerLen int, terminalWidth int) { - const minIDLen = 10 - maxStatusLen := maxStatusLength(lines) - - // Iteratively truncate until all lines fit - for range 100 { // safety limit - maxBeforeStatus := maxBeforeStatusWidth(lines) - overflow := computeOverflow(lines, maxBeforeStatus, maxStatusLen, timerLen, terminalWidth) - - if overflow <= 0 { - break - } - - // First try to truncate details, then taskID - if !truncateDetails(lines, overflow) && !truncateLongestTaskID(lines, overflow, minIDLen) { - break // Can't truncate further - } - } -} - -// maxStatusLength returns the maximum status text length across all lines. -func maxStatusLength(lines []lineData) int { - var maxLen int - for i := range lines { - if len(lines[i].status) > maxLen { - maxLen = len(lines[i].status) - } - } - return maxLen -} - -// maxBeforeStatusWidth computes the maximum width before statusPad across all lines. -// This is: space(1) + spinner(1) + prefix + space(1) + taskID + progress -func maxBeforeStatusWidth(lines []lineData) int { - var maxWidth int - for i := range lines { - l := &lines[i] - width := 3 + lenAnsi(l.prefix) + len(l.taskID) + lenAnsi(l.progress) - if width > maxWidth { - maxWidth = width - } - } - return maxWidth -} - -// computeOverflow calculates how many characters the widest line exceeds the terminal width. -// Returns 0 or negative if all lines fit. -func computeOverflow(lines []lineData, maxBeforeStatus, maxStatusLen, timerLen, terminalWidth int) int { - var maxOverflow int - for i := range lines { - l := &lines[i] - detailsLen := len(l.details) - if detailsLen > 0 { - detailsLen++ // space before details - } - // Line width: maxBeforeStatus + space(1) + status + details + minTimerPad(1) + timer - lineWidth := maxBeforeStatus + 1 + maxStatusLen + detailsLen + 1 + timerLen - overflow := lineWidth - terminalWidth - if overflow > maxOverflow { - maxOverflow = overflow - } - } - return maxOverflow -} - -// truncateDetails tries to truncate the first line's details to reduce overflow. -// Returns true if any truncation was performed. -func truncateDetails(lines []lineData, overflow int) bool { - for i := range lines { - l := &lines[i] - if len(l.details) > 3 { - reduction := overflow - if reduction > len(l.details)-3 { - reduction = len(l.details) - 3 - } - l.details = l.details[:len(l.details)-reduction-3] + "..." - return true - } else if l.details != "" { - l.details = "" - return true - } - } - return false -} - -// truncateLongestTaskID truncates the longest taskID to reduce overflow. -// Returns true if truncation was performed. -func truncateLongestTaskID(lines []lineData, overflow, minIDLen int) bool { - longestIdx := -1 - longestLen := minIDLen - for i := range lines { - if len(lines[i].taskID) > longestLen { - longestLen = len(lines[i].taskID) - longestIdx = i - } - } - - if longestIdx < 0 { - return false - } - - l := &lines[longestIdx] - reduction := overflow + 3 // account for "..." - newLen := len(l.taskID) - reduction - if newLen < minIDLen-3 { - newLen = minIDLen - 3 - } - if newLen > 0 { - l.taskID = l.taskID[:newLen] + "..." - } - return true -} - -func (w *ttyWriter) prepareLineData(t *task) lineData { - endTime := time.Now() - if t.status != api.Working { - endTime = t.startTime - if (t.endTime != time.Time{}) { - endTime = t.endTime - } - } - - prefix := "" - if w.dryRun { - prefix = PrefixColor(DRYRUN_PREFIX) - } - - elapsed := endTime.Sub(t.startTime).Seconds() - - var ( - hideDetails bool - total int64 - current int64 - completion []string - ) - - // only show the aggregated progress while the root operation is in-progress - if t.status == api.Working { - for child := range w.childrenTasks(t.ID) { - if child.status == api.Working && child.total == 0 { - hideDetails = true - } - total += child.total - current += child.current - r := len(percentChars) - 1 - p := child.percent - if p > 100 { - p = 100 - } - completion = append(completion, percentChars[r*p/100]) - } - } - - if total == 0 { - hideDetails = true - } - - var progress string - if len(completion) > 0 { - progress = " [" + SuccessColor(strings.Join(completion, "")) + "]" - if !hideDetails { - progress += fmt.Sprintf(" %7s / %-7s", units.HumanSize(float64(current)), units.HumanSize(float64(total))) - } - } - - return lineData{ - spinner: spinner(t), - prefix: prefix, - taskID: t.ID, - progress: progress, - status: t.text, - statusColor: colorFn(t.status), - details: t.details, - timer: fmt.Sprintf("%.1fs", elapsed), - } -} - -func lineText(l lineData) string { - var sb strings.Builder - sb.WriteString(" ") - sb.WriteString(l.spinner) - sb.WriteString(l.prefix) - sb.WriteString(" ") - sb.WriteString(l.taskID) - sb.WriteString(l.progress) - sb.WriteString(strings.Repeat(" ", l.statusPad)) - sb.WriteString(" ") - sb.WriteString(l.statusColor(l.status)) - if l.details != "" { - sb.WriteString(" ") - sb.WriteString(l.details) - } - sb.WriteString(strings.Repeat(" ", l.timerPad)) - sb.WriteString(TimerColor(l.timer)) - sb.WriteString("\n") - return sb.String() -} - -var ( - spinnerDone = "✔" - spinnerWarning = "!" - spinnerError = "✘" -) - -func spinner(t *task) string { - switch t.status { - case api.Done: - return SuccessColor(spinnerDone) - case api.Warning: - return WarningColor(spinnerWarning) - case api.Error: - return ErrorColor(spinnerError) - default: - return CountColor(t.spinner.String()) - } -} - -func colorFn(s api.EventStatus) colorFunc { - switch s { - case api.Done: - return SuccessColor - case api.Warning: - return WarningColor - case api.Error: - return ErrorColor - default: - return nocolor - } -} - -func numDone(tasks map[string]*task) int { - i := 0 - for _, t := range tasks { - if t.status != api.Working { - i++ - } - } - return i -} - -// lenAnsi count of user-perceived characters in ANSI string. -func lenAnsi(s string) int { - length := 0 - ansiCode := false - for _, r := range s { - if r == '\x1b' { - ansiCode = true - continue - } - if ansiCode && r == 'm' { - ansiCode = false - continue - } - if !ansiCode { - length++ - } - } - return length -} - -var percentChars = strings.Split("⠀⡀⣀⣄⣤⣦⣶⣷⣿", "") diff --git a/cmd/display/tty_test.go b/cmd/display/tty_test.go deleted file mode 100644 index 875f5f029f7..00000000000 --- a/cmd/display/tty_test.go +++ /dev/null @@ -1,424 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "bytes" - "strings" - "sync" - "testing" - "time" - "unicode/utf8" - - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/api" -) - -func newTestWriter() (*ttyWriter, *bytes.Buffer) { - var buf bytes.Buffer - w := &ttyWriter{ - out: &buf, - info: &buf, - tasks: map[string]*task{}, - done: make(chan bool), - mtx: &sync.Mutex{}, - operation: "pull", - } - return w, &buf -} - -func addTask(w *ttyWriter, id, text, details string, status api.EventStatus) { - t := &task{ - ID: id, - parents: make(map[string]struct{}), - startTime: time.Now(), - text: text, - details: details, - status: status, - spinner: NewSpinner(), - } - w.tasks[id] = t - w.ids = append(w.ids, id) -} - -// extractLines parses the output buffer and returns lines without ANSI control sequences -func extractLines(buf *bytes.Buffer) []string { - content := buf.String() - // Split by newline - rawLines := strings.Split(content, "\n") - var lines []string - for _, line := range rawLines { - // Skip empty lines and lines that are just ANSI codes - if lenAnsi(line) > 0 { - lines = append(lines, line) - } - } - return lines -} - -func TestPrintWithDimensions_LinesFitTerminalWidth(t *testing.T) { - testCases := []struct { - name string - taskID string - status string - details string - terminalWidth int - }{ - { - name: "short task fits wide terminal", - taskID: "Image foo", - status: "Pulling", - details: "layer abc123", - terminalWidth: 100, - }, - { - name: "long details truncated to fit", - taskID: "Image foo", - status: "Pulling", - details: "downloading layer sha256:abc123def456789xyz0123456789abcdef", - terminalWidth: 50, - }, - { - name: "long taskID truncated to fit", - taskID: "very-long-image-name-that-exceeds-terminal-width", - status: "Pulling", - details: "", - terminalWidth: 40, - }, - { - name: "both long taskID and details", - taskID: "my-very-long-service-name-here", - status: "Downloading", - details: "layer sha256:abc123def456789xyz0123456789", - terminalWidth: 50, - }, - { - name: "narrow terminal", - taskID: "service-name", - status: "Pulling", - details: "some details", - terminalWidth: 35, - }, - } - - for _, tc := range testCases { - t.Run(tc.name, func(t *testing.T) { - w, buf := newTestWriter() - addTask(w, tc.taskID, tc.status, tc.details, api.Working) - - w.printWithDimensions(tc.terminalWidth, 24) - - lines := extractLines(buf) - for i, line := range lines { - lineLen := lenAnsi(line) - assert.Assert(t, lineLen <= tc.terminalWidth, - "line %d has length %d which exceeds terminal width %d: %q", - i, lineLen, tc.terminalWidth, line) - } - }) - } -} - -func TestPrintWithDimensions_MultipleTasksFitTerminalWidth(t *testing.T) { - w, buf := newTestWriter() - - // Add multiple tasks with varying lengths - addTask(w, "Image nginx", "Pulling", "layer sha256:abc123", api.Working) - addTask(w, "Image postgres-database", "Pulling", "downloading", api.Working) - addTask(w, "Image redis", "Pulled", "", api.Done) - - terminalWidth := 60 - w.printWithDimensions(terminalWidth, 24) - - lines := extractLines(buf) - for i, line := range lines { - lineLen := lenAnsi(line) - assert.Assert(t, lineLen <= terminalWidth, - "line %d has length %d which exceeds terminal width %d: %q", - i, lineLen, terminalWidth, line) - } -} - -func TestPrintWithDimensions_VeryNarrowTerminal(t *testing.T) { - w, buf := newTestWriter() - addTask(w, "Image nginx", "Pulling", "details", api.Working) - - terminalWidth := 30 - w.printWithDimensions(terminalWidth, 24) - - lines := extractLines(buf) - for i, line := range lines { - lineLen := lenAnsi(line) - assert.Assert(t, lineLen <= terminalWidth, - "line %d has length %d which exceeds terminal width %d: %q", - i, lineLen, terminalWidth, line) - } -} - -func TestPrintWithDimensions_TaskWithProgress(t *testing.T) { - w, buf := newTestWriter() - - // Create parent task - parent := &task{ - ID: "Image nginx", - parents: make(map[string]struct{}), - startTime: time.Now(), - text: "Pulling", - status: api.Working, - spinner: NewSpinner(), - } - w.tasks["Image nginx"] = parent - w.ids = append(w.ids, "Image nginx") - - // Create child tasks to trigger progress display - for i := 0; i < 3; i++ { - child := &task{ - ID: "layer" + string(rune('a'+i)), - parents: map[string]struct{}{"Image nginx": {}}, - startTime: time.Now(), - text: "Downloading", - status: api.Working, - total: 1000, - current: 500, - percent: 50, - spinner: NewSpinner(), - } - w.tasks[child.ID] = child - w.ids = append(w.ids, child.ID) - } - - terminalWidth := 80 - w.printWithDimensions(terminalWidth, 24) - - lines := extractLines(buf) - for i, line := range lines { - lineLen := lenAnsi(line) - assert.Assert(t, lineLen <= terminalWidth, - "line %d has length %d which exceeds terminal width %d: %q", - i, lineLen, terminalWidth, line) - } -} - -func TestAdjustLineWidth_DetailsCorrectlyTruncated(t *testing.T) { - w := &ttyWriter{} - lines := []lineData{ - { - taskID: "Image foo", - status: "Pulling", - details: "downloading layer sha256:abc123def456789xyz", - }, - } - - terminalWidth := 50 - timerLen := 5 - w.adjustLineWidth(lines, timerLen, terminalWidth) - - // Verify the line fits - detailsLen := len(lines[0].details) - if detailsLen > 0 { - detailsLen++ // space before details - } - // widthWithoutDetails = 5 + prefix(0) + taskID(9) + progress(0) + status(7) + timer(5) = 26 - lineWidth := 5 + len(lines[0].taskID) + len(lines[0].status) + detailsLen + timerLen - - assert.Assert(t, lineWidth <= terminalWidth, - "line width %d should not exceed terminal width %d (taskID=%q, details=%q)", - lineWidth, terminalWidth, lines[0].taskID, lines[0].details) - - // Verify details were truncated (not removed entirely) - assert.Assert(t, lines[0].details != "", "details should be truncated, not removed") - assert.Assert(t, strings.HasSuffix(lines[0].details, "..."), "truncated details should end with ...") -} - -func TestAdjustLineWidth_TaskIDCorrectlyTruncated(t *testing.T) { - w := &ttyWriter{} - lines := []lineData{ - { - taskID: "very-long-image-name-that-exceeds-minimum-length", - status: "Pulling", - details: "", - }, - } - - terminalWidth := 40 - timerLen := 5 - w.adjustLineWidth(lines, timerLen, terminalWidth) - - lineWidth := 5 + len(lines[0].taskID) + 7 + timerLen - - assert.Assert(t, lineWidth <= terminalWidth, - "line width %d should not exceed terminal width %d (taskID=%q)", - lineWidth, terminalWidth, lines[0].taskID) - - assert.Assert(t, strings.HasSuffix(lines[0].taskID, "..."), "truncated taskID should end with ...") -} - -func TestAdjustLineWidth_NoTruncationNeeded(t *testing.T) { - w := &ttyWriter{} - originalDetails := "short" - originalTaskID := "Image foo" - lines := []lineData{ - { - taskID: originalTaskID, - status: "Pulling", - details: originalDetails, - }, - } - - // Wide terminal, nothing should be truncated - w.adjustLineWidth(lines, 5, 100) - - assert.Equal(t, originalTaskID, lines[0].taskID, "taskID should not be modified") - assert.Equal(t, originalDetails, lines[0].details, "details should not be modified") -} - -func TestAdjustLineWidth_DetailsRemovedWhenTooShort(t *testing.T) { - w := &ttyWriter{} - lines := []lineData{ - { - taskID: "Image foo", - status: "Pulling", - details: "abc", // Very short, can't be meaningfully truncated - }, - } - - // Terminal so narrow that even minimal details + "..." wouldn't help - w.adjustLineWidth(lines, 5, 28) - - assert.Equal(t, "", lines[0].details, "details should be removed entirely when too short to truncate") -} - -// stripAnsi removes ANSI escape codes from a string -func stripAnsi(s string) string { - var result strings.Builder - inAnsi := false - for _, r := range s { - if r == '\x1b' { - inAnsi = true - continue - } - if inAnsi { - // ANSI sequences end with a letter (m, h, l, G, etc.) - if (r >= 'A' && r <= 'Z') || (r >= 'a' && r <= 'z') { - inAnsi = false - } - continue - } - result.WriteRune(r) - } - return result.String() -} - -func TestPrintWithDimensions_PulledAndPullingWithLongIDs(t *testing.T) { - w, buf := newTestWriter() - - // Add a completed task with long ID - completedTask := &task{ - ID: "Image docker.io/library/nginx-long-name", - parents: make(map[string]struct{}), - startTime: time.Now().Add(-2 * time.Second), - endTime: time.Now(), - text: "Pulled", - status: api.Done, - spinner: NewSpinner(), - } - completedTask.spinner.Stop() - w.tasks[completedTask.ID] = completedTask - w.ids = append(w.ids, completedTask.ID) - - // Add a pending task with long ID - pendingTask := &task{ - ID: "Image docker.io/library/postgres-database", - parents: make(map[string]struct{}), - startTime: time.Now(), - text: "Pulling", - status: api.Working, - spinner: NewSpinner(), - } - w.tasks[pendingTask.ID] = pendingTask - w.ids = append(w.ids, pendingTask.ID) - - terminalWidth := 50 - w.printWithDimensions(terminalWidth, 24) - - // Strip all ANSI codes from output and split by newline - stripped := stripAnsi(buf.String()) - lines := strings.Split(stripped, "\n") - - // Filter non-empty lines - var nonEmptyLines []string - for _, line := range lines { - if strings.TrimSpace(line) != "" { - nonEmptyLines = append(nonEmptyLines, line) - } - } - - // Expected output format (50 runes per task line) - expected := `[+] pull 1/2 - ✔ Image docker.io/library/nginx-l... Pulled 2.0s - ⠋ Image docker.io/library/postgre... Pulling 0.0s` - - expectedLines := strings.Split(expected, "\n") - - // Debug output - t.Logf("Actual output:\n") - for i, line := range nonEmptyLines { - t.Logf(" line %d (%2d runes): %q", i, utf8.RuneCountInString(line), line) - } - - // Verify number of lines - assert.Equal(t, len(expectedLines), len(nonEmptyLines), "number of lines should match") - - // Verify each line matches expected - for i, line := range nonEmptyLines { - if i < len(expectedLines) { - assert.Equal(t, expectedLines[i], line, - "line %d should match expected", i) - } - } - - // Verify task lines fit within terminal width (strict - no tolerance) - for i, line := range nonEmptyLines { - if i > 0 { // Skip header line - runeCount := utf8.RuneCountInString(line) - assert.Assert(t, runeCount <= terminalWidth, - "line %d has %d runes which exceeds terminal width %d: %q", - i, runeCount, terminalWidth, line) - } - } -} - -func TestLenAnsi(t *testing.T) { - testCases := []struct { - input string - expected int - }{ - {"hello", 5}, - {"\x1b[32mhello\x1b[0m", 5}, - {"\x1b[1;32mgreen\x1b[0m text", 10}, - {"", 0}, - {"\x1b[0m", 0}, - } - - for _, tc := range testCases { - t.Run(tc.input, func(t *testing.T) { - result := lenAnsi(tc.input) - assert.Equal(t, tc.expected, result) - }) - } -} diff --git a/cmd/formatter/ansi.go b/cmd/formatter/ansi.go deleted file mode 100644 index ad1031946ef..00000000000 --- a/cmd/formatter/ansi.go +++ /dev/null @@ -1,99 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "fmt" - - "github.com/acarl005/stripansi" - "github.com/morikuni/aec" -) - -var disableAnsi bool - -func saveCursor() { - if disableAnsi { - return - } - // see https://github.com/morikuni/aec/pull/5 - fmt.Print(aec.Save) -} - -func restoreCursor() { - if disableAnsi { - return - } - // see https://github.com/morikuni/aec/pull/5 - fmt.Print(aec.Restore) -} - -func showCursor() { - if disableAnsi { - return - } - fmt.Print(aec.Show) -} - -func moveCursor(y, x int) { - if disableAnsi { - return - } - fmt.Print(aec.Position(uint(y), uint(x))) -} - -func carriageReturn() { - if disableAnsi { - return - } - fmt.Print(aec.Column(0)) -} - -func clearLine() { - if disableAnsi { - return - } - // Does not move cursor from its current position - fmt.Print(aec.EraseLine(aec.EraseModes.Tail)) -} - -func moveCursorUp(lines int) { - if disableAnsi { - return - } - // Does not add new lines - fmt.Print(aec.Up(uint(lines))) -} - -func moveCursorDown(lines int) { - if disableAnsi { - return - } - // Does not add new lines - fmt.Print(aec.Down(uint(lines))) -} - -func newLine() { - // Like \n - fmt.Print("\012") -} - -func lenAnsi(s string) int { - // len has into consideration ansi codes, if we want - // the len of the actual len(string) we need to strip - // all ansi codes - return len(stripansi.Strip(s)) -} diff --git a/cmd/formatter/colors.go b/cmd/formatter/colors.go deleted file mode 100644 index ea0e1a26362..00000000000 --- a/cmd/formatter/colors.go +++ /dev/null @@ -1,148 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "fmt" - "strconv" - "strings" - "sync" - - "github.com/docker/cli/cli/command" -) - -var names = []string{ - "grey", - "red", - "green", - "yellow", - "blue", - "magenta", - "cyan", - "white", -} - -const ( - BOLD = "1" - FAINT = "2" - ITALIC = "3" - UNDERLINE = "4" -) - -const ( - RESET = "0" - CYAN = "36" -) - -const ( - // Never use ANSI codes - Never = "never" - - // Always use ANSI codes - Always = "always" - - // Auto detect terminal is a tty and can use ANSI codes - Auto = "auto" -) - -// ansiColorOffset is the offset for basic foreground colors in ANSI escape codes. -const ansiColorOffset = 30 - -// SetANSIMode configure formatter for colored output on ANSI-compliant console -func SetANSIMode(streams command.Streams, ansi string) { - if !useAnsi(streams, ansi) { - nextColor = func() colorFunc { - return monochrome - } - disableAnsi = true - } -} - -func useAnsi(streams command.Streams, ansi string) bool { - switch ansi { - case Always: - return true - case Auto: - return streams.Out().IsTerminal() - } - return false -} - -// colorFunc use ANSI codes to render colored text on console -type colorFunc func(s string) string - -var monochrome = func(s string) string { - return s -} - -func ansiColor(code, s string, formatOpts ...string) string { - return fmt.Sprintf("%s%s%s", ansiColorCode(code, formatOpts...), s, ansiColorCode("0")) -} - -// Everything about ansiColorCode color https://hyperskill.org/learn/step/18193 -func ansiColorCode(code string, formatOpts ...string) string { - var sb strings.Builder - sb.WriteString("\033[") - for _, c := range formatOpts { - sb.WriteString(c) - sb.WriteString(";") - } - sb.WriteString(code) - sb.WriteString("m") - return sb.String() -} - -func makeColorFunc(code string) colorFunc { - return func(s string) string { - return ansiColor(code, s) - } -} - -var ( - nextColor = rainbowColor - rainbow []colorFunc - currentIndex = 0 - mutex sync.Mutex -) - -func rainbowColor() colorFunc { - mutex.Lock() - defer mutex.Unlock() - result := rainbow[currentIndex] - currentIndex = (currentIndex + 1) % len(rainbow) - return result -} - -func init() { - colors := map[string]colorFunc{} - for i, name := range names { - colors[name] = makeColorFunc(strconv.Itoa(ansiColorOffset + i)) - colors["intense_"+name] = makeColorFunc(strconv.Itoa(ansiColorOffset+i) + ";1") - } - rainbow = []colorFunc{ - colors["cyan"], - colors["yellow"], - colors["green"], - colors["magenta"], - colors["blue"], - colors["intense_cyan"], - colors["intense_yellow"], - colors["intense_green"], - colors["intense_magenta"], - colors["intense_blue"], - } -} diff --git a/cmd/formatter/consts.go b/cmd/formatter/consts.go deleted file mode 100644 index 0ae447c432f..00000000000 --- a/cmd/formatter/consts.go +++ /dev/null @@ -1,30 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -const ( - // JSON Print in JSON format - JSON = "json" - // TemplateLegacyJSON the legacy json formatting value using go template - TemplateLegacyJSON = "{{json.}}" - // PRETTY is the constant for default formats on list commands - // - // Deprecated: use TABLE - PRETTY = "pretty" - // TABLE Print output in table format with column headers (default) - TABLE = "table" -) diff --git a/cmd/formatter/container.go b/cmd/formatter/container.go deleted file mode 100644 index 488caf4afad..00000000000 --- a/cmd/formatter/container.go +++ /dev/null @@ -1,288 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "fmt" - "strconv" - "strings" - "time" - - "github.com/docker/cli/cli/command/formatter" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/pkg/stringid" - "github.com/docker/go-units" - - "github.com/docker/compose/v5/pkg/api" -) - -const ( - defaultContainerTableFormat = "table {{.Name}}\t{{.Image}}\t{{.Command}}\t{{.Service}}\t{{.RunningFor}}\t{{.Status}}\t{{.Ports}}" - - nameHeader = "NAME" - projectHeader = "PROJECT" - serviceHeader = "SERVICE" - commandHeader = "COMMAND" - runningForHeader = "CREATED" - mountsHeader = "MOUNTS" - localVolumes = "LOCAL VOLUMES" - networksHeader = "NETWORKS" -) - -// NewContainerFormat returns a Format for rendering using a Context -func NewContainerFormat(source string, quiet bool, size bool) formatter.Format { - switch source { - case formatter.TableFormatKey, "": // table formatting is the default if none is set. - if quiet { - return formatter.DefaultQuietFormat - } - format := defaultContainerTableFormat - if size { - format += `\t{{.Size}}` - } - return formatter.Format(format) - case formatter.RawFormatKey: - if quiet { - return `container_id: {{.ID}}` - } - format := `container_id: {{.ID}} -image: {{.Image}} -command: {{.Command}} -created_at: {{.CreatedAt}} -state: {{- pad .State 1 0}} -status: {{- pad .Status 1 0}} -names: {{.Names}} -labels: {{- pad .Labels 1 0}} -ports: {{- pad .Ports 1 0}} -` - if size { - format += `size: {{.Size}}\n` - } - return formatter.Format(format) - default: // custom format - if quiet { - return formatter.DefaultQuietFormat - } - return formatter.Format(source) - } -} - -// ContainerWrite renders the context for a list of containers -func ContainerWrite(ctx formatter.Context, containers []api.ContainerSummary) error { - render := func(format func(subContext formatter.SubContext) error) error { - for _, container := range containers { - err := format(&ContainerContext{trunc: ctx.Trunc, c: container}) - if err != nil { - return err - } - } - return nil - } - return ctx.Write(NewContainerContext(), render) -} - -// ContainerContext is a struct used for rendering a list of containers in a Go template. -type ContainerContext struct { - formatter.HeaderContext - trunc bool - c api.ContainerSummary - - // FieldsUsed is used in the pre-processing step to detect which fields are - // used in the template. It's currently only used to detect use of the .Size - // field which (if used) automatically sets the '--size' option when making - // the API call. - FieldsUsed map[string]any -} - -// NewContainerContext creates a new context for rendering containers -func NewContainerContext() *ContainerContext { - containerCtx := ContainerContext{} - containerCtx.Header = formatter.SubHeaderContext{ - "ID": formatter.ContainerIDHeader, - "Name": nameHeader, - "Project": projectHeader, - "Service": serviceHeader, - "Image": formatter.ImageHeader, - "Command": commandHeader, - "CreatedAt": formatter.CreatedAtHeader, - "RunningFor": runningForHeader, - "Ports": formatter.PortsHeader, - "State": formatter.StateHeader, - "Status": formatter.StatusHeader, - "Size": formatter.SizeHeader, - "Labels": formatter.LabelsHeader, - } - return &containerCtx -} - -// MarshalJSON makes ContainerContext implement json.Marshaler -func (c *ContainerContext) MarshalJSON() ([]byte, error) { - return formatter.MarshalJSON(c) -} - -// ID returns the container's ID as a string. Depending on the `--no-trunc` -// option being set, the full or truncated ID is returned. -func (c *ContainerContext) ID() string { - if c.trunc { - return stringid.TruncateID(c.c.ID) - } - return c.c.ID -} - -func (c *ContainerContext) Name() string { - return c.c.Name -} - -// Names returns a comma-separated string of the container's names, with their -// slash (/) prefix stripped. Additional names for the container (related to the -// legacy `--link` feature) are omitted. -func (c *ContainerContext) Names() string { - names := formatter.StripNamePrefix(c.c.Names) - if c.trunc { - for _, name := range names { - if len(strings.Split(name, "/")) == 1 { - names = []string{name} - break - } - } - } - return strings.Join(names, ",") -} - -func (c *ContainerContext) Service() string { - return c.c.Service -} - -func (c *ContainerContext) Project() string { - return c.c.Project -} - -func (c *ContainerContext) Image() string { - return c.c.Image -} - -func (c *ContainerContext) Command() string { - command := c.c.Command - if c.trunc { - command = formatter.Ellipsis(command, 20) - } - return strconv.Quote(command) -} - -func (c *ContainerContext) CreatedAt() string { - return time.Unix(c.c.Created, 0).String() -} - -func (c *ContainerContext) RunningFor() string { - createdAt := time.Unix(c.c.Created, 0) - return units.HumanDuration(time.Now().UTC().Sub(createdAt)) + " ago" -} - -func (c *ContainerContext) ExitCode() int { - return c.c.ExitCode -} - -func (c *ContainerContext) State() string { - return c.c.State -} - -func (c *ContainerContext) Status() string { - return c.c.Status -} - -func (c *ContainerContext) Health() string { - return c.c.Health -} - -func (c *ContainerContext) Publishers() api.PortPublishers { - return c.c.Publishers -} - -func (c *ContainerContext) Ports() string { - var ports []container.Port - for _, publisher := range c.c.Publishers { - ports = append(ports, container.Port{ - IP: publisher.URL, - PrivatePort: uint16(publisher.TargetPort), - PublicPort: uint16(publisher.PublishedPort), - Type: publisher.Protocol, - }) - } - return formatter.DisplayablePorts(ports) -} - -// Labels returns a comma-separated string of labels present on the container. -func (c *ContainerContext) Labels() string { - if c.c.Labels == nil { - return "" - } - - var joinLabels []string - for k, v := range c.c.Labels { - joinLabels = append(joinLabels, fmt.Sprintf("%s=%s", k, v)) - } - return strings.Join(joinLabels, ",") -} - -// Label returns the value of the label with the given name or an empty string -// if the given label does not exist. -func (c *ContainerContext) Label(name string) string { - if c.c.Labels == nil { - return "" - } - return c.c.Labels[name] -} - -// Mounts returns a comma-separated string of mount names present on the container. -// If the trunc option is set, names can be truncated (ellipsized). -func (c *ContainerContext) Mounts() string { - var mounts []string - for _, name := range c.c.Mounts { - if c.trunc { - name = formatter.Ellipsis(name, 15) - } - mounts = append(mounts, name) - } - return strings.Join(mounts, ",") -} - -// LocalVolumes returns the number of volumes using the "local" volume driver. -func (c *ContainerContext) LocalVolumes() string { - return fmt.Sprintf("%d", c.c.LocalVolumes) -} - -// Networks returns a comma-separated string of networks that the container is -// attached to. -func (c *ContainerContext) Networks() string { - return strings.Join(c.c.Networks, ",") -} - -// Size returns the container's size and virtual size (e.g. "2B (virtual 21.5MB)") -func (c *ContainerContext) Size() string { - if c.FieldsUsed == nil { - c.FieldsUsed = map[string]any{} - } - c.FieldsUsed["Size"] = struct{}{} - srw := units.HumanSizeWithPrecision(float64(c.c.SizeRw), 3) - sv := units.HumanSizeWithPrecision(float64(c.c.SizeRootFs), 3) - - sf := srw - if c.c.SizeRootFs > 0 { - sf = fmt.Sprintf("%s (virtual %s)", srw, sv) - } - return sf -} diff --git a/cmd/formatter/formatter.go b/cmd/formatter/formatter.go deleted file mode 100644 index 6af92c478db..00000000000 --- a/cmd/formatter/formatter.go +++ /dev/null @@ -1,71 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "fmt" - "io" - "reflect" - "strings" - - "github.com/docker/compose/v5/pkg/api" -) - -// Print prints formatted lists in different formats -func Print(toJSON any, format string, outWriter io.Writer, writerFn func(w io.Writer), headers ...string) error { - switch strings.ToLower(format) { - case TABLE, PRETTY, "": - return PrintPrettySection(outWriter, writerFn, headers...) - case TemplateLegacyJSON: - switch reflect.TypeOf(toJSON).Kind() { - case reflect.Slice: - s := reflect.ValueOf(toJSON) - for i := 0; i < s.Len(); i++ { - obj := s.Index(i).Interface() - outJSON, err := ToJSON(obj, "", "") - if err != nil { - return err - } - _, _ = fmt.Fprint(outWriter, outJSON) - } - default: - outJSON, err := ToStandardJSON(toJSON) - if err != nil { - return err - } - _, _ = fmt.Fprintln(outWriter, outJSON) - } - case JSON: - switch reflect.TypeOf(toJSON).Kind() { - case reflect.Slice: - outJSON, err := ToJSON(toJSON, "", "") - if err != nil { - return err - } - _, _ = fmt.Fprint(outWriter, outJSON) - default: - outJSON, err := ToStandardJSON(toJSON) - if err != nil { - return err - } - _, _ = fmt.Fprintln(outWriter, outJSON) - } - default: - return fmt.Errorf("format value %q could not be parsed: %w", format, api.ErrParsingFailed) - } - return nil -} diff --git a/cmd/formatter/formatter_test.go b/cmd/formatter/formatter_test.go deleted file mode 100644 index 07b152559c1..00000000000 --- a/cmd/formatter/formatter_test.go +++ /dev/null @@ -1,78 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "bytes" - "fmt" - "io" - "testing" - - "go.uber.org/goleak" - "gotest.tools/v3/assert" -) - -type testStruct struct { - Name string - Status string -} - -// Print prints formatted lists in different formats -func TestPrint(t *testing.T) { - testList := []testStruct{ - { - Name: "myName1", - Status: "myStatus1", - }, - { - Name: "myName2", - Status: "myStatus2", - }, - } - - b := &bytes.Buffer{} - assert.NilError(t, Print(testList, TABLE, b, func(w io.Writer) { - for _, t := range testList { - _, _ = fmt.Fprintf(w, "%s\t%s\n", t.Name, t.Status) - } - }, "NAME", "STATUS")) - assert.Equal(t, b.String(), "NAME STATUS\nmyName1 myStatus1\nmyName2 myStatus2\n") - - b.Reset() - assert.NilError(t, Print(testList, JSON, b, func(w io.Writer) { - for _, t := range testList { - _, _ = fmt.Fprintf(w, "%s\t%s\n", t.Name, t.Status) - } - }, "NAME", "STATUS")) - assert.Equal(t, b.String(), `[{"Name":"myName1","Status":"myStatus1"},{"Name":"myName2","Status":"myStatus2"}] -`) - - b.Reset() - assert.NilError(t, Print(testList, TemplateLegacyJSON, b, func(w io.Writer) { - for _, t := range testList { - _, _ = fmt.Fprintf(w, "%s\t%s\n", t.Name, t.Status) - } - }, "NAME", "STATUS")) - json := b.String() - assert.Equal(t, json, `{"Name":"myName1","Status":"myStatus1"} -{"Name":"myName2","Status":"myStatus2"} -`) -} - -func TestColorsGoroutinesLeak(t *testing.T) { - goleak.VerifyNone(t) -} diff --git a/cmd/formatter/json.go b/cmd/formatter/json.go deleted file mode 100644 index b09e721aa27..00000000000 --- a/cmd/formatter/json.go +++ /dev/null @@ -1,39 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "bytes" - "encoding/json" -) - -const standardIndentation = " " - -// ToStandardJSON return a string with the JSON representation of the interface{} -func ToStandardJSON(i any) (string, error) { - return ToJSON(i, "", standardIndentation) -} - -// ToJSON return a string with the JSON representation of the interface{} -func ToJSON(i any, prefix string, indentation string) (string, error) { - buffer := &bytes.Buffer{} - encoder := json.NewEncoder(buffer) - encoder.SetEscapeHTML(false) - encoder.SetIndent(prefix, indentation) - err := encoder.Encode(i) - return buffer.String(), err -} diff --git a/cmd/formatter/logs.go b/cmd/formatter/logs.go deleted file mode 100644 index 1ee35e96b32..00000000000 --- a/cmd/formatter/logs.go +++ /dev/null @@ -1,186 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "context" - "fmt" - "io" - "strconv" - "strings" - "sync" - "time" - - "github.com/buger/goterm" - "github.com/docker/docker/pkg/jsonmessage" - - "github.com/docker/compose/v5/pkg/api" -) - -// LogConsumer consume logs from services and format them -type logConsumer struct { - ctx context.Context - presenters sync.Map // map[string]*presenter - width int - stdout io.Writer - stderr io.Writer - color bool - prefix bool - timestamp bool -} - -// NewLogConsumer creates a new LogConsumer -func NewLogConsumer(ctx context.Context, stdout, stderr io.Writer, color, prefix, timestamp bool) api.LogConsumer { - return &logConsumer{ - ctx: ctx, - presenters: sync.Map{}, - width: 0, - stdout: stdout, - stderr: stderr, - color: color, - prefix: prefix, - timestamp: timestamp, - } -} - -func (l *logConsumer) register(name string) *presenter { - var p *presenter - root, _, found := strings.Cut(name, " ") - if found { - parent := l.getPresenter(root) - p = &presenter{ - colors: parent.colors, - name: name, - prefix: parent.prefix, - } - } else { - cf := monochrome - if l.color { - switch name { - case "": - cf = monochrome - case api.WatchLogger: - cf = makeColorFunc("92") - default: - cf = nextColor() - } - } - p = &presenter{ - colors: cf, - name: name, - } - } - l.presenters.Store(name, p) - l.computeWidth() - if l.prefix { - l.presenters.Range(func(key, value any) bool { - p := value.(*presenter) - p.setPrefix(l.width) - return true - }) - } - return p -} - -func (l *logConsumer) getPresenter(container string) *presenter { - p, ok := l.presenters.Load(container) - if !ok { // should have been registered, but ¯\_(ツ)_/¯ - return l.register(container) - } - return p.(*presenter) -} - -// Log formats a log message as received from name/container -func (l *logConsumer) Log(container, message string) { - l.write(l.stdout, container, message) -} - -// Err formats a log message as received from name/container -func (l *logConsumer) Err(container, message string) { - l.write(l.stderr, container, message) -} - -func (l *logConsumer) write(w io.Writer, container, message string) { - if l.ctx.Err() != nil { - return - } - p := l.getPresenter(container) - timestamp := time.Now().Format(jsonmessage.RFC3339NanoFixed) - for line := range strings.SplitSeq(message, "\n") { - if l.timestamp { - _, _ = fmt.Fprintf(w, "%s%s %s\n", p.prefix, timestamp, line) - } else { - _, _ = fmt.Fprintf(w, "%s%s\n", p.prefix, line) - } - } -} - -func (l *logConsumer) Status(container, msg string) { - p := l.getPresenter(container) - s := p.colors(fmt.Sprintf("%s%s %s\n", goterm.RESET_LINE, container, msg)) - l.stdout.Write([]byte(s)) //nolint:errcheck -} - -func (l *logConsumer) computeWidth() { - width := 0 - l.presenters.Range(func(key, value any) bool { - p := value.(*presenter) - if len(p.name) > width { - width = len(p.name) - } - return true - }) - l.width = width + 1 -} - -type presenter struct { - colors colorFunc - name string - prefix string -} - -func (p *presenter) setPrefix(width int) { - if p.name == api.WatchLogger { - p.prefix = p.colors(strings.Repeat(" ", width) + " ⦿ ") - return - } - p.prefix = p.colors(fmt.Sprintf("%-"+strconv.Itoa(width)+"s | ", p.name)) -} - -type logDecorator struct { - decorated api.LogConsumer - Before func() - After func() -} - -func (l logDecorator) Log(containerName, message string) { - l.Before() - l.decorated.Log(containerName, message) - l.After() -} - -func (l logDecorator) Err(containerName, message string) { - l.Before() - l.decorated.Err(containerName, message) - l.After() -} - -func (l logDecorator) Status(container, msg string) { - l.Before() - l.decorated.Status(container, msg) - l.After() -} diff --git a/cmd/formatter/pretty.go b/cmd/formatter/pretty.go deleted file mode 100644 index bb85dedef61..00000000000 --- a/cmd/formatter/pretty.go +++ /dev/null @@ -1,32 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "fmt" - "io" - "strings" - "text/tabwriter" -) - -// PrintPrettySection prints a tabbed section on the writer parameter -func PrintPrettySection(out io.Writer, printer func(writer io.Writer), headers ...string) error { - w := tabwriter.NewWriter(out, 20, 1, 3, ' ', 0) - _, _ = fmt.Fprintln(w, strings.Join(headers, "\t")) - printer(w) - return w.Flush() -} diff --git a/cmd/formatter/shortcut.go b/cmd/formatter/shortcut.go deleted file mode 100644 index 16e8fc2e7af..00000000000 --- a/cmd/formatter/shortcut.go +++ /dev/null @@ -1,365 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "context" - "errors" - "fmt" - "math" - "os" - "strings" - "syscall" - "time" - - "github.com/buger/goterm" - "github.com/compose-spec/compose-go/v2/types" - "github.com/eiannone/keyboard" - "github.com/skratchdot/open-golang/open" - - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" -) - -const DISPLAY_ERROR_TIME = 10 - -type KeyboardError struct { - err error - timeStart time.Time -} - -func (ke *KeyboardError) shouldDisplay() bool { - return ke.err != nil && int(time.Since(ke.timeStart).Seconds()) < DISPLAY_ERROR_TIME -} - -func (ke *KeyboardError) printError(height int, info string) { - if ke.shouldDisplay() { - errMessage := ke.err.Error() - - moveCursor(height-1-extraLines(info)-extraLines(errMessage), 0) - clearLine() - - fmt.Print(errMessage) - } -} - -func (ke *KeyboardError) addError(prefix string, err error) { - ke.timeStart = time.Now() - - prefix = ansiColor(CYAN, fmt.Sprintf("%s →", prefix), BOLD) - errorString := fmt.Sprintf("%s %s", prefix, err.Error()) - - ke.err = errors.New(errorString) -} - -func (ke *KeyboardError) error() string { - return ke.err.Error() -} - -type KeyboardWatch struct { - Watching bool - Watcher Feature -} - -// Feature is an compose feature that can be started/stopped by a menu command -type Feature interface { - Start(context.Context) error - Stop() error -} - -type KEYBOARD_LOG_LEVEL int - -const ( - NONE KEYBOARD_LOG_LEVEL = 0 - INFO KEYBOARD_LOG_LEVEL = 1 - DEBUG KEYBOARD_LOG_LEVEL = 2 -) - -type LogKeyboard struct { - kError KeyboardError - Watch *KeyboardWatch - Detach func() - IsDockerDesktopActive bool - logLevel KEYBOARD_LOG_LEVEL - signalChannel chan<- os.Signal -} - -func NewKeyboardManager(isDockerDesktopActive bool, sc chan<- os.Signal) *LogKeyboard { - return &LogKeyboard{ - IsDockerDesktopActive: isDockerDesktopActive, - logLevel: INFO, - signalChannel: sc, - } -} - -func (lk *LogKeyboard) Decorate(l api.LogConsumer) api.LogConsumer { - return logDecorator{ - decorated: l, - Before: lk.clearNavigationMenu, - After: lk.PrintKeyboardInfo, - } -} - -func (lk *LogKeyboard) PrintKeyboardInfo() { - if lk.logLevel == INFO { - lk.printNavigationMenu() - } -} - -// Creates space to print error and menu string -func (lk *LogKeyboard) createBuffer(lines int) { - if lk.kError.shouldDisplay() { - extraLines := extraLines(lk.kError.error()) + 1 - lines += extraLines - } - - // get the string - infoMessage := lk.navigationMenu() - // calculate how many lines we need to display the menu info - // might be needed a line break - extraLines := extraLines(infoMessage) + 1 - lines += extraLines - - if lines > 0 { - allocateSpace(lines) - moveCursorUp(lines) - } -} - -func (lk *LogKeyboard) printNavigationMenu() { - offset := 1 - lk.clearNavigationMenu() - lk.createBuffer(offset) - - if lk.logLevel == INFO { - height := goterm.Height() - menu := lk.navigationMenu() - - carriageReturn() - saveCursor() - - lk.kError.printError(height, menu) - - moveCursor(height-extraLines(menu), 0) - clearLine() - fmt.Print(menu) - - carriageReturn() - restoreCursor() - } -} - -func (lk *LogKeyboard) navigationMenu() string { - var items []string - if lk.IsDockerDesktopActive { - items = append(items, shortcutKeyColor("v")+navColor(" View in Docker Desktop")) - } - - if lk.IsDockerDesktopActive { - items = append(items, shortcutKeyColor("o")+navColor(" View Config")) - } - - isEnabled := " Enable" - if lk.Watch != nil && lk.Watch.Watching { - isEnabled = " Disable" - } - items = append(items, shortcutKeyColor("w")+navColor(isEnabled+" Watch")) - items = append(items, shortcutKeyColor("d")+navColor(" Detach")) - - return strings.Join(items, " ") -} - -func (lk *LogKeyboard) clearNavigationMenu() { - height := goterm.Height() - carriageReturn() - saveCursor() - - // clearLine() - for i := 0; i < height; i++ { - moveCursorDown(1) - clearLine() - } - restoreCursor() -} - -func (lk *LogKeyboard) openDockerDesktop(ctx context.Context, project *types.Project) { - if !lk.IsDockerDesktopActive { - return - } - go func() { - _ = tracing.EventWrapFuncForErrGroup(ctx, "menu/gui", tracing.SpanOptions{}, - func(ctx context.Context) error { - link := fmt.Sprintf("docker-desktop://dashboard/apps/%s", project.Name) - err := open.Run(link) - if err != nil { - err = fmt.Errorf("could not open Docker Desktop") - lk.keyboardError("View", err) - } - return err - })() - }() -} - -func (lk *LogKeyboard) openDDComposeUI(ctx context.Context, project *types.Project) { - if !lk.IsDockerDesktopActive { - return - } - go func() { - _ = tracing.EventWrapFuncForErrGroup(ctx, "menu/gui/composeview", tracing.SpanOptions{}, - func(ctx context.Context) error { - link := fmt.Sprintf("docker-desktop://dashboard/docker-compose/%s", project.Name) - err := open.Run(link) - if err != nil { - err = fmt.Errorf("could not open Docker Desktop Compose UI") - lk.keyboardError("View Config", err) - } - return err - })() - }() -} - -func (lk *LogKeyboard) openDDWatchDocs(ctx context.Context, project *types.Project) { - go func() { - _ = tracing.EventWrapFuncForErrGroup(ctx, "menu/gui/watch", tracing.SpanOptions{}, - func(ctx context.Context) error { - link := fmt.Sprintf("docker-desktop://dashboard/docker-compose/%s/watch", project.Name) - err := open.Run(link) - if err != nil { - err = fmt.Errorf("could not open Docker Desktop Compose UI") - lk.keyboardError("Watch Docs", err) - } - return err - })() - }() -} - -func (lk *LogKeyboard) keyboardError(prefix string, err error) { - lk.kError.addError(prefix, err) - - lk.printNavigationMenu() - timer1 := time.NewTimer((DISPLAY_ERROR_TIME + 1) * time.Second) - go func() { - <-timer1.C - lk.printNavigationMenu() - }() -} - -func (lk *LogKeyboard) ToggleWatch(ctx context.Context, options api.UpOptions) { - if lk.Watch == nil { - return - } - if lk.Watch.Watching { - err := lk.Watch.Watcher.Stop() - if err != nil { - options.Start.Attach.Err(api.WatchLogger, err.Error()) - } else { - lk.Watch.Watching = false - } - } else { - go func() { - _ = tracing.EventWrapFuncForErrGroup(ctx, "menu/watch", tracing.SpanOptions{}, - func(ctx context.Context) error { - err := lk.Watch.Watcher.Start(ctx) - if err != nil { - options.Start.Attach.Err(api.WatchLogger, err.Error()) - } else { - lk.Watch.Watching = true - } - return err - })() - }() - } -} - -func (lk *LogKeyboard) HandleKeyEvents(ctx context.Context, event keyboard.KeyEvent, project *types.Project, options api.UpOptions) { - switch kRune := event.Rune; kRune { - case 'd': - lk.clearNavigationMenu() - lk.Detach() - case 'v': - lk.openDockerDesktop(ctx, project) - case 'w': - if lk.Watch == nil { - // we try to open watch docs if DD is installed - if lk.IsDockerDesktopActive { - lk.openDDWatchDocs(ctx, project) - } - // either way we mark menu/watch as an error - go func() { - _ = tracing.EventWrapFuncForErrGroup(ctx, "menu/watch", tracing.SpanOptions{}, - func(ctx context.Context) error { - err := fmt.Errorf("watch is not yet configured. Learn more: %s", ansiColor(CYAN, "https://docs.docker.com/compose/file-watch/")) - lk.keyboardError("Watch", err) - return err - })() - }() - } - lk.ToggleWatch(ctx, options) - case 'o': - lk.openDDComposeUI(ctx, project) - } - switch key := event.Key; key { - case keyboard.KeyCtrlC: - _ = keyboard.Close() - lk.clearNavigationMenu() - showCursor() - - lk.logLevel = NONE - // will notify main thread to kill and will handle gracefully - lk.signalChannel <- syscall.SIGINT - case keyboard.KeyCtrlZ: - handleCtrlZ() - case keyboard.KeyEnter: - newLine() - lk.printNavigationMenu() - } -} - -func (lk *LogKeyboard) EnableWatch(enabled bool, watcher Feature) { - lk.Watch = &KeyboardWatch{ - Watching: enabled, - Watcher: watcher, - } -} - -func (lk *LogKeyboard) EnableDetach(detach func()) { - lk.Detach = detach -} - -func allocateSpace(lines int) { - for i := 0; i < lines; i++ { - clearLine() - newLine() - carriageReturn() - } -} - -func extraLines(s string) int { - return int(math.Floor(float64(lenAnsi(s)) / float64(goterm.Width()))) -} - -func shortcutKeyColor(key string) string { - foreground := "38;2" - black := "0;0;0" - background := "48;2" - white := "255;255;255" - return ansiColor(foreground+";"+black+";"+background+";"+white, key, BOLD) -} - -func navColor(key string) string { - return ansiColor(FAINT, key) -} diff --git a/cmd/formatter/shortcut_unix.go b/cmd/formatter/shortcut_unix.go deleted file mode 100644 index 0baa3a949cc..00000000000 --- a/cmd/formatter/shortcut_unix.go +++ /dev/null @@ -1,25 +0,0 @@ -//go:build !windows - -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import "syscall" - -func handleCtrlZ() { - _ = syscall.Kill(0, syscall.SIGSTOP) -} diff --git a/cmd/formatter/shortcut_windows.go b/cmd/formatter/shortcut_windows.go deleted file mode 100644 index 1efa14cc2dd..00000000000 --- a/cmd/formatter/shortcut_windows.go +++ /dev/null @@ -1,25 +0,0 @@ -//go:build windows - -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -// handleCtrlZ is a no-op on Windows as SIGSTOP is not supported -func handleCtrlZ() { - // Windows doesn't support SIGSTOP/SIGCONT signals - // Ctrl+Z behavior is handled differently by the Windows terminal -} diff --git a/cmd/main.go b/cmd/main.go deleted file mode 100644 index 7a5ec58d60b..00000000000 --- a/cmd/main.go +++ /dev/null @@ -1,85 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package main - -import ( - "os" - - dockercli "github.com/docker/cli/cli" - "github.com/docker/cli/cli-plugins/metadata" - "github.com/docker/cli/cli-plugins/plugin" - "github.com/docker/cli/cli/command" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/cmdtrace" - "github.com/docker/compose/v5/cmd/compatibility" - commands "github.com/docker/compose/v5/cmd/compose" - "github.com/docker/compose/v5/cmd/prompt" - "github.com/docker/compose/v5/internal" - "github.com/docker/compose/v5/pkg/compose" -) - -func pluginMain() { - plugin.Run( - func(cli command.Cli) *cobra.Command { - backendOptions := &commands.BackendOptions{ - Options: []compose.Option{ - compose.WithPrompt(prompt.NewPrompt(cli.In(), cli.Out()).Confirm), - }, - } - - cmd := commands.RootCommand(cli, backendOptions) - originalPreRunE := cmd.PersistentPreRunE - cmd.PersistentPreRunE = func(cmd *cobra.Command, args []string) error { - // initialize the cli instance - if err := plugin.PersistentPreRunE(cmd, args); err != nil { - return err - } - if err := cmdtrace.Setup(cmd, cli, os.Args[1:]); err != nil { - logrus.Debugf("failed to enable tracing: %v", err) - } - - if originalPreRunE != nil { - return originalPreRunE(cmd, args) - } - return nil - } - - cmd.SetFlagErrorFunc(func(c *cobra.Command, err error) error { - return dockercli.StatusError{ - StatusCode: 1, - Status: err.Error(), - } - }) - return cmd - }, - metadata.Metadata{ - SchemaVersion: "0.1.0", - Vendor: "Docker Inc.", - Version: internal.Version, - }, - command.WithUserAgent("compose/"+internal.Version), - ) -} - -func main() { - if plugin.RunningStandalone() { - os.Args = append([]string{"docker"}, compatibility.Convert(os.Args[1:])...) - } - pluginMain() -} diff --git a/cmd/prompt/prompt.go b/cmd/prompt/prompt.go deleted file mode 100644 index 87379f5eb61..00000000000 --- a/cmd/prompt/prompt.go +++ /dev/null @@ -1,102 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package prompt - -import ( - "fmt" - "io" - - "github.com/AlecAivazis/survey/v2" - "github.com/docker/cli/cli/streams" - - "github.com/docker/compose/v5/pkg/utils" -) - -//go:generate mockgen -destination=./prompt_mock.go -self_package "github.com/docker/compose/v5/pkg/prompt" -package=prompt . UI - -// UI - prompt user input -type UI interface { - Confirm(message string, defaultValue bool) (bool, error) -} - -func NewPrompt(stdin *streams.In, stdout *streams.Out) UI { - if stdin.IsTerminal() { - return User{stdin: streamsFileReader{stdin}, stdout: streamsFileWriter{stdout}} - } - return Pipe{stdin: stdin, stdout: stdout} -} - -// User - in a terminal -type User struct { - stdout streamsFileWriter - stdin streamsFileReader -} - -// adapt streams.Out to terminal.FileWriter -type streamsFileWriter struct { - stream *streams.Out -} - -func (s streamsFileWriter) Write(p []byte) (n int, err error) { - return s.stream.Write(p) -} - -func (s streamsFileWriter) Fd() uintptr { - return s.stream.FD() -} - -// adapt streams.In to terminal.FileReader -type streamsFileReader struct { - stream *streams.In -} - -func (s streamsFileReader) Read(p []byte) (n int, err error) { - return s.stream.Read(p) -} - -func (s streamsFileReader) Fd() uintptr { - return s.stream.FD() -} - -// Confirm asks for yes or no input -func (u User) Confirm(message string, defaultValue bool) (bool, error) { - qs := &survey.Confirm{ - Message: message, - Default: defaultValue, - } - var b bool - err := survey.AskOne(qs, &b, func(options *survey.AskOptions) error { - options.Stdio.In = u.stdin - options.Stdio.Out = u.stdout - return nil - }) - return b, err -} - -// Pipe - aggregates prompt methods -type Pipe struct { - stdout io.Writer - stdin io.Reader -} - -// Confirm asks for yes or no input -func (u Pipe) Confirm(message string, defaultValue bool) (bool, error) { - _, _ = fmt.Fprint(u.stdout, message) - var answer string - _, _ = fmt.Fscanln(u.stdin, &answer) - return utils.StringToBool(answer), nil -} diff --git a/cmd/prompt/prompt_mock.go b/cmd/prompt/prompt_mock.go deleted file mode 100644 index 83b0ff1189b..00000000000 --- a/cmd/prompt/prompt_mock.go +++ /dev/null @@ -1,94 +0,0 @@ -// Code generated by MockGen. DO NOT EDIT. -// Source: github.com/docker/compose-cli/pkg/prompt (interfaces: UI) - -// Package prompt is a generated GoMock package. -package prompt - -import ( - reflect "reflect" - - gomock "go.uber.org/mock/gomock" -) - -// MockUI is a mock of UI interface -type MockUI struct { - ctrl *gomock.Controller - recorder *MockUIMockRecorder -} - -// MockUIMockRecorder is the mock recorder for MockUI -type MockUIMockRecorder struct { - mock *MockUI -} - -// NewMockUI creates a new mock instance -func NewMockUI(ctrl *gomock.Controller) *MockUI { - mock := &MockUI{ctrl: ctrl} - mock.recorder = &MockUIMockRecorder{mock} - return mock -} - -// EXPECT returns an object that allows the caller to indicate expected use -func (m *MockUI) EXPECT() *MockUIMockRecorder { - return m.recorder -} - -// Confirm mocks base method -func (m *MockUI) Confirm(arg0 string, arg1 bool) (bool, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Confirm", arg0, arg1) - ret0, _ := ret[0].(bool) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Confirm indicates an expected call of Confirm -func (mr *MockUIMockRecorder) Confirm(arg0, arg1 interface{}) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Confirm", reflect.TypeOf((*MockUI)(nil).Confirm), arg0, arg1) -} - -// Input mocks base method -func (m *MockUI) Input(arg0, arg1 string) (string, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Input", arg0, arg1) - ret0, _ := ret[0].(string) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Input indicates an expected call of Input -func (mr *MockUIMockRecorder) Input(arg0, arg1 interface{}) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Input", reflect.TypeOf((*MockUI)(nil).Input), arg0, arg1) -} - -// Password mocks base method -func (m *MockUI) Password(arg0 string) (string, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Password", arg0) - ret0, _ := ret[0].(string) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Password indicates an expected call of Password -func (mr *MockUIMockRecorder) Password(arg0 interface{}) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Password", reflect.TypeOf((*MockUI)(nil).Password), arg0) -} - -// Select mocks base method -func (m *MockUI) Select(arg0 string, arg1 []string) (int, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Select", arg0, arg1) - ret0, _ := ret[0].(int) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Select indicates an expected call of Select -func (mr *MockUIMockRecorder) Select(arg0, arg1 interface{}) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Select", reflect.TypeOf((*MockUI)(nil).Select), arg0, arg1) -} diff --git a/codecov.yml b/codecov.yml deleted file mode 100644 index a66912f32e0..00000000000 --- a/codecov.yml +++ /dev/null @@ -1,21 +0,0 @@ -coverage: - status: - project: - default: - informational: true - target: auto - threshold: 2% - patch: - default: - informational: true - -comment: - require_changes: true - -ignore: - - "packaging" - - "docs" - - "bin" - - "e2e" - - "pkg/e2e" - - "**/*_test.go" diff --git a/compose/__init__.py b/compose/__init__.py new file mode 100644 index 00000000000..444d27edb84 --- /dev/null +++ b/compose/__init__.py @@ -0,0 +1 @@ +__version__ = '1.28.0dev' diff --git a/compose/__main__.py b/compose/__main__.py new file mode 100644 index 00000000000..199ba2ae9b4 --- /dev/null +++ b/compose/__main__.py @@ -0,0 +1,3 @@ +from compose.cli.main import main + +main() diff --git a/pkg/e2e/fixtures/environment/env-priority/.env.empty b/compose/cli/__init__.py similarity index 100% rename from pkg/e2e/fixtures/environment/env-priority/.env.empty rename to compose/cli/__init__.py diff --git a/compose/cli/colors.py b/compose/cli/colors.py new file mode 100644 index 00000000000..a4983a9f51a --- /dev/null +++ b/compose/cli/colors.py @@ -0,0 +1,46 @@ +from ..const import IS_WINDOWS_PLATFORM + +NAMES = [ + 'grey', + 'red', + 'green', + 'yellow', + 'blue', + 'magenta', + 'cyan', + 'white' +] + + +def get_pairs(): + for i, name in enumerate(NAMES): + yield (name, str(30 + i)) + yield ('intense_' + name, str(30 + i) + ';1') + + +def ansi(code): + return '\033[{}m'.format(code) + + +def ansi_color(code, s): + return '{}{}{}'.format(ansi(code), s, ansi(0)) + + +def make_color_fn(code): + return lambda s: ansi_color(code, s) + + +if IS_WINDOWS_PLATFORM: + import colorama + colorama.init(strip=False) +for (name, code) in get_pairs(): + globals()[name] = make_color_fn(code) + + +def rainbow(): + cs = ['cyan', 'yellow', 'green', 'magenta', 'blue', + 'intense_cyan', 'intense_yellow', 'intense_green', + 'intense_magenta', 'intense_blue'] + + for c in cs: + yield globals()[c] diff --git a/compose/cli/command.py b/compose/cli/command.py new file mode 100644 index 00000000000..599df9969a7 --- /dev/null +++ b/compose/cli/command.py @@ -0,0 +1,210 @@ +import logging +import os +import re + +from . import errors +from .. import config +from .. import parallel +from ..config.environment import Environment +from ..const import LABEL_CONFIG_FILES +from ..const import LABEL_ENVIRONMENT_FILE +from ..const import LABEL_WORKING_DIR +from ..project import Project +from .docker_client import get_client +from .docker_client import load_context +from .docker_client import make_context +from .errors import UserError + +log = logging.getLogger(__name__) + +SILENT_COMMANDS = { + 'events', + 'exec', + 'kill', + 'logs', + 'pause', + 'ps', + 'restart', + 'rm', + 'start', + 'stop', + 'top', + 'unpause', +} + + +def project_from_options(project_dir, options, additional_options=None): + additional_options = additional_options or {} + override_dir = get_project_dir(options) + environment_file = options.get('--env-file') + environment = Environment.from_env_file(override_dir or project_dir, environment_file) + environment.silent = options.get('COMMAND', None) in SILENT_COMMANDS + set_parallel_limit(environment) + + # get the context for the run + context = None + context_name = options.get('--context', None) + if context_name: + context = load_context(context_name) + if not context: + raise UserError("Context '{}' not found".format(context_name)) + + host = options.get('--host', None) + if host is not None: + if context: + raise UserError( + "-H, --host and -c, --context are mutually exclusive. Only one should be set.") + host = host.lstrip('=') + context = make_context(host, options, environment) + + return get_project( + project_dir, + get_config_path_from_options(options, environment), + project_name=options.get('--project-name'), + verbose=options.get('--verbose'), + context=context, + environment=environment, + override_dir=override_dir, + interpolate=(not additional_options.get('--no-interpolate')), + environment_file=environment_file, + enabled_profiles=get_profiles_from_options(options, environment) + ) + + +def set_parallel_limit(environment): + parallel_limit = environment.get('COMPOSE_PARALLEL_LIMIT') + if parallel_limit: + try: + parallel_limit = int(parallel_limit) + except ValueError: + raise errors.UserError( + 'COMPOSE_PARALLEL_LIMIT must be an integer (found: "{}")'.format( + environment.get('COMPOSE_PARALLEL_LIMIT') + ) + ) + if parallel_limit <= 1: + raise errors.UserError('COMPOSE_PARALLEL_LIMIT can not be less than 2') + parallel.GlobalLimit.set_global_limit(parallel_limit) + + +def get_project_dir(options): + override_dir = None + files = get_config_path_from_options(options, os.environ) + if files: + if files[0] == '-': + return '.' + override_dir = os.path.dirname(files[0]) + return options.get('--project-directory') or override_dir + + +def get_config_from_options(base_dir, options, additional_options=None): + additional_options = additional_options or {} + override_dir = get_project_dir(options) + environment_file = options.get('--env-file') + environment = Environment.from_env_file(override_dir or base_dir, environment_file) + config_path = get_config_path_from_options(options, environment) + return config.load( + config.find(base_dir, config_path, environment, override_dir), + not additional_options.get('--no-interpolate') + ) + + +def get_config_path_from_options(options, environment): + def unicode_paths(paths): + return [p.decode('utf-8') if isinstance(p, bytes) else p for p in paths] + + file_option = options.get('--file') + if file_option: + return unicode_paths(file_option) + + config_files = environment.get('COMPOSE_FILE') + if config_files: + pathsep = environment.get('COMPOSE_PATH_SEPARATOR', os.pathsep) + return unicode_paths(config_files.split(pathsep)) + return None + + +def get_profiles_from_options(options, environment): + profile_option = options.get('--profile') + if profile_option: + return profile_option + + profiles = environment.get('COMPOSE_PROFILE') + if profiles: + return profiles.split(',') + + return [] + + +def get_project(project_dir, config_path=None, project_name=None, verbose=False, + context=None, environment=None, override_dir=None, + interpolate=True, environment_file=None, enabled_profiles=None): + if not environment: + environment = Environment.from_env_file(project_dir) + config_details = config.find(project_dir, config_path, environment, override_dir) + project_name = get_project_name( + config_details.working_dir, project_name, environment + ) + config_data = config.load(config_details, interpolate) + + api_version = environment.get('COMPOSE_API_VERSION') + + client = get_client( + verbose=verbose, version=api_version, context=context, environment=environment + ) + + with errors.handle_connection_errors(client): + return Project.from_config( + project_name, + config_data, + client, + environment.get('DOCKER_DEFAULT_PLATFORM'), + execution_context_labels(config_details, environment_file), + enabled_profiles, + ) + + +def execution_context_labels(config_details, environment_file): + extra_labels = [ + '{}={}'.format(LABEL_WORKING_DIR, os.path.abspath(config_details.working_dir)) + ] + + if not use_config_from_stdin(config_details): + extra_labels.append('{}={}'.format(LABEL_CONFIG_FILES, config_files_label(config_details))) + + if environment_file is not None: + extra_labels.append('{}={}'.format( + LABEL_ENVIRONMENT_FILE, + os.path.normpath(environment_file)) + ) + return extra_labels + + +def use_config_from_stdin(config_details): + for c in config_details.config_files: + if not c.filename: + return True + return False + + +def config_files_label(config_details): + return ",".join( + os.path.normpath(c.filename) for c in config_details.config_files + ) + + +def get_project_name(working_dir, project_name=None, environment=None): + def normalize_name(name): + return re.sub(r'[^-_a-z0-9]', '', name.lower()) + + if not environment: + environment = Environment.from_env_file(working_dir) + project_name = project_name or environment.get('COMPOSE_PROJECT_NAME') + if project_name: + return normalize_name(project_name) + + project = os.path.basename(os.path.abspath(working_dir)) + if project: + return normalize_name(project) + + return 'default' diff --git a/compose/cli/docker_client.py b/compose/cli/docker_client.py new file mode 100644 index 00000000000..e4a0fea61b4 --- /dev/null +++ b/compose/cli/docker_client.py @@ -0,0 +1,173 @@ +import logging +import os.path +import ssl + +from docker import APIClient +from docker import Context +from docker import ContextAPI +from docker import TLSConfig +from docker.errors import TLSParameterError +from docker.utils import kwargs_from_env +from docker.utils.config import home_dir + +from . import verbose_proxy +from ..config.environment import Environment +from ..const import HTTP_TIMEOUT +from ..utils import unquote_path +from .errors import UserError +from .utils import generate_user_agent +from .utils import get_version_info + +log = logging.getLogger(__name__) + + +def default_cert_path(): + return os.path.join(home_dir(), '.docker') + + +def make_context(host, options, environment): + tls = tls_config_from_options(options, environment) + ctx = Context("compose", host=host, tls=tls.verify if tls else False) + if tls: + ctx.set_endpoint("docker", host, tls, skip_tls_verify=not tls.verify) + return ctx + + +def load_context(name=None): + return ContextAPI.get_context(name) + + +def get_client(environment, verbose=False, version=None, context=None): + client = docker_client( + version=version, context=context, + environment=environment, tls_version=get_tls_version(environment) + ) + if verbose: + version_info = client.version().items() + log.info(get_version_info('full')) + log.info("Docker base_url: %s", client.base_url) + log.info("Docker version: %s", + ", ".join("%s=%s" % item for item in version_info)) + return verbose_proxy.VerboseProxy('docker', client) + return client + + +def get_tls_version(environment): + compose_tls_version = environment.get('COMPOSE_TLS_VERSION', None) + if not compose_tls_version: + return None + + tls_attr_name = "PROTOCOL_{}".format(compose_tls_version) + if not hasattr(ssl, tls_attr_name): + log.warning( + 'The "{}" protocol is unavailable. You may need to update your ' + 'version of Python or OpenSSL. Falling back to TLSv1 (default).' + .format(compose_tls_version) + ) + return None + + return getattr(ssl, tls_attr_name) + + +def tls_config_from_options(options, environment=None): + environment = environment or Environment() + cert_path = environment.get('DOCKER_CERT_PATH') or None + + tls = options.get('--tls', False) + ca_cert = unquote_path(options.get('--tlscacert')) + cert = unquote_path(options.get('--tlscert')) + key = unquote_path(options.get('--tlskey')) + # verify is a special case - with docopt `--tlsverify` = False means it + # wasn't used, so we set it if either the environment or the flag is True + # see https://github.com/docker/compose/issues/5632 + verify = options.get('--tlsverify') or environment.get_boolean('DOCKER_TLS_VERIFY') + + skip_hostname_check = options.get('--skip-hostname-check', False) + if cert_path is not None and not any((ca_cert, cert, key)): + # FIXME: Modify TLSConfig to take a cert_path argument and do this internally + cert = os.path.join(cert_path, 'cert.pem') + key = os.path.join(cert_path, 'key.pem') + ca_cert = os.path.join(cert_path, 'ca.pem') + + if verify and not any((ca_cert, cert, key)): + # Default location for cert files is ~/.docker + ca_cert = os.path.join(default_cert_path(), 'ca.pem') + cert = os.path.join(default_cert_path(), 'cert.pem') + key = os.path.join(default_cert_path(), 'key.pem') + + tls_version = get_tls_version(environment) + + advanced_opts = any([ca_cert, cert, key, verify, tls_version]) + + if tls is True and not advanced_opts: + return True + elif advanced_opts: # --tls is a noop + client_cert = None + if cert or key: + client_cert = (cert, key) + + return TLSConfig( + client_cert=client_cert, verify=verify, ca_cert=ca_cert, + assert_hostname=False if skip_hostname_check else None, + ssl_version=tls_version + ) + + return None + + +def docker_client(environment, version=None, context=None, tls_version=None): + """ + Returns a docker-py client configured using environment variables + according to the same logic as the official Docker client. + """ + try: + kwargs = kwargs_from_env(environment=environment, ssl_version=tls_version) + except TLSParameterError: + raise UserError( + "TLS configuration is invalid - make sure your DOCKER_TLS_VERIFY " + "and DOCKER_CERT_PATH are set correctly.\n" + "You might need to run `eval \"$(docker-machine env default)\"`") + + if not context: + # check env for DOCKER_HOST and certs path + host = kwargs.get("base_url", None) + tls = kwargs.get("tls", None) + verify = False if not tls else tls.verify + if host: + context = Context("compose", host=host, tls=verify) + else: + context = ContextAPI.get_current_context() + if tls: + context.set_endpoint("docker", host=host, tls_cfg=tls, skip_tls_verify=not verify) + + if not context.is_docker_host(): + raise UserError( + "The platform targeted with the current context is not supported.\n" + "Make sure the context in use targets a Docker Engine.\n") + + kwargs['base_url'] = context.Host + if context.TLSConfig: + kwargs['tls'] = context.TLSConfig + + if version: + kwargs['version'] = version + + timeout = environment.get('COMPOSE_HTTP_TIMEOUT') + if timeout: + kwargs['timeout'] = int(timeout) + else: + kwargs['timeout'] = HTTP_TIMEOUT + + kwargs['user_agent'] = generate_user_agent() + + # Workaround for + # https://pyinstaller.readthedocs.io/en/v3.3.1/runtime-information.html#ld-library-path-libpath-considerations + if 'LD_LIBRARY_PATH_ORIG' in environment: + kwargs['credstore_env'] = { + 'LD_LIBRARY_PATH': environment.get('LD_LIBRARY_PATH_ORIG'), + } + use_paramiko_ssh = int(environment.get('COMPOSE_PARAMIKO_SSH', 0)) + client = APIClient(use_ssh_client=not use_paramiko_ssh, **kwargs) + client._original_base_url = kwargs.get('base_url') + + return client diff --git a/compose/cli/docopt_command.py b/compose/cli/docopt_command.py new file mode 100644 index 00000000000..e56b37835bc --- /dev/null +++ b/compose/cli/docopt_command.py @@ -0,0 +1,62 @@ +from inspect import getdoc + +from docopt import docopt +from docopt import DocoptExit + + +def docopt_full_help(docstring, *args, **kwargs): + try: + return docopt(docstring, *args, **kwargs) + except DocoptExit: + raise SystemExit(docstring) + + +class DocoptDispatcher: + + def __init__(self, command_class, options): + self.command_class = command_class + self.options = options + + @classmethod + def get_command_and_options(cls, doc_entity, argv, options): + command_help = getdoc(doc_entity) + opt = docopt_full_help(command_help, argv, **options) + command = opt['COMMAND'] + return command_help, opt, command + + def parse(self, argv): + command_help, options, command = DocoptDispatcher.get_command_and_options( + self.command_class, argv, self.options) + + if command is None: + raise SystemExit(command_help) + + handler = get_handler(self.command_class, command) + docstring = getdoc(handler) + + if docstring is None: + raise NoSuchCommand(command, self) + + command_options = docopt_full_help(docstring, options['ARGS'], options_first=True) + return options, handler, command_options + + +def get_handler(command_class, command): + command = command.replace('-', '_') + # we certainly want to have "exec" command, since that's what docker client has + # but in python exec is a keyword + if command == "exec": + command = "exec_command" + + if not hasattr(command_class, command): + raise NoSuchCommand(command, command_class) + + return getattr(command_class, command) + + +class NoSuchCommand(Exception): + def __init__(self, command, supercommand): + super().__init__("No such command: %s" % command) + + self.command = command + self.supercommand = supercommand diff --git a/compose/cli/errors.py b/compose/cli/errors.py new file mode 100644 index 00000000000..a807c7d1c78 --- /dev/null +++ b/compose/cli/errors.py @@ -0,0 +1,165 @@ +import contextlib +import logging +import socket +from distutils.spawn import find_executable +from textwrap import dedent + +from docker.errors import APIError +from requests.exceptions import ConnectionError as RequestsConnectionError +from requests.exceptions import ReadTimeout +from requests.exceptions import SSLError +from requests.packages.urllib3.exceptions import ReadTimeoutError + +from ..const import API_VERSION_TO_ENGINE_VERSION +from .utils import binarystr_to_unicode +from .utils import is_docker_for_mac_installed +from .utils import is_mac +from .utils import is_ubuntu +from .utils import is_windows + + +log = logging.getLogger(__name__) + + +class UserError(Exception): + + def __init__(self, msg): + self.msg = dedent(msg).strip() + + def __str__(self): + return self.msg + + +class ConnectionError(Exception): + pass + + +@contextlib.contextmanager +def handle_connection_errors(client): + try: + yield + except SSLError as e: + log.error('SSL error: %s' % e) + raise ConnectionError() + except RequestsConnectionError as e: + if e.args and isinstance(e.args[0], ReadTimeoutError): + log_timeout_error(client.timeout) + raise ConnectionError() + exit_with_error(get_conn_error_message(client.base_url)) + except APIError as e: + log_api_error(e, client.api_version) + raise ConnectionError() + except (ReadTimeout, socket.timeout): + log_timeout_error(client.timeout) + raise ConnectionError() + except Exception as e: + if is_windows(): + import pywintypes + if isinstance(e, pywintypes.error): + log_windows_pipe_error(e) + raise ConnectionError() + raise + + +def log_windows_pipe_error(exc): + if exc.winerror == 2: + log.error("Couldn't connect to Docker daemon. You might need to start Docker for Windows.") + elif exc.winerror == 232: # https://github.com/docker/compose/issues/5005 + log.error( + "The current Compose file version is not compatible with your engine version. " + "Please upgrade your Compose file to a more recent version, or set " + "a COMPOSE_API_VERSION in your environment." + ) + else: + log.error( + "Windows named pipe error: {} (code: {})".format( + binarystr_to_unicode(exc.strerror), exc.winerror + ) + ) + + +def log_timeout_error(timeout): + log.error( + "An HTTP request took too long to complete. Retry with --verbose to " + "obtain debug information.\n" + "If you encounter this issue regularly because of slow network " + "conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher " + "value (current value: %s)." % timeout) + + +def log_api_error(e, client_version): + explanation = binarystr_to_unicode(e.explanation) + + if 'client is newer than server' not in explanation: + log.error(explanation) + return + + version = API_VERSION_TO_ENGINE_VERSION.get(client_version) + if not version: + # They've set a custom API version + log.error(explanation) + return + + log.error( + "The Docker Engine version is less than the minimum required by " + "Compose. Your current project requires a Docker Engine of " + "version {version} or greater.".format(version=version) + ) + + +def exit_with_error(msg): + log.error(dedent(msg).strip()) + raise ConnectionError() + + +def get_conn_error_message(url): + try: + if find_executable('docker') is None: + return docker_not_found_msg("Couldn't connect to Docker daemon.") + if is_docker_for_mac_installed(): + return conn_error_docker_for_mac + if find_executable('docker-machine') is not None: + return conn_error_docker_machine + except UnicodeDecodeError: + # https://github.com/docker/compose/issues/5442 + # Ignore the error and print the generic message instead. + pass + return conn_error_generic.format(url=url) + + +def docker_not_found_msg(problem): + return "{} You might need to install Docker:\n\n{}".format( + problem, docker_install_url()) + + +def docker_install_url(): + if is_mac(): + return docker_install_url_mac + elif is_ubuntu(): + return docker_install_url_ubuntu + elif is_windows(): + return docker_install_url_windows + else: + return docker_install_url_generic + + +docker_install_url_mac = "https://docs.docker.com/engine/installation/mac/" +docker_install_url_ubuntu = "https://docs.docker.com/engine/installation/ubuntulinux/" +docker_install_url_windows = "https://docs.docker.com/engine/installation/windows/" +docker_install_url_generic = "https://docs.docker.com/engine/installation/" + + +conn_error_docker_machine = """ + Couldn't connect to Docker daemon - you might need to run `docker-machine start default`. +""" + +conn_error_docker_for_mac = """ + Couldn't connect to Docker daemon. You might need to start Docker for Mac. +""" + + +conn_error_generic = """ + Couldn't connect to Docker daemon at {url} - is it running? + + If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable. +""" diff --git a/compose/cli/formatter.py b/compose/cli/formatter.py new file mode 100644 index 00000000000..ff81ee65163 --- /dev/null +++ b/compose/cli/formatter.py @@ -0,0 +1,54 @@ +import logging +from shutil import get_terminal_size + +import texttable + +from compose.cli import colors + + +def get_tty_width(): + try: + # get_terminal_size can't determine the size if compose is piped + # to another command. But in such case it doesn't make sense to + # try format the output by terminal size as this output is consumed + # by another command. So let's pretend we have a huge terminal so + # output is single-lined + width, _ = get_terminal_size(fallback=(999, 0)) + return int(width) + except OSError: + return 0 + + +class Formatter: + """Format tabular data for printing.""" + + @staticmethod + def table(headers, rows): + table = texttable.Texttable(max_width=get_tty_width()) + table.set_cols_dtype(['t' for h in headers]) + table.add_rows([headers] + rows) + table.set_deco(table.HEADER) + table.set_chars(['-', '|', '+', '-']) + + return table.draw() + + +class ConsoleWarningFormatter(logging.Formatter): + """A logging.Formatter which prints WARNING and ERROR messages with + a prefix of the log level colored appropriate for the log level. + """ + + def get_level_message(self, record): + separator = ': ' + if record.levelno >= logging.ERROR: + return colors.red(record.levelname) + separator + if record.levelno >= logging.WARNING: + return colors.yellow(record.levelname) + separator + + return '' + + def format(self, record): + if isinstance(record.msg, bytes): + record.msg = record.msg.decode('utf-8') + message = super().format(record) + return '{}{}'.format(self.get_level_message(record), message) diff --git a/compose/cli/log_printer.py b/compose/cli/log_printer.py new file mode 100644 index 00000000000..c49b817de7e --- /dev/null +++ b/compose/cli/log_printer.py @@ -0,0 +1,271 @@ +import _thread as thread +import sys +from collections import namedtuple +from itertools import cycle +from operator import attrgetter +from queue import Empty +from queue import Queue +from threading import Thread + +from docker.errors import APIError + +from . import colors +from compose.cli.signals import ShutdownException +from compose.utils import split_buffer + + +class LogPresenter: + + def __init__(self, prefix_width, color_func, keep_prefix=True): + self.prefix_width = prefix_width + self.color_func = color_func + self.keep_prefix = keep_prefix + + def present(self, container, line): + to_log = '{line}'.format(line=line) + + if self.keep_prefix: + prefix = container.name_without_project.ljust(self.prefix_width) + to_log = '{prefix} '.format(prefix=self.color_func(prefix + ' |')) + to_log + + return to_log + + +def build_log_presenters(service_names, monochrome, keep_prefix=True): + """Return an iterable of functions. + + Each function can be used to format the logs output of a container. + """ + prefix_width = max_name_width(service_names) + + def no_color(text): + return text + + for color_func in cycle([no_color] if monochrome else colors.rainbow()): + yield LogPresenter(prefix_width, color_func, keep_prefix) + + +def max_name_width(service_names, max_index_width=3): + """Calculate the maximum width of container names so we can make the log + prefixes line up like so: + + db_1 | Listening + web_1 | Listening + """ + return max(len(name) for name in service_names) + max_index_width + + +class LogPrinter: + """Print logs from many containers to a single output stream.""" + + def __init__(self, + containers, + presenters, + event_stream, + output=sys.stdout, + cascade_stop=False, + log_args=None): + self.containers = containers + self.presenters = presenters + self.event_stream = event_stream + self.output = output + self.cascade_stop = cascade_stop + self.log_args = log_args or {} + + def run(self): + if not self.containers: + return + + queue = Queue() + thread_args = queue, self.log_args + thread_map = build_thread_map(self.containers, self.presenters, thread_args) + start_producer_thread(( + thread_map, + self.event_stream, + self.presenters, + thread_args)) + + for line in consume_queue(queue, self.cascade_stop): + remove_stopped_threads(thread_map) + + if self.cascade_stop: + matching_container = [cont.name for cont in self.containers if cont.name == line] + if line in matching_container: + # Returning the name of the container that started the + # the cascade_stop so we can return the correct exit code + return line + + if not line: + if not thread_map: + # There are no running containers left to tail, so exit + return + # We got an empty line because of a timeout, but there are still + # active containers to tail, so continue + continue + + self.write(line) + + def write(self, line): + try: + self.output.write(line) + except UnicodeEncodeError: + # This may happen if the user's locale settings don't support UTF-8 + # and UTF-8 characters are present in the log line. The following + # will output a "degraded" log with unsupported characters + # replaced by `?` + self.output.write(line.encode('ascii', 'replace').decode()) + self.output.flush() + + +def remove_stopped_threads(thread_map): + for container_id, tailer_thread in list(thread_map.items()): + if not tailer_thread.is_alive(): + thread_map.pop(container_id, None) + + +def build_thread(container, presenter, queue, log_args): + tailer = Thread( + target=tail_container_logs, + args=(container, presenter, queue, log_args)) + tailer.daemon = True + tailer.start() + return tailer + + +def build_thread_map(initial_containers, presenters, thread_args): + return { + container.id: build_thread(container, next(presenters), *thread_args) + # Container order is unspecified, so they are sorted by name in order to make + # container:presenter (log color) assignment deterministic when given a list of containers + # with the same names. + for container in sorted(initial_containers, key=attrgetter('name')) + } + + +class QueueItem(namedtuple('_QueueItem', 'item is_stop exc')): + + @classmethod + def new(cls, item): + return cls(item, None, None) + + @classmethod + def exception(cls, exc): + return cls(None, None, exc) + + @classmethod + def stop(cls, item=None): + return cls(item, True, None) + + +def tail_container_logs(container, presenter, queue, log_args): + generator = get_log_generator(container) + + try: + for item in generator(container, log_args): + queue.put(QueueItem.new(presenter.present(container, item))) + except Exception as e: + queue.put(QueueItem.exception(e)) + return + if log_args.get('follow'): + queue.put(QueueItem.new(presenter.color_func(wait_on_exit(container)))) + queue.put(QueueItem.stop(container.name)) + + +def get_log_generator(container): + if container.has_api_logs: + return build_log_generator + return build_no_log_generator + + +def build_no_log_generator(container, log_args): + """Return a generator that prints a warning about logs and waits for + container to exit. + """ + yield "WARNING: no logs are available with the '{}' log driver\n".format( + container.log_driver) + + +def build_log_generator(container, log_args): + # if the container doesn't have a log_stream we need to attach to container + # before log printer starts running + if container.log_stream is None: + stream = container.logs(stdout=True, stderr=True, stream=True, **log_args) + else: + stream = container.log_stream + + return split_buffer(stream) + + +def wait_on_exit(container): + try: + exit_code = container.wait() + return "{} exited with code {}\n".format(container.name, exit_code) + except APIError as e: + return "Unexpected API error for {} (HTTP code {})\nResponse body:\n{}\n".format( + container.name, e.response.status_code, + e.response.text or '[empty]' + ) + + +def start_producer_thread(thread_args): + producer = Thread(target=watch_events, args=thread_args) + producer.daemon = True + producer.start() + + +def watch_events(thread_map, event_stream, presenters, thread_args): + crashed_containers = set() + for event in event_stream: + if event['action'] == 'stop': + thread_map.pop(event['id'], None) + + if event['action'] == 'die': + thread_map.pop(event['id'], None) + crashed_containers.add(event['id']) + + if event['action'] != 'start': + continue + + if event['id'] in thread_map: + if thread_map[event['id']].is_alive(): + continue + # Container was stopped and started, we need a new thread + thread_map.pop(event['id'], None) + + # Container crashed so we should reattach to it + if event['id'] in crashed_containers: + container = event['container'] + if not container.is_restarting: + try: + container.attach_log_stream() + except APIError: + # Just ignore errors when reattaching to already crashed containers + pass + crashed_containers.remove(event['id']) + + thread_map[event['id']] = build_thread( + event['container'], + next(presenters), + *thread_args + ) + + +def consume_queue(queue, cascade_stop): + """Consume the queue by reading lines off of it and yielding them.""" + while True: + try: + item = queue.get(timeout=0.1) + except Empty: + yield None + continue + # See https://github.com/docker/compose/issues/189 + except thread.error: + raise ShutdownException() + + if item.exc: + raise item.exc + + if item.is_stop and not cascade_stop: + continue + + yield item.item diff --git a/compose/cli/main.py b/compose/cli/main.py new file mode 100644 index 00000000000..37521cc7b3d --- /dev/null +++ b/compose/cli/main.py @@ -0,0 +1,1657 @@ +import contextlib +import functools +import json +import logging +import os +import pipes +import re +import subprocess +import sys +from distutils.spawn import find_executable +from inspect import getdoc +from operator import attrgetter + +import docker.errors +import docker.utils + +from . import errors +from . import signals +from .. import __version__ +from ..config import ConfigurationError +from ..config import parse_environment +from ..config import parse_labels +from ..config import resolve_build_args +from ..config.environment import Environment +from ..config.serialize import serialize_config +from ..config.types import VolumeSpec +from ..const import IS_WINDOWS_PLATFORM +from ..errors import StreamParseError +from ..metrics.decorator import metrics +from ..progress_stream import StreamOutputError +from ..project import get_image_digests +from ..project import MissingDigests +from ..project import NoSuchService +from ..project import OneOffFilter +from ..project import ProjectError +from ..service import BuildAction +from ..service import BuildError +from ..service import ConvergenceStrategy +from ..service import ImageType +from ..service import NeedsBuildError +from ..service import OperationFailedError +from ..utils import filter_attached_for_up +from .command import get_config_from_options +from .command import get_project_dir +from .command import project_from_options +from .docopt_command import DocoptDispatcher +from .docopt_command import get_handler +from .docopt_command import NoSuchCommand +from .errors import UserError +from .formatter import ConsoleWarningFormatter +from .formatter import Formatter +from .log_printer import build_log_presenters +from .log_printer import LogPrinter +from .utils import get_version_info +from .utils import human_readable_file_size +from .utils import yesno +from compose.metrics.client import MetricsCommand +from compose.metrics.client import Status + + +if not IS_WINDOWS_PLATFORM: + from dockerpty.pty import PseudoTerminal, RunOperation, ExecOperation + +log = logging.getLogger(__name__) +console_handler = logging.StreamHandler(sys.stderr) + + +def main(): # noqa: C901 + signals.ignore_sigpipe() + command = None + try: + _, opts, command = DocoptDispatcher.get_command_and_options( + TopLevelCommand, + get_filtered_args(sys.argv[1:]), + {'options_first': True, 'version': get_version_info('compose')}) + except Exception: + pass + try: + command_func = dispatch() + command_func() + except (KeyboardInterrupt, signals.ShutdownException): + exit_with_metrics(command, "Aborting.", status=Status.FAILURE) + except (UserError, NoSuchService, ConfigurationError, + ProjectError, OperationFailedError) as e: + exit_with_metrics(command, e.msg, status=Status.FAILURE) + except BuildError as e: + reason = "" + if e.reason: + reason = " : " + e.reason + exit_with_metrics(command, + "Service '{}' failed to build{}".format(e.service.name, reason), + status=Status.FAILURE) + except StreamOutputError as e: + exit_with_metrics(command, e, status=Status.FAILURE) + except NeedsBuildError as e: + exit_with_metrics(command, + "Service '{}' needs to be built, but --no-build was passed.".format( + e.service.name), status=Status.FAILURE) + except NoSuchCommand as e: + commands = "\n".join(parse_doc_section("commands:", getdoc(e.supercommand))) + exit_with_metrics(e.command, "No such command: {}\n\n{}".format(e.command, commands)) + except (errors.ConnectionError, StreamParseError): + exit_with_metrics(command, status=Status.FAILURE) + except SystemExit as e: + status = Status.SUCCESS + if len(sys.argv) > 1 and '--help' not in sys.argv: + status = Status.FAILURE + + if command and len(sys.argv) >= 3 and sys.argv[2] == '--help': + command = '--help ' + command + + if not command and len(sys.argv) >= 2 and sys.argv[1] == '--help': + command = '--help' + + msg = e.args[0] if len(e.args) else "" + code = 0 + if isinstance(e.code, int): + code = e.code + exit_with_metrics(command, log_msg=msg, status=status, + exit_code=code) + + +def get_filtered_args(args): + if args[0] in ('-h', '--help'): + return [] + if args[0] == '--version': + return ['version'] + + +def exit_with_metrics(command, log_msg=None, status=Status.SUCCESS, exit_code=1): + if log_msg: + if not exit_code: + log.info(log_msg) + else: + log.error(log_msg) + + MetricsCommand(command, status=status).send_metrics() + sys.exit(exit_code) + + +def dispatch(): + setup_logging() + dispatcher = DocoptDispatcher( + TopLevelCommand, + {'options_first': True, 'version': get_version_info('compose')}) + + options, handler, command_options = dispatcher.parse(sys.argv[1:]) + setup_console_handler(console_handler, + options.get('--verbose'), + set_no_color_if_clicolor(options.get('--no-ansi')), + options.get("--log-level")) + setup_parallel_logger(set_no_color_if_clicolor(options.get('--no-ansi'))) + if options.get('--no-ansi'): + command_options['--no-color'] = True + return functools.partial(perform_command, options, handler, command_options) + + +def perform_command(options, handler, command_options): + if options['COMMAND'] in ('help', 'version'): + # Skip looking up the compose file. + handler(command_options) + return + + if options['COMMAND'] == 'config': + command = TopLevelCommand(None, options=options) + handler(command, command_options) + return + + project = project_from_options('.', options) + command = TopLevelCommand(project, options=options) + with errors.handle_connection_errors(project.client): + handler(command, command_options) + + +def setup_logging(): + root_logger = logging.getLogger() + root_logger.addHandler(console_handler) + root_logger.setLevel(logging.DEBUG) + + # Disable requests and docker-py logging + logging.getLogger("urllib3").propagate = False + logging.getLogger("requests").propagate = False + logging.getLogger("docker").propagate = False + + +def setup_parallel_logger(noansi): + if noansi: + import compose.parallel + compose.parallel.ParallelStreamWriter.set_noansi() + + +def setup_console_handler(handler, verbose, noansi=False, level=None): + if handler.stream.isatty() and noansi is False: + format_class = ConsoleWarningFormatter + else: + format_class = logging.Formatter + + if verbose: + handler.setFormatter(format_class('%(name)s.%(funcName)s: %(message)s')) + loglevel = logging.DEBUG + else: + handler.setFormatter(format_class()) + loglevel = logging.INFO + + if level is not None: + levels = { + 'DEBUG': logging.DEBUG, + 'INFO': logging.INFO, + 'WARNING': logging.WARNING, + 'ERROR': logging.ERROR, + 'CRITICAL': logging.CRITICAL, + } + loglevel = levels.get(level.upper()) + if loglevel is None: + raise UserError( + 'Invalid value for --log-level. Expected one of DEBUG, INFO, WARNING, ERROR, CRITICAL.' + ) + + handler.setLevel(loglevel) + + +# stolen from docopt master +def parse_doc_section(name, source): + pattern = re.compile('^([^\n]*' + name + '[^\n]*\n?(?:[ \t].*?(?:\n|$))*)', + re.IGNORECASE | re.MULTILINE) + return [s.strip() for s in pattern.findall(source)] + + +class TopLevelCommand: + """Define and run multi-container applications with Docker. + + Usage: + docker-compose [-f <arg>...] [--profile <name>...] [options] [--] [COMMAND] [ARGS...] + docker-compose -h|--help + + Options: + -f, --file FILE Specify an alternate compose file + (default: docker-compose.yml) + -p, --project-name NAME Specify an alternate project name + (default: directory name) + --profile NAME Specify a profile to enable + -c, --context NAME Specify a context name + --verbose Show more output + --log-level LEVEL Set log level (DEBUG, INFO, WARNING, ERROR, CRITICAL) + --no-ansi Do not print ANSI control characters + -v, --version Print version and exit + -H, --host HOST Daemon socket to connect to + + --tls Use TLS; implied by --tlsverify + --tlscacert CA_PATH Trust certs signed only by this CA + --tlscert CLIENT_CERT_PATH Path to TLS certificate file + --tlskey TLS_KEY_PATH Path to TLS key file + --tlsverify Use TLS and verify the remote + --skip-hostname-check Don't check the daemon's hostname against the + name specified in the client certificate + --project-directory PATH Specify an alternate working directory + (default: the path of the Compose file) + --compatibility If set, Compose will attempt to convert keys + in v3 files to their non-Swarm equivalent (DEPRECATED) + --env-file PATH Specify an alternate environment file + + Commands: + build Build or rebuild services + config Validate and view the Compose file + create Create services + down Stop and remove resources + events Receive real time events from containers + exec Execute a command in a running container + help Get help on a command + images List images + kill Kill containers + logs View output from containers + pause Pause services + port Print the public port for a port binding + ps List containers + pull Pull service images + push Push service images + restart Restart services + rm Remove stopped containers + run Run a one-off command + scale Set number of containers for a service + start Start services + stop Stop services + top Display the running processes + unpause Unpause services + up Create and start containers + version Show version information and quit + """ + + def __init__(self, project, options=None): + self.project = project + self.toplevel_options = options or {} + + @property + def project_dir(self): + return get_project_dir(self.toplevel_options) + + @property + def toplevel_environment(self): + environment_file = self.toplevel_options.get('--env-file') + return Environment.from_env_file(self.project_dir, environment_file) + + @metrics() + def build(self, options): + """ + Build or rebuild services. + + Services are built once and then tagged as `project_service`, + e.g. `composetest_db`. If you change a service's `Dockerfile` or the + contents of its build directory, you can run `docker-compose build` to rebuild it. + + Usage: build [options] [--build-arg key=val...] [--] [SERVICE...] + + Options: + --build-arg key=val Set build-time variables for services. + --compress Compress the build context using gzip. + --force-rm Always remove intermediate containers. + -m, --memory MEM Set memory limit for the build container. + --no-cache Do not use cache when building the image. + --no-rm Do not remove intermediate containers after a successful build. + --parallel Build images in parallel. + --progress string Set type of progress output (auto, plain, tty). + --pull Always attempt to pull a newer version of the image. + -q, --quiet Don't print anything to STDOUT + """ + service_names = options['SERVICE'] + build_args = options.get('--build-arg', None) + if build_args: + if not service_names and docker.utils.version_lt(self.project.client.api_version, '1.25'): + raise UserError( + '--build-arg is only supported when services are specified for API version < 1.25.' + ' Please use a Compose file version > 2.2 or specify which services to build.' + ) + build_args = resolve_build_args(build_args, self.toplevel_environment) + + native_builder = self.toplevel_environment.get_boolean('COMPOSE_DOCKER_CLI_BUILD', True) + + self.project.build( + service_names=options['SERVICE'], + no_cache=bool(options.get('--no-cache', False)), + pull=bool(options.get('--pull', False)), + force_rm=bool(options.get('--force-rm', False)), + memory=options.get('--memory'), + rm=not bool(options.get('--no-rm', False)), + build_args=build_args, + gzip=options.get('--compress', False), + parallel_build=options.get('--parallel', False), + silent=options.get('--quiet', False), + cli=native_builder, + progress=options.get('--progress'), + ) + + @metrics() + def config(self, options): + """ + Validate and view the Compose file. + + Usage: config [options] + + Options: + --resolve-image-digests Pin image tags to digests. + --no-interpolate Don't interpolate environment variables. + -q, --quiet Only validate the configuration, don't print + anything. + --services Print the service names, one per line. + --volumes Print the volume names, one per line. + --hash="*" Print the service config hash, one per line. + Set "service1,service2" for a list of specified services + or use the wildcard symbol to display all services. + """ + + additional_options = {'--no-interpolate': options.get('--no-interpolate')} + compose_config = get_config_from_options('.', self.toplevel_options, additional_options) + image_digests = None + + if options['--resolve-image-digests']: + self.project = project_from_options('.', self.toplevel_options, additional_options) + with errors.handle_connection_errors(self.project.client): + image_digests = image_digests_for_project(self.project) + + if options['--quiet']: + return + + if options['--services']: + print('\n'.join(service['name'] for service in compose_config.services)) + return + + if options['--volumes']: + print('\n'.join(volume for volume in compose_config.volumes)) + return + + if options['--hash'] is not None: + h = options['--hash'] + self.project = project_from_options('.', self.toplevel_options, additional_options) + services = [svc for svc in options['--hash'].split(',')] if h != '*' else None + with errors.handle_connection_errors(self.project.client): + for service in self.project.get_services(services): + print('{} {}'.format(service.name, service.config_hash)) + return + + print(serialize_config(compose_config, image_digests, not options['--no-interpolate'])) + + @metrics() + def create(self, options): + """ + Creates containers for a service. + This command is deprecated. Use the `up` command with `--no-start` instead. + + Usage: create [options] [SERVICE...] + + Options: + --force-recreate Recreate containers even if their configuration and + image haven't changed. Incompatible with --no-recreate. + --no-recreate If containers already exist, don't recreate them. + Incompatible with --force-recreate. + --no-build Don't build an image, even if it's missing. + --build Build images before creating containers. + """ + service_names = options['SERVICE'] + + log.warning( + 'The create command is deprecated. ' + 'Use the up command with the --no-start flag instead.' + ) + + self.project.create( + service_names=service_names, + strategy=convergence_strategy_from_opts(options), + do_build=build_action_from_opts(options), + ) + + @metrics() + def down(self, options): + """ + Stops containers and removes containers, networks, volumes, and images + created by `up`. + + By default, the only things removed are: + + - Containers for services defined in the Compose file + - Networks defined in the `networks` section of the Compose file + - The default network, if one is used + + Networks and volumes defined as `external` are never removed. + + Usage: down [options] + + Options: + --rmi type Remove images. Type must be one of: + 'all': Remove all images used by any service. + 'local': Remove only images that don't have a + custom tag set by the `image` field. + -v, --volumes Remove named volumes declared in the `volumes` + section of the Compose file and anonymous volumes + attached to containers. + --remove-orphans Remove containers for services not defined in the + Compose file + -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. + (default: 10) + """ + ignore_orphans = self.toplevel_environment.get_boolean('COMPOSE_IGNORE_ORPHANS') + + if ignore_orphans and options['--remove-orphans']: + raise UserError("COMPOSE_IGNORE_ORPHANS and --remove-orphans cannot be combined.") + + image_type = image_type_from_opt('--rmi', options['--rmi']) + timeout = timeout_from_opts(options) + self.project.down( + image_type, + options['--volumes'], + options['--remove-orphans'], + timeout=timeout, + ignore_orphans=ignore_orphans) + + def events(self, options): + """ + Receive real time events from containers. + + Usage: events [options] [--] [SERVICE...] + + Options: + --json Output events as a stream of json objects + """ + + def format_event(event): + attributes = ["%s=%s" % item for item in event['attributes'].items()] + return ("{time} {type} {action} {id} ({attrs})").format( + attrs=", ".join(sorted(attributes)), + **event) + + def json_format_event(event): + event['time'] = event['time'].isoformat() + event.pop('container') + return json.dumps(event) + + for event in self.project.events(): + formatter = json_format_event if options['--json'] else format_event + print(formatter(event)) + sys.stdout.flush() + + @metrics("exec") + def exec_command(self, options): + """ + Execute a command in a running container + + Usage: exec [options] [-e KEY=VAL...] [--] SERVICE COMMAND [ARGS...] + + Options: + -d, --detach Detached mode: Run command in the background. + --privileged Give extended privileges to the process. + -u, --user USER Run the command as this user. + -T Disable pseudo-tty allocation. By default `docker-compose exec` + allocates a TTY. + --index=index index of the container if there are multiple + instances of a service [default: 1] + -e, --env KEY=VAL Set environment variables (can be used multiple times, + not supported in API < 1.25) + -w, --workdir DIR Path to workdir directory for this command. + """ + use_cli = not self.toplevel_environment.get_boolean('COMPOSE_INTERACTIVE_NO_CLI') + index = int(options.get('--index')) + service = self.project.get_service(options['SERVICE']) + detach = options.get('--detach') + + if options['--env'] and docker.utils.version_lt(self.project.client.api_version, '1.25'): + raise UserError("Setting environment for exec is not supported in API < 1.25 (%s)" + % self.project.client.api_version) + + if options['--workdir'] and docker.utils.version_lt(self.project.client.api_version, '1.35'): + raise UserError("Setting workdir for exec is not supported in API < 1.35 (%s)" + % self.project.client.api_version) + + try: + container = service.get_container(number=index) + except ValueError as e: + raise UserError(str(e)) + command = [options['COMMAND']] + options['ARGS'] + tty = not options["-T"] + + if IS_WINDOWS_PLATFORM or use_cli and not detach: + sys.exit(call_docker( + build_exec_command(options, container.id, command), + self.toplevel_options, self.toplevel_environment) + ) + + create_exec_options = { + "privileged": options["--privileged"], + "user": options["--user"], + "tty": tty, + "stdin": True, + "workdir": options["--workdir"], + } + + if docker.utils.version_gte(self.project.client.api_version, '1.25'): + create_exec_options["environment"] = options["--env"] + + exec_id = container.create_exec(command, **create_exec_options) + + if detach: + container.start_exec(exec_id, tty=tty, stream=True) + return + + signals.set_signal_handler_to_shutdown() + try: + operation = ExecOperation( + self.project.client, + exec_id, + interactive=tty, + ) + pty = PseudoTerminal(self.project.client, operation) + pty.start() + except signals.ShutdownException: + log.info("received shutdown exception: closing") + exit_code = self.project.client.exec_inspect(exec_id).get("ExitCode") + sys.exit(exit_code) + + @classmethod + @metrics() + def help(cls, options): + """ + Get help on a command. + + Usage: help [COMMAND] + """ + if options['COMMAND']: + subject = get_handler(cls, options['COMMAND']) + else: + subject = cls + + print(getdoc(subject)) + + @metrics() + def images(self, options): + """ + List images used by the created containers. + Usage: images [options] [--] [SERVICE...] + + Options: + -q, --quiet Only display IDs + """ + containers = sorted( + self.project.containers(service_names=options['SERVICE'], stopped=True) + + self.project.containers(service_names=options['SERVICE'], one_off=OneOffFilter.only), + key=attrgetter('name')) + + if options['--quiet']: + for image in {c.image for c in containers}: + print(image.split(':')[1]) + return + + def add_default_tag(img_name): + if ':' not in img_name.split('/')[-1]: + return '{}:latest'.format(img_name) + return img_name + + headers = [ + 'Container', + 'Repository', + 'Tag', + 'Image Id', + 'Size' + ] + rows = [] + for container in containers: + image_config = container.image_config + service = self.project.get_service(container.service) + index = 0 + img_name = add_default_tag(service.image_name) + if img_name in image_config['RepoTags']: + index = image_config['RepoTags'].index(img_name) + repo_tags = ( + image_config['RepoTags'][index].rsplit(':', 1) if image_config['RepoTags'] + else ('<none>', '<none>') + ) + + image_id = image_config['Id'].split(':')[1][:12] + size = human_readable_file_size(image_config['Size']) + rows.append([ + container.name, + repo_tags[0], + repo_tags[1], + image_id, + size + ]) + print(Formatter.table(headers, rows)) + + @metrics() + def kill(self, options): + """ + Force stop service containers. + + Usage: kill [options] [--] [SERVICE...] + + Options: + -s SIGNAL SIGNAL to send to the container. + Default signal is SIGKILL. + """ + signal = options.get('-s', 'SIGKILL') + + self.project.kill(service_names=options['SERVICE'], signal=signal) + + @metrics() + def logs(self, options): + """ + View output from containers. + + Usage: logs [options] [--] [SERVICE...] + + Options: + --no-color Produce monochrome output. + -f, --follow Follow log output. + -t, --timestamps Show timestamps. + --tail="all" Number of lines to show from the end of the logs + for each container. + --no-log-prefix Don't print prefix in logs. + """ + containers = self.project.containers(service_names=options['SERVICE'], stopped=True) + + tail = options['--tail'] + if tail is not None: + if tail.isdigit(): + tail = int(tail) + elif tail != 'all': + raise UserError("tail flag must be all or a number") + log_args = { + 'follow': options['--follow'], + 'tail': tail, + 'timestamps': options['--timestamps'] + } + print("Attaching to", list_containers(containers)) + log_printer_from_project( + self.project, + containers, + set_no_color_if_clicolor(options['--no-color']), + log_args, + event_stream=self.project.events(service_names=options['SERVICE']), + keep_prefix=not options['--no-log-prefix']).run() + + @metrics() + def pause(self, options): + """ + Pause services. + + Usage: pause [SERVICE...] + """ + containers = self.project.pause(service_names=options['SERVICE']) + exit_if(not containers, 'No containers to pause', 1) + + @metrics() + def port(self, options): + """ + Print the public port for a port binding. + + Usage: port [options] [--] SERVICE PRIVATE_PORT + + Options: + --protocol=proto tcp or udp [default: tcp] + --index=index index of the container if there are multiple + instances of a service [default: 1] + """ + index = int(options.get('--index')) + service = self.project.get_service(options['SERVICE']) + try: + container = service.get_container(number=index) + except ValueError as e: + raise UserError(str(e)) + print(container.get_local_port( + options['PRIVATE_PORT'], + protocol=options.get('--protocol') or 'tcp') or '') + + @metrics() + def ps(self, options): + """ + List containers. + + Usage: ps [options] [--] [SERVICE...] + + Options: + -q, --quiet Only display IDs + --services Display services + --filter KEY=VAL Filter services by a property + -a, --all Show all stopped containers (including those created by the run command) + """ + if options['--quiet'] and options['--services']: + raise UserError('--quiet and --services cannot be combined') + + if options['--services']: + filt = build_filter(options.get('--filter')) + services = self.project.services + if filt: + services = filter_services(filt, services, self.project) + print('\n'.join(service.name for service in services)) + return + + if options['--all']: + containers = sorted(self.project.containers(service_names=options['SERVICE'], + one_off=OneOffFilter.include, stopped=True), + key=attrgetter('name')) + else: + containers = sorted( + self.project.containers(service_names=options['SERVICE'], stopped=True) + + self.project.containers(service_names=options['SERVICE'], one_off=OneOffFilter.only), + key=attrgetter('name')) + + if options['--quiet']: + for container in containers: + print(container.id) + else: + headers = [ + 'Name', + 'Command', + 'State', + 'Ports', + ] + rows = [] + for container in containers: + command = container.human_readable_command + if len(command) > 30: + command = '%s ...' % command[:26] + rows.append([ + container.name, + command, + container.human_readable_state, + container.human_readable_ports, + ]) + print(Formatter.table(headers, rows)) + + @metrics() + def pull(self, options): + """ + Pulls images for services defined in a Compose file, but does not start the containers. + + Usage: pull [options] [--] [SERVICE...] + + Options: + --ignore-pull-failures Pull what it can and ignores images with pull failures. + --parallel Deprecated, pull multiple images in parallel (enabled by default). + --no-parallel Disable parallel pulling. + -q, --quiet Pull without printing progress information + --include-deps Also pull services declared as dependencies + """ + if options.get('--parallel'): + log.warning('--parallel option is deprecated and will be removed in future versions.') + self.project.pull( + service_names=options['SERVICE'], + ignore_pull_failures=options.get('--ignore-pull-failures'), + parallel_pull=not options.get('--no-parallel'), + silent=options.get('--quiet'), + include_deps=options.get('--include-deps'), + ) + + @metrics() + def push(self, options): + """ + Pushes images for services. + + Usage: push [options] [--] [SERVICE...] + + Options: + --ignore-push-failures Push what it can and ignores images with push failures. + """ + self.project.push( + service_names=options['SERVICE'], + ignore_push_failures=options.get('--ignore-push-failures') + ) + + @metrics() + def rm(self, options): + """ + Removes stopped service containers. + + By default, anonymous volumes attached to containers will not be removed. You + can override this with `-v`. To list all volumes, use `docker volume ls`. + + Any data which is not in a volume will be lost. + + Usage: rm [options] [--] [SERVICE...] + + Options: + -f, --force Don't ask to confirm removal + -s, --stop Stop the containers, if required, before removing + -v Remove any anonymous volumes attached to containers + -a, --all Deprecated - no effect. + """ + if options.get('--all'): + log.warning( + '--all flag is obsolete. This is now the default behavior ' + 'of `docker-compose rm`' + ) + one_off = OneOffFilter.include + + if options.get('--stop'): + self.project.stop(service_names=options['SERVICE'], one_off=one_off) + + all_containers = self.project.containers( + service_names=options['SERVICE'], stopped=True, one_off=one_off + ) + stopped_containers = [c for c in all_containers if not c.is_running] + + if len(stopped_containers) > 0: + print("Going to remove", list_containers(stopped_containers)) + if options.get('--force') \ + or yesno("Are you sure? [yN] ", default=False): + self.project.remove_stopped( + service_names=options['SERVICE'], + v=options.get('-v', False), + one_off=one_off + ) + else: + print("No stopped containers") + + @metrics() + def run(self, options): + """ + Run a one-off command on a service. + + For example: + + $ docker-compose run web python manage.py shell + + By default, linked services will be started, unless they are already + running. If you do not want to start linked services, use + `docker-compose run --no-deps SERVICE COMMAND [ARGS...]`. + + Usage: + run [options] [-v VOLUME...] [-p PORT...] [-e KEY=VAL...] [-l KEY=VALUE...] [--] + SERVICE [COMMAND] [ARGS...] + + Options: + -d, --detach Detached mode: Run container in the background, print + new container name. + --name NAME Assign a name to the container + --entrypoint CMD Override the entrypoint of the image. + -e KEY=VAL Set an environment variable (can be used multiple times) + -l, --label KEY=VAL Add or override a label (can be used multiple times) + -u, --user="" Run as specified username or uid + --no-deps Don't start linked services. + --rm Remove container after run. Ignored in detached mode. + -p, --publish=[] Publish a container's port(s) to the host + --service-ports Run command with the service's ports enabled and mapped + to the host. + --use-aliases Use the service's network aliases in the network(s) the + container connects to. + -v, --volume=[] Bind mount a volume (default []) + -T Disable pseudo-tty allocation. By default `docker-compose run` + allocates a TTY. + -w, --workdir="" Working directory inside the container + """ + service = self.project.get_service(options['SERVICE']) + detach = options.get('--detach') + + if options['--publish'] and options['--service-ports']: + raise UserError( + 'Service port mapping and manual port mapping ' + 'can not be used together' + ) + + if options['COMMAND'] is not None: + command = [options['COMMAND']] + options['ARGS'] + elif options['--entrypoint'] is not None: + command = [] + else: + command = service.options.get('command') + + options['stdin_open'] = service.options.get('stdin_open', True) + + container_options = build_one_off_container_options(options, detach, command) + run_one_off_container( + container_options, self.project, service, options, + self.toplevel_options, self.toplevel_environment + ) + + @metrics() + def scale(self, options): + """ + Set number of containers to run for a service. + + Numbers are specified in the form `service=num` as arguments. + For example: + + $ docker-compose scale web=2 worker=3 + + This command is deprecated. Use the up command with the `--scale` flag + instead. + + Usage: scale [options] [SERVICE=NUM...] + + Options: + -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. + (default: 10) + """ + timeout = timeout_from_opts(options) + + log.warning( + 'The scale command is deprecated. ' + 'Use the up command with the --scale flag instead.' + ) + + for service_name, num in parse_scale_args(options['SERVICE=NUM']).items(): + self.project.get_service(service_name).scale(num, timeout=timeout) + + @metrics() + def start(self, options): + """ + Start existing containers. + + Usage: start [SERVICE...] + """ + containers = self.project.start(service_names=options['SERVICE']) + exit_if(not containers, 'No containers to start', 1) + + @metrics() + def stop(self, options): + """ + Stop running containers without removing them. + + They can be started again with `docker-compose start`. + + Usage: stop [options] [--] [SERVICE...] + + Options: + -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. + (default: 10) + """ + timeout = timeout_from_opts(options) + self.project.stop(service_names=options['SERVICE'], timeout=timeout) + + @metrics() + def restart(self, options): + """ + Restart running containers. + + Usage: restart [options] [--] [SERVICE...] + + Options: + -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. + (default: 10) + """ + timeout = timeout_from_opts(options) + containers = self.project.restart(service_names=options['SERVICE'], timeout=timeout) + exit_if(not containers, 'No containers to restart', 1) + + @metrics() + def top(self, options): + """ + Display the running processes + + Usage: top [SERVICE...] + + """ + containers = sorted( + self.project.containers(service_names=options['SERVICE'], stopped=False) + + self.project.containers(service_names=options['SERVICE'], one_off=OneOffFilter.only), + key=attrgetter('name') + ) + + for idx, container in enumerate(containers): + if idx > 0: + print() + + top_data = self.project.client.top(container.name) + headers = top_data.get("Titles") + rows = [] + + for process in top_data.get("Processes", []): + rows.append(process) + + print(container.name) + print(Formatter.table(headers, rows)) + + @metrics() + def unpause(self, options): + """ + Unpause services. + + Usage: unpause [SERVICE...] + """ + containers = self.project.unpause(service_names=options['SERVICE']) + exit_if(not containers, 'No containers to unpause', 1) + + @metrics() + def up(self, options): + """ + Builds, (re)creates, starts, and attaches to containers for a service. + + Unless they are already running, this command also starts any linked services. + + The `docker-compose up` command aggregates the output of each container. When + the command exits, all containers are stopped. Running `docker-compose up -d` + starts the containers in the background and leaves them running. + + If there are existing containers for a service, and the service's configuration + or image was changed after the container's creation, `docker-compose up` picks + up the changes by stopping and recreating the containers (preserving mounted + volumes). To prevent Compose from picking up changes, use the `--no-recreate` + flag. + + If you want to force Compose to stop and recreate all containers, use the + `--force-recreate` flag. + + Usage: up [options] [--scale SERVICE=NUM...] [--] [SERVICE...] + + Options: + -d, --detach Detached mode: Run containers in the background, + print new container names. Incompatible with + --abort-on-container-exit. + --no-color Produce monochrome output. + --quiet-pull Pull without printing progress information + --no-deps Don't start linked services. + --force-recreate Recreate containers even if their configuration + and image haven't changed. + --always-recreate-deps Recreate dependent containers. + Incompatible with --no-recreate. + --no-recreate If containers already exist, don't recreate + them. Incompatible with --force-recreate and -V. + --no-build Don't build an image, even if it's missing. + --no-start Don't start the services after creating them. + --build Build images before starting containers. + --abort-on-container-exit Stops all containers if any container was + stopped. Incompatible with -d. + --attach-dependencies Attach to dependent containers. + -t, --timeout TIMEOUT Use this timeout in seconds for container + shutdown when attached or when containers are + already running. (default: 10) + -V, --renew-anon-volumes Recreate anonymous volumes instead of retrieving + data from the previous containers. + --remove-orphans Remove containers for services not defined + in the Compose file. + --exit-code-from SERVICE Return the exit code of the selected service + container. Implies --abort-on-container-exit. + --scale SERVICE=NUM Scale SERVICE to NUM instances. Overrides the + `scale` setting in the Compose file if present. + --no-log-prefix Don't print prefix in logs. + """ + start_deps = not options['--no-deps'] + always_recreate_deps = options['--always-recreate-deps'] + exit_value_from = exitval_from_opts(options, self.project) + cascade_stop = options['--abort-on-container-exit'] + service_names = options['SERVICE'] + timeout = timeout_from_opts(options) + remove_orphans = options['--remove-orphans'] + detached = options.get('--detach') + no_start = options.get('--no-start') + attach_dependencies = options.get('--attach-dependencies') + keep_prefix = not options['--no-log-prefix'] + + if detached and (cascade_stop or exit_value_from or attach_dependencies): + raise UserError( + "-d cannot be combined with --abort-on-container-exit or --attach-dependencies.") + + ignore_orphans = self.toplevel_environment.get_boolean('COMPOSE_IGNORE_ORPHANS') + + if ignore_orphans and remove_orphans: + raise UserError("COMPOSE_IGNORE_ORPHANS and --remove-orphans cannot be combined.") + + opts = ['--detach', '--abort-on-container-exit', '--exit-code-from', '--attach-dependencies'] + for excluded in [x for x in opts if options.get(x) and no_start]: + raise UserError('--no-start and {} cannot be combined.'.format(excluded)) + + native_builder = self.toplevel_environment.get_boolean('COMPOSE_DOCKER_CLI_BUILD', True) + + with up_shutdown_context(self.project, service_names, timeout, detached): + warn_for_swarm_mode(self.project.client) + + def up(rebuild): + return self.project.up( + service_names=service_names, + start_deps=start_deps, + strategy=convergence_strategy_from_opts(options), + do_build=build_action_from_opts(options), + timeout=timeout, + detached=detached, + remove_orphans=remove_orphans, + ignore_orphans=ignore_orphans, + scale_override=parse_scale_args(options['--scale']), + start=not no_start, + always_recreate_deps=always_recreate_deps, + reset_container_image=rebuild, + renew_anonymous_volumes=options.get('--renew-anon-volumes'), + silent=options.get('--quiet-pull'), + cli=native_builder, + attach_dependencies=attach_dependencies, + ) + + try: + to_attach = up(False) + except docker.errors.ImageNotFound as e: + log.error( + "The image for the service you're trying to recreate has been removed. " + "If you continue, volume data could be lost. Consider backing up your data " + "before continuing.\n" + ) + res = yesno("Continue with the new image? [yN]", False) + if res is None or not res: + raise e + + to_attach = up(True) + + if detached or no_start: + return + + attached_containers = filter_attached_containers( + to_attach, + service_names, + attach_dependencies) + + log_printer = log_printer_from_project( + self.project, + attached_containers, + set_no_color_if_clicolor(options['--no-color']), + {'follow': True}, + cascade_stop, + event_stream=self.project.events(service_names=service_names), + keep_prefix=keep_prefix) + print("Attaching to", list_containers(log_printer.containers)) + cascade_starter = log_printer.run() + + if cascade_stop: + print("Aborting on container exit...") + all_containers = self.project.containers(service_names=options['SERVICE'], stopped=True) + exit_code = compute_exit_code( + exit_value_from, attached_containers, cascade_starter, all_containers + ) + + self.project.stop(service_names=service_names, timeout=timeout) + if exit_value_from: + exit_code = compute_service_exit_code(exit_value_from, attached_containers) + + sys.exit(exit_code) + + @classmethod + @metrics() + def version(cls, options): + """ + Show version information and quit. + + Usage: version [--short] + + Options: + --short Shows only Compose's version number. + """ + if options['--short']: + print(__version__) + else: + print(get_version_info('full')) + + +def compute_service_exit_code(exit_value_from, attached_containers): + candidates = list(filter( + lambda c: c.service == exit_value_from, + attached_containers)) + if not candidates: + log.error( + 'No containers matching the spec "{}" ' + 'were run.'.format(exit_value_from) + ) + return 2 + if len(candidates) > 1: + exit_values = filter( + lambda e: e != 0, + [c.inspect()['State']['ExitCode'] for c in candidates] + ) + + return exit_values[0] + return candidates[0].inspect()['State']['ExitCode'] + + +def compute_exit_code(exit_value_from, attached_containers, cascade_starter, all_containers): + exit_code = 0 + for e in all_containers: + if (not e.is_running and cascade_starter == e.name): + if not e.exit_code == 0: + exit_code = e.exit_code + break + + return exit_code + + +def convergence_strategy_from_opts(options): + no_recreate = options['--no-recreate'] + force_recreate = options['--force-recreate'] + renew_anonymous_volumes = options.get('--renew-anon-volumes') + if force_recreate and no_recreate: + raise UserError("--force-recreate and --no-recreate cannot be combined.") + + if no_recreate and renew_anonymous_volumes: + raise UserError('--no-recreate and --renew-anon-volumes cannot be combined.') + + if force_recreate or renew_anonymous_volumes: + return ConvergenceStrategy.always + + if no_recreate: + return ConvergenceStrategy.never + + return ConvergenceStrategy.changed + + +def timeout_from_opts(options): + timeout = options.get('--timeout') + return None if timeout is None else int(timeout) + + +def image_digests_for_project(project): + try: + return get_image_digests(project) + + except MissingDigests as e: + def list_images(images): + return "\n".join(" {}".format(name) for name in sorted(images)) + + paras = ["Some images are missing digests."] + + if e.needs_push: + command_hint = ( + "Use `docker push {}` to push them. " + .format(" ".join(sorted(e.needs_push))) + ) + paras += [ + "The following images can be pushed:", + list_images(e.needs_push), + command_hint, + ] + + if e.needs_pull: + command_hint = ( + "Use `docker pull {}` to pull them. " + .format(" ".join(sorted(e.needs_pull))) + ) + + paras += [ + "The following images need to be pulled:", + list_images(e.needs_pull), + command_hint, + ] + + raise UserError("\n\n".join(paras)) + + +def exitval_from_opts(options, project): + exit_value_from = options.get('--exit-code-from') + if exit_value_from: + if not options.get('--abort-on-container-exit'): + log.warning('using --exit-code-from implies --abort-on-container-exit') + options['--abort-on-container-exit'] = True + if exit_value_from not in [s.name for s in project.get_services()]: + log.error('No service named "%s" was found in your compose file.', + exit_value_from) + sys.exit(2) + return exit_value_from + + +def image_type_from_opt(flag, value): + if not value: + return ImageType.none + try: + return ImageType[value] + except KeyError: + raise UserError("%s flag must be one of: all, local" % flag) + + +def build_action_from_opts(options): + if options['--build'] and options['--no-build']: + raise UserError("--build and --no-build can not be combined.") + + if options['--build']: + return BuildAction.force + + if options['--no-build']: + return BuildAction.skip + + return BuildAction.none + + +def build_one_off_container_options(options, detach, command): + container_options = { + 'command': command, + 'tty': not (detach or options['-T'] or not sys.stdin.isatty()), + 'stdin_open': options.get('stdin_open'), + 'detach': detach, + } + + if options['-e']: + container_options['environment'] = Environment.from_command_line( + parse_environment(options['-e']) + ) + + if options['--label']: + container_options['labels'] = parse_labels(options['--label']) + + if options.get('--entrypoint') is not None: + container_options['entrypoint'] = ( + [""] if options['--entrypoint'] == '' else options['--entrypoint'] + ) + + # Ensure that run command remains one-off (issue #6302) + container_options['restart'] = None + + if options['--user']: + container_options['user'] = options.get('--user') + + if not options['--service-ports']: + container_options['ports'] = [] + + if options['--publish']: + container_options['ports'] = options.get('--publish') + + if options['--name']: + container_options['name'] = options['--name'] + + if options['--workdir']: + container_options['working_dir'] = options['--workdir'] + + if options['--volume']: + volumes = [VolumeSpec.parse(i) for i in options['--volume']] + container_options['volumes'] = volumes + + return container_options + + +def run_one_off_container(container_options, project, service, options, toplevel_options, + toplevel_environment): + native_builder = toplevel_environment.get_boolean('COMPOSE_DOCKER_CLI_BUILD') + detach = options.get('--detach') + use_network_aliases = options.get('--use-aliases') + service.scale_num = 1 + containers = project.up( + service_names=[service.name], + start_deps=not options['--no-deps'], + strategy=ConvergenceStrategy.never, + detached=True, + rescale=False, + cli=native_builder, + one_off=True, + override_options=container_options, + ) + try: + container = next(c for c in containers if c.service == service.name) + except StopIteration: + raise OperationFailedError('Could not bring up the requested service') + + if detach: + service.start_container(container, use_network_aliases) + print(container.name) + return + + def remove_container(): + if options['--rm']: + project.client.remove_container(container.id, force=True, v=True) + + use_cli = not toplevel_environment.get_boolean('COMPOSE_INTERACTIVE_NO_CLI') + + signals.set_signal_handler_to_shutdown() + signals.set_signal_handler_to_hang_up() + try: + try: + if IS_WINDOWS_PLATFORM or use_cli: + service.connect_container_to_networks(container, use_network_aliases) + exit_code = call_docker( + get_docker_start_call(container_options, container.id), + toplevel_options, toplevel_environment + ) + else: + operation = RunOperation( + project.client, + container.id, + interactive=not options['-T'], + logs=False, + ) + pty = PseudoTerminal(project.client, operation) + sockets = pty.sockets() + service.start_container(container, use_network_aliases) + pty.start(sockets) + exit_code = container.wait() + except (signals.ShutdownException): + project.client.stop(container.id) + exit_code = 1 + except (signals.ShutdownException, signals.HangUpException): + project.client.kill(container.id) + remove_container() + sys.exit(2) + + remove_container() + sys.exit(exit_code) + + +def get_docker_start_call(container_options, container_id): + docker_call = ["start"] + if not container_options.get('detach'): + docker_call.append("--attach") + if container_options.get('stdin_open'): + docker_call.append("--interactive") + docker_call.append(container_id) + return docker_call + + +def log_printer_from_project( + project, + containers, + monochrome, + log_args, + cascade_stop=False, + event_stream=None, + keep_prefix=True, +): + return LogPrinter( + containers, + build_log_presenters(project.service_names, monochrome, keep_prefix), + event_stream or project.events(), + cascade_stop=cascade_stop, + log_args=log_args) + + +def filter_attached_containers(containers, service_names, attach_dependencies=False): + return filter_attached_for_up( + containers, + service_names, + attach_dependencies, + lambda container: container.service) + + +@contextlib.contextmanager +def up_shutdown_context(project, service_names, timeout, detached): + if detached: + yield + return + + signals.set_signal_handler_to_shutdown() + try: + try: + yield + except signals.ShutdownException: + print("Gracefully stopping... (press Ctrl+C again to force)") + project.stop(service_names=service_names, timeout=timeout) + except signals.ShutdownException: + project.kill(service_names=service_names) + sys.exit(2) + + +def list_containers(containers): + return ", ".join(c.name for c in containers) + + +def exit_if(condition, message, exit_code): + if condition: + log.error(message) + raise SystemExit(exit_code) + + +def call_docker(args, dockeropts, environment): + executable_path = find_executable('docker') + if not executable_path: + raise UserError(errors.docker_not_found_msg("Couldn't find `docker` binary.")) + + tls = dockeropts.get('--tls', False) + ca_cert = dockeropts.get('--tlscacert') + cert = dockeropts.get('--tlscert') + key = dockeropts.get('--tlskey') + verify = dockeropts.get('--tlsverify') + host = dockeropts.get('--host') + context = dockeropts.get('--context') + tls_options = [] + if tls: + tls_options.append('--tls') + if ca_cert: + tls_options.extend(['--tlscacert', ca_cert]) + if cert: + tls_options.extend(['--tlscert', cert]) + if key: + tls_options.extend(['--tlskey', key]) + if verify: + tls_options.append('--tlsverify') + if host: + tls_options.extend( + ['--host', re.sub(r'^https?://', 'tcp://', host.lstrip('='))] + ) + if context: + tls_options.extend( + ['--context', context] + ) + + args = [executable_path] + tls_options + args + log.debug(" ".join(map(pipes.quote, args))) + + filtered_env = {k: v for k, v in environment.items() if v is not None} + + return subprocess.call(args, env=filtered_env) + + +def parse_scale_args(options): + res = {} + for s in options: + if '=' not in s: + raise UserError('Arguments to scale should be in the form service=num') + service_name, num = s.split('=', 1) + try: + num = int(num) + except ValueError: + raise UserError( + 'Number of containers for service "%s" is not a number' % service_name + ) + res[service_name] = num + return res + + +def build_exec_command(options, container_id, command): + args = ["exec"] + + if options["--detach"]: + args += ["--detach"] + else: + args += ["--interactive"] + + if not options["-T"]: + args += ["--tty"] + + if options["--privileged"]: + args += ["--privileged"] + + if options["--user"]: + args += ["--user", options["--user"]] + + if options["--env"]: + for env_variable in options["--env"]: + args += ["--env", env_variable] + + if options["--workdir"]: + args += ["--workdir", options["--workdir"]] + + args += [container_id] + args += command + return args + + +def has_container_with_state(containers, state): + states = { + 'running': lambda c: c.is_running, + 'stopped': lambda c: not c.is_running, + 'paused': lambda c: c.is_paused, + 'restarting': lambda c: c.is_restarting, + } + for container in containers: + if state not in states: + raise UserError("Invalid state: %s" % state) + if states[state](container): + return True + + +def filter_services(filt, services, project): + def should_include(service): + for f in filt: + if f == 'status': + state = filt[f] + containers = project.containers([service.name], stopped=True) + if not has_container_with_state(containers, state): + return False + elif f == 'source': + source = filt[f] + if source == 'image' or source == 'build': + if source not in service.options: + return False + else: + raise UserError("Invalid value for source filter: %s" % source) + else: + raise UserError("Invalid filter: %s" % f) + return True + + return filter(should_include, services) + + +def build_filter(arg): + filt = {} + if arg is not None: + if '=' not in arg: + raise UserError("Arguments to --filter should be in form KEY=VAL") + key, val = arg.split('=', 1) + filt[key] = val + return filt + + +def warn_for_swarm_mode(client): + info = client.info() + if info.get('Swarm', {}).get('LocalNodeState') == 'active': + if info.get('ServerVersion', '').startswith('ucp'): + # UCP does multi-node scheduling with traditional Compose files. + return + + log.warning( + "The Docker Engine you're using is running in swarm mode.\n\n" + "Compose does not use swarm mode to deploy services to multiple nodes in a swarm. " + "All containers will be scheduled on the current node.\n\n" + "To deploy your application across the swarm, " + "use `docker stack deploy`.\n" + ) + + +def set_no_color_if_clicolor(no_color_flag): + return no_color_flag or os.environ.get('CLICOLOR') == "0" diff --git a/compose/cli/signals.py b/compose/cli/signals.py new file mode 100644 index 00000000000..0244e70189a --- /dev/null +++ b/compose/cli/signals.py @@ -0,0 +1,41 @@ +import signal + +from ..const import IS_WINDOWS_PLATFORM + + +class ShutdownException(Exception): + pass + + +class HangUpException(Exception): + pass + + +def shutdown(signal, frame): + raise ShutdownException() + + +def set_signal_handler(handler): + signal.signal(signal.SIGINT, handler) + signal.signal(signal.SIGTERM, handler) + + +def set_signal_handler_to_shutdown(): + set_signal_handler(shutdown) + + +def hang_up(signal, frame): + raise HangUpException() + + +def set_signal_handler_to_hang_up(): + # on Windows a ValueError will be raised if trying to set signal handler for SIGHUP + if not IS_WINDOWS_PLATFORM: + signal.signal(signal.SIGHUP, hang_up) + + +def ignore_sigpipe(): + # Restore default behavior for SIGPIPE instead of raising + # an exception when encountered. + if not IS_WINDOWS_PLATFORM: + signal.signal(signal.SIGPIPE, signal.SIG_DFL) diff --git a/compose/cli/utils.py b/compose/cli/utils.py new file mode 100644 index 00000000000..6a4615a9660 --- /dev/null +++ b/compose/cli/utils.py @@ -0,0 +1,144 @@ +import math +import os +import platform +import ssl +import subprocess +import sys + +import distro +import docker + +import compose +from ..const import IS_WINDOWS_PLATFORM + + +def yesno(prompt, default=None): + """ + Prompt the user for a yes or no. + + Can optionally specify a default value, which will only be + used if they enter a blank line. + + Unrecognised input (anything other than "y", "n", "yes", + "no" or "") will return None. + """ + answer = input(prompt).strip().lower() + + if answer == "y" or answer == "yes": + return True + elif answer == "n" or answer == "no": + return False + elif answer == "": + return default + else: + return None + + +def input(prompt): + """ + Version of input (raw_input in Python 2) which forces a flush of sys.stdout + to avoid problems where the prompt fails to appear due to line buffering + """ + sys.stdout.write(prompt) + sys.stdout.flush() + return sys.stdin.readline().rstrip('\n') + + +def call_silently(*args, **kwargs): + """ + Like subprocess.call(), but redirects stdout and stderr to /dev/null. + """ + with open(os.devnull, 'w') as shutup: + try: + return subprocess.call(*args, stdout=shutup, stderr=shutup, **kwargs) + except OSError: + # On Windows, subprocess.call() can still raise exceptions. Normalize + # to POSIXy behaviour by returning a nonzero exit code. + return 1 + + +def is_mac(): + return platform.system() == 'Darwin' + + +def is_ubuntu(): + return platform.system() == 'Linux' and distro.linux_distribution()[0] == 'Ubuntu' + + +def is_windows(): + return IS_WINDOWS_PLATFORM + + +def get_version_info(scope): + versioninfo = 'docker-compose version {}, build {}'.format( + compose.__version__, + get_build_version()) + + if scope == 'compose': + return versioninfo + if scope == 'full': + return ( + "{}\n" + "docker-py version: {}\n" + "{} version: {}\n" + "OpenSSL version: {}" + ).format( + versioninfo, + docker.version, + platform.python_implementation(), + platform.python_version(), + ssl.OPENSSL_VERSION) + + raise ValueError("{} is not a valid version scope".format(scope)) + + +def get_build_version(): + filename = os.path.join(os.path.dirname(compose.__file__), 'GITSHA') + if not os.path.exists(filename): + return 'unknown' + + with open(filename) as fh: + return fh.read().strip() + + +def is_docker_for_mac_installed(): + return is_mac() and os.path.isdir('/Applications/Docker.app') + + +def generate_user_agent(): + parts = [ + "docker-compose/{}".format(compose.__version__), + "docker-py/{}".format(docker.__version__), + ] + try: + p_system = platform.system() + p_release = platform.release() + except OSError: + pass + else: + parts.append("{}/{}".format(p_system, p_release)) + return " ".join(parts) + + +def human_readable_file_size(size): + suffixes = ['B', 'kB', 'MB', 'GB', 'TB', 'PB', 'EB', ] + order = int(math.log(size, 1000)) if size else 0 + if order >= len(suffixes): + order = len(suffixes) - 1 + + return '{:.4g} {}'.format( + size / pow(10, order * 3), + suffixes[order] + ) + + +def binarystr_to_unicode(s): + if not isinstance(s, bytes): + return s + + if IS_WINDOWS_PLATFORM: + try: + return s.decode('windows-1250') + except UnicodeDecodeError: + pass + return s.decode('utf-8', 'replace') diff --git a/compose/cli/verbose_proxy.py b/compose/cli/verbose_proxy.py new file mode 100644 index 00000000000..c9340c4e0d2 --- /dev/null +++ b/compose/cli/verbose_proxy.py @@ -0,0 +1,55 @@ +import functools +import logging +import pprint +from itertools import chain + + +def format_call(args, kwargs): + args = (repr(a) for a in args) + kwargs = ("{!s}={!r}".format(*item) for item in kwargs.items()) + return "({})".format(", ".join(chain(args, kwargs))) + + +def format_return(result, max_lines): + if isinstance(result, (list, tuple, set)): + return "({} with {} items)".format(type(result).__name__, len(result)) + + if result: + lines = pprint.pformat(result).split('\n') + extra = '\n...' if len(lines) > max_lines else '' + return '\n'.join(lines[:max_lines]) + extra + + return result + + +class VerboseProxy: + """Proxy all function calls to another class and log method name, arguments + and return values for each call. + """ + + def __init__(self, obj_name, obj, log_name=None, max_lines=10): + self.obj_name = obj_name + self.obj = obj + self.max_lines = max_lines + self.log = logging.getLogger(log_name or __name__) + + def __getattr__(self, name): + attr = getattr(self.obj, name) + + if not callable(attr): + return attr + + return functools.partial(self.proxy_callable, name) + + def proxy_callable(self, call_name, *args, **kwargs): + self.log.info("%s %s <- %s", + self.obj_name, + call_name, + format_call(args, kwargs)) + + result = getattr(self.obj, call_name)(*args, **kwargs) + self.log.info("%s %s -> %s", + self.obj_name, + call_name, + format_return(result, self.max_lines)) + return result diff --git a/compose/config/__init__.py b/compose/config/__init__.py new file mode 100644 index 00000000000..855b2401d94 --- /dev/null +++ b/compose/config/__init__.py @@ -0,0 +1,12 @@ +# flake8: noqa +from . import environment +from .config import ConfigurationError +from .config import DOCKER_CONFIG_KEYS +from .config import find +from .config import is_url +from .config import load +from .config import merge_environment +from .config import merge_labels +from .config import parse_environment +from .config import parse_labels +from .config import resolve_build_args diff --git a/compose/config/compose_spec.json b/compose/config/compose_spec.json new file mode 100644 index 00000000000..6d892376795 --- /dev/null +++ b/compose/config/compose_spec.json @@ -0,0 +1,812 @@ +{ + "$schema": "http://json-schema.org/draft/2019-09/schema#", + "id": "compose_spec.json", + "type": "object", + "title": "Compose Specification", + "description": "The Compose file is a YAML file defining a multi-containers based application.", + + "properties": { + "version": { + "type": "string", + "description": "Version of the Compose specification used. Tools not implementing required version MUST reject the configuration file." + }, + + "services": { + "id": "#/properties/services", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/service" + } + }, + "additionalProperties": false + }, + + "networks": { + "id": "#/properties/networks", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/network" + } + } + }, + + "volumes": { + "id": "#/properties/volumes", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/volume" + } + }, + "additionalProperties": false + }, + + "secrets": { + "id": "#/properties/secrets", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/secret" + } + }, + "additionalProperties": false + }, + + "configs": { + "id": "#/properties/configs", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/config" + } + }, + "additionalProperties": false + } + }, + + "patternProperties": {"^x-": {}}, + "additionalProperties": false, + + "definitions": { + + "service": { + "id": "#/definitions/service", + "type": "object", + + "properties": { + "deploy": {"$ref": "#/definitions/deployment"}, + "build": { + "oneOf": [ + {"type": "string"}, + { + "type": "object", + "properties": { + "context": {"type": "string"}, + "dockerfile": {"type": "string"}, + "args": {"$ref": "#/definitions/list_or_dict"}, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "cache_from": {"type": "array", "items": {"type": "string"}}, + "network": {"type": "string"}, + "target": {"type": "string"}, + "shm_size": {"type": ["integer", "string"]}, + "extra_hosts": {"$ref": "#/definitions/list_or_dict"}, + "isolation": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + ] + }, + "blkio_config": { + "type": "object", + "properties": { + "device_read_bps": { + "type": "array", + "items": {"$ref": "#/definitions/blkio_limit"} + }, + "device_read_iops": { + "type": "array", + "items": {"$ref": "#/definitions/blkio_limit"} + }, + "device_write_bps": { + "type": "array", + "items": {"$ref": "#/definitions/blkio_limit"} + }, + "device_write_iops": { + "type": "array", + "items": {"$ref": "#/definitions/blkio_limit"} + }, + "weight": {"type": "integer"}, + "weight_device": { + "type": "array", + "items": {"$ref": "#/definitions/blkio_weight"} + } + }, + "additionalProperties": false + }, + "cap_add": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "cap_drop": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "cgroup_parent": {"type": "string"}, + "command": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "configs": { + "type": "array", + "items": { + "oneOf": [ + {"type": "string"}, + { + "type": "object", + "properties": { + "source": {"type": "string"}, + "target": {"type": "string"}, + "uid": {"type": "string"}, + "gid": {"type": "string"}, + "mode": {"type": "number"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + ] + } + }, + "container_name": {"type": "string"}, + "cpu_count": {"type": "integer", "minimum": 0}, + "cpu_percent": {"type": "integer", "minimum": 0, "maximum": 100}, + "cpu_shares": {"type": ["number", "string"]}, + "cpu_quota": {"type": ["number", "string"]}, + "cpu_period": {"type": ["number", "string"]}, + "cpu_rt_period": {"type": ["number", "string"]}, + "cpu_rt_runtime": {"type": ["number", "string"]}, + "cpus": {"type": ["number", "string"]}, + "cpuset": {"type": "string"}, + "credential_spec": { + "type": "object", + "properties": { + "config": {"type": "string"}, + "file": {"type": "string"}, + "registry": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "depends_on": { + "oneOf": [ + {"$ref": "#/definitions/list_of_strings"}, + { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "type": "object", + "additionalProperties": false, + "properties": { + "condition": { + "type": "string", + "enum": ["service_started", "service_healthy"] + } + }, + "required": ["condition"] + } + } + } + ] + }, + "device_cgroup_rules": {"$ref": "#/definitions/list_of_strings"}, + "devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "dns": {"$ref": "#/definitions/string_or_list"}, + "dns_opt": {"type": "array","items": {"type": "string"}, "uniqueItems": true}, + "dns_search": {"$ref": "#/definitions/string_or_list"}, + "domainname": {"type": "string"}, + "entrypoint": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "env_file": {"$ref": "#/definitions/string_or_list"}, + "environment": {"$ref": "#/definitions/list_or_dict"}, + + "expose": { + "type": "array", + "items": { + "type": ["string", "number"], + "format": "expose" + }, + "uniqueItems": true + }, + "extends": { + "oneOf": [ + {"type": "string"}, + { + "type": "object", + + "properties": { + "service": {"type": "string"}, + "file": {"type": "string"} + }, + "required": ["service"], + "additionalProperties": false + } + ] + }, + "external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "extra_hosts": {"$ref": "#/definitions/list_or_dict"}, + "group_add": { + "type": "array", + "items": { + "type": ["string", "number"] + }, + "uniqueItems": true + }, + "healthcheck": {"$ref": "#/definitions/healthcheck"}, + "hostname": {"type": "string"}, + "image": {"type": "string"}, + "init": {"type": "boolean"}, + "ipc": {"type": "string"}, + "isolation": {"type": "string"}, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "logging": { + "type": "object", + + "properties": { + "driver": {"type": "string"}, + "options": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number", "null"]} + } + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "mac_address": {"type": "string"}, + "mem_limit": {"type": ["number", "string"]}, + "mem_reservation": {"type": ["string", "integer"]}, + "mem_swappiness": {"type": "integer"}, + "memswap_limit": {"type": ["number", "string"]}, + "network_mode": {"type": "string"}, + "networks": { + "oneOf": [ + {"$ref": "#/definitions/list_of_strings"}, + { + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "oneOf": [ + { + "type": "object", + "properties": { + "aliases": {"$ref": "#/definitions/list_of_strings"}, + "ipv4_address": {"type": "string"}, + "ipv6_address": {"type": "string"}, + "link_local_ips": {"$ref": "#/definitions/list_of_strings"}, + "priority": {"type": "number"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + {"type": "null"} + ] + } + }, + "additionalProperties": false + } + ] + }, + "oom_kill_disable": {"type": "boolean"}, + "oom_score_adj": {"type": "integer", "minimum": -1000, "maximum": 1000}, + "pid": {"type": ["string", "null"]}, + "pids_limit": {"type": ["number", "string"]}, + "platform": {"type": "string"}, + "ports": { + "type": "array", + "items": { + "oneOf": [ + {"type": "number", "format": "ports"}, + {"type": "string", "format": "ports"}, + { + "type": "object", + "properties": { + "mode": {"type": "string"}, + "target": {"type": "integer"}, + "published": {"type": "integer"}, + "protocol": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + ] + }, + "uniqueItems": true + }, + "privileged": {"type": "boolean"}, + "profiles": {"$ref": "#/definitions/list_of_strings"}, + "pull_policy": {"type": "string", "enum": [ + "always", "never", "if_not_present" + ]}, + "read_only": {"type": "boolean"}, + "restart": {"type": "string"}, + "runtime": { + "deprecated": true, + "type": "string" + }, + "scale": { + "type": "integer" + }, + "security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "shm_size": {"type": ["number", "string"]}, + "secrets": { + "type": "array", + "items": { + "oneOf": [ + {"type": "string"}, + { + "type": "object", + "properties": { + "source": {"type": "string"}, + "target": {"type": "string"}, + "uid": {"type": "string"}, + "gid": {"type": "string"}, + "mode": {"type": "number"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + ] + } + }, + "sysctls": {"$ref": "#/definitions/list_or_dict"}, + "stdin_open": {"type": "boolean"}, + "stop_grace_period": {"type": "string", "format": "duration"}, + "stop_signal": {"type": "string"}, + "tmpfs": {"$ref": "#/definitions/string_or_list"}, + "tty": {"type": "boolean"}, + "ulimits": { + "type": "object", + "patternProperties": { + "^[a-z]+$": { + "oneOf": [ + {"type": "integer"}, + { + "type": "object", + "properties": { + "hard": {"type": "integer"}, + "soft": {"type": "integer"} + }, + "required": ["soft", "hard"], + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + ] + } + } + }, + "user": {"type": "string"}, + "userns_mode": {"type": "string"}, + "volumes": { + "type": "array", + "items": { + "oneOf": [ + {"type": "string"}, + { + "type": "object", + "required": ["type"], + "properties": { + "type": {"type": "string"}, + "source": {"type": "string"}, + "target": {"type": "string"}, + "read_only": {"type": "boolean"}, + "consistency": {"type": "string"}, + "bind": { + "type": "object", + "properties": { + "propagation": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "volume": { + "type": "object", + "properties": { + "nocopy": {"type": "boolean"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "tmpfs": { + "type": "object", + "properties": { + "size": { + "type": "integer", + "minimum": 0 + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + ] + }, + "uniqueItems": true + }, + "volumes_from": { + "type": "array", + "items": {"type": "string"}, + "uniqueItems": true + }, + "working_dir": {"type": "string"} + }, + "patternProperties": {"^x-": {}}, + "additionalProperties": false + }, + + "healthcheck": { + "id": "#/definitions/healthcheck", + "type": "object", + "properties": { + "disable": {"type": "boolean"}, + "interval": {"type": "string", "format": "duration"}, + "retries": {"type": "number"}, + "test": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "timeout": {"type": "string", "format": "duration"}, + "start_period": {"type": "string", "format": "duration"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "deployment": { + "id": "#/definitions/deployment", + "type": ["object", "null"], + "properties": { + "mode": {"type": "string"}, + "endpoint_mode": {"type": "string"}, + "replicas": {"type": "integer"}, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "rollback_config": { + "type": "object", + "properties": { + "parallelism": {"type": "integer"}, + "delay": {"type": "string", "format": "duration"}, + "failure_action": {"type": "string"}, + "monitor": {"type": "string", "format": "duration"}, + "max_failure_ratio": {"type": "number"}, + "order": {"type": "string", "enum": [ + "start-first", "stop-first" + ]} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "update_config": { + "type": "object", + "properties": { + "parallelism": {"type": "integer"}, + "delay": {"type": "string", "format": "duration"}, + "failure_action": {"type": "string"}, + "monitor": {"type": "string", "format": "duration"}, + "max_failure_ratio": {"type": "number"}, + "order": {"type": "string", "enum": [ + "start-first", "stop-first" + ]} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "resources": { + "type": "object", + "properties": { + "limits": { + "type": "object", + "properties": { + "cpus": {"type": ["number", "string"]}, + "memory": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "reservations": { + "type": "object", + "properties": { + "cpus": {"type": ["number", "string"]}, + "memory": {"type": "string"}, + "generic_resources": {"$ref": "#/definitions/generic_resources"}, + "devices": {"$ref": "#/definitions/devices"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "restart_policy": { + "type": "object", + "properties": { + "condition": {"type": "string"}, + "delay": {"type": "string", "format": "duration"}, + "max_attempts": {"type": "integer"}, + "window": {"type": "string", "format": "duration"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "placement": { + "type": "object", + "properties": { + "constraints": {"type": "array", "items": {"type": "string"}}, + "preferences": { + "type": "array", + "items": { + "type": "object", + "properties": { + "spread": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + "max_replicas_per_node": {"type": "integer"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + + "generic_resources": { + "id": "#/definitions/generic_resources", + "type": "array", + "items": { + "type": "object", + "properties": { + "discrete_resource_spec": { + "type": "object", + "properties": { + "kind": {"type": "string"}, + "value": {"type": "number"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + + "devices": { + "id": "#/definitions/devices", + "type": "array", + "items": { + "type": "object", + "properties": { + "capabilities": {"$ref": "#/definitions/list_of_strings"}, + "count": {"type": ["string", "integer"]}, + "device_ids": {"$ref": "#/definitions/list_of_strings"}, + "driver":{"type": "string"}, + "options":{"$ref": "#/definitions/list_or_dict"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + + "network": { + "id": "#/definitions/network", + "type": ["object", "null"], + "properties": { + "name": {"type": "string"}, + "driver": {"type": "string"}, + "driver_opts": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number"]} + } + }, + "ipam": { + "type": "object", + "properties": { + "driver": {"type": "string"}, + "config": { + "type": "array", + "items": { + "type": "object", + "properties": { + "subnet": {"type": "string", "format": "subnet_ip_address"}, + "ip_range": {"type": "string"}, + "gateway": {"type": "string"}, + "aux_addresses": { + "type": "object", + "additionalProperties": false, + "patternProperties": {"^.+$": {"type": "string"}} + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + "options": { + "type": "object", + "additionalProperties": false, + "patternProperties": {"^.+$": {"type": "string"}} + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "external": { + "type": ["boolean", "object"], + "properties": { + "name": { + "deprecated": true, + "type": "string" + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "internal": {"type": "boolean"}, + "enable_ipv6": {"type": "boolean"}, + "attachable": {"type": "boolean"}, + "labels": {"$ref": "#/definitions/list_or_dict"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + + "volume": { + "id": "#/definitions/volume", + "type": ["object", "null"], + "properties": { + "name": {"type": "string"}, + "driver": {"type": "string"}, + "driver_opts": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number"]} + } + }, + "external": { + "type": ["boolean", "object"], + "properties": { + "name": { + "deprecated": true, + "type": "string" + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "labels": {"$ref": "#/definitions/list_or_dict"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + + "secret": { + "id": "#/definitions/secret", + "type": "object", + "properties": { + "name": {"type": "string"}, + "file": {"type": "string"}, + "external": { + "type": ["boolean", "object"], + "properties": { + "name": {"type": "string"} + } + }, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "driver": {"type": "string"}, + "driver_opts": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number"]} + } + }, + "template_driver": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + + "config": { + "id": "#/definitions/config", + "type": "object", + "properties": { + "name": {"type": "string"}, + "file": {"type": "string"}, + "external": { + "type": ["boolean", "object"], + "properties": { + "name": { + "deprecated": true, + "type": "string" + } + } + }, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "template_driver": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + + "string_or_list": { + "oneOf": [ + {"type": "string"}, + {"$ref": "#/definitions/list_of_strings"} + ] + }, + + "list_of_strings": { + "type": "array", + "items": {"type": "string"}, + "uniqueItems": true + }, + + "list_or_dict": { + "oneOf": [ + { + "type": "object", + "patternProperties": { + ".+": { + "type": ["string", "number", "null"] + } + }, + "additionalProperties": false + }, + {"type": "array", "items": {"type": "string"}, "uniqueItems": true} + ] + }, + + "blkio_limit": { + "type": "object", + "properties": { + "path": {"type": "string"}, + "rate": {"type": ["integer", "string"]} + }, + "additionalProperties": false + }, + "blkio_weight": { + "type": "object", + "properties": { + "path": {"type": "string"}, + "weight": {"type": "integer"} + }, + "additionalProperties": false + }, + + "constraints": { + "service": { + "id": "#/definitions/constraints/service", + "anyOf": [ + {"required": ["build"]}, + {"required": ["image"]} + ], + "properties": { + "build": { + "required": ["context"] + } + } + } + } + } +} diff --git a/compose/config/config.py b/compose/config/config.py new file mode 100644 index 00000000000..39c815a9842 --- /dev/null +++ b/compose/config/config.py @@ -0,0 +1,1480 @@ +import functools +import logging +import os +import re +import string +import sys +from collections import namedtuple +from itertools import chain +from operator import attrgetter +from operator import itemgetter + +import yaml +from cached_property import cached_property + +from . import types +from ..const import COMPOSE_SPEC as VERSION +from ..const import COMPOSEFILE_V1 as V1 +from ..utils import build_string_dict +from ..utils import json_hash +from ..utils import parse_bytes +from ..utils import parse_nanoseconds_int +from ..utils import splitdrive +from ..version import ComposeVersion +from .environment import env_vars_from_file +from .environment import Environment +from .environment import split_env +from .errors import CircularReference +from .errors import ComposeFileNotFound +from .errors import ConfigurationError +from .errors import DuplicateOverrideFileFound +from .errors import VERSION_EXPLANATION +from .interpolation import interpolate_environment_variables +from .sort_services import get_container_name_from_network_mode +from .sort_services import get_service_name_from_network_mode +from .sort_services import sort_service_dicts +from .types import MountSpec +from .types import parse_extra_hosts +from .types import parse_restart_spec +from .types import SecurityOpt +from .types import ServiceLink +from .types import ServicePort +from .types import VolumeFromSpec +from .types import VolumeSpec +from .validation import match_named_volumes +from .validation import validate_against_config_schema +from .validation import validate_config_section +from .validation import validate_cpu +from .validation import validate_credential_spec +from .validation import validate_depends_on +from .validation import validate_extends_file_path +from .validation import validate_healthcheck +from .validation import validate_ipc_mode +from .validation import validate_links +from .validation import validate_network_mode +from .validation import validate_pid_mode +from .validation import validate_service_constraints +from .validation import validate_top_level_object +from .validation import validate_ulimits + + +DOCKER_CONFIG_KEYS = [ + 'cap_add', + 'cap_drop', + 'cgroup_parent', + 'command', + 'cpu_count', + 'cpu_percent', + 'cpu_period', + 'cpu_quota', + 'cpu_rt_period', + 'cpu_rt_runtime', + 'cpu_shares', + 'cpus', + 'cpuset', + 'detach', + 'device_cgroup_rules', + 'devices', + 'dns', + 'dns_search', + 'dns_opt', + 'domainname', + 'entrypoint', + 'env_file', + 'environment', + 'extra_hosts', + 'group_add', + 'hostname', + 'healthcheck', + 'image', + 'ipc', + 'isolation', + 'labels', + 'links', + 'mac_address', + 'mem_limit', + 'mem_reservation', + 'memswap_limit', + 'mem_swappiness', + 'net', + 'oom_score_adj', + 'oom_kill_disable', + 'pid', + 'ports', + 'privileged', + 'read_only', + 'restart', + 'runtime', + 'secrets', + 'security_opt', + 'shm_size', + 'pids_limit', + 'stdin_open', + 'stop_signal', + 'sysctls', + 'tty', + 'user', + 'userns_mode', + 'volume_driver', + 'volumes', + 'volumes_from', + 'working_dir', +] + +ALLOWED_KEYS = DOCKER_CONFIG_KEYS + [ + 'blkio_config', + 'build', + 'container_name', + 'credential_spec', + 'dockerfile', + 'init', + 'log_driver', + 'log_opt', + 'logging', + 'network_mode', + 'platform', + 'profiles', + 'scale', + 'stop_grace_period', +] + +DOCKER_VALID_URL_PREFIXES = ( + 'http://', + 'https://', + 'git://', + 'github.com/', + 'git@', +) + +SUPPORTED_FILENAMES = [ + 'docker-compose.yml', + 'docker-compose.yaml', +] + +DEFAULT_OVERRIDE_FILENAMES = ('docker-compose.override.yml', 'docker-compose.override.yaml') + + +log = logging.getLogger(__name__) + + +class ConfigDetails(namedtuple('_ConfigDetails', 'working_dir config_files environment')): + """ + :param working_dir: the directory to use for relative paths in the config + :type working_dir: string + :param config_files: list of configuration files to load + :type config_files: list of :class:`ConfigFile` + :param environment: computed environment values for this project + :type environment: :class:`environment.Environment` + """ + def __new__(cls, working_dir, config_files, environment=None): + if environment is None: + environment = Environment.from_env_file(working_dir) + return super().__new__( + cls, working_dir, config_files, environment + ) + + +class ConfigFile(namedtuple('_ConfigFile', 'filename config')): + """ + :param filename: filename of the config file + :type filename: string + :param config: contents of the config file + :type config: :class:`dict` + """ + + @classmethod + def from_filename(cls, filename): + return cls(filename, load_yaml(filename)) + + @cached_property + def config_version(self): + version = self.config.get('version', None) + if isinstance(version, dict): + return V1 + return ComposeVersion(version) if version else self.version + + @cached_property + def version(self): + version = self.config.get('version', None) + if not version: + # no version is specified in the config file + services = self.config.get('services', None) + networks = self.config.get('networks', None) + volumes = self.config.get('volumes', None) + if services or networks or volumes: + # validate V2/V3 structure + for section in ['services', 'networks', 'volumes']: + validate_config_section( + self.filename, self.config.get(section, {}), section) + return VERSION + + # validate V1 structure + validate_config_section( + self.filename, self.config, 'services') + return V1 + + if isinstance(version, dict): + log.warning('Unexpected type for "version" key in "{}". Assuming ' + '"version" is the name of a service, and defaulting to ' + 'Compose file version {}.'.format(self.filename, V1)) + return V1 + + if not isinstance(version, str): + raise ConfigurationError( + 'Version in "{}" is invalid - it should be a string.' + .format(self.filename)) + + if isinstance(version, str): + version_pattern = re.compile(r"^[1-3]+(\.\d+)?$") + if not version_pattern.match(version): + raise ConfigurationError( + 'Version "{}" in "{}" is invalid.' + .format(version, self.filename)) + + if version.startswith("1"): + raise ConfigurationError( + 'Version in "{}" is invalid. {}' + .format(self.filename, VERSION_EXPLANATION) + ) + + return VERSION + + def get_service(self, name): + return self.get_service_dicts()[name] + + def get_service_dicts(self): + if self.version == V1: + return self.config + return self.config.get('services', {}) + + def get_volumes(self): + return {} if self.version == V1 else self.config.get('volumes', {}) + + def get_networks(self): + return {} if self.version == V1 else self.config.get('networks', {}) + + def get_secrets(self): + return {} if self.version == V1 else self.config.get('secrets', {}) + + def get_configs(self): + return {} if self.version == V1 else self.config.get('configs', {}) + + +class Config(namedtuple('_Config', 'config_version version services volumes networks secrets configs')): + """ + :param config_version: configuration file version + :type config_version: int + :param version: configuration version + :type version: int + :param services: List of service description dictionaries + :type services: :class:`list` + :param volumes: Dictionary mapping volume names to description dictionaries + :type volumes: :class:`dict` + :param networks: Dictionary mapping network names to description dictionaries + :type networks: :class:`dict` + :param secrets: Dictionary mapping secret names to description dictionaries + :type secrets: :class:`dict` + :param configs: Dictionary mapping config names to description dictionaries + :type configs: :class:`dict` + """ + + +class ServiceConfig(namedtuple('_ServiceConfig', 'working_dir filename name config')): + + @classmethod + def with_abs_paths(cls, working_dir, filename, name, config): + if not working_dir: + raise ValueError("No working_dir for ServiceConfig.") + + return cls( + os.path.abspath(working_dir), + os.path.abspath(filename) if filename else filename, + name, + config) + + +def find(base_dir, filenames, environment, override_dir=None): + if filenames == ['-']: + return ConfigDetails( + os.path.abspath(override_dir) if override_dir else os.getcwd(), + [ConfigFile(None, yaml.safe_load(sys.stdin))], + environment + ) + + if filenames: + filenames = [os.path.join(base_dir, f) for f in filenames] + else: + filenames = get_default_config_files(base_dir) + + log.debug("Using configuration files: {}".format(",".join(filenames))) + return ConfigDetails( + override_dir if override_dir else os.path.dirname(filenames[0]), + [ConfigFile.from_filename(f) for f in filenames], + environment + ) + + +def validate_config_version(config_files): + main_file = config_files[0] + validate_top_level_object(main_file) + + for next_file in config_files[1:]: + validate_top_level_object(next_file) + + if main_file.version != next_file.version: + raise ConfigurationError( + "Version mismatch: file {} specifies version {} but " + "extension file {} uses version {}".format( + main_file.filename, + main_file.version, + next_file.filename, + next_file.version)) + + +def get_default_config_files(base_dir): + (candidates, path) = find_candidates_in_parent_dirs(SUPPORTED_FILENAMES, base_dir) + + if not candidates: + raise ComposeFileNotFound(SUPPORTED_FILENAMES) + + winner = candidates[0] + + if len(candidates) > 1: + log.warning("Found multiple config files with supported names: %s", ", ".join(candidates)) + log.warning("Using %s\n", winner) + + return [os.path.join(path, winner)] + get_default_override_file(path) + + +def get_default_override_file(path): + override_files_in_path = [os.path.join(path, override_filename) for override_filename + in DEFAULT_OVERRIDE_FILENAMES + if os.path.exists(os.path.join(path, override_filename))] + if len(override_files_in_path) > 1: + raise DuplicateOverrideFileFound(override_files_in_path) + return override_files_in_path + + +def find_candidates_in_parent_dirs(filenames, path): + """ + Given a directory path to start, looks for filenames in the + directory, and then each parent directory successively, + until found. + + Returns tuple (candidates, path). + """ + candidates = [filename for filename in filenames + if os.path.exists(os.path.join(path, filename))] + + if not candidates: + parent_dir = os.path.join(path, '..') + if os.path.abspath(parent_dir) != os.path.abspath(path): + return find_candidates_in_parent_dirs(filenames, parent_dir) + + return (candidates, path) + + +def check_swarm_only_config(service_dicts): + warning_template = ( + "Some services ({services}) use the '{key}' key, which will be ignored. " + "Compose does not support '{key}' configuration - use " + "`docker stack deploy` to deploy to a swarm." + ) + key = 'configs' + services = [s for s in service_dicts if s.get(key)] + if services: + log.warning( + warning_template.format( + services=", ".join(sorted(s['name'] for s in services)), + key=key + ) + ) + + +def load(config_details, interpolate=True): + """Load the configuration from a working directory and a list of + configuration files. Files are loaded in order, and merged on top + of each other to create the final configuration. + + Return a fully interpolated, extended and validated configuration. + """ + + # validate against latest version and if fails do it against v1 schema + validate_config_version(config_details.config_files) + + processed_files = [ + process_config_file(config_file, config_details.environment, interpolate=interpolate) + for config_file in config_details.config_files + ] + config_details = config_details._replace(config_files=processed_files) + + main_file = config_details.config_files[0] + volumes = load_mapping( + config_details.config_files, 'get_volumes', 'Volume' + ) + networks = load_mapping( + config_details.config_files, 'get_networks', 'Network' + ) + secrets = load_mapping( + config_details.config_files, 'get_secrets', 'Secret', config_details.working_dir + ) + configs = load_mapping( + config_details.config_files, 'get_configs', 'Config', config_details.working_dir + ) + service_dicts = load_services(config_details, main_file, interpolate=interpolate) + + if main_file.version != V1: + for service_dict in service_dicts: + match_named_volumes(service_dict, volumes) + + check_swarm_only_config(service_dicts) + + return Config(main_file.config_version, main_file.version, + service_dicts, volumes, networks, secrets, configs) + + +def load_mapping(config_files, get_func, entity_type, working_dir=None): + mapping = {} + + for config_file in config_files: + for name, config in getattr(config_file, get_func)().items(): + mapping[name] = config or {} + if not config: + continue + + external = config.get('external') + if external: + validate_external(entity_type, name, config, config_file.version) + if isinstance(external, dict): + config['name'] = external.get('name') + elif not config.get('name'): + config['name'] = name + + if 'labels' in config: + config['labels'] = parse_labels(config['labels']) + + if 'file' in config: + config['file'] = expand_path(working_dir, config['file']) + + if 'driver_opts' in config: + config['driver_opts'] = build_string_dict( + config['driver_opts'] + ) + device = format_device_option(entity_type, config) + if device: + config['driver_opts']['device'] = device + return mapping + + +def format_device_option(entity_type, config): + if entity_type != 'Volume': + return + # default driver is 'local' + driver = config.get('driver', 'local') + if driver != 'local': + return + o = config['driver_opts'].get('o') + device = config['driver_opts'].get('device') + if o and o == 'bind' and device: + fullpath = os.path.abspath(os.path.expanduser(device)) + return fullpath + + +def validate_external(entity_type, name, config, version): + for k in config.keys(): + if entity_type == 'Network' and k == 'driver': + continue + if k not in ['external', 'name']: + raise ConfigurationError( + "{} {} declared as external but specifies additional attributes " + "({}).".format( + entity_type, name, ', '.join(k for k in config if k != 'external'))) + + +def load_services(config_details, config_file, interpolate=True): + def build_service(service_name, service_dict, service_names): + service_config = ServiceConfig.with_abs_paths( + config_details.working_dir, + config_file.filename, + service_name, + service_dict) + resolver = ServiceExtendsResolver( + service_config, config_file, environment=config_details.environment + ) + service_dict = process_service(resolver.run()) + + service_config = service_config._replace(config=service_dict) + validate_service(service_config, service_names, config_file) + service_dict = finalize_service( + service_config, + service_names, + config_file.version, + config_details.environment, + interpolate + ) + return service_dict + + def build_services(service_config): + service_names = service_config.keys() + return sort_service_dicts([ + build_service(name, service_dict, service_names) + for name, service_dict in service_config.items() + ]) + + def merge_services(base, override): + all_service_names = set(base) | set(override) + return { + name: merge_service_dicts_from_files( + base.get(name, {}), + override.get(name, {}), + config_file.version) + for name in all_service_names + } + + service_configs = [ + file.get_service_dicts() for file in config_details.config_files + ] + + service_config = functools.reduce(merge_services, service_configs) + + return build_services(service_config) + + +def interpolate_config_section(config_file, config, section, environment): + return interpolate_environment_variables( + config_file.version, + config, + section, + environment + ) + + +def process_config_section(config_file, config, section, environment, interpolate): + validate_config_section(config_file.filename, config, section) + if interpolate: + return interpolate_environment_variables( + config_file.version, + config, + section, + environment + ) + else: + return config + + +def process_config_file(config_file, environment, service_name=None, interpolate=True): + services = process_config_section( + config_file, + config_file.get_service_dicts(), + 'service', + environment, + interpolate, + ) + + if config_file.version > V1: + processed_config = dict(config_file.config) + processed_config['services'] = services + processed_config['volumes'] = process_config_section( + config_file, + config_file.get_volumes(), + 'volume', + environment, + interpolate, + ) + processed_config['networks'] = process_config_section( + config_file, + config_file.get_networks(), + 'network', + environment, + interpolate, + ) + processed_config['secrets'] = process_config_section( + config_file, + config_file.get_secrets(), + 'secret', + environment, + interpolate, + ) + processed_config['configs'] = process_config_section( + config_file, + config_file.get_configs(), + 'config', + environment, + interpolate, + ) + else: + processed_config = services + + config_file = config_file._replace(config=processed_config) + validate_against_config_schema(config_file, config_file.version) + + if service_name and service_name not in services: + raise ConfigurationError( + "Cannot extend service '{}' in {}: Service not found".format( + service_name, config_file.filename)) + + return config_file + + +class ServiceExtendsResolver: + def __init__(self, service_config, config_file, environment, already_seen=None): + self.service_config = service_config + self.working_dir = service_config.working_dir + self.already_seen = already_seen or [] + self.config_file = config_file + self.environment = environment + + @property + def signature(self): + return self.service_config.filename, self.service_config.name + + def detect_cycle(self): + if self.signature in self.already_seen: + raise CircularReference(self.already_seen + [self.signature]) + + def run(self): + self.detect_cycle() + + if 'extends' in self.service_config.config: + service_dict = self.resolve_extends(*self.validate_and_construct_extends()) + return self.service_config._replace(config=service_dict) + + return self.service_config + + def validate_and_construct_extends(self): + extends = self.service_config.config['extends'] + if not isinstance(extends, dict): + extends = {'service': extends} + + config_path = self.get_extended_config_path(extends) + service_name = extends['service'] + + if config_path == os.path.abspath(self.config_file.filename): + try: + service_config = self.config_file.get_service(service_name) + except KeyError: + raise ConfigurationError( + "Cannot extend service '{}' in {}: Service not found".format( + service_name, config_path) + ) + else: + extends_file = ConfigFile.from_filename(config_path) + validate_config_version([self.config_file, extends_file]) + extended_file = process_config_file( + extends_file, self.environment, service_name=service_name + ) + service_config = extended_file.get_service(service_name) + + return config_path, service_config, service_name + + def resolve_extends(self, extended_config_path, service_dict, service_name): + resolver = ServiceExtendsResolver( + ServiceConfig.with_abs_paths( + os.path.dirname(extended_config_path), + extended_config_path, + service_name, + service_dict), + self.config_file, + already_seen=self.already_seen + [self.signature], + environment=self.environment + ) + + service_config = resolver.run() + other_service_dict = process_service(service_config) + validate_extended_service_dict( + other_service_dict, + extended_config_path, + service_name) + + return merge_service_dicts( + other_service_dict, + self.service_config.config, + self.config_file.version) + + def get_extended_config_path(self, extends_options): + """Service we are extending either has a value for 'file' set, which we + need to obtain a full path too or we are extending from a service + defined in our own file. + """ + filename = self.service_config.filename + validate_extends_file_path( + self.service_config.name, + extends_options, + filename) + if 'file' in extends_options: + return expand_path(self.working_dir, extends_options['file']) + return filename + + +def resolve_environment(service_dict, environment=None, interpolate=True): + """Unpack any environment variables from an env_file, if set. + Interpolate environment values if set. + """ + env = {} + for env_file in service_dict.get('env_file', []): + env.update(env_vars_from_file(env_file, interpolate)) + + env.update(parse_environment(service_dict.get('environment'))) + return dict(resolve_env_var(k, v, environment) for k, v in env.items()) + + +def resolve_build_args(buildargs, environment): + args = parse_build_arguments(buildargs) + return dict(resolve_env_var(k, v, environment) for k, v in args.items()) + + +def validate_extended_service_dict(service_dict, filename, service): + error_prefix = "Cannot extend service '{}' in {}:".format(service, filename) + + if 'links' in service_dict: + raise ConfigurationError( + "%s services with 'links' cannot be extended" % error_prefix) + + if 'volumes_from' in service_dict: + raise ConfigurationError( + "%s services with 'volumes_from' cannot be extended" % error_prefix) + + if 'net' in service_dict: + if get_container_name_from_network_mode(service_dict['net']): + raise ConfigurationError( + "%s services with 'net: container' cannot be extended" % error_prefix) + + if 'network_mode' in service_dict: + if get_service_name_from_network_mode(service_dict['network_mode']): + raise ConfigurationError( + "%s services with 'network_mode: service' cannot be extended" % error_prefix) + + if 'depends_on' in service_dict: + raise ConfigurationError( + "%s services with 'depends_on' cannot be extended" % error_prefix) + + +def validate_service(service_config, service_names, config_file): + def build_image(): + args = sys.argv[1:] + if 'pull' in args: + return False + + if '--no-build' in args: + return False + + return True + + service_dict, service_name = service_config.config, service_config.name + validate_service_constraints(service_dict, service_name, config_file) + + if build_image(): + # We only care about valid paths when actually building images + validate_paths(service_dict) + + validate_cpu(service_config) + validate_ulimits(service_config) + validate_ipc_mode(service_config, service_names) + validate_network_mode(service_config, service_names) + validate_pid_mode(service_config, service_names) + validate_depends_on(service_config, service_names) + validate_links(service_config, service_names) + validate_healthcheck(service_config) + validate_credential_spec(service_config) + + if not service_dict.get('image') and has_uppercase(service_name): + raise ConfigurationError( + "Service '{name}' contains uppercase characters which are not valid " + "as part of an image name. Either use a lowercase service name or " + "use the `image` field to set a custom name for the service image." + .format(name=service_name)) + + +def process_service(service_config): + working_dir = service_config.working_dir + service_dict = dict(service_config.config) + + if 'env_file' in service_dict: + service_dict['env_file'] = [ + expand_path(working_dir, path) + for path in to_list(service_dict['env_file']) + ] + + if 'build' in service_dict: + process_build_section(service_dict, working_dir) + + if 'volumes' in service_dict and service_dict.get('volume_driver') is None: + service_dict['volumes'] = resolve_volume_paths(working_dir, service_dict) + + if 'sysctls' in service_dict: + service_dict['sysctls'] = build_string_dict(parse_sysctls(service_dict['sysctls'])) + + if 'labels' in service_dict: + service_dict['labels'] = parse_labels(service_dict['labels']) + + service_dict = process_depends_on(service_dict) + + for field in ['dns', 'dns_search', 'tmpfs']: + if field in service_dict: + service_dict[field] = to_list(service_dict[field]) + + service_dict = process_security_opt(process_blkio_config(process_ports( + process_healthcheck(service_dict) + ))) + + return service_dict + + +def process_build_section(service_dict, working_dir): + if isinstance(service_dict['build'], str): + service_dict['build'] = resolve_build_path(working_dir, service_dict['build']) + elif isinstance(service_dict['build'], dict): + if 'context' in service_dict['build']: + path = service_dict['build']['context'] + service_dict['build']['context'] = resolve_build_path(working_dir, path) + if 'labels' in service_dict['build']: + service_dict['build']['labels'] = parse_labels(service_dict['build']['labels']) + + +def process_ports(service_dict): + if 'ports' not in service_dict: + return service_dict + + ports = [] + for port_definition in service_dict['ports']: + if isinstance(port_definition, ServicePort): + ports.append(port_definition) + else: + ports.extend(ServicePort.parse(port_definition)) + service_dict['ports'] = ports + return service_dict + + +def process_depends_on(service_dict): + if 'depends_on' in service_dict and not isinstance(service_dict['depends_on'], dict): + service_dict['depends_on'] = { + svc: {'condition': 'service_started'} for svc in service_dict['depends_on'] + } + return service_dict + + +def process_blkio_config(service_dict): + if not service_dict.get('blkio_config'): + return service_dict + + for field in ['device_read_bps', 'device_write_bps']: + if field in service_dict['blkio_config']: + for v in service_dict['blkio_config'].get(field, []): + rate = v.get('rate', 0) + v['rate'] = parse_bytes(rate) + if v['rate'] is None: + raise ConfigurationError('Invalid format for bytes value: "{}"'.format(rate)) + + for field in ['device_read_iops', 'device_write_iops']: + if field in service_dict['blkio_config']: + for v in service_dict['blkio_config'].get(field, []): + try: + v['rate'] = int(v.get('rate', 0)) + except ValueError: + raise ConfigurationError( + 'Invalid IOPS value: "{}". Must be a positive integer.'.format(v.get('rate')) + ) + + return service_dict + + +def process_healthcheck(service_dict): + if 'healthcheck' not in service_dict: + return service_dict + + hc = service_dict['healthcheck'] + + if 'disable' in hc: + del hc['disable'] + hc['test'] = ['NONE'] + + for field in ['interval', 'timeout', 'start_period']: + if field not in hc or isinstance(hc[field], int): + continue + hc[field] = parse_nanoseconds_int(hc[field]) + + return service_dict + + +def finalize_service_volumes(service_dict, environment): + if 'volumes' in service_dict: + finalized_volumes = [] + normalize = environment.get_boolean('COMPOSE_CONVERT_WINDOWS_PATHS') + win_host = environment.get_boolean('COMPOSE_FORCE_WINDOWS_HOST') + for v in service_dict['volumes']: + if isinstance(v, dict): + finalized_volumes.append(MountSpec.parse(v, normalize, win_host)) + else: + finalized_volumes.append(VolumeSpec.parse(v, normalize, win_host)) + + duplicate_mounts = [] + mounts = [v.as_volume_spec() if isinstance(v, MountSpec) else v for v in finalized_volumes] + for mount in mounts: + if list(map(attrgetter('internal'), mounts)).count(mount.internal) > 1: + duplicate_mounts.append(mount.repr()) + + if duplicate_mounts: + raise ConfigurationError("Duplicate mount points: [%s]" % ( + ', '.join(duplicate_mounts))) + + service_dict['volumes'] = finalized_volumes + + return service_dict + + +def finalize_service(service_config, service_names, version, environment, + interpolate=True): + service_dict = dict(service_config.config) + + if 'environment' in service_dict or 'env_file' in service_dict: + service_dict['environment'] = resolve_environment(service_dict, environment, interpolate) + service_dict.pop('env_file', None) + + if 'volumes_from' in service_dict: + service_dict['volumes_from'] = [ + VolumeFromSpec.parse(vf, service_names, version) + for vf in service_dict['volumes_from'] + ] + + service_dict = finalize_service_volumes(service_dict, environment) + + if 'net' in service_dict: + network_mode = service_dict.pop('net') + container_name = get_container_name_from_network_mode(network_mode) + if container_name and container_name in service_names: + service_dict['network_mode'] = 'service:{}'.format(container_name) + else: + service_dict['network_mode'] = network_mode + + if 'networks' in service_dict: + service_dict['networks'] = parse_networks(service_dict['networks']) + + if 'restart' in service_dict: + service_dict['restart'] = parse_restart_spec(service_dict['restart']) + + if 'secrets' in service_dict: + service_dict['secrets'] = [ + types.ServiceSecret.parse(s) for s in service_dict['secrets'] + ] + + if 'configs' in service_dict: + service_dict['configs'] = [ + types.ServiceConfig.parse(c) for c in service_dict['configs'] + ] + + normalize_build(service_dict, service_config.working_dir, environment) + + service_dict['name'] = service_config.name + return normalize_v1_service_format(service_dict) + + +def normalize_v1_service_format(service_dict): + if 'log_driver' in service_dict or 'log_opt' in service_dict: + if 'logging' not in service_dict: + service_dict['logging'] = {} + if 'log_driver' in service_dict: + service_dict['logging']['driver'] = service_dict['log_driver'] + del service_dict['log_driver'] + if 'log_opt' in service_dict: + service_dict['logging']['options'] = service_dict['log_opt'] + del service_dict['log_opt'] + + if 'dockerfile' in service_dict: + service_dict['build'] = service_dict.get('build', {}) + service_dict['build'].update({ + 'dockerfile': service_dict.pop('dockerfile') + }) + + return service_dict + + +def merge_service_dicts_from_files(base, override, version): + """When merging services from multiple files we need to merge the `extends` + field. This is not handled by `merge_service_dicts()` which is used to + perform the `extends`. + """ + new_service = merge_service_dicts(base, override, version) + if 'extends' in override: + new_service['extends'] = override['extends'] + elif 'extends' in base: + new_service['extends'] = base['extends'] + return new_service + + +class MergeDict(dict): + """A dict-like object responsible for merging two dicts into one.""" + + def __init__(self, base, override): + self.base = base + self.override = override + + def needs_merge(self, field): + return field in self.base or field in self.override + + def merge_field(self, field, merge_func, default=None): + if not self.needs_merge(field): + return + + self[field] = merge_func( + self.base.get(field, default), + self.override.get(field, default)) + + def merge_mapping(self, field, parse_func=None): + if not self.needs_merge(field): + return + + if parse_func is None: + def parse_func(m): + return m or {} + + self[field] = parse_func(self.base.get(field)) + self[field].update(parse_func(self.override.get(field))) + + def merge_sequence(self, field, parse_func): + def parse_sequence_func(seq): + return to_mapping((parse_func(item) for item in seq), 'merge_field') + + if not self.needs_merge(field): + return + + merged = parse_sequence_func(self.base.get(field, [])) + merged.update(parse_sequence_func(self.override.get(field, []))) + self[field] = [item.repr() for item in sorted(merged.values())] + + def merge_scalar(self, field): + if self.needs_merge(field): + self[field] = self.override.get(field, self.base.get(field)) + + +def merge_service_dicts(base, override, version): + md = MergeDict(base, override) + + md.merge_mapping('environment', parse_environment) + md.merge_mapping('labels', parse_labels) + md.merge_mapping('ulimits', parse_flat_dict) + md.merge_mapping('sysctls', parse_sysctls) + md.merge_mapping('depends_on', parse_depends_on) + md.merge_mapping('storage_opt', parse_flat_dict) + md.merge_sequence('links', ServiceLink.parse) + md.merge_sequence('secrets', types.ServiceSecret.parse) + md.merge_sequence('configs', types.ServiceConfig.parse) + md.merge_sequence('security_opt', types.SecurityOpt.parse) + md.merge_mapping('extra_hosts', parse_extra_hosts) + + md.merge_field('networks', merge_networks, default={}) + for field in ['volumes', 'devices']: + md.merge_field(field, merge_path_mappings) + + for field in [ + 'cap_add', 'cap_drop', 'expose', 'external_links', + 'volumes_from', 'device_cgroup_rules', 'profiles', + ]: + md.merge_field(field, merge_unique_items_lists, default=[]) + + for field in ['dns', 'dns_search', 'env_file', 'tmpfs']: + md.merge_field(field, merge_list_or_string) + + md.merge_field('logging', merge_logging, default={}) + merge_ports(md, base, override) + md.merge_field('blkio_config', merge_blkio_config, default={}) + md.merge_field('healthcheck', merge_healthchecks, default={}) + md.merge_field('deploy', merge_deploy, default={}) + + for field in set(ALLOWED_KEYS) - set(md): + md.merge_scalar(field) + + if version == V1: + legacy_v1_merge_image_or_build(md, base, override) + elif md.needs_merge('build'): + md['build'] = merge_build(md, base, override) + + return dict(md) + + +def merge_unique_items_lists(base, override): + override = (str(o) for o in override) + base = (str(b) for b in base) + return sorted(set(chain(base, override))) + + +def merge_healthchecks(base, override): + if override.get('disabled') is True: + return override + result = base.copy() + result.update(override) + return result + + +def merge_ports(md, base, override): + def parse_sequence_func(seq): + acc = [s for item in seq for s in ServicePort.parse(item)] + return to_mapping(acc, 'merge_field') + + field = 'ports' + + if not md.needs_merge(field): + return + + merged = parse_sequence_func(md.base.get(field, [])) + merged.update(parse_sequence_func(md.override.get(field, []))) + md[field] = [item for item in sorted(merged.values(), key=attrgetter("target"))] + + +def merge_build(output, base, override): + def to_dict(service): + build_config = service.get('build', {}) + if isinstance(build_config, str): + return {'context': build_config} + return build_config + + md = MergeDict(to_dict(base), to_dict(override)) + md.merge_scalar('context') + md.merge_scalar('dockerfile') + md.merge_scalar('network') + md.merge_scalar('target') + md.merge_scalar('shm_size') + md.merge_scalar('isolation') + md.merge_mapping('args', parse_build_arguments) + md.merge_field('cache_from', merge_unique_items_lists, default=[]) + md.merge_mapping('labels', parse_labels) + md.merge_mapping('extra_hosts', parse_extra_hosts) + return dict(md) + + +def merge_deploy(base, override): + md = MergeDict(base or {}, override or {}) + md.merge_scalar('mode') + md.merge_scalar('endpoint_mode') + md.merge_scalar('replicas') + md.merge_mapping('labels', parse_labels) + md.merge_mapping('update_config') + md.merge_mapping('rollback_config') + md.merge_mapping('restart_policy') + if md.needs_merge('resources'): + resources_md = MergeDict(md.base.get('resources') or {}, md.override.get('resources') or {}) + resources_md.merge_mapping('limits') + resources_md.merge_field('reservations', merge_reservations, default={}) + md['resources'] = dict(resources_md) + if md.needs_merge('placement'): + placement_md = MergeDict(md.base.get('placement') or {}, md.override.get('placement') or {}) + placement_md.merge_scalar('max_replicas_per_node') + placement_md.merge_field('constraints', merge_unique_items_lists, default=[]) + placement_md.merge_field('preferences', merge_unique_objects_lists, default=[]) + md['placement'] = dict(placement_md) + + return dict(md) + + +def merge_networks(base, override): + merged_networks = {} + all_network_names = set(base) | set(override) + base = {k: {} for k in base} if isinstance(base, list) else base + override = {k: {} for k in override} if isinstance(override, list) else override + for network_name in all_network_names: + md = MergeDict(base.get(network_name) or {}, override.get(network_name) or {}) + md.merge_field('aliases', merge_unique_items_lists, []) + md.merge_field('link_local_ips', merge_unique_items_lists, []) + md.merge_scalar('priority') + md.merge_scalar('ipv4_address') + md.merge_scalar('ipv6_address') + merged_networks[network_name] = dict(md) + return merged_networks + + +def merge_reservations(base, override): + md = MergeDict(base, override) + md.merge_scalar('cpus') + md.merge_scalar('memory') + md.merge_sequence('generic_resources', types.GenericResource.parse) + md.merge_field('devices', merge_unique_objects_lists, default=[]) + return dict(md) + + +def merge_unique_objects_lists(base, override): + result = {json_hash(i): i for i in base + override} + return [i[1] for i in sorted(((k, v) for k, v in result.items()), key=itemgetter(0))] + + +def merge_blkio_config(base, override): + md = MergeDict(base, override) + md.merge_scalar('weight') + + def merge_blkio_limits(base, override): + get_path = itemgetter('path') + index = {get_path(b): b for b in base} + index.update((get_path(o), o) for o in override) + + return sorted(index.values(), key=get_path) + + for field in [ + "device_read_bps", "device_read_iops", "device_write_bps", + "device_write_iops", "weight_device", + ]: + md.merge_field(field, merge_blkio_limits, default=[]) + + return dict(md) + + +def merge_logging(base, override): + md = MergeDict(base, override) + md.merge_scalar('driver') + if md.get('driver') == base.get('driver') or base.get('driver') is None: + md.merge_mapping('options', lambda m: m or {}) + elif override.get('options'): + md['options'] = override.get('options', {}) + return dict(md) + + +def legacy_v1_merge_image_or_build(output, base, override): + output.pop('image', None) + output.pop('build', None) + if 'image' in override: + output['image'] = override['image'] + elif 'build' in override: + output['build'] = override['build'] + elif 'image' in base: + output['image'] = base['image'] + elif 'build' in base: + output['build'] = base['build'] + + +def merge_environment(base, override): + env = parse_environment(base) + env.update(parse_environment(override)) + return env + + +def merge_labels(base, override): + labels = parse_labels(base) + labels.update(parse_labels(override)) + return labels + + +def split_kv(kvpair): + if '=' in kvpair: + return kvpair.split('=', 1) + else: + return kvpair, '' + + +def parse_dict_or_list(split_func, type_name, arguments): + if not arguments: + return {} + + if isinstance(arguments, list): + return dict(split_func(e) for e in arguments) + + if isinstance(arguments, dict): + return dict(arguments) + + raise ConfigurationError( + "%s \"%s\" must be a list or mapping," % + (type_name, arguments) + ) + + +parse_build_arguments = functools.partial(parse_dict_or_list, split_env, 'build arguments') +parse_environment = functools.partial(parse_dict_or_list, split_env, 'environment') +parse_labels = functools.partial(parse_dict_or_list, split_kv, 'labels') +parse_networks = functools.partial(parse_dict_or_list, lambda k: (k, None), 'networks') +parse_sysctls = functools.partial(parse_dict_or_list, split_kv, 'sysctls') +parse_depends_on = functools.partial( + parse_dict_or_list, lambda k: (k, {'condition': 'service_started'}), 'depends_on' +) + + +def parse_flat_dict(d): + if not d: + return {} + + if isinstance(d, dict): + return dict(d) + + raise ConfigurationError("Invalid type: expected mapping") + + +def resolve_env_var(key, val, environment): + if val is not None: + return key, val + elif environment and key in environment: + return key, environment[key] + else: + return key, None + + +def resolve_volume_paths(working_dir, service_dict): + return [ + resolve_volume_path(working_dir, volume) + for volume in service_dict['volumes'] + ] + + +def resolve_volume_path(working_dir, volume): + if isinstance(volume, dict): + if volume.get('source', '').startswith(('.', '~')) and volume['type'] == 'bind': + volume['source'] = expand_path(working_dir, volume['source']) + return volume + + mount_params = None + container_path, mount_params = split_path_mapping(volume) + + if mount_params is not None: + host_path, mode = mount_params + if host_path is None: + return container_path + if host_path.startswith('.'): + host_path = expand_path(working_dir, host_path) + host_path = os.path.expanduser(host_path) + return "{}:{}{}".format(host_path, container_path, (':' + mode if mode else '')) + + return container_path + + +def normalize_build(service_dict, working_dir, environment): + + if 'build' in service_dict: + build = {} + # Shortcut where specifying a string is treated as the build context + if isinstance(service_dict['build'], str): + build['context'] = service_dict.pop('build') + else: + build.update(service_dict['build']) + if 'args' in build: + build['args'] = build_string_dict( + resolve_build_args(build.get('args'), environment) + ) + + service_dict['build'] = build + + +def resolve_build_path(working_dir, build_path): + if is_url(build_path): + return build_path + return expand_path(working_dir, build_path) + + +def is_url(build_path): + return build_path.startswith(DOCKER_VALID_URL_PREFIXES) + + +def validate_paths(service_dict): + if 'build' in service_dict: + build = service_dict.get('build', {}) + + if isinstance(build, str): + build_path = build + elif isinstance(build, dict) and 'context' in build: + build_path = build['context'] + else: + # We have a build section but no context, so nothing to validate + return + + if ( + not is_url(build_path) and + (not os.path.exists(build_path) or not os.access(build_path, os.R_OK)) + ): + raise ConfigurationError( + "build path %s either does not exist, is not accessible, " + "or is not a valid URL." % build_path) + + +def merge_path_mappings(base, override): + d = dict_from_path_mappings(base) + d.update(dict_from_path_mappings(override)) + return path_mappings_from_dict(d) + + +def dict_from_path_mappings(path_mappings): + if path_mappings: + return dict(split_path_mapping(v) for v in path_mappings) + else: + return {} + + +def path_mappings_from_dict(d): + return [join_path_mapping(v) for v in sorted(d.items())] + + +def split_path_mapping(volume_path): + """ + Ascertain if the volume_path contains a host path as well as a container + path. Using splitdrive so windows absolute paths won't cause issues with + splitting on ':'. + """ + if isinstance(volume_path, dict): + return (volume_path.get('target'), volume_path) + drive, volume_config = splitdrive(volume_path) + + if ':' in volume_config: + (host, container) = volume_config.split(':', 1) + container_drive, container_path = splitdrive(container) + mode = None + if ':' in container_path: + container_path, mode = container_path.rsplit(':', 1) + + return (container_drive + container_path, (drive + host, mode)) + else: + return (volume_path, None) + + +def process_security_opt(service_dict): + security_opts = service_dict.get('security_opt', []) + result = [] + for value in security_opts: + result.append(SecurityOpt.parse(value)) + if result: + service_dict['security_opt'] = result + return service_dict + + +def join_path_mapping(pair): + (container, host) = pair + if isinstance(host, dict): + return host + elif host is None: + return container + else: + host, mode = host + result = ":".join((host, container)) + if mode: + result += ":" + mode + return result + + +def expand_path(working_dir, path): + return os.path.abspath(os.path.join(working_dir, os.path.expanduser(path))) + + +def merge_list_or_string(base, override): + return to_list(base) + to_list(override) + + +def to_list(value): + if value is None: + return [] + elif isinstance(value, str): + return [value] + else: + return value + + +def to_mapping(sequence, key_field): + return {getattr(item, key_field): item for item in sequence} + + +def has_uppercase(name): + return any(char in string.ascii_uppercase for char in name) + + +def load_yaml(filename, encoding=None, binary=True): + try: + with open(filename, 'rb' if binary else 'r', encoding=encoding) as fh: + return yaml.safe_load(fh) + except (OSError, yaml.YAMLError, UnicodeDecodeError) as e: + if encoding is None: + # Sometimes the user's locale sets an encoding that doesn't match + # the YAML files. Im such cases, retry once with the "default" + # UTF-8 encoding + return load_yaml(filename, encoding='utf-8-sig', binary=False) + error_name = getattr(e, '__module__', '') + '.' + e.__class__.__name__ + raise ConfigurationError("{}: {}".format(error_name, e)) diff --git a/compose/config/config_schema_v1.json b/compose/config/config_schema_v1.json new file mode 100644 index 00000000000..2771f9958fb --- /dev/null +++ b/compose/config/config_schema_v1.json @@ -0,0 +1,203 @@ +{ + "$schema": "http://json-schema.org/draft-04/schema#", + "id": "config_schema_v1.json", + + "type": "object", + + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/service" + } + }, + + "additionalProperties": false, + + "definitions": { + "service": { + "id": "#/definitions/service", + "type": "object", + + "properties": { + "build": {"type": "string"}, + "cap_add": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "cap_drop": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "cgroup_parent": {"type": "string"}, + "command": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "container_name": {"type": "string"}, + "cpu_shares": {"type": ["number", "string"]}, + "cpu_quota": {"type": ["number", "string"]}, + "cpuset": {"type": "string"}, + "devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "dns": {"$ref": "#/definitions/string_or_list"}, + "dns_search": {"$ref": "#/definitions/string_or_list"}, + "dockerfile": {"type": "string"}, + "domainname": {"type": "string"}, + "entrypoint": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "env_file": {"$ref": "#/definitions/string_or_list"}, + "environment": {"$ref": "#/definitions/list_or_dict"}, + + "expose": { + "type": "array", + "items": { + "type": ["string", "number"], + "format": "expose" + }, + "uniqueItems": true + }, + + "extends": { + "oneOf": [ + { + "type": "string" + }, + { + "type": "object", + + "properties": { + "service": {"type": "string"}, + "file": {"type": "string"} + }, + "required": ["service"], + "additionalProperties": false + } + ] + }, + + "extra_hosts": {"$ref": "#/definitions/list_or_dict"}, + "external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "hostname": {"type": "string"}, + "image": {"type": "string"}, + "ipc": {"type": "string"}, + "labels": {"$ref": "#/definitions/labels"}, + "links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "log_driver": {"type": "string"}, + "log_opt": {"type": "object"}, + "mac_address": {"type": "string"}, + "mem_limit": {"type": ["number", "string"]}, + "memswap_limit": {"type": ["number", "string"]}, + "mem_swappiness": {"type": "integer"}, + "net": {"type": "string"}, + "pid": {"type": ["string", "null"]}, + + "ports": { + "type": "array", + "items": { + "type": ["string", "number"], + "format": "ports" + }, + "uniqueItems": true + }, + + "privileged": {"type": "boolean"}, + "read_only": {"type": "boolean"}, + "restart": {"type": "string"}, + "security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "shm_size": {"type": ["number", "string"]}, + "stdin_open": {"type": "boolean"}, + "stop_signal": {"type": "string"}, + "tty": {"type": "boolean"}, + "ulimits": { + "type": "object", + "patternProperties": { + "^[a-z]+$": { + "oneOf": [ + {"type": "integer"}, + { + "type":"object", + "properties": { + "hard": {"type": "integer"}, + "soft": {"type": "integer"} + }, + "required": ["soft", "hard"], + "additionalProperties": false + } + ] + } + } + }, + "user": {"type": "string"}, + "volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "volume_driver": {"type": "string"}, + "volumes_from": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "working_dir": {"type": "string"} + }, + + "dependencies": { + "memswap_limit": ["mem_limit"] + }, + "additionalProperties": false + }, + + "string_or_list": { + "oneOf": [ + {"type": "string"}, + {"$ref": "#/definitions/list_of_strings"} + ] + }, + + "list_of_strings": { + "type": "array", + "items": {"type": "string"}, + "uniqueItems": true + }, + + "list_or_dict": { + "oneOf": [ + { + "type": "object", + "patternProperties": { + ".+": { + "type": ["string", "number", "null"] + } + }, + "additionalProperties": false + }, + {"type": "array", "items": {"type": "string"}, "uniqueItems": true} + ] + }, + + "labels": { + "oneOf": [ + { + "type": "object", + "patternProperties": { + ".+": { + "type": "string" + } + }, + "additionalProperties": false + }, + {"type": "array", "items": {"type": "string"}, "uniqueItems": true} + ] + }, + + "constraints": { + "service": { + "id": "#/definitions/constraints/service", + "anyOf": [ + { + "required": ["build"], + "not": {"required": ["image"]} + }, + { + "required": ["image"], + "not": {"anyOf": [ + {"required": ["build"]}, + {"required": ["dockerfile"]} + ]} + } + ] + } + } + } +} diff --git a/compose/config/environment.py b/compose/config/environment.py new file mode 100644 index 00000000000..8769df9f8f7 --- /dev/null +++ b/compose/config/environment.py @@ -0,0 +1,125 @@ +import logging +import os +import re + +import dotenv + +from ..const import IS_WINDOWS_PLATFORM +from .errors import ConfigurationError +from .errors import EnvFileNotFound + +log = logging.getLogger(__name__) + + +def split_env(env): + if isinstance(env, bytes): + env = env.decode('utf-8', 'replace') + key = value = None + if '=' in env: + key, value = env.split('=', 1) + else: + key = env + if re.search(r'\s', key): + raise ConfigurationError( + "environment variable name '{}' may not contain whitespace.".format(key) + ) + return key, value + + +def env_vars_from_file(filename, interpolate=True): + """ + Read in a line delimited file of environment variables. + """ + if not os.path.exists(filename): + raise EnvFileNotFound("Couldn't find env file: {}".format(filename)) + elif not os.path.isfile(filename): + raise EnvFileNotFound("{} is not a file.".format(filename)) + + env = dotenv.dotenv_values(dotenv_path=filename, encoding='utf-8-sig', interpolate=interpolate) + for k, v in env.items(): + env[k] = v if interpolate else v.replace('$', '$$') + return env + + +class Environment(dict): + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.missing_keys = [] + self.silent = False + + @classmethod + def from_env_file(cls, base_dir, env_file=None): + def _initialize(): + result = cls() + if base_dir is None: + return result + if env_file: + env_file_path = os.path.join(base_dir, env_file) + else: + env_file_path = os.path.join(base_dir, '.env') + try: + return cls(env_vars_from_file(env_file_path)) + except EnvFileNotFound: + pass + return result + + instance = _initialize() + instance.update(os.environ) + return instance + + @classmethod + def from_command_line(cls, parsed_env_opts): + result = cls() + for k, v in parsed_env_opts.items(): + # Values from the command line take priority, unless they're unset + # in which case they take the value from the system's environment + if v is None and k in os.environ: + result[k] = os.environ[k] + else: + result[k] = v + return result + + def __getitem__(self, key): + try: + return super().__getitem__(key) + except KeyError: + if IS_WINDOWS_PLATFORM: + try: + return super().__getitem__(key.upper()) + except KeyError: + pass + if not self.silent and key not in self.missing_keys: + log.warning( + "The {} variable is not set. Defaulting to a blank string." + .format(key) + ) + self.missing_keys.append(key) + + return "" + + def __contains__(self, key): + result = super().__contains__(key) + if IS_WINDOWS_PLATFORM: + return ( + result or super().__contains__(key.upper()) + ) + return result + + def get(self, key, *args, **kwargs): + if IS_WINDOWS_PLATFORM: + return super().get( + key, + super().get(key.upper(), *args, **kwargs) + ) + return super().get(key, *args, **kwargs) + + def get_boolean(self, key, default=False): + # Convert a value to a boolean using "common sense" rules. + # Unset, empty, "0" and "false" (i-case) yield False. + # All other values yield True. + value = self.get(key) + if not value: + return default + if value.lower() in ['0', 'false']: + return False + return True diff --git a/compose/config/errors.py b/compose/config/errors.py new file mode 100644 index 00000000000..b66433a7998 --- /dev/null +++ b/compose/config/errors.py @@ -0,0 +1,55 @@ +VERSION_EXPLANATION = ( + 'You might be seeing this error because you\'re using the wrong Compose file version. ' + 'Either specify a supported version (e.g "2.2" or "3.3") and place ' + 'your service definitions under the `services` key, or omit the `version` key ' + 'and place your service definitions at the root of the file to use ' + 'version 1.\nFor more on the Compose file format versions, see ' + 'https://docs.docker.com/compose/compose-file/') + + +class ConfigurationError(Exception): + def __init__(self, msg): + self.msg = msg + + def __str__(self): + return self.msg + + +class EnvFileNotFound(ConfigurationError): + pass + + +class DependencyError(ConfigurationError): + pass + + +class CircularReference(ConfigurationError): + def __init__(self, trail): + self.trail = trail + + @property + def msg(self): + lines = [ + "{} in {}".format(service_name, filename) + for (filename, service_name) in self.trail + ] + return "Circular reference:\n {}".format("\n extends ".join(lines)) + + +class ComposeFileNotFound(ConfigurationError): + def __init__(self, supported_filenames): + super().__init__(""" + Can't find a suitable configuration file in this directory or any + parent. Are you in the right directory? + + Supported filenames: %s + """ % ", ".join(supported_filenames)) + + +class DuplicateOverrideFileFound(ConfigurationError): + def __init__(self, override_filenames): + self.override_filenames = override_filenames + super().__init__( + "Multiple override files found: {}. You may only use a single " + "override file.".format(", ".join(override_filenames)) + ) diff --git a/compose/config/interpolation.py b/compose/config/interpolation.py new file mode 100644 index 00000000000..a464f946569 --- /dev/null +++ b/compose/config/interpolation.py @@ -0,0 +1,295 @@ +import logging +import re +from string import Template + +from .errors import ConfigurationError +from compose.const import COMPOSEFILE_V1 as V1 +from compose.utils import parse_bytes +from compose.utils import parse_nanoseconds_int + + +log = logging.getLogger(__name__) + + +class Interpolator: + + def __init__(self, templater, mapping): + self.templater = templater + self.mapping = mapping + + def interpolate(self, string): + try: + return self.templater(string).substitute(self.mapping) + except ValueError: + raise InvalidInterpolation(string) + + +def interpolate_environment_variables(version, config, section, environment): + if version == V1: + interpolator = Interpolator(Template, environment) + else: + interpolator = Interpolator(TemplateWithDefaults, environment) + + def process_item(name, config_dict): + return { + key: interpolate_value(name, key, val, section, interpolator) + for key, val in (config_dict or {}).items() + } + + return { + name: process_item(name, config_dict or {}) + for name, config_dict in config.items() + } + + +def get_config_path(config_key, section, name): + return '{}/{}/{}'.format(section, name, config_key) + + +def interpolate_value(name, config_key, value, section, interpolator): + try: + return recursive_interpolate(value, interpolator, get_config_path(config_key, section, name)) + except InvalidInterpolation as e: + raise ConfigurationError( + 'Invalid interpolation format for "{config_key}" option ' + 'in {section} "{name}": "{string}"'.format( + config_key=config_key, + name=name, + section=section, + string=e.string)) + except UnsetRequiredSubstitution as e: + raise ConfigurationError( + 'Missing mandatory value for "{config_key}" option interpolating {value} ' + 'in {section} "{name}": {err}'.format(config_key=config_key, + value=value, + name=name, + section=section, + err=e.err) + ) + + +def recursive_interpolate(obj, interpolator, config_path): + def append(config_path, key): + return '{}/{}'.format(config_path, key) + + if isinstance(obj, str): + return converter.convert(config_path, interpolator.interpolate(obj)) + if isinstance(obj, dict): + return { + key: recursive_interpolate(val, interpolator, append(config_path, key)) + for key, val in obj.items() + } + if isinstance(obj, list): + return [recursive_interpolate(val, interpolator, config_path) for val in obj] + return converter.convert(config_path, obj) + + +class TemplateWithDefaults(Template): + pattern = r""" + {delim}(?: + (?P<escaped>{delim}) | + (?P<named>{id}) | + {{(?P<braced>{bid})}} | + (?P<invalid>) + ) + """.format( + delim=re.escape('$'), + id=r'[_a-z][_a-z0-9]*', + bid=r'[_a-z][_a-z0-9]*(?:(?P<sep>:?[-?])[^}]*)?', + ) + + @staticmethod + def process_braced_group(braced, sep, mapping): + if ':-' == sep: + var, _, default = braced.partition(':-') + return mapping.get(var) or default + elif '-' == sep: + var, _, default = braced.partition('-') + return mapping.get(var, default) + + elif ':?' == sep: + var, _, err = braced.partition(':?') + result = mapping.get(var) + if not result: + err = err or var + raise UnsetRequiredSubstitution(err) + return result + elif '?' == sep: + var, _, err = braced.partition('?') + if var in mapping: + return mapping.get(var) + err = err or var + raise UnsetRequiredSubstitution(err) + + # Modified from python2.7/string.py + def substitute(self, mapping): + # Helper function for .sub() + + def convert(mo): + named = mo.group('named') or mo.group('braced') + braced = mo.group('braced') + if braced is not None: + sep = mo.group('sep') + if sep: + return self.process_braced_group(braced, sep, mapping) + + if named is not None: + val = mapping[named] + if isinstance(val, bytes): + val = val.decode('utf-8') + return '{}'.format(val) + if mo.group('escaped') is not None: + return self.delimiter + if mo.group('invalid') is not None: + self._invalid(mo) + raise ValueError('Unrecognized named group in pattern', + self.pattern) + return self.pattern.sub(convert, self.template) + + +class InvalidInterpolation(Exception): + def __init__(self, string): + self.string = string + + +class UnsetRequiredSubstitution(Exception): + def __init__(self, custom_err_msg): + self.err = custom_err_msg + + +PATH_JOKER = '[^/]+' +FULL_JOKER = '.+' + + +def re_path(*args): + return re.compile('^{}$'.format('/'.join(args))) + + +def re_path_basic(section, name): + return re_path(section, PATH_JOKER, name) + + +def service_path(*args): + return re_path('service', PATH_JOKER, *args) + + +def to_boolean(s): + if not isinstance(s, str): + return s + s = s.lower() + if s in ['y', 'yes', 'true', 'on']: + return True + elif s in ['n', 'no', 'false', 'off']: + return False + raise ValueError('"{}" is not a valid boolean value'.format(s)) + + +def to_int(s): + if not isinstance(s, str): + return s + + # We must be able to handle octal representation for `mode` values notably + if re.match('^0[0-9]+$', s.strip()): + s = '0o' + s[1:] + try: + return int(s, base=0) + except ValueError: + raise ValueError('"{}" is not a valid integer'.format(s)) + + +def to_float(s): + if not isinstance(s, str): + return s + + try: + return float(s) + except ValueError: + raise ValueError('"{}" is not a valid float'.format(s)) + + +def to_str(o): + if isinstance(o, (bool, float, int)): + return '{}'.format(o) + return o + + +def bytes_to_int(s): + v = parse_bytes(s) + if v is None: + raise ValueError('"{}" is not a valid byte value'.format(s)) + return v + + +def to_microseconds(v): + if not isinstance(v, str): + return v + return int(parse_nanoseconds_int(v) / 1000) + + +class ConversionMap: + map = { + service_path('blkio_config', 'weight'): to_int, + service_path('blkio_config', 'weight_device', 'weight'): to_int, + service_path('build', 'labels', FULL_JOKER): to_str, + service_path('cpus'): to_float, + service_path('cpu_count'): to_int, + service_path('cpu_quota'): to_microseconds, + service_path('cpu_period'): to_microseconds, + service_path('cpu_rt_period'): to_microseconds, + service_path('cpu_rt_runtime'): to_microseconds, + service_path('configs', 'mode'): to_int, + service_path('secrets', 'mode'): to_int, + service_path('healthcheck', 'retries'): to_int, + service_path('healthcheck', 'disable'): to_boolean, + service_path('deploy', 'labels', PATH_JOKER): to_str, + service_path('deploy', 'replicas'): to_int, + service_path('deploy', 'resources', 'limits', "cpus"): to_float, + service_path('deploy', 'update_config', 'parallelism'): to_int, + service_path('deploy', 'update_config', 'max_failure_ratio'): to_float, + service_path('deploy', 'rollback_config', 'parallelism'): to_int, + service_path('deploy', 'rollback_config', 'max_failure_ratio'): to_float, + service_path('deploy', 'restart_policy', 'max_attempts'): to_int, + service_path('mem_swappiness'): to_int, + service_path('labels', FULL_JOKER): to_str, + service_path('oom_kill_disable'): to_boolean, + service_path('oom_score_adj'): to_int, + service_path('ports', 'target'): to_int, + service_path('ports', 'published'): to_int, + service_path('scale'): to_int, + service_path('ulimits', PATH_JOKER): to_int, + service_path('ulimits', PATH_JOKER, 'soft'): to_int, + service_path('ulimits', PATH_JOKER, 'hard'): to_int, + service_path('privileged'): to_boolean, + service_path('read_only'): to_boolean, + service_path('stdin_open'): to_boolean, + service_path('tty'): to_boolean, + service_path('volumes', 'read_only'): to_boolean, + service_path('volumes', 'volume', 'nocopy'): to_boolean, + service_path('volumes', 'tmpfs', 'size'): bytes_to_int, + re_path_basic('network', 'attachable'): to_boolean, + re_path_basic('network', 'external'): to_boolean, + re_path_basic('network', 'internal'): to_boolean, + re_path('network', PATH_JOKER, 'labels', FULL_JOKER): to_str, + re_path_basic('volume', 'external'): to_boolean, + re_path('volume', PATH_JOKER, 'labels', FULL_JOKER): to_str, + re_path_basic('secret', 'external'): to_boolean, + re_path('secret', PATH_JOKER, 'labels', FULL_JOKER): to_str, + re_path_basic('config', 'external'): to_boolean, + re_path('config', PATH_JOKER, 'labels', FULL_JOKER): to_str, + } + + def convert(self, path, value): + for rexp in self.map.keys(): + if rexp.match(path): + try: + return self.map[rexp](value) + except ValueError as e: + raise ConfigurationError( + 'Error while attempting to convert {} to appropriate type: {}'.format( + path.replace('/', '.'), e + ) + ) + return value + + +converter = ConversionMap() diff --git a/compose/config/serialize.py b/compose/config/serialize.py new file mode 100644 index 00000000000..e3295df78ea --- /dev/null +++ b/compose/config/serialize.py @@ -0,0 +1,149 @@ +import yaml + +from compose.config import types +from compose.const import COMPOSE_SPEC as VERSION +from compose.const import COMPOSEFILE_V1 as V1 + + +def serialize_config_type(dumper, data): + representer = dumper.represent_str + return representer(data.repr()) + + +def serialize_dict_type(dumper, data): + return dumper.represent_dict(data.repr()) + + +def serialize_string(dumper, data): + """ Ensure boolean-like strings are quoted in the output """ + representer = dumper.represent_str + + if isinstance(data, bytes): + data = data.decode('utf-8') + + if data.lower() in ('y', 'n', 'yes', 'no', 'on', 'off', 'true', 'false'): + # Empirically only y/n appears to be an issue, but this might change + # depending on which PyYaml version is being used. Err on safe side. + return dumper.represent_scalar('tag:yaml.org,2002:str', data, style='"') + return representer(data) + + +def serialize_string_escape_dollar(dumper, data): + """ Ensure boolean-like strings are quoted in the output and escape $ characters """ + data = data.replace('$', '$$') + return serialize_string(dumper, data) + + +yaml.SafeDumper.add_representer(types.MountSpec, serialize_dict_type) +yaml.SafeDumper.add_representer(types.VolumeFromSpec, serialize_config_type) +yaml.SafeDumper.add_representer(types.VolumeSpec, serialize_config_type) +yaml.SafeDumper.add_representer(types.SecurityOpt, serialize_config_type) +yaml.SafeDumper.add_representer(types.ServiceSecret, serialize_dict_type) +yaml.SafeDumper.add_representer(types.ServiceConfig, serialize_dict_type) +yaml.SafeDumper.add_representer(types.ServicePort, serialize_dict_type) + + +def denormalize_config(config, image_digests=None): + result = {'version': str(config.config_version)} + denormalized_services = [ + denormalize_service_dict( + service_dict, + config.version, + image_digests[service_dict['name']] if image_digests else None) + for service_dict in config.services + ] + result['services'] = { + service_dict.pop('name'): service_dict + for service_dict in denormalized_services + } + + for key in ('networks', 'volumes', 'secrets', 'configs'): + config_dict = getattr(config, key) + if not config_dict: + continue + result[key] = config_dict.copy() + for name, conf in result[key].items(): + if 'external_name' in conf: + del conf['external_name'] + + if 'name' in conf: + if 'external' in conf: + conf['external'] = bool(conf['external']) + return result + + +def serialize_config(config, image_digests=None, escape_dollar=True): + if escape_dollar: + yaml.SafeDumper.add_representer(str, serialize_string_escape_dollar) + yaml.SafeDumper.add_representer(str, serialize_string_escape_dollar) + else: + yaml.SafeDumper.add_representer(str, serialize_string) + yaml.SafeDumper.add_representer(str, serialize_string) + return yaml.safe_dump( + denormalize_config(config, image_digests), + default_flow_style=False, + indent=2, + width=80, + allow_unicode=True + ) + + +def serialize_ns_time_value(value): + result = (value, 'ns') + table = [ + (1000., 'us'), + (1000., 'ms'), + (1000., 's'), + (60., 'm'), + (60., 'h') + ] + for stage in table: + tmp = value / stage[0] + if tmp == int(value / stage[0]): + value = tmp + result = (int(value), stage[1]) + else: + break + return '{}{}'.format(*result) + + +def denormalize_service_dict(service_dict, version, image_digest=None): + service_dict = service_dict.copy() + + if image_digest: + service_dict['image'] = image_digest + + if 'restart' in service_dict: + service_dict['restart'] = types.serialize_restart_spec( + service_dict['restart'] + ) + + if version == V1 and 'network_mode' not in service_dict: + service_dict['network_mode'] = 'bridge' + + if 'healthcheck' in service_dict: + if 'interval' in service_dict['healthcheck']: + service_dict['healthcheck']['interval'] = serialize_ns_time_value( + service_dict['healthcheck']['interval'] + ) + if 'timeout' in service_dict['healthcheck']: + service_dict['healthcheck']['timeout'] = serialize_ns_time_value( + service_dict['healthcheck']['timeout'] + ) + + if 'start_period' in service_dict['healthcheck']: + service_dict['healthcheck']['start_period'] = serialize_ns_time_value( + service_dict['healthcheck']['start_period'] + ) + + if 'ports' in service_dict: + service_dict['ports'] = [ + p.legacy_repr() if p.external_ip or version < VERSION else p + for p in service_dict['ports'] + ] + if 'volumes' in service_dict and (version == V1): + service_dict['volumes'] = [ + v.legacy_repr() if isinstance(v, types.MountSpec) else v for v in service_dict['volumes'] + ] + + return service_dict diff --git a/compose/config/sort_services.py b/compose/config/sort_services.py new file mode 100644 index 00000000000..0a7eb2b4fda --- /dev/null +++ b/compose/config/sort_services.py @@ -0,0 +1,71 @@ +from compose.config.errors import DependencyError + + +def get_service_name_from_network_mode(network_mode): + return get_source_name_from_network_mode(network_mode, 'service') + + +def get_container_name_from_network_mode(network_mode): + return get_source_name_from_network_mode(network_mode, 'container') + + +def get_source_name_from_network_mode(network_mode, source_type): + if not network_mode: + return + + if not network_mode.startswith(source_type+':'): + return + + _, net_name = network_mode.split(':', 1) + return net_name + + +def get_service_names(links): + return [link.split(':', 1)[0] for link in links] + + +def get_service_names_from_volumes_from(volumes_from): + return [volume_from.source for volume_from in volumes_from] + + +def get_service_dependents(service_dict, services): + name = service_dict['name'] + return [ + service for service in services + if (name in get_service_names(service.get('links', [])) or + name in get_service_names_from_volumes_from(service.get('volumes_from', [])) or + name == get_service_name_from_network_mode(service.get('network_mode')) or + name == get_service_name_from_network_mode(service.get('pid')) or + name == get_service_name_from_network_mode(service.get('ipc')) or + name in service.get('depends_on', [])) + ] + + +def sort_service_dicts(services): + # Topological sort (Cormen/Tarjan algorithm). + unmarked = services[:] + temporary_marked = set() + sorted_services = [] + + def visit(n): + if n['name'] in temporary_marked: + if n['name'] in get_service_names(n.get('links', [])): + raise DependencyError('A service can not link to itself: %s' % n['name']) + if n['name'] in n.get('volumes_from', []): + raise DependencyError('A service can not mount itself as volume: %s' % n['name']) + if n['name'] in n.get('depends_on', []): + raise DependencyError('A service can not depend on itself: %s' % n['name']) + raise DependencyError('Circular dependency between %s' % ' and '.join(temporary_marked)) + + if n in unmarked: + temporary_marked.add(n['name']) + for m in get_service_dependents(n, services): + visit(m) + temporary_marked.remove(n['name']) + unmarked.remove(n) + sorted_services.insert(0, n) + + while unmarked: + visit(unmarked[-1]) + + return sorted_services diff --git a/compose/config/types.py b/compose/config/types.py new file mode 100644 index 00000000000..f52b5654139 --- /dev/null +++ b/compose/config/types.py @@ -0,0 +1,500 @@ +""" +Types for objects parsed from the configuration. +""" +import json +import ntpath +import os +import re +from collections import namedtuple + +from docker.utils.ports import build_port_bindings + +from ..const import COMPOSEFILE_V1 as V1 +from ..utils import unquote_path +from .errors import ConfigurationError +from compose.const import IS_WINDOWS_PLATFORM +from compose.utils import splitdrive + +win32_root_path_pattern = re.compile(r'^[A-Za-z]\:\\.*') + + +class VolumeFromSpec(namedtuple('_VolumeFromSpec', 'source mode type')): + + # TODO: drop service_names arg when v1 is removed + @classmethod + def parse(cls, volume_from_config, service_names, version): + func = cls.parse_v1 if version == V1 else cls.parse_v2 + return func(service_names, volume_from_config) + + @classmethod + def parse_v1(cls, service_names, volume_from_config): + parts = volume_from_config.split(':') + if len(parts) > 2: + raise ConfigurationError( + "volume_from {} has incorrect format, should be " + "service[:mode]".format(volume_from_config)) + + if len(parts) == 1: + source = parts[0] + mode = 'rw' + else: + source, mode = parts + + type = 'service' if source in service_names else 'container' + return cls(source, mode, type) + + @classmethod + def parse_v2(cls, service_names, volume_from_config): + parts = volume_from_config.split(':') + if len(parts) > 3: + raise ConfigurationError( + "volume_from {} has incorrect format, should be one of " + "'<service name>[:<mode>]' or " + "'container:<container name>[:<mode>]'".format(volume_from_config)) + + if len(parts) == 1: + source = parts[0] + return cls(source, 'rw', 'service') + + if len(parts) == 2: + if parts[0] == 'container': + type, source = parts + return cls(source, 'rw', type) + + source, mode = parts + return cls(source, mode, 'service') + + if len(parts) == 3: + type, source, mode = parts + if type not in ('service', 'container'): + raise ConfigurationError( + "Unknown volumes_from type '{}' in '{}'".format( + type, + volume_from_config)) + + return cls(source, mode, type) + + def repr(self): + return '{v.type}:{v.source}:{v.mode}'.format(v=self) + + +def parse_restart_spec(restart_config): + if not restart_config: + return None + parts = restart_config.split(':') + if len(parts) > 2: + raise ConfigurationError( + "Restart %s has incorrect format, should be " + "mode[:max_retry]" % restart_config) + if len(parts) == 2: + name, max_retry_count = parts + else: + name, = parts + max_retry_count = 0 + + return {'Name': name, 'MaximumRetryCount': int(max_retry_count)} + + +def serialize_restart_spec(restart_spec): + if not restart_spec: + return '' + parts = [restart_spec['Name']] + if restart_spec['MaximumRetryCount']: + parts.append(str(restart_spec['MaximumRetryCount'])) + return ':'.join(parts) + + +def parse_extra_hosts(extra_hosts_config): + if not extra_hosts_config: + return {} + + if isinstance(extra_hosts_config, dict): + return dict(extra_hosts_config) + + if isinstance(extra_hosts_config, list): + extra_hosts_dict = {} + for extra_hosts_line in extra_hosts_config: + # TODO: validate string contains ':' ? + host, ip = extra_hosts_line.split(':', 1) + extra_hosts_dict[host.strip()] = ip.strip() + return extra_hosts_dict + + +def normalize_path_for_engine(path): + """Windows paths, c:\\my\\path\\shiny, need to be changed to be compatible with + the Engine. Volume paths are expected to be linux style /c/my/path/shiny/ + """ + drive, tail = splitdrive(path) + + if drive: + path = '/' + drive.lower().rstrip(':') + tail + + return path.replace('\\', '/') + + +def normpath(path, win_host=False): + """ Custom path normalizer that handles Compose-specific edge cases like + UNIX paths on Windows hosts and vice-versa. """ + + sysnorm = ntpath.normpath if win_host else os.path.normpath + # If a path looks like a UNIX absolute path on Windows, it probably is; + # we'll need to revert the backslashes to forward slashes after normalization + flip_slashes = path.startswith('/') and IS_WINDOWS_PLATFORM + path = sysnorm(path) + if flip_slashes: + path = path.replace('\\', '/') + return path + + +class MountSpec: + options_map = { + 'volume': { + 'nocopy': 'no_copy' + }, + 'bind': { + 'propagation': 'propagation' + }, + 'tmpfs': { + 'size': 'tmpfs_size' + } + } + _fields = ['type', 'source', 'target', 'read_only', 'consistency'] + + @classmethod + def parse(cls, mount_dict, normalize=False, win_host=False): + if mount_dict.get('source'): + if mount_dict['type'] == 'tmpfs': + raise ConfigurationError('tmpfs mounts can not specify a source') + + mount_dict['source'] = normpath(mount_dict['source'], win_host) + if normalize: + mount_dict['source'] = normalize_path_for_engine(mount_dict['source']) + + return cls(**mount_dict) + + def __init__(self, type, source=None, target=None, read_only=None, consistency=None, **kwargs): + self.type = type + self.source = source + self.target = target + self.read_only = read_only + self.consistency = consistency + self.options = None + if self.type in kwargs: + self.options = kwargs[self.type] + + def as_volume_spec(self): + mode = 'ro' if self.read_only else 'rw' + return VolumeSpec(external=self.source, internal=self.target, mode=mode) + + def legacy_repr(self): + return self.as_volume_spec().repr() + + def repr(self): + res = {} + for field in self._fields: + if getattr(self, field, None): + res[field] = getattr(self, field) + if self.options: + res[self.type] = self.options + return res + + @property + def is_named_volume(self): + return self.type == 'volume' and self.source + + @property + def is_tmpfs(self): + return self.type == 'tmpfs' + + @property + def external(self): + return self.source + + +class VolumeSpec(namedtuple('_VolumeSpec', 'external internal mode')): + win32 = False + + @classmethod + def _parse_unix(cls, volume_config): + parts = volume_config.split(':') + + if len(parts) > 3: + raise ConfigurationError( + "Volume %s has incorrect format, should be " + "external:internal[:mode]" % volume_config) + + if len(parts) == 1: + external = None + internal = os.path.normpath(parts[0]) + else: + external = os.path.normpath(parts[0]) + internal = os.path.normpath(parts[1]) + + mode = 'rw' + if len(parts) == 3: + mode = parts[2] + + return cls(external, internal, mode) + + @classmethod + def _parse_win32(cls, volume_config, normalize): + # relative paths in windows expand to include the drive, eg C:\ + # so we join the first 2 parts back together to count as one + mode = 'rw' + + def separate_next_section(volume_config): + drive, tail = splitdrive(volume_config) + parts = tail.split(':', 1) + if drive: + parts[0] = drive + parts[0] + return parts + + parts = separate_next_section(volume_config) + if len(parts) == 1: + internal = parts[0] + external = None + else: + external = parts[0] + parts = separate_next_section(parts[1]) + external = normpath(external, True) + internal = parts[0] + if len(parts) > 1: + if ':' in parts[1]: + raise ConfigurationError( + "Volume %s has incorrect format, should be " + "external:internal[:mode]" % volume_config + ) + mode = parts[1] + + if normalize: + external = normalize_path_for_engine(external) if external else None + + result = cls(external, internal, mode) + result.win32 = True + return result + + @classmethod + def parse(cls, volume_config, normalize=False, win_host=False): + """Parse a volume_config path and split it into external:internal[:mode] + parts to be returned as a valid VolumeSpec. + """ + if IS_WINDOWS_PLATFORM or win_host: + return cls._parse_win32(volume_config, normalize) + else: + return cls._parse_unix(volume_config) + + def repr(self): + external = self.external + ':' if self.external else '' + mode = ':' + self.mode if self.external else '' + return '{ext}{v.internal}{mode}'.format(mode=mode, ext=external, v=self) + + @property + def is_named_volume(self): + res = self.external and not self.external.startswith(('.', '/', '~')) + if not self.win32: + return res + + return ( + res and not self.external.startswith('\\') and + not win32_root_path_pattern.match(self.external) + ) + + +class ServiceLink(namedtuple('_ServiceLink', 'target alias')): + + @classmethod + def parse(cls, link_spec): + target, _, alias = link_spec.partition(':') + if not alias: + alias = target + return cls(target, alias) + + def repr(self): + if self.target == self.alias: + return self.target + return '{s.target}:{s.alias}'.format(s=self) + + @property + def merge_field(self): + return self.alias + + +class ServiceConfigBase(namedtuple('_ServiceConfigBase', 'source target uid gid mode name')): + @classmethod + def parse(cls, spec): + if isinstance(spec, str): + return cls(spec, None, None, None, None, None) + return cls( + spec.get('source'), + spec.get('target'), + spec.get('uid'), + spec.get('gid'), + spec.get('mode'), + spec.get('name') + ) + + @property + def merge_field(self): + return self.source + + def repr(self): + return { + k: v for k, v in zip(self._fields, self) if v is not None + } + + +class ServiceSecret(ServiceConfigBase): + pass + + +class ServiceConfig(ServiceConfigBase): + pass + + +class ServicePort(namedtuple('_ServicePort', 'target published protocol mode external_ip')): + def __new__(cls, target, published, *args, **kwargs): + try: + if target: + target = int(target) + except ValueError: + raise ConfigurationError('Invalid target port: {}'.format(target)) + + if published: + if isinstance(published, str) and '-' in published: # "x-y:z" format + a, b = published.split('-', 1) + if not a.isdigit() or not b.isdigit(): + raise ConfigurationError('Invalid published port: {}'.format(published)) + else: + try: + published = int(published) + except ValueError: + raise ConfigurationError('Invalid published port: {}'.format(published)) + + return super().__new__( + cls, target, published, *args, **kwargs + ) + + @classmethod + def parse(cls, spec): + if isinstance(spec, cls): + # When extending a service with ports, the port definitions have already been parsed + return [spec] + + if not isinstance(spec, dict): + result = [] + try: + for k, v in build_port_bindings([spec]).items(): + if '/' in k: + target, proto = k.split('/', 1) + else: + target, proto = (k, None) + for pub in v: + if pub is None: + result.append( + cls(target, None, proto, None, None) + ) + elif isinstance(pub, tuple): + result.append( + cls(target, pub[1], proto, None, pub[0]) + ) + else: + result.append( + cls(target, pub, proto, None, None) + ) + except ValueError as e: + raise ConfigurationError(str(e)) + + return result + + return [cls( + spec.get('target'), + spec.get('published'), + spec.get('protocol'), + spec.get('mode'), + None + )] + + @property + def merge_field(self): + return (self.target, self.published, self.external_ip, self.protocol) + + def repr(self): + return { + k: v for k, v in zip(self._fields, self) if v is not None + } + + def legacy_repr(self): + return normalize_port_dict(self.repr()) + + +class GenericResource(namedtuple('_GenericResource', 'kind value')): + @classmethod + def parse(cls, dct): + if 'discrete_resource_spec' not in dct: + raise ConfigurationError( + 'generic_resource entry must include a discrete_resource_spec key' + ) + if 'kind' not in dct['discrete_resource_spec']: + raise ConfigurationError( + 'generic_resource entry must include a discrete_resource_spec.kind subkey' + ) + return cls( + dct['discrete_resource_spec']['kind'], + dct['discrete_resource_spec'].get('value') + ) + + def repr(self): + return { + 'discrete_resource_spec': { + 'kind': self.kind, + 'value': self.value, + } + } + + @property + def merge_field(self): + return self.kind + + +def normalize_port_dict(port): + return '{external_ip}{has_ext_ip}{published}{is_pub}{target}/{protocol}'.format( + published=port.get('published', ''), + is_pub=(':' if port.get('published') is not None or port.get('external_ip') else ''), + target=port.get('target'), + protocol=port.get('protocol', 'tcp'), + external_ip=port.get('external_ip', ''), + has_ext_ip=(':' if port.get('external_ip') else ''), + ) + + +class SecurityOpt(namedtuple('_SecurityOpt', 'value src_file')): + @classmethod + def parse(cls, value): + if not isinstance(value, str): + return value + # based on https://github.com/docker/cli/blob/9de1b162f/cli/command/container/opts.go#L673-L697 + con = value.split('=', 2) + if len(con) == 1 and con[0] != 'no-new-privileges': + if ':' not in value: + raise ConfigurationError('Invalid security_opt: {}'.format(value)) + con = value.split(':', 2) + + if con[0] == 'seccomp' and con[1] != 'unconfined': + try: + with open(unquote_path(con[1])) as f: + seccomp_data = json.load(f) + except (OSError, ValueError) as e: + raise ConfigurationError('Error reading seccomp profile: {}'.format(e)) + return cls( + 'seccomp={}'.format(json.dumps(seccomp_data)), con[1] + ) + return cls(value, None) + + def repr(self): + if self.src_file is not None: + return 'seccomp:{}'.format(self.src_file) + return self.value + + @property + def merge_field(self): + return self.value diff --git a/compose/config/validation.py b/compose/config/validation.py new file mode 100644 index 00000000000..d9aaeda4bd2 --- /dev/null +++ b/compose/config/validation.py @@ -0,0 +1,569 @@ +import json +import logging +import os +import re +import sys + +from docker.utils.ports import split_port +from jsonschema import Draft4Validator +from jsonschema import FormatChecker +from jsonschema import RefResolver +from jsonschema import ValidationError + +from ..const import COMPOSEFILE_V1 as V1 +from ..const import NANOCPUS_SCALE +from .errors import ConfigurationError +from .errors import VERSION_EXPLANATION +from .sort_services import get_service_name_from_network_mode + + +log = logging.getLogger(__name__) + + +DOCKER_CONFIG_HINTS = { + 'cpu_share': 'cpu_shares', + 'add_host': 'extra_hosts', + 'hosts': 'extra_hosts', + 'extra_host': 'extra_hosts', + 'device': 'devices', + 'link': 'links', + 'memory_swap': 'memswap_limit', + 'port': 'ports', + 'privilege': 'privileged', + 'priviliged': 'privileged', + 'privilige': 'privileged', + 'volume': 'volumes', + 'workdir': 'working_dir', +} + + +VALID_NAME_CHARS = r'[a-zA-Z0-9\._\-]' +VALID_EXPOSE_FORMAT = r'^\d+(\-\d+)?(\/[a-zA-Z]+)?$' + +VALID_IPV4_SEG = r'(\d{1,2}|1\d{2}|2[0-4]\d|25[0-5])' +VALID_IPV4_ADDR = r"({IPV4_SEG}\.){{3}}{IPV4_SEG}".format(IPV4_SEG=VALID_IPV4_SEG) +VALID_REGEX_IPV4_CIDR = r"^{IPV4_ADDR}/(\d|[1-2]\d|3[0-2])$".format(IPV4_ADDR=VALID_IPV4_ADDR) + +VALID_IPV6_SEG = r'[0-9a-fA-F]{1,4}' +VALID_REGEX_IPV6_CIDR = "".join(r""" +^ +( + (({IPV6_SEG}:){{7}}{IPV6_SEG})| + (({IPV6_SEG}:){{1,7}}:)| + (({IPV6_SEG}:){{1,6}}(:{IPV6_SEG}){{1,1}})| + (({IPV6_SEG}:){{1,5}}(:{IPV6_SEG}){{1,2}})| + (({IPV6_SEG}:){{1,4}}(:{IPV6_SEG}){{1,3}})| + (({IPV6_SEG}:){{1,3}}(:{IPV6_SEG}){{1,4}})| + (({IPV6_SEG}:){{1,2}}(:{IPV6_SEG}){{1,5}})| + (({IPV6_SEG}:){{1,1}}(:{IPV6_SEG}){{1,6}})| + (:((:{IPV6_SEG}){{1,7}}|:))| + (fe80:(:{IPV6_SEG}){{0,4}}%[0-9a-zA-Z]{{1,}})| + (::(ffff(:0{{1,4}}){{0,1}}:){{0,1}}{IPV4_ADDR})| + (({IPV6_SEG}:){{1,4}}:{IPV4_ADDR}) +) +/(\d|[1-9]\d|1[0-1]\d|12[0-8]) +$ +""".format(IPV6_SEG=VALID_IPV6_SEG, IPV4_ADDR=VALID_IPV4_ADDR).split()) + + +@FormatChecker.cls_checks(format="ports", raises=ValidationError) +def format_ports(instance): + try: + split_port(instance) + except ValueError as e: + raise ValidationError(str(e)) + return True + + +@FormatChecker.cls_checks(format="expose", raises=ValidationError) +def format_expose(instance): + if isinstance(instance, str): + if not re.match(VALID_EXPOSE_FORMAT, instance): + raise ValidationError( + "should be of the format 'PORT[/PROTOCOL]'") + + return True + + +@FormatChecker.cls_checks("subnet_ip_address", raises=ValidationError) +def format_subnet_ip_address(instance): + if isinstance(instance, str): + if not re.match(VALID_REGEX_IPV4_CIDR, instance) and \ + not re.match(VALID_REGEX_IPV6_CIDR, instance): + raise ValidationError("should use the CIDR format") + + return True + + +def match_named_volumes(service_dict, project_volumes): + service_volumes = service_dict.get('volumes', []) + for volume_spec in service_volumes: + if volume_spec.is_named_volume and volume_spec.external not in project_volumes: + raise ConfigurationError( + 'Named volume "{}" is used in service "{}" but no' + ' declaration was found in the volumes section.'.format( + volume_spec.repr(), service_dict.get('name') + ) + ) + + +def python_type_to_yaml_type(type_): + type_name = type(type_).__name__ + return { + 'dict': 'mapping', + 'list': 'array', + 'int': 'number', + 'float': 'number', + 'bool': 'boolean', + 'unicode': 'string', + 'str': 'string', + 'bytes': 'string', + }.get(type_name, type_name) + + +def validate_config_section(filename, config, section): + """Validate the structure of a configuration section. This must be done + before interpolation so it's separate from schema validation. + """ + if not isinstance(config, dict): + raise ConfigurationError( + "In file '{filename}', {section} must be a mapping, not " + "{type}.".format( + filename=filename, + section=section, + type=anglicize_json_type(python_type_to_yaml_type(config)))) + + for key, value in config.items(): + if not isinstance(key, str): + raise ConfigurationError( + "In file '{filename}', the {section} name {name} must be a " + "quoted string, i.e. '{name}'.".format( + filename=filename, + section=section, + name=key)) + + if not isinstance(value, (dict, type(None))): + raise ConfigurationError( + "In file '{filename}', {section} '{name}' must be a mapping not " + "{type}.".format( + filename=filename, + section=section, + name=key, + type=anglicize_json_type(python_type_to_yaml_type(value)))) + + +def validate_top_level_object(config_file): + if not isinstance(config_file.config, dict): + raise ConfigurationError( + "Top level object in '{}' needs to be an object not '{}'.".format( + config_file.filename, + type(config_file.config))) + + +def validate_ulimits(service_config): + ulimit_config = service_config.config.get('ulimits', {}) + for limit_name, soft_hard_values in ulimit_config.items(): + if isinstance(soft_hard_values, dict): + if not soft_hard_values['soft'] <= soft_hard_values['hard']: + raise ConfigurationError( + "Service '{s.name}' has invalid ulimit '{ulimit}'. " + "'soft' value can not be greater than 'hard' value ".format( + s=service_config, + ulimit=ulimit_config)) + + +def validate_extends_file_path(service_name, extends_options, filename): + """ + The service to be extended must either be defined in the config key 'file', + or within 'filename'. + """ + error_prefix = "Invalid 'extends' configuration for %s:" % service_name + + if 'file' not in extends_options and filename is None: + raise ConfigurationError( + "%s you need to specify a 'file', e.g. 'file: something.yml'" % error_prefix + ) + + +def validate_network_mode(service_config, service_names): + network_mode = service_config.config.get('network_mode') + if not network_mode: + return + + if 'networks' in service_config.config: + raise ConfigurationError("'network_mode' and 'networks' cannot be combined") + + dependency = get_service_name_from_network_mode(network_mode) + if not dependency: + return + + if dependency not in service_names: + raise ConfigurationError( + "Service '{s.name}' uses the network stack of service '{dep}' which " + "is undefined.".format(s=service_config, dep=dependency)) + + +def validate_pid_mode(service_config, service_names): + pid_mode = service_config.config.get('pid') + if not pid_mode: + return + + dependency = get_service_name_from_network_mode(pid_mode) + if not dependency: + return + if dependency not in service_names: + raise ConfigurationError( + "Service '{s.name}' uses the PID namespace of service '{dep}' which " + "is undefined.".format(s=service_config, dep=dependency) + ) + + +def validate_ipc_mode(service_config, service_names): + ipc_mode = service_config.config.get('ipc') + if not ipc_mode: + return + + dependency = get_service_name_from_network_mode(ipc_mode) + if not dependency: + return + if dependency not in service_names: + raise ConfigurationError( + "Service '{s.name}' uses the IPC namespace of service '{dep}' which " + "is undefined.".format(s=service_config, dep=dependency) + ) + + +def validate_links(service_config, service_names): + for link in service_config.config.get('links', []): + if link.split(':')[0] not in service_names: + raise ConfigurationError( + "Service '{s.name}' has a link to service '{link}' which is " + "undefined.".format(s=service_config, link=link)) + + +def validate_depends_on(service_config, service_names): + deps = service_config.config.get('depends_on', {}) + for dependency in deps.keys(): + if dependency not in service_names: + raise ConfigurationError( + "Service '{s.name}' depends on service '{dep}' which is " + "undefined.".format(s=service_config, dep=dependency) + ) + + +def validate_credential_spec(service_config): + credential_spec = service_config.config.get('credential_spec') + if not credential_spec: + return + + if 'registry' not in credential_spec and 'file' not in credential_spec: + raise ConfigurationError( + "Service '{s.name}' is missing 'credential_spec.file' or " + "credential_spec.registry'".format(s=service_config) + ) + + +def get_unsupported_config_msg(path, error_key): + msg = "Unsupported config option for {}: '{}'".format(path_string(path), error_key) + if error_key in DOCKER_CONFIG_HINTS: + msg += " (did you mean '{}'?)".format(DOCKER_CONFIG_HINTS[error_key]) + return msg + + +def anglicize_json_type(json_type): + if json_type.startswith(('a', 'e', 'i', 'o', 'u')): + return 'an ' + json_type + return 'a ' + json_type + + +def is_service_dict_schema(schema_id): + return schema_id in ('config_schema_v1.json', '#/properties/services') + + +def handle_error_for_schema_with_id(error, path): + schema_id = error.schema['id'] + + if is_service_dict_schema(schema_id) and error.validator == 'additionalProperties': + return "Invalid service name '{}' - only {} characters are allowed".format( + # The service_name is one of the keys in the json object + [i for i in list(error.instance) if not i or any(filter( + lambda c: not re.match(VALID_NAME_CHARS, c), i + ))][0], + VALID_NAME_CHARS + ) + + if error.validator == 'additionalProperties': + if schema_id == '#/definitions/service': + invalid_config_key = parse_key_from_error_msg(error) + return get_unsupported_config_msg(path, invalid_config_key) + + if schema_id.startswith('config_schema_'): + invalid_config_key = parse_key_from_error_msg(error) + return ('Invalid top-level property "{key}". Valid top-level ' + 'sections for this Compose file are: {properties}, and ' + 'extensions starting with "x-".\n\n{explanation}').format( + key=invalid_config_key, + properties=', '.join(error.schema['properties'].keys()), + explanation=VERSION_EXPLANATION + ) + + if not error.path: + return '{}\n\n{}'.format(error.message, VERSION_EXPLANATION) + + +def handle_generic_error(error, path): + msg_format = None + error_msg = error.message + + if error.validator == 'oneOf': + msg_format = "{path} {msg}" + config_key, error_msg = _parse_oneof_validator(error) + if config_key: + path.append(config_key) + + elif error.validator == 'type': + msg_format = "{path} contains an invalid type, it should be {msg}" + error_msg = _parse_valid_types_from_validator(error.validator_value) + + elif error.validator == 'required': + error_msg = ", ".join(error.validator_value) + msg_format = "{path} is invalid, {msg} is required." + + elif error.validator == 'dependencies': + config_key = list(error.validator_value.keys())[0] + required_keys = ",".join(error.validator_value[config_key]) + + msg_format = "{path} is invalid: {msg}" + path.append(config_key) + error_msg = "when defining '{}' you must set '{}' as well".format( + config_key, + required_keys) + + elif error.cause: + error_msg = str(error.cause) + msg_format = "{path} is invalid: {msg}" + + elif error.path: + msg_format = "{path} value {msg}" + + if msg_format: + return msg_format.format(path=path_string(path), msg=error_msg) + + return error.message + + +def parse_key_from_error_msg(error): + try: + return error.message.split("'")[1] + except IndexError: + return error.message.split('(')[1].split(' ')[0].strip("'") + + +def path_string(path): + return ".".join(c for c in path if isinstance(c, str)) + + +def _parse_valid_types_from_validator(validator): + """A validator value can be either an array of valid types or a string of + a valid type. Parse the valid types and prefix with the correct article. + """ + if not isinstance(validator, list): + return anglicize_json_type(validator) + + if len(validator) == 1: + return anglicize_json_type(validator[0]) + + return "{}, or {}".format( + ", ".join([anglicize_json_type(validator[0])] + validator[1:-1]), + anglicize_json_type(validator[-1])) + + +def _parse_oneof_validator(error): + """oneOf has multiple schemas, so we need to reason about which schema, sub + schema or constraint the validation is failing on. + Inspecting the context value of a ValidationError gives us information about + which sub schema failed and which kind of error it is. + """ + types = [] + for context in error.context: + if context.validator == 'oneOf': + _, error_msg = _parse_oneof_validator(context) + return path_string(context.path), error_msg + + if context.validator == 'required': + return (None, context.message) + + if context.validator == 'additionalProperties': + invalid_config_key = parse_key_from_error_msg(context) + return (None, "contains unsupported option: '{}'".format(invalid_config_key)) + + if context.validator == 'uniqueItems': + return ( + path_string(context.path) if context.path else None, + "contains non-unique items, please remove duplicates from {}".format( + context.instance), + ) + + if context.path: + return ( + path_string(context.path), + "contains {}, which is an invalid type, it should be {}".format( + json.dumps(context.instance), + _parse_valid_types_from_validator(context.validator_value)), + ) + + if context.validator == 'type': + types.append(context.validator_value) + + valid_types = _parse_valid_types_from_validator(types) + return (None, "contains an invalid type, it should be {}".format(valid_types)) + + +def process_service_constraint_errors(error, service_name, version): + if version == V1: + if 'image' in error.instance and 'build' in error.instance: + return ( + "Service {} has both an image and build path specified. " + "A service can either be built to image or use an existing " + "image, not both.".format(service_name)) + + if 'image' in error.instance and 'dockerfile' in error.instance: + return ( + "Service {} has both an image and alternate Dockerfile. " + "A service can either be built to image or use an existing " + "image, not both.".format(service_name)) + + if 'image' not in error.instance and 'build' not in error.instance: + return ( + "Service {} has neither an image nor a build context specified. " + "At least one must be provided.".format(service_name)) + + +def process_config_schema_errors(error): + path = list(error.path) + + if 'id' in error.schema: + error_msg = handle_error_for_schema_with_id(error, path) + if error_msg: + return error_msg + + return handle_generic_error(error, path) + + +def keys_to_str(config_file): + """ + Non-string keys may break validator with patterned fields. + """ + d = {} + for k, v in config_file.items(): + d[str(k)] = v + if isinstance(v, dict): + d[str(k)] = keys_to_str(v) + return d + + +def validate_against_config_schema(config_file, version): + schema = load_jsonschema(version) + config = keys_to_str(config_file.config) + + format_checker = FormatChecker(["ports", "expose", "subnet_ip_address"]) + validator = Draft4Validator( + schema, + resolver=RefResolver(get_resolver_path(), schema), + format_checker=format_checker) + handle_errors( + validator.iter_errors(config), + process_config_schema_errors, + config_file.filename) + + +def validate_service_constraints(config, service_name, config_file): + def handler(errors): + return process_service_constraint_errors( + errors, service_name, config_file.version) + + schema = load_jsonschema(config_file.version) + validator = Draft4Validator(schema['definitions']['constraints']['service']) + handle_errors(validator.iter_errors(config), handler, None) + + +def validate_cpu(service_config): + cpus = service_config.config.get('cpus') + if not cpus: + return + nano_cpus = cpus * NANOCPUS_SCALE + if isinstance(nano_cpus, float) and not nano_cpus.is_integer(): + raise ConfigurationError( + "cpus must have nine or less digits after decimal point") + + +def get_schema_path(): + return os.path.dirname(os.path.abspath(__file__)) + + +def load_jsonschema(version): + name = "compose_spec" + if version == V1: + name = "config_schema_v1" + + filename = os.path.join( + get_schema_path(), + "{}.json".format(name)) + + if not os.path.exists(filename): + raise ConfigurationError( + 'Version in "{}" is unsupported. {}' + .format(filename, VERSION_EXPLANATION)) + with open(filename) as fh: + return json.load(fh) + + +def get_resolver_path(): + schema_path = get_schema_path() + if sys.platform == "win32": + scheme = "///" + # TODO: why is this necessary? + schema_path = schema_path.replace('\\', '/') + else: + scheme = "//" + return "file:{}{}/".format(scheme, schema_path) + + +def handle_errors(errors, format_error_func, filename): + """jsonschema returns an error tree full of information to explain what has + gone wrong. Process each error and pull out relevant information and re-write + helpful error messages that are relevant. + """ + errors = sorted(errors, key=str) + if not errors: + return + + error_msg = '\n'.join(format_error_func(error) for error in errors) + raise ConfigurationError( + "The Compose file{file_msg} is invalid because:\n{error_msg}".format( + file_msg=" '{}'".format(filename) if filename else "", + error_msg=error_msg)) + + +def validate_healthcheck(service_config): + healthcheck = service_config.config.get('healthcheck', {}) + + if 'test' in healthcheck and isinstance(healthcheck['test'], list): + if len(healthcheck['test']) == 0: + raise ConfigurationError( + 'Service "{}" defines an invalid healthcheck: ' + '"test" is an empty list' + .format(service_config.name)) + + # when disable is true config.py::process_healthcheck adds "test: ['NONE']" to service_config + elif healthcheck['test'][0] == 'NONE' and len(healthcheck) > 1: + raise ConfigurationError( + 'Service "{}" defines an invalid healthcheck: ' + '"disable: true" cannot be combined with other options' + .format(service_config.name)) + + elif healthcheck['test'][0] not in ('NONE', 'CMD', 'CMD-SHELL'): + raise ConfigurationError( + 'Service "{}" defines an invalid healthcheck: ' + 'when "test" is a list the first item must be either NONE, CMD or CMD-SHELL' + .format(service_config.name)) diff --git a/compose/const.py b/compose/const.py new file mode 100644 index 00000000000..04342980255 --- /dev/null +++ b/compose/const.py @@ -0,0 +1,39 @@ +import sys + +from .version import ComposeVersion + +DEFAULT_TIMEOUT = 10 +HTTP_TIMEOUT = 60 +IS_WINDOWS_PLATFORM = (sys.platform == "win32") +LABEL_CONTAINER_NUMBER = 'com.docker.compose.container-number' +LABEL_ONE_OFF = 'com.docker.compose.oneoff' +LABEL_PROJECT = 'com.docker.compose.project' +LABEL_WORKING_DIR = 'com.docker.compose.project.working_dir' +LABEL_CONFIG_FILES = 'com.docker.compose.project.config_files' +LABEL_ENVIRONMENT_FILE = 'com.docker.compose.project.environment_file' +LABEL_SERVICE = 'com.docker.compose.service' +LABEL_NETWORK = 'com.docker.compose.network' +LABEL_VERSION = 'com.docker.compose.version' +LABEL_SLUG = 'com.docker.compose.slug' +LABEL_VOLUME = 'com.docker.compose.volume' +LABEL_CONFIG_HASH = 'com.docker.compose.config-hash' +NANOCPUS_SCALE = 1000000000 +PARALLEL_LIMIT = 64 + +SECRETS_PATH = '/run/secrets' +WINDOWS_LONGPATH_PREFIX = '\\\\?\\' + +COMPOSEFILE_V1 = ComposeVersion('1') +COMPOSE_SPEC = ComposeVersion('3.9') + +# minimum DOCKER ENGINE API version needed to support +# features for each compose schema version +API_VERSIONS = { + COMPOSEFILE_V1: '1.21', + COMPOSE_SPEC: '1.38', +} + +API_VERSION_TO_ENGINE_VERSION = { + API_VERSIONS[COMPOSEFILE_V1]: '1.9.0', + API_VERSIONS[COMPOSE_SPEC]: '18.06.0', +} diff --git a/compose/container.py b/compose/container.py new file mode 100644 index 00000000000..00626b6190e --- /dev/null +++ b/compose/container.py @@ -0,0 +1,331 @@ +from functools import reduce + +from docker.errors import ImageNotFound + +from .const import LABEL_CONTAINER_NUMBER +from .const import LABEL_ONE_OFF +from .const import LABEL_PROJECT +from .const import LABEL_SERVICE +from .const import LABEL_SLUG +from .const import LABEL_VERSION +from .utils import truncate_id +from .version import ComposeVersion + + +class Container: + """ + Represents a Docker container, constructed from the output of + GET /containers/:id:/json. + """ + def __init__(self, client, dictionary, has_been_inspected=False): + self.client = client + self.dictionary = dictionary + self.has_been_inspected = has_been_inspected + self.log_stream = None + + @classmethod + def from_ps(cls, client, dictionary, **kwargs): + """ + Construct a container object from the output of GET /containers/json. + """ + name = get_container_name(dictionary) + if name is None: + return None + + new_dictionary = { + 'Id': dictionary['Id'], + 'Image': dictionary['Image'], + 'Name': '/' + name, + } + return cls(client, new_dictionary, **kwargs) + + @classmethod + def from_id(cls, client, id): + return cls(client, client.inspect_container(id), has_been_inspected=True) + + @classmethod + def create(cls, client, **options): + response = client.create_container(**options) + return cls.from_id(client, response['Id']) + + @property + def id(self): + return self.dictionary['Id'] + + @property + def image(self): + return self.dictionary['Image'] + + @property + def image_config(self): + return self.client.inspect_image(self.image) + + @property + def short_id(self): + return self.id[:12] + + @property + def name(self): + return self.dictionary['Name'][1:] + + @property + def project(self): + return self.labels.get(LABEL_PROJECT) + + @property + def service(self): + return self.labels.get(LABEL_SERVICE) + + @property + def name_without_project(self): + if self.name.startswith('{}_{}'.format(self.project, self.service)): + return '{}_{}'.format(self.service, self.number if self.number is not None else self.slug) + else: + return self.name + + @property + def number(self): + if self.one_off: + # One-off containers are no longer assigned numbers and use slugs instead. + return None + + number = self.labels.get(LABEL_CONTAINER_NUMBER) + if not number: + raise ValueError("Container {} does not have a {} label".format( + self.short_id, LABEL_CONTAINER_NUMBER)) + return int(number) + + @property + def slug(self): + if not self.full_slug: + return None + return truncate_id(self.full_slug) + + @property + def full_slug(self): + return self.labels.get(LABEL_SLUG) + + @property + def one_off(self): + return self.labels.get(LABEL_ONE_OFF) == 'True' + + @property + def ports(self): + self.inspect_if_not_inspected() + return self.get('NetworkSettings.Ports') or {} + + @property + def human_readable_ports(self): + def format_port(private, public): + if not public: + return [private] + return [ + '{HostIp}:{HostPort}->{private}'.format(private=private, **pub) + for pub in public + ] + + return ', '.join( + ','.join(format_port(*item)) + for item in sorted(self.ports.items()) + ) + + @property + def labels(self): + return self.get('Config.Labels') or {} + + @property + def stop_signal(self): + return self.get('Config.StopSignal') + + @property + def log_config(self): + return self.get('HostConfig.LogConfig') or None + + @property + def human_readable_state(self): + if self.is_paused: + return 'Paused' + if self.is_restarting: + return 'Restarting' + if self.is_running: + return 'Ghost' if self.get('State.Ghost') else self.human_readable_health_status + else: + return 'Exit %s' % self.get('State.ExitCode') + + @property + def human_readable_command(self): + entrypoint = self.get('Config.Entrypoint') or [] + cmd = self.get('Config.Cmd') or [] + return ' '.join(entrypoint + cmd) + + @property + def environment(self): + def parse_env(var): + if '=' in var: + return var.split("=", 1) + return var, None + return dict(parse_env(var) for var in self.get('Config.Env') or []) + + @property + def exit_code(self): + return self.get('State.ExitCode') + + @property + def is_running(self): + return self.get('State.Running') + + @property + def is_restarting(self): + return self.get('State.Restarting') + + @property + def is_paused(self): + return self.get('State.Paused') + + @property + def log_driver(self): + return self.get('HostConfig.LogConfig.Type') + + @property + def has_api_logs(self): + log_type = self.log_driver + return not log_type or log_type in ('json-file', 'journald', 'local') + + @property + def human_readable_health_status(self): + """ Generate UP status string with up time and health + """ + status_string = 'Up' + container_status = self.get('State.Health.Status') + if container_status == 'starting': + status_string += ' (health: starting)' + elif container_status is not None: + status_string += ' (%s)' % container_status + return status_string + + def attach_log_stream(self): + """A log stream can only be attached if the container uses a + json-file, journald or local log driver. + """ + if self.has_api_logs: + self.log_stream = self.attach(stdout=True, stderr=True, stream=True) + + def get(self, key): + """Return a value from the container or None if the value is not set. + + :param key: a string using dotted notation for nested dictionary + lookups + """ + self.inspect_if_not_inspected() + + def get_value(dictionary, key): + return (dictionary or {}).get(key) + + return reduce(get_value, key.split('.'), self.dictionary) + + def get_local_port(self, port, protocol='tcp'): + port = self.ports.get("{}/{}".format(port, protocol)) + return "{HostIp}:{HostPort}".format(**port[0]) if port else None + + def get_mount(self, mount_dest): + for mount in self.get('Mounts'): + if mount['Destination'] == mount_dest: + return mount + return None + + def start(self, **options): + return self.client.start(self.id, **options) + + def stop(self, **options): + return self.client.stop(self.id, **options) + + def pause(self, **options): + return self.client.pause(self.id, **options) + + def unpause(self, **options): + return self.client.unpause(self.id, **options) + + def kill(self, **options): + return self.client.kill(self.id, **options) + + def restart(self, **options): + return self.client.restart(self.id, **options) + + def remove(self, **options): + return self.client.remove_container(self.id, **options) + + def create_exec(self, command, **options): + return self.client.exec_create(self.id, command, **options) + + def start_exec(self, exec_id, **options): + return self.client.exec_start(exec_id, **options) + + def rename_to_tmp_name(self): + """Rename the container to a hopefully unique temporary container name + by prepending the short id. + """ + if not self.name.startswith(self.short_id): + self.client.rename( + self.id, '{}_{}'.format(self.short_id, self.name) + ) + + def inspect_if_not_inspected(self): + if not self.has_been_inspected: + self.inspect() + + def wait(self): + return self.client.wait(self.id).get('StatusCode', 127) + + def logs(self, *args, **kwargs): + return self.client.logs(self.id, *args, **kwargs) + + def inspect(self): + self.dictionary = self.client.inspect_container(self.id) + self.has_been_inspected = True + return self.dictionary + + def image_exists(self): + try: + self.client.inspect_image(self.image) + except ImageNotFound: + return False + + return True + + def reset_image(self, img_id): + """ If this container's image has been removed, temporarily replace the old image ID + with `img_id`. + """ + if not self.image_exists(): + self.dictionary['Image'] = img_id + + def attach(self, *args, **kwargs): + return self.client.attach(self.id, *args, **kwargs) + + def has_legacy_proj_name(self, project_name): + return ( + ComposeVersion(self.labels.get(LABEL_VERSION)) < ComposeVersion('1.21.0') and + self.project != project_name + ) + + def __repr__(self): + return '<Container: {} ({})>'.format(self.name, self.id[:6]) + + def __eq__(self, other): + if type(self) != type(other): + return False + return self.id == other.id + + def __hash__(self): + return self.id.__hash__() + + +def get_container_name(container): + if not container.get('Name') and not container.get('Names'): + return None + # inspect + if 'Name' in container: + return container['Name'] + # ps + shortest_name = min(container['Names'], key=lambda n: len(n.split('/'))) + return shortest_name.split('/')[-1] diff --git a/compose/errors.py b/compose/errors.py new file mode 100644 index 00000000000..d4fead251ad --- /dev/null +++ b/compose/errors.py @@ -0,0 +1,29 @@ +class OperationFailedError(Exception): + def __init__(self, reason): + self.msg = reason + + +class StreamParseError(RuntimeError): + def __init__(self, reason): + self.msg = reason + + +class HealthCheckException(Exception): + def __init__(self, reason): + self.msg = reason + + +class HealthCheckFailed(HealthCheckException): + def __init__(self, container_id): + super().__init__( + 'Container "{}" is unhealthy.'.format(container_id) + ) + + +class NoHealthCheckConfigured(HealthCheckException): + def __init__(self, service_name): + super().__init__( + 'Service "{}" is missing a healthcheck configuration'.format( + service_name + ) + ) diff --git a/compose/metrics/__init__.py b/compose/metrics/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/compose/metrics/client.py b/compose/metrics/client.py new file mode 100644 index 00000000000..513e22ce61c --- /dev/null +++ b/compose/metrics/client.py @@ -0,0 +1,64 @@ +import os +from enum import Enum + +import requests +from docker import ContextAPI +from docker.transport import UnixHTTPAdapter + +from compose.const import IS_WINDOWS_PLATFORM + +if IS_WINDOWS_PLATFORM: + from docker.transport import NpipeHTTPAdapter + + +class Status(Enum): + SUCCESS = "success" + FAILURE = "failure" + CANCELED = "canceled" + + +class MetricsSource: + CLI = "docker-compose" + + +if IS_WINDOWS_PLATFORM: + METRICS_SOCKET_FILE = 'npipe://\\\\.\\pipe\\docker_cli' +else: + METRICS_SOCKET_FILE = 'http+unix:///var/run/docker-cli.sock' + + +class MetricsCommand(requests.Session): + """ + Representation of a command in the metrics. + """ + + def __init__(self, command, + context_type=None, status=Status.SUCCESS, + source=MetricsSource.CLI, uri=None): + super().__init__() + self.command = "compose " + command if command else "compose --help" + self.context = context_type or ContextAPI.get_current_context().context_type or 'moby' + self.source = source + self.status = status.value + self.uri = uri or os.environ.get("METRICS_SOCKET_FILE", METRICS_SOCKET_FILE) + if IS_WINDOWS_PLATFORM: + self.mount("http+unix://", NpipeHTTPAdapter(self.uri)) + else: + self.mount("http+unix://", UnixHTTPAdapter(self.uri)) + + def send_metrics(self): + try: + return self.post("http+unix://localhost/usage", + json=self.to_map(), + timeout=.05, + headers={'Content-Type': 'application/json'}) + except Exception as e: + return e + + def to_map(self): + return { + 'command': self.command, + 'context': self.context, + 'source': self.source, + 'status': self.status, + } diff --git a/compose/metrics/decorator.py b/compose/metrics/decorator.py new file mode 100644 index 00000000000..3126e6941fa --- /dev/null +++ b/compose/metrics/decorator.py @@ -0,0 +1,21 @@ +import functools + +from compose.metrics.client import MetricsCommand +from compose.metrics.client import Status + + +class metrics: + def __init__(self, command_name=None): + self.command_name = command_name + + def __call__(self, fn): + @functools.wraps(fn, + assigned=functools.WRAPPER_ASSIGNMENTS, + updated=functools.WRAPPER_UPDATES) + def wrapper(*args, **kwargs): + if not self.command_name: + self.command_name = fn.__name__ + result = fn(*args, **kwargs) + MetricsCommand(self.command_name, status=Status.SUCCESS).send_metrics() + return result + return wrapper diff --git a/compose/network.py b/compose/network.py new file mode 100644 index 00000000000..a67c703c01f --- /dev/null +++ b/compose/network.py @@ -0,0 +1,332 @@ +import logging +import re +from collections import OrderedDict +from operator import itemgetter + +from docker.errors import NotFound +from docker.types import IPAMConfig +from docker.types import IPAMPool +from docker.utils import version_gte +from docker.utils import version_lt + +from . import __version__ +from .config import ConfigurationError +from .const import LABEL_NETWORK +from .const import LABEL_PROJECT +from .const import LABEL_VERSION + + +log = logging.getLogger(__name__) + +OPTS_EXCEPTIONS = [ + 'com.docker.network.driver.overlay.vxlanid_list', + 'com.docker.network.windowsshim.hnsid', + 'com.docker.network.windowsshim.networkname' +] + + +class Network: + def __init__(self, client, project, name, driver=None, driver_opts=None, + ipam=None, external=False, internal=False, enable_ipv6=False, + labels=None, custom_name=False): + self.client = client + self.project = project + self.name = name + self.driver = driver + self.driver_opts = driver_opts + self.ipam = create_ipam_config_from_dict(ipam) + self.external = external + self.internal = internal + self.enable_ipv6 = enable_ipv6 + self.labels = labels + self.custom_name = custom_name + self.legacy = None + + def ensure(self): + if self.external: + if self.driver == 'overlay': + # Swarm nodes do not register overlay networks that were + # created on a different node unless they're in use. + # See docker/compose#4399 + return + try: + self.inspect() + log.debug( + 'Network {} declared as external. No new ' + 'network will be created.'.format(self.name) + ) + except NotFound: + raise ConfigurationError( + 'Network {name} declared as external, but could' + ' not be found. Please create the network manually' + ' using `{command} {name}` and try again.'.format( + name=self.full_name, + command='docker network create' + ) + ) + return + + self._set_legacy_flag() + try: + data = self.inspect(legacy=self.legacy) + check_remote_network_config(data, self) + except NotFound: + driver_name = 'the default driver' + if self.driver: + driver_name = 'driver "{}"'.format(self.driver) + + log.info( + 'Creating network "{}" with {}'.format(self.full_name, driver_name) + ) + + self.client.create_network( + name=self.full_name, + driver=self.driver, + options=self.driver_opts, + ipam=self.ipam, + internal=self.internal, + enable_ipv6=self.enable_ipv6, + labels=self._labels, + attachable=version_gte(self.client._version, '1.24') or None, + check_duplicate=True, + ) + + def remove(self): + if self.external: + log.info("Network %s is external, skipping", self.true_name) + return + + log.info("Removing network {}".format(self.true_name)) + self.client.remove_network(self.true_name) + + def inspect(self, legacy=False): + if legacy: + return self.client.inspect_network(self.legacy_full_name) + return self.client.inspect_network(self.full_name) + + @property + def legacy_full_name(self): + if self.custom_name: + return self.name + return '{}_{}'.format( + re.sub(r'[_-]', '', self.project), self.name + ) + + @property + def full_name(self): + if self.custom_name: + return self.name + return '{}_{}'.format(self.project, self.name) + + @property + def true_name(self): + self._set_legacy_flag() + if self.legacy: + return self.legacy_full_name + return self.full_name + + @property + def _labels(self): + if version_lt(self.client._version, '1.23'): + return None + labels = self.labels.copy() if self.labels else {} + labels.update({ + LABEL_PROJECT: self.project, + LABEL_NETWORK: self.name, + LABEL_VERSION: __version__, + }) + return labels + + def _set_legacy_flag(self): + if self.legacy is not None: + return + try: + data = self.inspect(legacy=True) + self.legacy = data is not None + except NotFound: + self.legacy = False + + +def create_ipam_config_from_dict(ipam_dict): + if not ipam_dict: + return None + + return IPAMConfig( + driver=ipam_dict.get('driver') or 'default', + pool_configs=[ + IPAMPool( + subnet=config.get('subnet'), + iprange=config.get('ip_range'), + gateway=config.get('gateway'), + aux_addresses=config.get('aux_addresses'), + ) + for config in ipam_dict.get('config', []) + ], + options=ipam_dict.get('options') + ) + + +class NetworkConfigChangedError(ConfigurationError): + def __init__(self, net_name, property_name): + super().__init__( + 'Network "{}" needs to be recreated - {} has changed'.format( + net_name, property_name + ) + ) + + +def check_remote_ipam_config(remote, local): + remote_ipam = remote.get('IPAM') + ipam_dict = create_ipam_config_from_dict(local.ipam) + if local.ipam.get('driver') and local.ipam.get('driver') != remote_ipam.get('Driver'): + raise NetworkConfigChangedError(local.true_name, 'IPAM driver') + if len(ipam_dict['Config']) != 0: + if len(ipam_dict['Config']) != len(remote_ipam['Config']): + raise NetworkConfigChangedError(local.true_name, 'IPAM configs') + remote_configs = sorted(remote_ipam['Config'], key='Subnet') + local_configs = sorted(ipam_dict['Config'], key='Subnet') + while local_configs: + lc = local_configs.pop() + rc = remote_configs.pop() + if lc.get('Subnet') != rc.get('Subnet'): + raise NetworkConfigChangedError(local.true_name, 'IPAM config subnet') + if lc.get('Gateway') is not None and lc.get('Gateway') != rc.get('Gateway'): + raise NetworkConfigChangedError(local.true_name, 'IPAM config gateway') + if lc.get('IPRange') != rc.get('IPRange'): + raise NetworkConfigChangedError(local.true_name, 'IPAM config ip_range') + if sorted(lc.get('AuxiliaryAddresses')) != sorted(rc.get('AuxiliaryAddresses')): + raise NetworkConfigChangedError(local.true_name, 'IPAM config aux_addresses') + + remote_opts = remote_ipam.get('Options') or {} + local_opts = local.ipam.get('Options') or {} + for k in set.union(set(remote_opts.keys()), set(local_opts.keys())): + if remote_opts.get(k) != local_opts.get(k): + raise NetworkConfigChangedError(local.true_name, 'IPAM option "{}"'.format(k)) + + +def check_remote_network_config(remote, local): + if local.driver and remote.get('Driver') != local.driver: + raise NetworkConfigChangedError(local.true_name, 'driver') + local_opts = local.driver_opts or {} + remote_opts = remote.get('Options') or {} + for k in set.union(set(remote_opts.keys()), set(local_opts.keys())): + if k in OPTS_EXCEPTIONS: + continue + if remote_opts.get(k) != local_opts.get(k): + raise NetworkConfigChangedError(local.true_name, 'option "{}"'.format(k)) + + if local.ipam is not None: + check_remote_ipam_config(remote, local) + + if local.internal is not None and local.internal != remote.get('Internal', False): + raise NetworkConfigChangedError(local.true_name, 'internal') + if local.enable_ipv6 is not None and local.enable_ipv6 != remote.get('EnableIPv6', False): + raise NetworkConfigChangedError(local.true_name, 'enable_ipv6') + + local_labels = local.labels or {} + remote_labels = remote.get('Labels') or {} + for k in set.union(set(remote_labels.keys()), set(local_labels.keys())): + if k.startswith('com.docker.'): # We are only interested in user-specified labels + continue + if remote_labels.get(k) != local_labels.get(k): + log.warning( + 'Network {}: label "{}" has changed. It may need to be' + ' recreated.'.format(local.true_name, k) + ) + + +def build_networks(name, config_data, client): + network_config = config_data.networks or {} + networks = { + network_name: Network( + client=client, project=name, + name=data.get('name', network_name), + driver=data.get('driver'), + driver_opts=data.get('driver_opts'), + ipam=data.get('ipam'), + external=bool(data.get('external', False)), + internal=data.get('internal'), + enable_ipv6=data.get('enable_ipv6'), + labels=data.get('labels'), + custom_name=data.get('name') is not None, + ) + for network_name, data in network_config.items() + } + + if 'default' not in networks: + networks['default'] = Network(client, name, 'default') + + return networks + + +class ProjectNetworks: + + def __init__(self, networks, use_networking): + self.networks = networks or {} + self.use_networking = use_networking + + @classmethod + def from_services(cls, services, networks, use_networking): + service_networks = { + network: networks.get(network) + for service in services + for network in get_network_names_for_service(service) + } + unused = set(networks) - set(service_networks) - {'default'} + if unused: + log.warning( + "Some networks were defined but are not used by any service: " + "{}".format(", ".join(unused))) + return cls(service_networks, use_networking) + + def remove(self): + if not self.use_networking: + return + for network in self.networks.values(): + try: + network.remove() + except NotFound: + log.warning("Network %s not found.", network.true_name) + + def initialize(self): + if not self.use_networking: + return + + for network in self.networks.values(): + network.ensure() + + +def get_network_defs_for_service(service_dict): + if 'network_mode' in service_dict: + return {} + networks = service_dict.get('networks', {'default': None}) + return { + net: (config or {}) + for net, config in networks.items() + } + + +def get_network_names_for_service(service_dict): + return get_network_defs_for_service(service_dict).keys() + + +def get_networks(service_dict, network_definitions): + networks = {} + for name, netdef in get_network_defs_for_service(service_dict).items(): + network = network_definitions.get(name) + if network: + networks[network.true_name] = netdef + else: + raise ConfigurationError( + 'Service "{}" uses an undefined network "{}"' + .format(service_dict['name'], name)) + + if any([v.get('priority') for v in networks.values()]): + return OrderedDict(sorted( + networks.items(), + key=lambda t: t[1].get('priority') or 0, reverse=True + )) + else: + # Ensure Compose will pick a consistent primary network if no + # priority is set + return OrderedDict(sorted(networks.items(), key=itemgetter(0))) diff --git a/compose/parallel.py b/compose/parallel.py new file mode 100644 index 00000000000..acf9e4a84cf --- /dev/null +++ b/compose/parallel.py @@ -0,0 +1,349 @@ +import _thread as thread +import logging +import operator +import sys +from queue import Empty +from queue import Queue +from threading import Lock +from threading import Semaphore +from threading import Thread + +from docker.errors import APIError +from docker.errors import ImageNotFound + +from compose.cli.colors import green +from compose.cli.colors import red +from compose.cli.signals import ShutdownException +from compose.const import PARALLEL_LIMIT +from compose.errors import HealthCheckFailed +from compose.errors import NoHealthCheckConfigured +from compose.errors import OperationFailedError + + +log = logging.getLogger(__name__) + +STOP = object() + + +class GlobalLimit: + """Simple class to hold a global semaphore limiter for a project. This class + should be treated as a singleton that is instantiated when the project is. + """ + + global_limiter = Semaphore(PARALLEL_LIMIT) + + @classmethod + def set_global_limit(cls, value): + if value is None: + value = PARALLEL_LIMIT + cls.global_limiter = Semaphore(value) + + +def parallel_execute_watch(events, writer, errors, results, msg, get_name, fail_check): + """ Watch events from a parallel execution, update status and fill errors and results. + Returns exception to re-raise. + """ + error_to_reraise = None + for obj, result, exception in events: + if exception is None: + if fail_check is not None and fail_check(obj): + writer.write(msg, get_name(obj), 'failed', red) + else: + writer.write(msg, get_name(obj), 'done', green) + results.append(result) + elif isinstance(exception, ImageNotFound): + # This is to bubble up ImageNotFound exceptions to the client so we + # can prompt the user if they want to rebuild. + errors[get_name(obj)] = exception.explanation + writer.write(msg, get_name(obj), 'error', red) + error_to_reraise = exception + elif isinstance(exception, APIError): + errors[get_name(obj)] = exception.explanation + writer.write(msg, get_name(obj), 'error', red) + elif isinstance(exception, (OperationFailedError, HealthCheckFailed, NoHealthCheckConfigured)): + errors[get_name(obj)] = exception.msg + writer.write(msg, get_name(obj), 'error', red) + elif isinstance(exception, UpstreamError): + writer.write(msg, get_name(obj), 'error', red) + else: + errors[get_name(obj)] = exception + error_to_reraise = exception + return error_to_reraise + + +def parallel_execute(objects, func, get_name, msg, get_deps=None, limit=None, fail_check=None): + """Runs func on objects in parallel while ensuring that func is + ran on object only after it is ran on all its dependencies. + + get_deps called on object must return a collection with its dependencies. + get_name called on object must return its name. + fail_check is an additional failure check for cases that should display as a failure + in the CLI logs, but don't raise an exception (such as attempting to start 0 containers) + """ + objects = list(objects) + stream = sys.stderr + + if ParallelStreamWriter.instance: + writer = ParallelStreamWriter.instance + else: + writer = ParallelStreamWriter(stream) + + for obj in objects: + writer.add_object(msg, get_name(obj)) + for obj in objects: + writer.write_initial(msg, get_name(obj)) + + events = parallel_execute_iter(objects, func, get_deps, limit) + + errors = {} + results = [] + error_to_reraise = parallel_execute_watch( + events, writer, errors, results, msg, get_name, fail_check + ) + + for obj_name, error in errors.items(): + stream.write("\nERROR: for {} {}\n".format(obj_name, error)) + + if error_to_reraise: + raise error_to_reraise + + return results, errors + + +def _no_deps(x): + return [] + + +class State: + """ + Holds the state of a partially-complete parallel operation. + + state.started: objects being processed + state.finished: objects which have been processed + state.failed: objects which either failed or whose dependencies failed + """ + def __init__(self, objects): + self.objects = objects + + self.started = set() + self.finished = set() + self.failed = set() + + def is_done(self): + return len(self.finished) + len(self.failed) >= len(self.objects) + + def pending(self): + return set(self.objects) - self.started - self.finished - self.failed + + +class NoLimit: + def __enter__(self): + pass + + def __exit__(self, *ex): + pass + + +def parallel_execute_iter(objects, func, get_deps, limit): + """ + Runs func on objects in parallel while ensuring that func is + ran on object only after it is ran on all its dependencies. + + Returns an iterator of tuples which look like: + + # if func returned normally when run on object + (object, result, None) + + # if func raised an exception when run on object + (object, None, exception) + + # if func raised an exception when run on one of object's dependencies + (object, None, UpstreamError()) + """ + if get_deps is None: + get_deps = _no_deps + + if limit is None: + limiter = NoLimit() + else: + limiter = Semaphore(limit) + + results = Queue() + state = State(objects) + + while True: + feed_queue(objects, func, get_deps, results, state, limiter) + + try: + event = results.get(timeout=0.1) + except Empty: + continue + # See https://github.com/docker/compose/issues/189 + except thread.error: + raise ShutdownException() + + if event is STOP: + break + + obj, _, exception = event + if exception is None: + log.debug('Finished processing: {}'.format(obj)) + state.finished.add(obj) + else: + log.debug('Failed: {}'.format(obj)) + state.failed.add(obj) + + yield event + + +def producer(obj, func, results, limiter): + """ + The entry point for a producer thread which runs func on a single object. + Places a tuple on the results queue once func has either returned or raised. + """ + with limiter, GlobalLimit.global_limiter: + try: + result = func(obj) + results.put((obj, result, None)) + except Exception as e: + results.put((obj, None, e)) + + +def feed_queue(objects, func, get_deps, results, state, limiter): + """ + Starts producer threads for any objects which are ready to be processed + (i.e. they have no dependencies which haven't been successfully processed). + + Shortcuts any objects whose dependencies have failed and places an + (object, None, UpstreamError()) tuple on the results queue. + """ + pending = state.pending() + log.debug('Pending: {}'.format(pending)) + + for obj in pending: + deps = get_deps(obj) + try: + if any(dep[0] in state.failed for dep in deps): + log.debug('{} has upstream errors - not processing'.format(obj)) + results.put((obj, None, UpstreamError())) + state.failed.add(obj) + elif all( + dep not in objects or ( + dep in state.finished and (not ready_check or ready_check(dep)) + ) for dep, ready_check in deps + ): + log.debug('Starting producer thread for {}'.format(obj)) + t = Thread(target=producer, args=(obj, func, results, limiter)) + t.daemon = True + t.start() + state.started.add(obj) + except (HealthCheckFailed, NoHealthCheckConfigured) as e: + log.debug( + 'Healthcheck for service(s) upstream of {} failed - ' + 'not processing'.format(obj) + ) + results.put((obj, None, e)) + + if state.is_done(): + results.put(STOP) + + +class UpstreamError(Exception): + pass + + +class ParallelStreamWriter: + """Write out messages for operations happening in parallel. + + Each operation has its own line, and ANSI code characters are used + to jump to the correct line, and write over the line. + """ + + noansi = False + lock = Lock() + instance = None + + @classmethod + def set_noansi(cls, value=True): + cls.noansi = value + + def __init__(self, stream): + self.stream = stream + self.lines = [] + self.width = 0 + ParallelStreamWriter.instance = self + + def add_object(self, msg, obj_index): + if msg is None: + return + self.lines.append(msg + obj_index) + self.width = max(self.width, len(msg + ' ' + obj_index)) + + def write_initial(self, msg, obj_index): + if msg is None: + return + return self._write_noansi(msg, obj_index, '') + + def _write_ansi(self, msg, obj_index, status): + self.lock.acquire() + position = self.lines.index(msg + obj_index) + diff = len(self.lines) - position + # move up + self.stream.write("%c[%dA" % (27, diff)) + # erase + self.stream.write("%c[2K\r" % 27) + self.stream.write("{:<{width}} ... {}\r".format(msg + ' ' + obj_index, + status, width=self.width)) + # move back down + self.stream.write("%c[%dB" % (27, diff)) + self.stream.flush() + self.lock.release() + + def _write_noansi(self, msg, obj_index, status): + self.stream.write( + "{:<{width}} ... {}\r\n".format( + msg + ' ' + obj_index, status, width=self.width + ) + ) + self.stream.flush() + + def write(self, msg, obj_index, status, color_func): + if msg is None: + return + if self.noansi: + self._write_noansi(msg, obj_index, status) + else: + self._write_ansi(msg, obj_index, color_func(status)) + + +def get_stream_writer(): + instance = ParallelStreamWriter.instance + if instance is None: + raise RuntimeError('ParallelStreamWriter has not yet been instantiated') + return instance + + +def parallel_operation(containers, operation, options, message): + parallel_execute( + containers, + operator.methodcaller(operation, **options), + operator.attrgetter('name'), + message, + ) + + +def parallel_remove(containers, options): + stopped_containers = [c for c in containers if not c.is_running] + parallel_operation(stopped_containers, 'remove', options, 'Removing') + + +def parallel_pause(containers, options): + parallel_operation(containers, 'pause', options, 'Pausing') + + +def parallel_unpause(containers, options): + parallel_operation(containers, 'unpause', options, 'Unpausing') + + +def parallel_kill(containers, options): + parallel_operation(containers, 'kill', options, 'Killing') diff --git a/compose/progress_stream.py b/compose/progress_stream.py new file mode 100644 index 00000000000..3c03cc4b5b9 --- /dev/null +++ b/compose/progress_stream.py @@ -0,0 +1,123 @@ +from compose import utils + + +class StreamOutputError(Exception): + pass + + +def write_to_stream(s, stream): + try: + stream.write(s) + except UnicodeEncodeError: + encoding = getattr(stream, 'encoding', 'ascii') + stream.write(s.encode(encoding, errors='replace').decode(encoding)) + + +def stream_output(output, stream): + is_terminal = hasattr(stream, 'isatty') and stream.isatty() + stream = stream + lines = {} + diff = 0 + + for event in utils.json_stream(output): + yield event + is_progress_event = 'progress' in event or 'progressDetail' in event + + if not is_progress_event: + print_output_event(event, stream, is_terminal) + stream.flush() + continue + + if not is_terminal: + continue + + # if it's a progress event and we have a terminal, then display the progress bars + image_id = event.get('id') + if not image_id: + continue + + if image_id not in lines: + lines[image_id] = len(lines) + write_to_stream("\n", stream) + + diff = len(lines) - lines[image_id] + + # move cursor up `diff` rows + write_to_stream("%c[%dA" % (27, diff), stream) + + print_output_event(event, stream, is_terminal) + + if 'id' in event: + # move cursor back down + write_to_stream("%c[%dB" % (27, diff), stream) + + stream.flush() + + +def print_output_event(event, stream, is_terminal): + if 'errorDetail' in event: + raise StreamOutputError(event['errorDetail']['message']) + + terminator = '' + + if is_terminal and 'stream' not in event: + # erase current line + write_to_stream("%c[2K\r" % 27, stream) + terminator = "\r" + elif 'progressDetail' in event: + return + + if 'time' in event: + write_to_stream("[%s] " % event['time'], stream) + + if 'id' in event: + write_to_stream("%s: " % event['id'], stream) + + if 'from' in event: + write_to_stream("(from %s) " % event['from'], stream) + + status = event.get('status', '') + + if 'progress' in event: + write_to_stream("{} {}{}".format(status, event['progress'], terminator), stream) + elif 'progressDetail' in event: + detail = event['progressDetail'] + total = detail.get('total') + if 'current' in detail and total: + percentage = float(detail['current']) / float(total) * 100 + write_to_stream('{} ({:.1f}%){}'.format(status, percentage, terminator), stream) + else: + write_to_stream('{}{}'.format(status, terminator), stream) + elif 'stream' in event: + write_to_stream("{}{}".format(event['stream'], terminator), stream) + else: + write_to_stream("{}{}\n".format(status, terminator), stream) + + +def get_digest_from_pull(events): + digest = None + for event in events: + status = event.get('status') + if not status or 'Digest' not in status: + continue + else: + digest = status.split(':', 1)[1].strip() + return digest + + +def get_digest_from_push(events): + for event in events: + digest = event.get('aux', {}).get('Digest') + if digest: + return digest + return None + + +def read_status(event): + status = event['status'].lower() + if 'progressDetail' in event: + detail = event['progressDetail'] + if 'current' in detail and 'total' in detail: + percentage = float(detail['current']) / float(detail['total']) + status = '{} ({:.1%})'.format(status, percentage) + return status diff --git a/compose/project.py b/compose/project.py new file mode 100644 index 00000000000..336aaa3fcac --- /dev/null +++ b/compose/project.py @@ -0,0 +1,1168 @@ +import datetime +import enum +import logging +import operator +import re +from functools import reduce +from os import path + +from docker.errors import APIError +from docker.errors import ImageNotFound +from docker.errors import NotFound +from docker.utils import version_lt + +from . import parallel +from .cli.errors import UserError +from .config import ConfigurationError +from .config.config import V1 +from .config.sort_services import get_container_name_from_network_mode +from .config.sort_services import get_service_name_from_network_mode +from .const import LABEL_ONE_OFF +from .const import LABEL_PROJECT +from .const import LABEL_SERVICE +from .container import Container +from .network import build_networks +from .network import get_networks +from .network import ProjectNetworks +from .progress_stream import read_status +from .service import BuildAction +from .service import ContainerIpcMode +from .service import ContainerNetworkMode +from .service import ContainerPidMode +from .service import ConvergenceStrategy +from .service import IpcMode +from .service import NetworkMode +from .service import NoSuchImageError +from .service import parse_repository_tag +from .service import PidMode +from .service import Service +from .service import ServiceIpcMode +from .service import ServiceNetworkMode +from .service import ServicePidMode +from .utils import filter_attached_for_up +from .utils import microseconds_from_time_nano +from .utils import truncate_string +from .volume import ProjectVolumes + +log = logging.getLogger(__name__) + + +@enum.unique +class OneOffFilter(enum.Enum): + include = 0 + exclude = 1 + only = 2 + + @classmethod + def update_labels(cls, value, labels): + if value == cls.only: + labels.append('{}={}'.format(LABEL_ONE_OFF, "True")) + elif value == cls.exclude: + labels.append('{}={}'.format(LABEL_ONE_OFF, "False")) + elif value == cls.include: + pass + else: + raise ValueError("Invalid value for one_off: {}".format(repr(value))) + + +class Project: + """ + A collection of services. + """ + def __init__(self, name, services, client, networks=None, volumes=None, config_version=None, + enabled_profiles=None): + self.name = name + self.services = services + self.client = client + self.volumes = volumes or ProjectVolumes({}) + self.networks = networks or ProjectNetworks({}, False) + self.config_version = config_version + self.enabled_profiles = enabled_profiles or [] + + def labels(self, one_off=OneOffFilter.exclude, legacy=False): + name = self.name + if legacy: + name = re.sub(r'[_-]', '', name) + labels = ['{}={}'.format(LABEL_PROJECT, name)] + + OneOffFilter.update_labels(one_off, labels) + return labels + + @classmethod + def from_config(cls, name, config_data, client, default_platform=None, extra_labels=None, + enabled_profiles=None): + """ + Construct a Project from a config.Config object. + """ + extra_labels = extra_labels or [] + use_networking = (config_data.version and config_data.version != V1) + networks = build_networks(name, config_data, client) + project_networks = ProjectNetworks.from_services( + config_data.services, + networks, + use_networking) + volumes = ProjectVolumes.from_config(name, config_data, client) + project = cls(name, [], client, project_networks, volumes, config_data.version, enabled_profiles) + + for service_dict in config_data.services: + service_dict = dict(service_dict) + if use_networking: + service_networks = get_networks(service_dict, networks) + else: + service_networks = {} + + service_dict.pop('networks', None) + links = project.get_links(service_dict) + ipc_mode = project.get_ipc_mode(service_dict) + network_mode = project.get_network_mode( + service_dict, list(service_networks.keys()) + ) + pid_mode = project.get_pid_mode(service_dict) + volumes_from = get_volumes_from(project, service_dict) + + if config_data.version != V1: + service_dict['volumes'] = [ + volumes.namespace_spec(volume_spec) + for volume_spec in service_dict.get('volumes', []) + ] + + secrets = get_secrets( + service_dict['name'], + service_dict.pop('secrets', None) or [], + config_data.secrets) + + service_dict['scale'] = project.get_service_scale(service_dict) + service_dict['device_requests'] = project.get_device_requests(service_dict) + service_dict = translate_credential_spec_to_security_opt(service_dict) + service_dict, ignored_keys = translate_deploy_keys_to_container_config( + service_dict + ) + if ignored_keys: + log.warning( + 'The following deploy sub-keys are not supported and have' + ' been ignored: {}'.format(', '.join(ignored_keys)) + ) + + project.services.append( + Service( + service_dict.pop('name'), + client=client, + project=name, + use_networking=use_networking, + networks=service_networks, + links=links, + network_mode=network_mode, + volumes_from=volumes_from, + secrets=secrets, + pid_mode=pid_mode, + ipc_mode=ipc_mode, + platform=service_dict.pop('platform', None), + default_platform=default_platform, + extra_labels=extra_labels, + **service_dict) + ) + + return project + + @property + def service_names(self): + return [service.name for service in self.services] + + def get_service(self, name): + """ + Retrieve a service by name. Raises NoSuchService + if the named service does not exist. + """ + for service in self.services: + if service.name == name: + return service + + raise NoSuchService(name) + + def validate_service_names(self, service_names): + """ + Validate that the given list of service names only contains valid + services. Raises NoSuchService if one of the names is invalid. + """ + valid_names = self.service_names + for name in service_names: + if name not in valid_names: + raise NoSuchService(name) + + def get_services(self, service_names=None, include_deps=False, auto_enable_profiles=True): + """ + Returns a list of this project's services filtered + by the provided list of names, or all services if service_names is None + or []. + + If include_deps is specified, returns a list including the dependencies for + service_names, in order of dependency. + + Preserves the original order of self.services where possible, + reordering as needed to resolve dependencies. + + Raises NoSuchService if any of the named services do not exist. + + Raises ConfigurationError if any service depended on is not enabled by active profiles + """ + # create a copy so we can *locally* add auto-enabled profiles later + enabled_profiles = self.enabled_profiles.copy() + + if service_names is None or len(service_names) == 0: + auto_enable_profiles = False + service_names = [ + service.name + for service in self.services + if service.enabled_for_profiles(enabled_profiles) + ] + + unsorted = [self.get_service(name) for name in service_names] + services = [s for s in self.services if s in unsorted] + + if auto_enable_profiles: + # enable profiles of explicitly targeted services + for service in services: + for profile in service.get_profiles(): + if profile not in enabled_profiles: + enabled_profiles.append(profile) + + if include_deps: + services = reduce( + lambda acc, s: self._inject_deps(acc, s, enabled_profiles), + services, + [] + ) + + uniques = [] + [uniques.append(s) for s in services if s not in uniques] + + return uniques + + def get_services_without_duplicate(self, service_names=None, include_deps=False): + services = self.get_services(service_names, include_deps) + for service in services: + service.remove_duplicate_containers() + return services + + def get_links(self, service_dict): + links = [] + if 'links' in service_dict: + for link in service_dict.get('links', []): + if ':' in link: + service_name, link_name = link.split(':', 1) + else: + service_name, link_name = link, None + try: + links.append((self.get_service(service_name), link_name)) + except NoSuchService: + raise ConfigurationError( + 'Service "%s" has a link to service "%s" which does not ' + 'exist.' % (service_dict['name'], service_name)) + del service_dict['links'] + return links + + def get_network_mode(self, service_dict, networks): + network_mode = service_dict.pop('network_mode', None) + if not network_mode: + if self.networks.use_networking: + return NetworkMode(networks[0]) if networks else NetworkMode('none') + return NetworkMode(None) + + service_name = get_service_name_from_network_mode(network_mode) + if service_name: + return ServiceNetworkMode(self.get_service(service_name)) + + container_name = get_container_name_from_network_mode(network_mode) + if container_name: + try: + return ContainerNetworkMode(Container.from_id(self.client, container_name)) + except APIError: + raise ConfigurationError( + "Service '{name}' uses the network stack of container '{dep}' which " + "does not exist.".format(name=service_dict['name'], dep=container_name)) + + return NetworkMode(network_mode) + + def get_pid_mode(self, service_dict): + pid_mode = service_dict.pop('pid', None) + if not pid_mode: + return PidMode(None) + + service_name = get_service_name_from_network_mode(pid_mode) + if service_name: + return ServicePidMode(self.get_service(service_name)) + + container_name = get_container_name_from_network_mode(pid_mode) + if container_name: + try: + return ContainerPidMode(Container.from_id(self.client, container_name)) + except APIError: + raise ConfigurationError( + "Service '{name}' uses the PID namespace of container '{dep}' which " + "does not exist.".format(name=service_dict['name'], dep=container_name) + ) + + return PidMode(pid_mode) + + def get_ipc_mode(self, service_dict): + ipc_mode = service_dict.pop('ipc', None) + if not ipc_mode: + return IpcMode(None) + + service_name = get_service_name_from_network_mode(ipc_mode) + if service_name: + return ServiceIpcMode(self.get_service(service_name)) + + container_name = get_container_name_from_network_mode(ipc_mode) + if container_name: + try: + return ContainerIpcMode(Container.from_id(self.client, container_name)) + except APIError: + raise ConfigurationError( + "Service '{name}' uses the IPC namespace of container '{dep}' which " + "does not exist.".format(name=service_dict['name'], dep=container_name) + ) + + return IpcMode(ipc_mode) + + def get_service_scale(self, service_dict): + # service.scale for v2 and deploy.replicas for v3 + scale = service_dict.get('scale', None) + deploy_dict = service_dict.get('deploy', None) + if not deploy_dict: + return 1 if scale is None else scale + + if deploy_dict.get('mode', 'replicated') != 'replicated': + return 1 if scale is None else scale + + replicas = deploy_dict.get('replicas', None) + if scale is not None and replicas is not None: + raise ConfigurationError( + "Both service.scale and service.deploy.replicas are set." + " Only one of them must be set." + ) + if replicas is not None: + scale = replicas + if scale is None: + return 1 + # deploy may contain placement constraints introduced in v3.8 + max_replicas = deploy_dict.get('placement', {}).get( + 'max_replicas_per_node', + scale) + + scale = min(scale, max_replicas) + if max_replicas < scale: + log.warning("Scale is limited to {} ('max_replicas_per_node' field).".format( + max_replicas)) + return scale + + def get_device_requests(self, service_dict): + deploy_dict = service_dict.get('deploy', None) + if not deploy_dict: + return + + resources = deploy_dict.get('resources', None) + if not resources or not resources.get('reservations', None): + return + devices = resources['reservations'].get('devices') + if not devices: + return + + for dev in devices: + count = dev.get("count", -1) + if not isinstance(count, int): + if count != "all": + raise ConfigurationError( + 'Invalid value "{}" for devices count'.format(dev["count"]), + '(expected integer or "all")') + dev["count"] = -1 + + if 'capabilities' in dev: + dev['capabilities'] = [dev['capabilities']] + return devices + + def start(self, service_names=None, **options): + containers = [] + + def start_service(service): + service_containers = service.start(quiet=True, **options) + containers.extend(service_containers) + + services = self.get_services(service_names) + + def get_deps(service): + return { + (self.get_service(dep), config) + for dep, config in service.get_dependency_configs().items() + } + + parallel.parallel_execute( + services, + start_service, + operator.attrgetter('name'), + 'Starting', + get_deps, + fail_check=lambda obj: not obj.containers(), + ) + + return containers + + def stop(self, service_names=None, one_off=OneOffFilter.exclude, **options): + containers = self.containers(service_names, one_off=one_off) + + def get_deps(container): + # actually returning inversed dependencies + return {(other, None) for other in containers + if container.service in + self.get_service(other.service).get_dependency_names()} + + parallel.parallel_execute( + containers, + self.build_container_operation_with_timeout_func('stop', options), + operator.attrgetter('name'), + 'Stopping', + get_deps, + ) + + def pause(self, service_names=None, **options): + containers = self.containers(service_names) + parallel.parallel_pause(reversed(containers), options) + return containers + + def unpause(self, service_names=None, **options): + containers = self.containers(service_names) + parallel.parallel_unpause(containers, options) + return containers + + def kill(self, service_names=None, **options): + parallel.parallel_kill(self.containers(service_names), options) + + def remove_stopped(self, service_names=None, one_off=OneOffFilter.exclude, **options): + parallel.parallel_remove(self.containers( + service_names, stopped=True, one_off=one_off + ), options) + + def down( + self, + remove_image_type, + include_volumes, + remove_orphans=False, + timeout=None, + ignore_orphans=False): + self.stop(one_off=OneOffFilter.include, timeout=timeout) + if not ignore_orphans: + self.find_orphan_containers(remove_orphans) + self.remove_stopped(v=include_volumes, one_off=OneOffFilter.include) + + self.networks.remove() + + if include_volumes: + self.volumes.remove() + + self.remove_images(remove_image_type) + + def remove_images(self, remove_image_type): + for service in self.services: + service.remove_image(remove_image_type) + + def restart(self, service_names=None, **options): + # filter service_names by enabled profiles + service_names = [s.name for s in self.get_services(service_names)] + containers = self.containers(service_names, stopped=True) + + parallel.parallel_execute( + containers, + self.build_container_operation_with_timeout_func('restart', options), + operator.attrgetter('name'), + 'Restarting', + ) + return containers + + def build(self, service_names=None, no_cache=False, pull=False, force_rm=False, memory=None, + build_args=None, gzip=False, parallel_build=False, rm=True, silent=False, cli=False, + progress=None): + + services = [] + for service in self.get_services(service_names): + if service.can_be_built(): + services.append(service) + elif not silent: + log.info('%s uses an image, skipping' % service.name) + + if cli: + log.info("Building with native build. Learn about native build in Compose here: " + "https://docs.docker.com/go/compose-native-build/") + if parallel_build: + log.warning("Flag '--parallel' is ignored when building with " + "COMPOSE_DOCKER_CLI_BUILD=1") + if gzip: + log.warning("Flag '--compress' is ignored when building with " + "COMPOSE_DOCKER_CLI_BUILD=1") + + def build_service(service): + service.build(no_cache, pull, force_rm, memory, build_args, gzip, rm, silent, cli, progress) + + if parallel_build: + _, errors = parallel.parallel_execute( + services, + build_service, + operator.attrgetter('name'), + 'Building', + limit=5, + ) + if len(errors): + combined_errors = '\n'.join([ + e.decode('utf-8') if isinstance(e, bytes) else e for e in errors.values() + ]) + raise ProjectError(combined_errors) + + else: + for service in services: + build_service(service) + + def create( + self, + service_names=None, + strategy=ConvergenceStrategy.changed, + do_build=BuildAction.none, + ): + services = self.get_services_without_duplicate(service_names, include_deps=True) + + for svc in services: + svc.ensure_image_exists(do_build=do_build) + plans = self._get_convergence_plans(services, strategy) + + for service in services: + service.execute_convergence_plan( + plans[service.name], + detached=True, + start=False) + + def _legacy_event_processor(self, service_names): + # Only for v1 files or when Compose is forced to use an older API version + def build_container_event(event, container): + time = datetime.datetime.fromtimestamp(event['time']) + time = time.replace( + microsecond=microseconds_from_time_nano(event['timeNano']) + ) + return { + 'time': time, + 'type': 'container', + 'action': event['status'], + 'id': container.id, + 'service': container.service, + 'attributes': { + 'name': container.name, + 'image': event['from'], + }, + 'container': container, + } + + service_names = set(service_names or self.service_names) + for event in self.client.events( + filters={'label': self.labels()}, + decode=True + ): + # This is a guard against some events broadcasted by swarm that + # don't have a status field. + # See https://github.com/docker/compose/issues/3316 + if 'status' not in event: + continue + + try: + # this can fail if the container has been removed or if the event + # refers to an image + container = Container.from_id(self.client, event['id']) + except APIError: + continue + if container.service not in service_names: + continue + yield build_container_event(event, container) + + def events(self, service_names=None): + if version_lt(self.client.api_version, '1.22'): + # New, better event API was introduced in 1.22. + return self._legacy_event_processor(service_names) + + def build_container_event(event): + container_attrs = event['Actor']['Attributes'] + time = datetime.datetime.fromtimestamp(event['time']) + time = time.replace( + microsecond=microseconds_from_time_nano(event['timeNano']) + ) + + container = None + try: + container = Container.from_id(self.client, event['id']) + except APIError: + # Container may have been removed (e.g. if this is a destroy event) + pass + + return { + 'time': time, + 'type': 'container', + 'action': event['status'], + 'id': event['Actor']['ID'], + 'service': container_attrs.get(LABEL_SERVICE), + 'attributes': { + k: v for k, v in container_attrs.items() + if not k.startswith('com.docker.compose.') + }, + 'container': container, + } + + def yield_loop(service_names): + for event in self.client.events( + filters={'label': self.labels()}, + decode=True + ): + # TODO: support other event types + if event.get('Type') != 'container': + continue + + try: + if event['Actor']['Attributes'][LABEL_SERVICE] not in service_names: + continue + except KeyError: + continue + yield build_container_event(event) + + return yield_loop(set(service_names) if service_names else self.service_names) + + def up(self, + service_names=None, + start_deps=True, + strategy=ConvergenceStrategy.changed, + do_build=BuildAction.none, + timeout=None, + detached=False, + remove_orphans=False, + ignore_orphans=False, + scale_override=None, + rescale=True, + start=True, + always_recreate_deps=False, + reset_container_image=False, + renew_anonymous_volumes=False, + silent=False, + cli=False, + one_off=False, + attach_dependencies=False, + override_options=None, + ): + + if cli: + log.info("Building with native build. Learn about native build in Compose here: " + "https://docs.docker.com/go/compose-native-build/") + + self.initialize() + if not ignore_orphans: + self.find_orphan_containers(remove_orphans) + + if scale_override is None: + scale_override = {} + + services = self.get_services_without_duplicate( + service_names, + include_deps=start_deps) + + for svc in services: + svc.ensure_image_exists(do_build=do_build, silent=silent, cli=cli) + plans = self._get_convergence_plans( + services, + strategy, + always_recreate_deps=always_recreate_deps, + one_off=service_names if one_off else [], + ) + + services_to_attach = filter_attached_for_up( + services, + service_names, + attach_dependencies, + lambda service: service.name) + + def do(service): + return service.execute_convergence_plan( + plans[service.name], + timeout=timeout, + detached=detached or (service not in services_to_attach), + scale_override=scale_override.get(service.name), + rescale=rescale, + start=start, + reset_container_image=reset_container_image, + renew_anonymous_volumes=renew_anonymous_volumes, + override_options=override_options, + ) + + def get_deps(service): + return { + (self.get_service(dep), config) + for dep, config in service.get_dependency_configs().items() + } + + results, errors = parallel.parallel_execute( + services, + do, + operator.attrgetter('name'), + None, + get_deps, + ) + if errors: + raise ProjectError( + 'Encountered errors while bringing up the project.' + ) + + return [ + container + for svc_containers in results + if svc_containers is not None + for container in svc_containers + ] + + def initialize(self): + self.networks.initialize() + self.volumes.initialize() + + def _get_convergence_plans(self, services, strategy, always_recreate_deps=False, one_off=None): + plans = {} + + for service in services: + updated_dependencies = [ + name + for name in service.get_dependency_names() + if name in plans and + plans[name].action in ('recreate', 'create') + ] + is_one_off = one_off and service.name in one_off + + if updated_dependencies and strategy.allows_recreate: + log.debug('%s has upstream changes (%s)', + service.name, + ", ".join(updated_dependencies)) + containers_stopped = any( + service.containers(stopped=True, filters={'status': ['created', 'exited']})) + service_has_links = any(service.get_link_names()) + container_has_links = any(c.get('HostConfig.Links') for c in service.containers()) + should_recreate_for_links = service_has_links ^ container_has_links + if always_recreate_deps or containers_stopped or should_recreate_for_links: + plan = service.convergence_plan(ConvergenceStrategy.always, is_one_off) + else: + plan = service.convergence_plan(strategy, is_one_off) + else: + plan = service.convergence_plan(strategy, is_one_off) + + plans[service.name] = plan + + return plans + + def pull(self, service_names=None, ignore_pull_failures=False, parallel_pull=True, silent=False, + include_deps=False): + services = self.get_services(service_names, include_deps) + + if parallel_pull: + self.parallel_pull(services, silent=silent) + + else: + must_build = [] + for service in services: + try: + service.pull(ignore_pull_failures, silent=silent) + except (ImageNotFound, NotFound): + if service.can_be_built(): + must_build.append(service.name) + else: + raise + + if len(must_build): + log.warning('Some service image(s) must be built from source by running:\n' + ' docker-compose build {}' + .format(' '.join(must_build))) + + def parallel_pull(self, services, ignore_pull_failures=False, silent=False): + msg = 'Pulling' if not silent else None + must_build = [] + + def pull_service(service): + strm = service.pull(ignore_pull_failures, True, stream=True) + + if strm is None: # Attempting to pull service with no `image` key is a no-op + return + + try: + writer = parallel.get_stream_writer() + for event in strm: + if 'status' not in event: + continue + status = read_status(event) + writer.write( + msg, service.name, truncate_string(status), lambda s: s + ) + except (ImageNotFound, NotFound): + if service.can_be_built(): + must_build.append(service.name) + else: + raise + + _, errors = parallel.parallel_execute( + services, + pull_service, + operator.attrgetter('name'), + msg, + limit=5, + ) + + if len(must_build): + log.warning('Some service image(s) must be built from source by running:\n' + ' docker-compose build {}' + .format(' '.join(must_build))) + if len(errors): + combined_errors = '\n'.join([ + e.decode('utf-8') if isinstance(e, bytes) else e for e in errors.values() + ]) + raise ProjectError(combined_errors) + + def push(self, service_names=None, ignore_push_failures=False): + unique_images = set() + for service in self.get_services(service_names, include_deps=False): + # Considering <image> and <image:latest> as the same + repo, tag, sep = parse_repository_tag(service.image_name) + service_image_name = sep.join((repo, tag)) if tag else sep.join((repo, 'latest')) + + if service_image_name not in unique_images: + service.push(ignore_push_failures) + unique_images.add(service_image_name) + + def _labeled_containers(self, stopped=False, one_off=OneOffFilter.exclude): + ctnrs = list(filter(None, [ + Container.from_ps(self.client, container) + for container in self.client.containers( + all=stopped, + filters={'label': self.labels(one_off=one_off)})]) + ) + if ctnrs: + return ctnrs + + return list(filter(lambda c: c.has_legacy_proj_name(self.name), filter(None, [ + Container.from_ps(self.client, container) + for container in self.client.containers( + all=stopped, + filters={'label': self.labels(one_off=one_off, legacy=True)})]) + )) + + def containers(self, service_names=None, stopped=False, one_off=OneOffFilter.exclude): + if service_names: + self.validate_service_names(service_names) + else: + service_names = self.service_names + + containers = self._labeled_containers(stopped, one_off) + + def matches_service_names(container): + return container.labels.get(LABEL_SERVICE) in service_names + + return [c for c in containers if matches_service_names(c)] + + def find_orphan_containers(self, remove_orphans): + def _find(): + containers = set(self._labeled_containers() + self._labeled_containers(stopped=True)) + for ctnr in containers: + service_name = ctnr.labels.get(LABEL_SERVICE) + if service_name not in self.service_names: + yield ctnr + orphans = list(_find()) + if not orphans: + return + if remove_orphans: + for ctnr in orphans: + log.info('Removing orphan container "{}"'.format(ctnr.name)) + try: + ctnr.kill() + except APIError: + pass + ctnr.remove(force=True) + else: + log.warning( + 'Found orphan containers ({}) for this project. If ' + 'you removed or renamed this service in your compose ' + 'file, you can run this command with the ' + '--remove-orphans flag to clean it up.'.format( + ', '.join(["{}".format(ctnr.name) for ctnr in orphans]) + ) + ) + + def _inject_deps(self, acc, service, enabled_profiles): + dep_names = service.get_dependency_names() + + if len(dep_names) > 0: + dep_services = self.get_services( + service_names=list(set(dep_names)), + include_deps=True, + auto_enable_profiles=False + ) + + for dep in dep_services: + if not dep.enabled_for_profiles(enabled_profiles): + raise ConfigurationError( + 'Service "{dep_name}" was pulled in as a dependency of ' + 'service "{service_name}" but is not enabled by the ' + 'active profiles. ' + 'You may fix this by adding a common profile to ' + '"{dep_name}" and "{service_name}".' + .format(dep_name=dep.name, service_name=service.name) + ) + else: + dep_services = [] + + dep_services.append(service) + return acc + dep_services + + def build_container_operation_with_timeout_func(self, operation, options): + def container_operation_with_timeout(container): + _options = options.copy() + if _options.get('timeout') is None: + service = self.get_service(container.service) + _options['timeout'] = service.stop_timeout(None) + return getattr(container, operation)(**_options) + return container_operation_with_timeout + + +def translate_credential_spec_to_security_opt(service_dict): + result = [] + + if 'credential_spec' in service_dict: + spec = convert_credential_spec_to_security_opt(service_dict['credential_spec']) + result.append('credentialspec={spec}'.format(spec=spec)) + + if result: + service_dict['security_opt'] = result + + return service_dict + + +def translate_resource_keys_to_container_config(resources_dict, service_dict): + if 'limits' in resources_dict: + service_dict['mem_limit'] = resources_dict['limits'].get('memory') + if 'cpus' in resources_dict['limits']: + service_dict['cpus'] = float(resources_dict['limits']['cpus']) + if 'reservations' in resources_dict: + service_dict['mem_reservation'] = resources_dict['reservations'].get('memory') + if 'cpus' in resources_dict['reservations']: + return ['resources.reservations.cpus'] + return [] + + +def convert_restart_policy(name): + try: + return { + 'any': 'always', + 'none': 'no', + 'on-failure': 'on-failure' + }[name] + except KeyError: + raise ConfigurationError('Invalid restart policy "{}"'.format(name)) + + +def convert_credential_spec_to_security_opt(credential_spec): + if 'file' in credential_spec: + return 'file://{file}'.format(file=credential_spec['file']) + return 'registry://{registry}'.format(registry=credential_spec['registry']) + + +def translate_deploy_keys_to_container_config(service_dict): + if 'credential_spec' in service_dict: + del service_dict['credential_spec'] + if 'configs' in service_dict: + del service_dict['configs'] + + if 'deploy' not in service_dict: + return service_dict, [] + + deploy_dict = service_dict['deploy'] + ignored_keys = [ + k for k in ['endpoint_mode', 'labels', 'update_config', 'rollback_config'] + if k in deploy_dict + ] + + if 'restart_policy' in deploy_dict: + service_dict['restart'] = { + 'Name': convert_restart_policy(deploy_dict['restart_policy'].get('condition', 'any')), + 'MaximumRetryCount': deploy_dict['restart_policy'].get('max_attempts', 0) + } + for k in deploy_dict['restart_policy'].keys(): + if k != 'condition' and k != 'max_attempts': + ignored_keys.append('restart_policy.{}'.format(k)) + + ignored_keys.extend( + translate_resource_keys_to_container_config( + deploy_dict.get('resources', {}), service_dict + ) + ) + del service_dict['deploy'] + return service_dict, ignored_keys + + +def get_volumes_from(project, service_dict): + volumes_from = service_dict.pop('volumes_from', None) + if not volumes_from: + return [] + + def build_volume_from(spec): + if spec.type == 'service': + try: + return spec._replace(source=project.get_service(spec.source)) + except NoSuchService: + pass + + if spec.type == 'container': + try: + container = Container.from_id(project.client, spec.source) + return spec._replace(source=container) + except APIError: + pass + + raise ConfigurationError( + "Service \"{}\" mounts volumes from \"{}\", which is not the name " + "of a service or container.".format( + service_dict['name'], + spec.source)) + + return [build_volume_from(vf) for vf in volumes_from] + + +def get_secrets(service, service_secrets, secret_defs): + secrets = [] + + for secret in service_secrets: + secret_def = secret_defs.get(secret.source) + if not secret_def: + raise ConfigurationError( + "Service \"{service}\" uses an undefined secret \"{secret}\" " + .format(service=service, secret=secret.source)) + + if secret_def.get('external'): + log.warning('Service "{service}" uses secret "{secret}" which is external. ' + 'External secrets are not available to containers created by ' + 'docker-compose.'.format(service=service, secret=secret.source)) + continue + + if secret.uid or secret.gid or secret.mode: + log.warning( + 'Service "{service}" uses secret "{secret}" with uid, ' + 'gid, or mode. These fields are not supported by this ' + 'implementation of the Compose file'.format( + service=service, secret=secret.source + ) + ) + + secret_file = secret_def.get('file') + if not path.isfile(str(secret_file)): + log.warning( + 'Service "{service}" uses an undefined secret file "{secret_file}", ' + 'the following file should be created "{secret_file}"'.format( + service=service, secret_file=secret_file + ) + ) + secrets.append({'secret': secret, 'file': secret_file}) + + return secrets + + +def get_image_digests(project): + digests = {} + needs_push = set() + needs_pull = set() + + for service in project.services: + try: + digests[service.name] = get_image_digest(service) + except NeedsPush as e: + needs_push.add(e.image_name) + except NeedsPull as e: + needs_pull.add(e.service_name) + + if needs_push or needs_pull: + raise MissingDigests(needs_push, needs_pull) + + return digests + + +def get_image_digest(service): + if 'image' not in service.options: + raise UserError( + "Service '{s.name}' doesn't define an image tag. An image name is " + "required to generate a proper image digest. Specify an image repo " + "and tag with the 'image' option.".format(s=service)) + + _, _, separator = parse_repository_tag(service.options['image']) + # Compose file already uses a digest, no lookup required + if separator == '@': + return service.options['image'] + + digest = get_digest(service) + + if digest: + return digest + + if 'build' not in service.options: + raise NeedsPull(service.image_name, service.name) + + raise NeedsPush(service.image_name) + + +def get_digest(service): + digest = None + try: + image = service.image() + # TODO: pick a digest based on the image tag if there are multiple + # digests + if image['RepoDigests']: + digest = image['RepoDigests'][0] + except NoSuchImageError: + try: + # Fetch the image digest from the registry + distribution = service.get_image_registry_data() + + if distribution['Descriptor']['digest']: + digest = '{image_name}@{digest}'.format( + image_name=service.image_name, + digest=distribution['Descriptor']['digest'] + ) + except NoSuchImageError: + raise UserError( + "Digest not found for service '{service}'. " + "Repository does not exist or may require 'docker login'" + .format(service=service.name)) + return digest + + +class MissingDigests(Exception): + def __init__(self, needs_push, needs_pull): + self.needs_push = needs_push + self.needs_pull = needs_pull + + +class NeedsPush(Exception): + def __init__(self, image_name): + self.image_name = image_name + + +class NeedsPull(Exception): + def __init__(self, image_name, service_name): + self.image_name = image_name + self.service_name = service_name + + +class NoSuchService(Exception): + def __init__(self, name): + if isinstance(name, bytes): + name = name.decode('utf-8') + self.name = name + self.msg = "No such service: %s" % self.name + + def __str__(self): + return self.msg + + +class ProjectError(Exception): + def __init__(self, msg): + self.msg = msg diff --git a/compose/service.py b/compose/service.py new file mode 100644 index 00000000000..df0d76fb913 --- /dev/null +++ b/compose/service.py @@ -0,0 +1,1930 @@ +import enum +import itertools +import json +import logging +import os +import re +import subprocess +import sys +import tempfile +from collections import namedtuple +from collections import OrderedDict +from operator import attrgetter + +from docker.errors import APIError +from docker.errors import ImageNotFound +from docker.errors import NotFound +from docker.types import LogConfig +from docker.types import Mount +from docker.utils import version_gte +from docker.utils import version_lt +from docker.utils.ports import build_port_bindings +from docker.utils.ports import split_port +from docker.utils.utils import convert_tmpfs_mounts + +from . import __version__ +from . import const +from . import progress_stream +from .config import DOCKER_CONFIG_KEYS +from .config import is_url +from .config import merge_environment +from .config import merge_labels +from .config.errors import DependencyError +from .config.types import MountSpec +from .config.types import ServicePort +from .config.types import VolumeSpec +from .const import DEFAULT_TIMEOUT +from .const import IS_WINDOWS_PLATFORM +from .const import LABEL_CONFIG_HASH +from .const import LABEL_CONTAINER_NUMBER +from .const import LABEL_ONE_OFF +from .const import LABEL_PROJECT +from .const import LABEL_SERVICE +from .const import LABEL_SLUG +from .const import LABEL_VERSION +from .const import NANOCPUS_SCALE +from .const import WINDOWS_LONGPATH_PREFIX +from .container import Container +from .errors import HealthCheckFailed +from .errors import NoHealthCheckConfigured +from .errors import OperationFailedError +from .parallel import parallel_execute +from .progress_stream import stream_output +from .progress_stream import StreamOutputError +from .utils import generate_random_id +from .utils import json_hash +from .utils import parse_bytes +from .utils import parse_seconds_float +from .utils import truncate_id +from .utils import unique_everseen +from compose.cli.utils import binarystr_to_unicode + + +log = logging.getLogger(__name__) + +HOST_CONFIG_KEYS = [ + 'cap_add', + 'cap_drop', + 'cgroup_parent', + 'cpu_count', + 'cpu_percent', + 'cpu_period', + 'cpu_quota', + 'cpu_rt_period', + 'cpu_rt_runtime', + 'cpu_shares', + 'cpus', + 'cpuset', + 'device_cgroup_rules', + 'devices', + 'device_requests', + 'dns', + 'dns_search', + 'dns_opt', + 'env_file', + 'extra_hosts', + 'group_add', + 'init', + 'ipc', + 'isolation', + 'read_only', + 'log_driver', + 'log_opt', + 'mem_limit', + 'mem_reservation', + 'memswap_limit', + 'mem_swappiness', + 'oom_kill_disable', + 'oom_score_adj', + 'pid', + 'pids_limit', + 'privileged', + 'restart', + 'runtime', + 'security_opt', + 'shm_size', + 'storage_opt', + 'sysctls', + 'userns_mode', + 'volumes_from', + 'volume_driver', +] + +CONDITION_STARTED = 'service_started' +CONDITION_HEALTHY = 'service_healthy' + + +class BuildError(Exception): + def __init__(self, service, reason): + self.service = service + self.reason = reason + + +class NeedsBuildError(Exception): + def __init__(self, service): + self.service = service + + +class NoSuchImageError(Exception): + pass + + +ServiceName = namedtuple('ServiceName', 'project service number') + +ConvergencePlan = namedtuple('ConvergencePlan', 'action containers') + + +@enum.unique +class ConvergenceStrategy(enum.Enum): + """Enumeration for all possible convergence strategies. Values refer to + when containers should be recreated. + """ + changed = 1 + always = 2 + never = 3 + + @property + def allows_recreate(self): + return self is not type(self).never + + +@enum.unique +class ImageType(enum.Enum): + """Enumeration for the types of images known to compose.""" + none = 0 + local = 1 + all = 2 + + +@enum.unique +class BuildAction(enum.Enum): + """Enumeration for the possible build actions.""" + none = 0 + force = 1 + skip = 2 + + +class Service: + def __init__( + self, + name, + client=None, + project='default', + use_networking=False, + links=None, + volumes_from=None, + network_mode=None, + networks=None, + secrets=None, + scale=1, + ipc_mode=None, + pid_mode=None, + default_platform=None, + extra_labels=None, + **options + ): + self.name = name + self.client = client + self.project = project + self.use_networking = use_networking + self.links = links or [] + self.volumes_from = volumes_from or [] + self.ipc_mode = ipc_mode or IpcMode(None) + self.network_mode = network_mode or NetworkMode(None) + self.pid_mode = pid_mode or PidMode(None) + self.networks = networks or {} + self.secrets = secrets or [] + self.scale_num = scale + self.default_platform = default_platform + self.options = options + self.extra_labels = extra_labels or [] + + def __repr__(self): + return '<Service: {}>'.format(self.name) + + def containers(self, stopped=False, one_off=False, filters=None, labels=None): + if filters is None: + filters = {} + filters.update({'label': self.labels(one_off=one_off) + (labels or [])}) + + result = list(filter(None, [ + Container.from_ps(self.client, container) + for container in self.client.containers( + all=stopped, + filters=filters)]) + ) + if result: + return result + + filters.update({'label': self.labels(one_off=one_off, legacy=True) + (labels or [])}) + return list( + filter( + lambda c: c.has_legacy_proj_name(self.project), filter(None, [ + Container.from_ps(self.client, container) + for container in self.client.containers( + all=stopped, + filters=filters)]) + ) + ) + + def get_container(self, number=1): + """Return a :class:`compose.container.Container` for this service. The + container must be active, and match `number`. + """ + for container in self.containers(labels=['{}={}'.format(LABEL_CONTAINER_NUMBER, number)]): + return container + + raise ValueError("No container found for {}_{}".format(self.name, number)) + + def start(self, **options): + containers = self.containers(stopped=True) + for c in containers: + self.start_container_if_stopped(c, **options) + return containers + + def show_scale_warnings(self, desired_num): + if self.custom_container_name and desired_num > 1: + log.warning('The "%s" service is using the custom container name "%s". ' + 'Docker requires each container to have a unique name. ' + 'Remove the custom name to scale the service.' + % (self.name, self.custom_container_name)) + + if self.specifies_host_port() and desired_num > 1: + log.warning('The "%s" service specifies a port on the host. If multiple containers ' + 'for this service are created on a single host, the port will clash.' + % self.name) + + def scale(self, desired_num, timeout=None): + """ + Adjusts the number of containers to the specified number and ensures + they are running. + + - creates containers until there are at least `desired_num` + - stops containers until there are at most `desired_num` running + - starts containers until there are at least `desired_num` running + - removes all stopped containers + """ + + self.show_scale_warnings(desired_num) + + running_containers = self.containers(stopped=False) + num_running = len(running_containers) + for c in running_containers: + if not c.has_legacy_proj_name(self.project): + continue + log.info('Recreating container with legacy name %s' % c.name) + self.recreate_container(c, timeout, start_new_container=False) + + if desired_num == num_running: + # do nothing as we already have the desired number + log.info('Desired container number already achieved') + return + + if desired_num > num_running: + all_containers = self.containers(stopped=True) + + if num_running != len(all_containers): + # we have some stopped containers, check for divergences + stopped_containers = [ + c for c in all_containers if not c.is_running + ] + + # Remove containers that have diverged + divergent_containers = [ + c for c in stopped_containers if self._containers_have_diverged([c]) + ] + for c in divergent_containers: + c.remove() + + all_containers = list(set(all_containers) - set(divergent_containers)) + + sorted_containers = sorted(all_containers, key=attrgetter('number')) + self._execute_convergence_start( + sorted_containers, desired_num, timeout, True, True + ) + + if desired_num < num_running: + num_to_stop = num_running - desired_num + + sorted_running_containers = sorted( + running_containers, + key=attrgetter('number')) + + self._downscale(sorted_running_containers[-num_to_stop:], timeout) + + def create_container(self, + one_off=False, + previous_container=None, + number=None, + quiet=False, + **override_options): + """ + Create a container for this service. If the image doesn't exist, attempt to pull + it. + """ + # This is only necessary for `scale` and `volumes_from` + # auto-creating containers to satisfy the dependency. + self.ensure_image_exists() + + container_options = self._get_container_create_options( + override_options, + number or self._next_container_number(one_off=one_off), + one_off=one_off, + previous_container=previous_container, + ) + + if 'name' in container_options and not quiet: + log.info("Creating %s" % container_options['name']) + + try: + return Container.create(self.client, **container_options) + except APIError as ex: + raise OperationFailedError("Cannot create container for service %s: %s" % + (self.name, binarystr_to_unicode(ex.explanation))) + + def ensure_image_exists(self, do_build=BuildAction.none, silent=False, cli=False): + if self.can_be_built() and do_build == BuildAction.force: + self.build(cli=cli) + return + + try: + self.image() + return + except NoSuchImageError: + pass + + if not self.can_be_built(): + self.pull(silent=silent) + return + + if do_build == BuildAction.skip: + raise NeedsBuildError(self) + + self.build(cli=cli) + log.warning( + "Image for service {} was built because it did not already exist. To " + "rebuild this image you must use `docker-compose build` or " + "`docker-compose up --build`.".format(self.name)) + + def get_image_registry_data(self): + try: + return self.client.inspect_distribution(self.image_name) + except APIError: + raise NoSuchImageError("Image '{}' not found".format(self.image_name)) + + def image(self): + try: + return self.client.inspect_image(self.image_name) + except ImageNotFound: + raise NoSuchImageError("Image '{}' not found".format(self.image_name)) + + @property + def image_name(self): + return self.options.get('image', '{project}_{s.name}'.format( + s=self, project=self.project.lstrip('_-') + )) + + @property + def platform(self): + platform = self.options.get('platform') + if not platform and version_gte(self.client.api_version, '1.35'): + platform = self.default_platform + return platform + + def convergence_plan(self, strategy=ConvergenceStrategy.changed, one_off=False): + containers = self.containers(stopped=True) + + if one_off: + return ConvergencePlan('one_off', []) + + if not containers: + return ConvergencePlan('create', []) + + if strategy is ConvergenceStrategy.never: + return ConvergencePlan('start', containers) + + if ( + strategy is ConvergenceStrategy.always or + self._containers_have_diverged(containers) + ): + return ConvergencePlan('recreate', containers) + + stopped = [c for c in containers if not c.is_running] + + if stopped: + return ConvergencePlan('start', containers) + + return ConvergencePlan('noop', containers) + + def _containers_have_diverged(self, containers): + config_hash = None + + try: + config_hash = self.config_hash + except NoSuchImageError as e: + log.debug( + 'Service %s has diverged: %s', + self.name, str(e), + ) + return True + + has_diverged = False + + for c in containers: + if c.has_legacy_proj_name(self.project): + log.debug('%s has diverged: Legacy project name' % c.name) + has_diverged = True + continue + container_config_hash = c.labels.get(LABEL_CONFIG_HASH, None) + if container_config_hash != config_hash: + log.debug( + '%s has diverged: %s != %s', + c.name, container_config_hash, config_hash, + ) + has_diverged = True + + return has_diverged + + def _execute_convergence_create(self, scale, detached, start, one_off=False, override_options=None): + + i = self._next_container_number() + + def create_and_start(service, n): + if one_off: + container = service.create_container(one_off=True, quiet=True, **override_options) + else: + container = service.create_container(number=n, quiet=True) + if not detached: + container.attach_log_stream() + if start and not one_off: + self.start_container(container) + return container + + def get_name(service_name): + if one_off: + return "_".join([ + service_name.project, + service_name.service, + "run", + ]) + return self.get_container_name(service_name.service, service_name.number) + + containers, errors = parallel_execute( + [ + ServiceName(self.project, self.name, index) + for index in range(i, i + scale) + ], + lambda service_name: create_and_start(self, service_name.number), + get_name, + "Creating" + ) + for error in errors.values(): + raise OperationFailedError(error) + + return containers + + def _execute_convergence_recreate(self, containers, scale, timeout, detached, start, + renew_anonymous_volumes): + if scale is not None and len(containers) > scale: + self._downscale(containers[scale:], timeout) + containers = containers[:scale] + + def recreate(container): + return self.recreate_container( + container, timeout=timeout, attach_logs=not detached, + start_new_container=start, renew_anonymous_volumes=renew_anonymous_volumes + ) + + containers, errors = parallel_execute( + containers, + recreate, + lambda c: c.name, + "Recreating", + ) + for error in errors.values(): + raise OperationFailedError(error) + + if scale is not None and len(containers) < scale: + containers.extend(self._execute_convergence_create( + scale - len(containers), detached, start + )) + return containers + + def _execute_convergence_start(self, containers, scale, timeout, detached, start): + if scale is not None and len(containers) > scale: + self._downscale(containers[scale:], timeout) + containers = containers[:scale] + if start: + stopped = [c for c in containers if not c.is_running] + _, errors = parallel_execute( + stopped, + lambda c: self.start_container_if_stopped(c, attach_logs=not detached, quiet=True), + lambda c: c.name, + "Starting", + ) + + for error in errors.values(): + raise OperationFailedError(error) + + if scale is not None and len(containers) < scale: + containers.extend(self._execute_convergence_create( + scale - len(containers), detached, start + )) + return containers + + def _downscale(self, containers, timeout=None): + def stop_and_remove(container): + container.stop(timeout=self.stop_timeout(timeout)) + container.remove() + + parallel_execute( + containers, + stop_and_remove, + lambda c: c.name, + "Stopping and removing", + ) + + def execute_convergence_plan(self, plan, timeout=None, detached=False, + start=True, scale_override=None, + rescale=True, reset_container_image=False, + renew_anonymous_volumes=False, override_options=None): + (action, containers) = plan + scale = scale_override if scale_override is not None else self.scale_num + containers = sorted(containers, key=attrgetter('number')) + + self.show_scale_warnings(scale) + + if action in ['create', 'one_off']: + return self._execute_convergence_create( + scale, + detached, + start, + one_off=(action == 'one_off'), + override_options=override_options + ) + + # The create action needs always needs an initial scale, but otherwise, + # we set scale to none in no-rescale scenarios (`run` dependencies) + if not rescale: + scale = None + + if action == 'recreate': + if reset_container_image: + # Updating the image ID on the container object lets us recover old volumes if + # the new image uses them as well + img_id = self.image()['Id'] + for c in containers: + c.reset_image(img_id) + return self._execute_convergence_recreate( + containers, scale, timeout, detached, start, + renew_anonymous_volumes, + ) + + if action == 'start': + return self._execute_convergence_start( + containers, scale, timeout, detached, start + ) + + if action == 'noop': + if scale != len(containers): + return self._execute_convergence_start( + containers, scale, timeout, detached, start + ) + for c in containers: + log.info("%s is up-to-date" % c.name) + + return containers + + raise Exception("Invalid action: {}".format(action)) + + def recreate_container(self, container, timeout=None, attach_logs=False, start_new_container=True, + renew_anonymous_volumes=False): + """Recreate a container. + + The original container is renamed to a temporary name so that data + volumes can be copied to the new container, before the original + container is removed. + """ + + container.stop(timeout=self.stop_timeout(timeout)) + container.rename_to_tmp_name() + new_container = self.create_container( + previous_container=container if not renew_anonymous_volumes else None, + number=container.number, + quiet=True, + ) + if attach_logs: + new_container.attach_log_stream() + if start_new_container: + self.start_container(new_container) + container.remove() + return new_container + + def stop_timeout(self, timeout): + if timeout is not None: + return timeout + timeout = parse_seconds_float(self.options.get('stop_grace_period')) + if timeout is not None: + return timeout + return DEFAULT_TIMEOUT + + def start_container_if_stopped(self, container, attach_logs=False, quiet=False): + if not container.is_running: + if not quiet: + log.info("Starting %s" % container.name) + if attach_logs: + container.attach_log_stream() + return self.start_container(container) + + def start_container(self, container, use_network_aliases=True): + self.connect_container_to_networks(container, use_network_aliases) + try: + container.start() + except APIError as ex: + expl = binarystr_to_unicode(ex.explanation) + if "driver failed programming external connectivity" in expl: + log.warn("Host is already in use by another container") + raise OperationFailedError("Cannot start service {}: {}".format(self.name, expl)) + return container + + @property + def prioritized_networks(self): + return OrderedDict( + sorted( + self.networks.items(), + key=lambda t: t[1].get('priority') or 0, reverse=True + ) + ) + + def connect_container_to_networks(self, container, use_network_aliases=True): + connected_networks = container.get('NetworkSettings.Networks') + + for network, netdefs in self.prioritized_networks.items(): + if network in connected_networks: + if short_id_alias_exists(container, network): + continue + self.client.disconnect_container_from_network(container.id, network) + + aliases = self._get_aliases(netdefs, container) if use_network_aliases else [] + + self.client.connect_container_to_network( + container.id, network, + aliases=aliases, + ipv4_address=netdefs.get('ipv4_address', None), + ipv6_address=netdefs.get('ipv6_address', None), + links=self._get_links(False), + link_local_ips=netdefs.get('link_local_ips', None), + ) + + def remove_duplicate_containers(self, timeout=None): + for c in self.duplicate_containers(): + log.info('Removing %s' % c.name) + c.stop(timeout=self.stop_timeout(timeout)) + c.remove() + + def duplicate_containers(self): + containers = sorted( + self.containers(stopped=True), + key=lambda c: c.get('Created'), + ) + + numbers = set() + + for c in containers: + if c.number in numbers: + yield c + else: + numbers.add(c.number) + + @property + def config_hash(self): + return json_hash(self.config_dict()) + + def config_dict(self): + def image_id(): + try: + return self.image()['Id'] + except NoSuchImageError: + return None + + return { + 'options': self.options, + 'image_id': image_id(), + 'links': self.get_link_names(), + 'net': self.network_mode.id, + 'networks': self.networks, + 'secrets': self.secrets, + 'volumes_from': [ + (v.source.name, v.mode) + for v in self.volumes_from if isinstance(v.source, Service) + ] + } + + def get_dependency_names(self): + net_name = self.network_mode.service_name + pid_namespace = self.pid_mode.service_name + ipc_namespace = self.ipc_mode.service_name + return ( + self.get_linked_service_names() + + self.get_volumes_from_names() + + ([net_name] if net_name else []) + + ([pid_namespace] if pid_namespace else []) + + ([ipc_namespace] if ipc_namespace else []) + + list(self.options.get('depends_on', {}).keys()) + ) + + def get_dependency_configs(self): + net_name = self.network_mode.service_name + pid_namespace = self.pid_mode.service_name + ipc_namespace = self.ipc_mode.service_name + + configs = { + name: None for name in self.get_linked_service_names() + } + configs.update( + (name, None) for name in self.get_volumes_from_names() + ) + configs.update({net_name: None} if net_name else {}) + configs.update({pid_namespace: None} if pid_namespace else {}) + configs.update({ipc_namespace: None} if ipc_namespace else {}) + configs.update(self.options.get('depends_on', {})) + for svc, config in self.options.get('depends_on', {}).items(): + if config['condition'] == CONDITION_STARTED: + configs[svc] = lambda s: True + elif config['condition'] == CONDITION_HEALTHY: + configs[svc] = lambda s: s.is_healthy() + else: + # The config schema already prevents this, but it might be + # bypassed if Compose is called programmatically. + raise ValueError( + 'depends_on condition "{}" is invalid.'.format( + config['condition'] + ) + ) + + return configs + + def get_linked_service_names(self): + return [service.name for (service, _) in self.links] + + def get_link_names(self): + return [(service.name, alias) for service, alias in self.links] + + def get_volumes_from_names(self): + return [s.source.name for s in self.volumes_from if isinstance(s.source, Service)] + + def _next_container_number(self, one_off=False): + if one_off: + return None + containers = itertools.chain( + self._fetch_containers( + all=True, + filters={'label': self.labels(one_off=False)} + ), self._fetch_containers( + all=True, + filters={'label': self.labels(one_off=False, legacy=True)} + ) + ) + numbers = [c.number for c in containers if c.number is not None] + return 1 if not numbers else max(numbers) + 1 + + def _fetch_containers(self, **fetch_options): + # Account for containers that might have been removed since we fetched + # the list. + def soft_inspect(container): + try: + return Container.from_id(self.client, container['Id']) + except NotFound: + return None + + return filter(None, [ + soft_inspect(container) + for container in self.client.containers(**fetch_options) + ]) + + def _get_aliases(self, network, container=None): + return list( + {self.name} | + ({container.short_id} if container else set()) | + set(network.get('aliases', ())) + ) + + def build_default_networking_config(self): + if not self.networks: + return {} + + network = self.networks[self.network_mode.id] + endpoint = { + 'Aliases': self._get_aliases(network), + 'IPAMConfig': {}, + } + + if network.get('ipv4_address'): + endpoint['IPAMConfig']['IPv4Address'] = network.get('ipv4_address') + if network.get('ipv6_address'): + endpoint['IPAMConfig']['IPv6Address'] = network.get('ipv6_address') + + return {"EndpointsConfig": {self.network_mode.id: endpoint}} + + def _get_links(self, link_to_self): + links = {} + + for service, link_name in self.links: + for container in service.containers(): + links[link_name or service.name] = container.name + links[container.name] = container.name + links[container.name_without_project] = container.name + + if link_to_self: + for container in self.containers(): + links[self.name] = container.name + links[container.name] = container.name + links[container.name_without_project] = container.name + + for external_link in self.options.get('external_links') or []: + if ':' not in external_link: + link_name = external_link + else: + external_link, link_name = external_link.split(':') + links[link_name] = external_link + + return [ + (alias, container_name) + for (container_name, alias) in links.items() + ] + + def _get_volumes_from(self): + return [build_volume_from(spec) for spec in self.volumes_from] + + def _get_container_create_options( + self, + override_options, + number, + one_off=False, + previous_container=None): + add_config_hash = (not one_off and not override_options) + slug = generate_random_id() if one_off else None + + container_options = { + k: self.options[k] + for k in DOCKER_CONFIG_KEYS if k in self.options} + override_volumes = override_options.pop('volumes', []) + container_options.update(override_options) + + if not container_options.get('name'): + container_options['name'] = self.get_container_name(self.name, number, slug) + + container_options.setdefault('detach', True) + + # If a qualified hostname was given, split it into an + # unqualified hostname and a domainname unless domainname + # was also given explicitly. This matches behavior + # until Docker Engine 1.11.0 - Docker API 1.23. + if (version_lt(self.client.api_version, '1.23') and + 'hostname' in container_options and + 'domainname' not in container_options and + '.' in container_options['hostname']): + parts = container_options['hostname'].partition('.') + container_options['hostname'] = parts[0] + container_options['domainname'] = parts[2] + + if (version_gte(self.client.api_version, '1.25') and + 'stop_grace_period' in self.options): + container_options['stop_timeout'] = self.stop_timeout(None) + + if 'ports' in container_options or 'expose' in self.options: + container_options['ports'] = build_container_ports( + formatted_ports(container_options.get('ports', [])), + self.options) + + if 'volumes' in container_options or override_volumes: + container_options['volumes'] = list(set( + container_options.get('volumes', []) + override_volumes + )) + + container_options['environment'] = merge_environment( + self._parse_proxy_config(), + merge_environment( + self.options.get('environment'), + override_options.get('environment') + ) + ) + + container_options['labels'] = merge_labels( + self.options.get('labels'), + override_options.get('labels')) + + container_options, override_options = self._build_container_volume_options( + previous_container, container_options, override_options + ) + + container_options['image'] = self.image_name + + container_options['labels'] = build_container_labels( + container_options.get('labels', {}), + self.labels(one_off=one_off) + self.extra_labels, + number, + self.config_hash if add_config_hash else None, + slug + ) + + # Delete options which are only used in HostConfig + for key in HOST_CONFIG_KEYS: + container_options.pop(key, None) + + container_options['host_config'] = self._get_container_host_config( + override_options, + one_off=one_off) + + networking_config = self.build_default_networking_config() + if networking_config: + container_options['networking_config'] = networking_config + + container_options['environment'] = format_environment( + container_options['environment']) + return container_options + + def _build_container_volume_options(self, previous_container, container_options, override_options): + container_volumes = [] + container_mounts = [] + if 'volumes' in container_options: + container_volumes = [ + v for v in container_options.get('volumes') if isinstance(v, VolumeSpec) + ] + container_mounts = [v for v in container_options.get('volumes') if isinstance(v, MountSpec)] + + binds, affinity = merge_volume_bindings( + container_volumes, self.options.get('tmpfs') or [], previous_container, + container_mounts + ) + container_options['environment'].update(affinity) + + container_options['volumes'] = {v.internal: {} for v in container_volumes or {}} + if version_gte(self.client.api_version, '1.30'): + override_options['mounts'] = [build_mount(v) for v in container_mounts] or None + else: + # Workaround for 3.2 format + override_options['tmpfs'] = self.options.get('tmpfs') or [] + for m in container_mounts: + if m.is_tmpfs: + override_options['tmpfs'].append(m.target) + else: + binds.append(m.legacy_repr()) + container_options['volumes'][m.target] = {} + + secret_volumes = self.get_secret_volumes() + if secret_volumes: + if version_lt(self.client.api_version, '1.30'): + binds.extend(v.legacy_repr() for v in secret_volumes) + container_options['volumes'].update( + (v.target, {}) for v in secret_volumes + ) + else: + override_options['mounts'] = override_options.get('mounts') or [] + override_options['mounts'].extend([build_mount(v) for v in secret_volumes]) + + # Remove possible duplicates (see e.g. https://github.com/docker/compose/issues/5885). + # unique_everseen preserves order. (see https://github.com/docker/compose/issues/6091). + override_options['binds'] = list(unique_everseen(binds)) + return container_options, override_options + + def _get_container_host_config(self, override_options, one_off=False): + options = dict(self.options, **override_options) + + logging_dict = options.get('logging', None) + blkio_config = convert_blkio_config(options.get('blkio_config', None)) + log_config = get_log_config(logging_dict) + init_path = None + if isinstance(options.get('init'), str): + init_path = options.get('init') + options['init'] = True + + security_opt = [ + o.value for o in options.get('security_opt') + ] if options.get('security_opt') else None + + nano_cpus = None + if 'cpus' in options: + nano_cpus = int(options.get('cpus') * NANOCPUS_SCALE) + + return self.client.create_host_config( + links=self._get_links(link_to_self=one_off), + port_bindings=build_port_bindings( + formatted_ports(options.get('ports', [])) + ), + binds=options.get('binds'), + volumes_from=self._get_volumes_from(), + privileged=options.get('privileged', False), + network_mode=self.network_mode.mode, + devices=options.get('devices'), + device_requests=options.get('device_requests'), + dns=options.get('dns'), + dns_opt=options.get('dns_opt'), + dns_search=options.get('dns_search'), + restart_policy=options.get('restart'), + runtime=options.get('runtime'), + cap_add=options.get('cap_add'), + cap_drop=options.get('cap_drop'), + mem_limit=options.get('mem_limit'), + mem_reservation=options.get('mem_reservation'), + memswap_limit=options.get('memswap_limit'), + ulimits=build_ulimits(options.get('ulimits')), + log_config=log_config, + extra_hosts=options.get('extra_hosts'), + read_only=options.get('read_only'), + pid_mode=self.pid_mode.mode, + security_opt=security_opt, + ipc_mode=self.ipc_mode.mode, + cgroup_parent=options.get('cgroup_parent'), + cpu_quota=options.get('cpu_quota'), + shm_size=options.get('shm_size'), + sysctls=options.get('sysctls'), + pids_limit=options.get('pids_limit'), + tmpfs=options.get('tmpfs'), + oom_kill_disable=options.get('oom_kill_disable'), + oom_score_adj=options.get('oom_score_adj'), + mem_swappiness=options.get('mem_swappiness'), + group_add=options.get('group_add'), + userns_mode=options.get('userns_mode'), + init=options.get('init', None), + init_path=init_path, + isolation=options.get('isolation'), + cpu_count=options.get('cpu_count'), + cpu_percent=options.get('cpu_percent'), + nano_cpus=nano_cpus, + volume_driver=options.get('volume_driver'), + cpuset_cpus=options.get('cpuset'), + cpu_shares=options.get('cpu_shares'), + storage_opt=options.get('storage_opt'), + blkio_weight=blkio_config.get('weight'), + blkio_weight_device=blkio_config.get('weight_device'), + device_read_bps=blkio_config.get('device_read_bps'), + device_read_iops=blkio_config.get('device_read_iops'), + device_write_bps=blkio_config.get('device_write_bps'), + device_write_iops=blkio_config.get('device_write_iops'), + mounts=options.get('mounts'), + device_cgroup_rules=options.get('device_cgroup_rules'), + cpu_period=options.get('cpu_period'), + cpu_rt_period=options.get('cpu_rt_period'), + cpu_rt_runtime=options.get('cpu_rt_runtime'), + ) + + def get_secret_volumes(self): + def build_spec(secret): + target = secret['secret'].target + if target is None: + target = '{}/{}'.format(const.SECRETS_PATH, secret['secret'].source) + elif not os.path.isabs(target): + target = '{}/{}'.format(const.SECRETS_PATH, target) + + return MountSpec('bind', secret['file'], target, read_only=True) + + return [build_spec(secret) for secret in self.secrets] + + def build(self, no_cache=False, pull=False, force_rm=False, memory=None, build_args_override=None, + gzip=False, rm=True, silent=False, cli=False, progress=None): + output_stream = open(os.devnull, 'w') + if not silent: + output_stream = sys.stdout + log.info('Building %s' % self.name) + + build_opts = self.options.get('build', {}) + + build_args = build_opts.get('args', {}).copy() + if build_args_override: + build_args.update(build_args_override) + + for k, v in self._parse_proxy_config().items(): + build_args.setdefault(k, v) + + path = rewrite_build_path(build_opts.get('context')) + if self.platform and version_lt(self.client.api_version, '1.35'): + raise OperationFailedError( + 'Impossible to perform platform-targeted builds for API version < 1.35' + ) + + builder = self.client if not cli else _CLIBuilder(progress) + build_output = builder.build( + path=path, + tag=self.image_name, + rm=rm, + forcerm=force_rm, + pull=pull, + nocache=no_cache, + dockerfile=build_opts.get('dockerfile', None), + cache_from=self.get_cache_from(build_opts), + labels=build_opts.get('labels', None), + buildargs=build_args, + network_mode=build_opts.get('network', None), + target=build_opts.get('target', None), + shmsize=parse_bytes(build_opts.get('shm_size')) if build_opts.get('shm_size') else None, + extra_hosts=build_opts.get('extra_hosts', None), + container_limits={ + 'memory': parse_bytes(memory) if memory else None + }, + gzip=gzip, + isolation=build_opts.get('isolation', self.options.get('isolation', None)), + platform=self.platform, + ) + + try: + all_events = list(stream_output(build_output, output_stream)) + except StreamOutputError as e: + raise BuildError(self, str(e)) + + # Ensure the HTTP connection is not reused for another + # streaming command, as the Docker daemon can sometimes + # complain about it + self.client.close() + + image_id = None + + for event in all_events: + if 'stream' in event: + match = re.search(r'Successfully built ([0-9a-f]+)', event.get('stream', '')) + if match: + image_id = match.group(1) + + if image_id is None: + raise BuildError(self, event if all_events else 'Unknown') + + return image_id + + def get_cache_from(self, build_opts): + cache_from = build_opts.get('cache_from', None) + if cache_from is not None: + cache_from = [tag for tag in cache_from if tag] + return cache_from + + def can_be_built(self): + return 'build' in self.options + + def labels(self, one_off=False, legacy=False): + proj_name = self.project if not legacy else re.sub(r'[_-]', '', self.project) + return [ + '{}={}'.format(LABEL_PROJECT, proj_name), + '{}={}'.format(LABEL_SERVICE, self.name), + '{}={}'.format(LABEL_ONE_OFF, "True" if one_off else "False"), + ] + + @property + def custom_container_name(self): + return self.options.get('container_name') + + def get_container_name(self, service_name, number, slug=None): + if self.custom_container_name and slug is None: + return self.custom_container_name + + container_name = build_container_name( + self.project, service_name, number, slug, + ) + ext_links_origins = [link.split(':')[0] for link in self.options.get('external_links', [])] + if container_name in ext_links_origins: + raise DependencyError( + 'Service {} has a self-referential external link: {}'.format( + self.name, container_name + ) + ) + return container_name + + def remove_image(self, image_type): + if not image_type or image_type == ImageType.none: + return False + if image_type == ImageType.local and self.options.get('image'): + return False + + log.info("Removing image %s", self.image_name) + try: + self.client.remove_image(self.image_name) + return True + except ImageNotFound: + log.warning("Image %s not found.", self.image_name) + return False + except APIError as e: + log.error("Failed to remove image for service %s: %s", self.name, e) + return False + + def specifies_host_port(self): + def has_host_port(binding): + if isinstance(binding, dict): + external_bindings = binding.get('published') + else: + _, external_bindings = split_port(binding) + + # there are no external bindings + if external_bindings is None: + return False + + # we only need to check the first binding from the range + external_binding = external_bindings[0] + + # non-tuple binding means there is a host port specified + if not isinstance(external_binding, tuple): + return True + + # extract actual host port from tuple of (host_ip, host_port) + _, host_port = external_binding + if host_port is not None: + return True + + return False + + return any(has_host_port(binding) for binding in self.options.get('ports', [])) + + def _do_pull(self, repo, pull_kwargs, silent, ignore_pull_failures): + try: + output = self.client.pull(repo, **pull_kwargs) + if silent: + with open(os.devnull, 'w') as devnull: + yield from stream_output(output, devnull) + else: + yield from stream_output(output, sys.stdout) + except (StreamOutputError, NotFound) as e: + if not ignore_pull_failures: + raise + else: + log.error(str(e)) + + def pull(self, ignore_pull_failures=False, silent=False, stream=False): + if 'image' not in self.options: + return + + repo, tag, separator = parse_repository_tag(self.options['image']) + kwargs = { + 'tag': tag or 'latest', + 'stream': True, + 'platform': self.platform, + } + if not silent: + log.info('Pulling {} ({}{}{})...'.format(self.name, repo, separator, tag)) + + if kwargs['platform'] and version_lt(self.client.api_version, '1.35'): + raise OperationFailedError( + 'Impossible to perform platform-targeted pulls for API version < 1.35' + ) + + event_stream = self._do_pull(repo, kwargs, silent, ignore_pull_failures) + if stream: + return event_stream + return progress_stream.get_digest_from_pull(event_stream) + + def push(self, ignore_push_failures=False): + if 'image' not in self.options or 'build' not in self.options: + return + + repo, tag, separator = parse_repository_tag(self.options['image']) + tag = tag or 'latest' + log.info('Pushing {} ({}{}{})...'.format(self.name, repo, separator, tag)) + output = self.client.push(repo, tag=tag, stream=True) + + try: + return progress_stream.get_digest_from_push( + stream_output(output, sys.stdout)) + except StreamOutputError as e: + if not ignore_push_failures: + raise + else: + log.error(str(e)) + + def is_healthy(self): + """ Check that all containers for this service report healthy. + Returns false if at least one healthcheck is pending. + If an unhealthy container is detected, raise a HealthCheckFailed + exception. + """ + result = True + for ctnr in self.containers(): + ctnr.inspect() + status = ctnr.get('State.Health.Status') + if status is None: + raise NoHealthCheckConfigured(self.name) + elif status == 'starting': + result = False + elif status == 'unhealthy': + raise HealthCheckFailed(ctnr.short_id) + return result + + def _parse_proxy_config(self): + client = self.client + if 'proxies' not in client._general_configs: + return {} + docker_host = getattr(client, '_original_base_url', client.base_url) + proxy_config = client._general_configs['proxies'].get( + docker_host, client._general_configs['proxies'].get('default') + ) or {} + + permitted = { + 'ftpProxy': 'FTP_PROXY', + 'httpProxy': 'HTTP_PROXY', + 'httpsProxy': 'HTTPS_PROXY', + 'noProxy': 'NO_PROXY', + } + + result = {} + + for k, v in proxy_config.items(): + if k not in permitted: + continue + result[permitted[k]] = result[permitted[k].lower()] = v + + return result + + def get_profiles(self): + if 'profiles' not in self.options: + return [] + + return self.options.get('profiles') + + def enabled_for_profiles(self, enabled_profiles): + # if service has no profiles specified it is always enabled + if 'profiles' not in self.options: + return True + + service_profiles = self.options.get('profiles') + for profile in enabled_profiles: + if profile in service_profiles: + return True + + return False + + +def short_id_alias_exists(container, network): + aliases = container.get( + 'NetworkSettings.Networks.{net}.Aliases'.format(net=network)) or () + return container.short_id in aliases + + +class IpcMode: + def __init__(self, mode): + self._mode = mode + + @property + def mode(self): + return self._mode + + @property + def service_name(self): + return None + + +class ServiceIpcMode(IpcMode): + def __init__(self, service): + self.service = service + + @property + def service_name(self): + return self.service.name + + @property + def mode(self): + containers = self.service.containers() + if containers: + return 'container:' + containers[0].id + + log.warning( + "Service %s is trying to use reuse the IPC namespace " + "of another service that is not running." % (self.service_name) + ) + return None + + +class ContainerIpcMode(IpcMode): + def __init__(self, container): + self.container = container + self._mode = 'container:{}'.format(container.id) + + +class PidMode: + def __init__(self, mode): + self._mode = mode + + @property + def mode(self): + return self._mode + + @property + def service_name(self): + return None + + +class ServicePidMode(PidMode): + def __init__(self, service): + self.service = service + + @property + def service_name(self): + return self.service.name + + @property + def mode(self): + containers = self.service.containers() + if containers: + return 'container:' + containers[0].id + + log.warning( + "Service %s is trying to use reuse the PID namespace " + "of another service that is not running." % (self.service_name) + ) + return None + + +class ContainerPidMode(PidMode): + def __init__(self, container): + self.container = container + self._mode = 'container:{}'.format(container.id) + + +class NetworkMode: + """A `standard` network mode (ex: host, bridge)""" + + service_name = None + + def __init__(self, network_mode): + self.network_mode = network_mode + + @property + def id(self): + return self.network_mode + + mode = id + + +class ContainerNetworkMode: + """A network mode that uses a container's network stack.""" + + service_name = None + + def __init__(self, container): + self.container = container + + @property + def id(self): + return self.container.id + + @property + def mode(self): + return 'container:' + self.container.id + + +class ServiceNetworkMode: + """A network mode that uses a service's network stack.""" + + def __init__(self, service): + self.service = service + + @property + def id(self): + return self.service.name + + service_name = id + + @property + def mode(self): + containers = self.service.containers() + if containers: + return 'container:' + containers[0].id + + log.warning("Service %s is trying to use reuse the network stack " + "of another service that is not running." % (self.id)) + return None + + +# Names + + +def build_container_name(project, service, number, slug=None): + bits = [project.lstrip('-_'), service] + if slug: + bits.extend(['run', truncate_id(slug)]) + else: + bits.append(str(number)) + return '_'.join(bits) + + +# Images + +def parse_repository_tag(repo_path): + """Splits image identification into base image path, tag/digest + and it's separator. + + Example: + + >>> parse_repository_tag('user/repo@sha256:digest') + ('user/repo', 'sha256:digest', '@') + >>> parse_repository_tag('user/repo:v1') + ('user/repo', 'v1', ':') + """ + tag_separator = ":" + digest_separator = "@" + + if digest_separator in repo_path: + repo, tag = repo_path.rsplit(digest_separator, 1) + return repo, tag, digest_separator + + repo, tag = repo_path, "" + if tag_separator in repo_path: + repo, tag = repo_path.rsplit(tag_separator, 1) + if "/" in tag: + repo, tag = repo_path, "" + + return repo, tag, tag_separator + + +# Volumes + + +def merge_volume_bindings(volumes, tmpfs, previous_container, mounts): + """ + Return a list of volume bindings for a container. Container data volumes + are replaced by those from the previous container. + Anonymous mounts are updated in place. + """ + affinity = {} + + volume_bindings = OrderedDict( + build_volume_binding(volume) + for volume in volumes + if volume.external + ) + + if previous_container: + old_volumes, old_mounts = get_container_data_volumes( + previous_container, volumes, tmpfs, mounts + ) + warn_on_masked_volume(volumes, old_volumes, previous_container.service) + volume_bindings.update( + build_volume_binding(volume) for volume in old_volumes + ) + + if old_volumes or old_mounts: + affinity = {'affinity:container': '=' + previous_container.id} + + return list(volume_bindings.values()), affinity + + +def get_container_data_volumes(container, volumes_option, tmpfs_option, mounts_option): + """ + Find the container data volumes that are in `volumes_option`, and return + a mapping of volume bindings for those volumes. + Anonymous volume mounts are updated in place instead. + """ + volumes = [] + volumes_option = volumes_option or [] + + container_mounts = { + mount['Destination']: mount + for mount in container.get('Mounts') or {} + } + + image_volumes = [ + VolumeSpec.parse(volume) + for volume in + container.image_config['ContainerConfig'].get('Volumes') or {} + ] + + for volume in set(volumes_option + image_volumes): + # No need to preserve host volumes + if volume.external: + continue + + # Attempting to rebind tmpfs volumes breaks: https://github.com/docker/compose/issues/4751 + if volume.internal in convert_tmpfs_mounts(tmpfs_option).keys(): + continue + + mount = container_mounts.get(volume.internal) + + # New volume, doesn't exist in the old container + if not mount: + continue + + # Volume was previously a host volume, now it's a container volume + if not mount.get('Name'): + continue + + # Volume (probably an image volume) is overridden by a mount in the service's config + # and would cause a duplicate mountpoint error + if volume.internal in [m.target for m in mounts_option]: + continue + + # Copy existing volume from old container + volume = volume._replace(external=mount['Name']) + volumes.append(volume) + + updated_mounts = False + for mount in mounts_option: + if mount.type != 'volume': + continue + + ctnr_mount = container_mounts.get(mount.target) + if not ctnr_mount or not ctnr_mount.get('Name'): + continue + + mount.source = ctnr_mount['Name'] + updated_mounts = True + + return volumes, updated_mounts + + +def warn_on_masked_volume(volumes_option, container_volumes, service): + container_volumes = { + volume.internal: volume.external + for volume in container_volumes} + + for volume in volumes_option: + if ( + volume.external and + volume.internal in container_volumes and + container_volumes.get(volume.internal) != volume.external + ): + log.warning(( + "Service \"{service}\" is using volume \"{volume}\" from the " + "previous container. Host mapping \"{host_path}\" has no effect. " + "Remove the existing containers (with `docker-compose rm {service}`) " + "to use the host volume mapping." + ).format( + service=service, + volume=volume.internal, + host_path=volume.external)) + + +def build_volume_binding(volume_spec): + return volume_spec.internal, volume_spec.repr() + + +def build_volume_from(volume_from_spec): + """ + volume_from can be either a service or a container. We want to return the + container.id and format it into a string complete with the mode. + """ + if isinstance(volume_from_spec.source, Service): + containers = volume_from_spec.source.containers(stopped=True) + if not containers: + return "{}:{}".format( + volume_from_spec.source.create_container().id, + volume_from_spec.mode) + + container = containers[0] + return "{}:{}".format(container.id, volume_from_spec.mode) + elif isinstance(volume_from_spec.source, Container): + return "{}:{}".format(volume_from_spec.source.id, volume_from_spec.mode) + + +def build_mount(mount_spec): + kwargs = {} + if mount_spec.options: + for option, sdk_name in mount_spec.options_map[mount_spec.type].items(): + if option in mount_spec.options: + kwargs[sdk_name] = mount_spec.options[option] + + return Mount( + type=mount_spec.type, target=mount_spec.target, source=mount_spec.source, + read_only=mount_spec.read_only, consistency=mount_spec.consistency, **kwargs + ) + + +# Labels + + +def build_container_labels(label_options, service_labels, number, config_hash, slug): + labels = dict(label_options or {}) + labels.update(label.split('=', 1) for label in service_labels) + if number is not None: + labels[LABEL_CONTAINER_NUMBER] = str(number) + if slug is not None: + labels[LABEL_SLUG] = slug + labels[LABEL_VERSION] = __version__ + + if config_hash: + log.debug("Added config hash: %s" % config_hash) + labels[LABEL_CONFIG_HASH] = config_hash + + return labels + + +# Ulimits + + +def build_ulimits(ulimit_config): + if not ulimit_config: + return None + ulimits = [] + for limit_name, soft_hard_values in ulimit_config.items(): + if isinstance(soft_hard_values, int): + ulimits.append({'name': limit_name, 'soft': soft_hard_values, 'hard': soft_hard_values}) + elif isinstance(soft_hard_values, dict): + ulimit_dict = {'name': limit_name} + ulimit_dict.update(soft_hard_values) + ulimits.append(ulimit_dict) + + return ulimits + + +def get_log_config(logging_dict): + log_driver = logging_dict.get('driver', "") if logging_dict else "" + log_options = logging_dict.get('options', None) if logging_dict else None + return LogConfig( + type=log_driver, + config=log_options + ) + + +# TODO: remove once fix is available in docker-py +def format_environment(environment): + def format_env(key, value): + if value is None: + return key + if isinstance(value, bytes): + value = value.decode('utf-8') + return '{key}={value}'.format(key=key, value=value) + + return [format_env(*item) for item in environment.items()] + + +# Ports +def formatted_ports(ports): + result = [] + for port in ports: + if isinstance(port, ServicePort): + result.append(port.legacy_repr()) + else: + result.append(port) + return result + + +def build_container_ports(container_ports, options): + ports = [] + all_ports = container_ports + options.get('expose', []) + for port_range in all_ports: + internal_range, _ = split_port(port_range) + for port in internal_range: + port = str(port) + if '/' in port: + port = tuple(port.split('/')) + ports.append(port) + return ports + + +def convert_blkio_config(blkio_config): + result = {} + if blkio_config is None: + return result + + result['weight'] = blkio_config.get('weight') + for field in [ + "device_read_bps", "device_read_iops", "device_write_bps", + "device_write_iops", "weight_device", + ]: + if field not in blkio_config: + continue + arr = [] + for item in blkio_config[field]: + arr.append({k.capitalize(): v for k, v in item.items()}) + result[field] = arr + return result + + +def rewrite_build_path(path): + if IS_WINDOWS_PLATFORM and not is_url(path) and not path.startswith(WINDOWS_LONGPATH_PREFIX): + path = WINDOWS_LONGPATH_PREFIX + os.path.normpath(path) + + return path + + +class _CLIBuilder: + def __init__(self, progress): + self._progress = progress + + def build(self, path, tag=None, quiet=False, fileobj=None, + nocache=False, rm=False, timeout=None, + custom_context=False, encoding=None, pull=False, + forcerm=False, dockerfile=None, container_limits=None, + decode=False, buildargs=None, gzip=False, shmsize=None, + labels=None, cache_from=None, target=None, network_mode=None, + squash=None, extra_hosts=None, platform=None, isolation=None, + use_config_proxy=True): + """ + Args: + path (str): Path to the directory containing the Dockerfile + buildargs (dict): A dictionary of build arguments + cache_from (:py:class:`list`): A list of images used for build + cache resolution + container_limits (dict): A dictionary of limits applied to each + container created by the build process. Valid keys: + - memory (int): set memory limit for build + - memswap (int): Total memory (memory + swap), -1 to disable + swap + - cpushares (int): CPU shares (relative weight) + - cpusetcpus (str): CPUs in which to allow execution, e.g., + ``"0-3"``, ``"0,1"`` + custom_context (bool): Optional if using ``fileobj`` + decode (bool): If set to ``True``, the returned stream will be + decoded into dicts on the fly. Default ``False`` + dockerfile (str): path within the build context to the Dockerfile + encoding (str): The encoding for a stream. Set to ``gzip`` for + compressing + extra_hosts (dict): Extra hosts to add to /etc/hosts in building + containers, as a mapping of hostname to IP address. + fileobj: A file object to use as the Dockerfile. (Or a file-like + object) + forcerm (bool): Always remove intermediate containers, even after + unsuccessful builds + isolation (str): Isolation technology used during build. + Default: `None`. + labels (dict): A dictionary of labels to set on the image + network_mode (str): networking mode for the run commands during + build + nocache (bool): Don't use the cache when set to ``True`` + platform (str): Platform in the format ``os[/arch[/variant]]`` + pull (bool): Downloads any updates to the FROM image in Dockerfiles + quiet (bool): Whether to return the status + rm (bool): Remove intermediate containers. The ``docker build`` + command now defaults to ``--rm=true``, but we have kept the old + default of `False` to preserve backward compatibility + shmsize (int): Size of `/dev/shm` in bytes. The size must be + greater than 0. If omitted the system uses 64MB + squash (bool): Squash the resulting images layers into a + single layer. + tag (str): A tag to add to the final image + target (str): Name of the build-stage to build in a multi-stage + Dockerfile + timeout (int): HTTP timeout + use_config_proxy (bool): If ``True``, and if the docker client + configuration file (``~/.docker/config.json`` by default) + contains a proxy configuration, the corresponding environment + variables will be set in the container being built. + Returns: + A generator for the build output. + """ + if dockerfile: + dockerfile = os.path.join(path, dockerfile) + iidfile = tempfile.mktemp() + + command_builder = _CommandBuilder() + command_builder.add_params("--build-arg", buildargs) + command_builder.add_list("--cache-from", cache_from) + command_builder.add_arg("--file", dockerfile) + command_builder.add_flag("--force-rm", forcerm) + command_builder.add_params("--label", labels) + command_builder.add_arg("--memory", container_limits.get("memory")) + command_builder.add_arg("--network", network_mode) + command_builder.add_flag("--no-cache", nocache) + command_builder.add_arg("--progress", self._progress) + command_builder.add_flag("--pull", pull) + command_builder.add_arg("--tag", tag) + command_builder.add_arg("--target", target) + command_builder.add_arg("--iidfile", iidfile) + args = command_builder.build([path]) + + magic_word = "Successfully built " + appear = False + with subprocess.Popen(args, stdout=subprocess.PIPE, + universal_newlines=True) as p: + while True: + line = p.stdout.readline() + if not line: + break + if line.startswith(magic_word): + appear = True + yield json.dumps({"stream": line}) + + p.communicate() + if p.returncode != 0: + raise StreamOutputError() + + with open(iidfile) as f: + line = f.readline() + image_id = line.split(":")[1].strip() + os.remove(iidfile) + + # In case of `DOCKER_BUILDKIT=1` + # there is no success message already present in the output. + # Since that's the way `Service::build` gets the `image_id` + # it has to be added `manually` + if not appear: + yield json.dumps({"stream": "{}{}\n".format(magic_word, image_id)}) + + +class _CommandBuilder: + def __init__(self): + self._args = ["docker", "build"] + + def add_arg(self, name, value): + if value: + self._args.extend([name, str(value)]) + + def add_flag(self, name, flag): + if flag: + self._args.extend([name]) + + def add_params(self, name, params): + if params: + for key, val in params.items(): + self._args.extend([name, "{}={}".format(key, val)]) + + def add_list(self, name, values): + if values: + for val in values: + self._args.extend([name, val]) + + def build(self, args): + return self._args + args diff --git a/compose/timeparse.py b/compose/timeparse.py new file mode 100644 index 00000000000..47744562519 --- /dev/null +++ b/compose/timeparse.py @@ -0,0 +1,92 @@ +#!/usr/bin/env python +''' +timeparse.py +(c) Will Roberts <wildwilhelm@gmail.com> 1 February, 2014 + +This is a vendored and modified copy of: +github.com/wroberts/pytimeparse @ cc0550d + +It has been modified to mimic the behaviour of +https://golang.org/pkg/time/#ParseDuration +''' +# MIT LICENSE +# +# Permission is hereby granted, free of charge, to any person +# obtaining a copy of this software and associated documentation files +# (the "Software"), to deal in the Software without restriction, +# including without limitation the rights to use, copy, modify, merge, +# publish, distribute, sublicense, and/or sell copies of the Software, +# and to permit persons to whom the Software is furnished to do so, +# subject to the following conditions: +# +# The above copyright notice and this permission notice shall be +# included in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS +# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN +# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. +import re + +HOURS = r'(?P<hours>[\d.]+)h' +MINS = r'(?P<mins>[\d.]+)m' +SECS = r'(?P<secs>[\d.]+)s' +MILLI = r'(?P<milli>[\d.]+)ms' +MICRO = r'(?P<micro>[\d.]+)(?:us|µs)' +NANO = r'(?P<nano>[\d.]+)ns' + + +def opt(x): + return r'(?:{x})?'.format(x=x) + + +TIMEFORMAT = r'{HOURS}{MINS}{SECS}{MILLI}{MICRO}{NANO}'.format( + HOURS=opt(HOURS), + MINS=opt(MINS), + SECS=opt(SECS), + MILLI=opt(MILLI), + MICRO=opt(MICRO), + NANO=opt(NANO), +) + +MULTIPLIERS = { + 'hours': 60 * 60, + 'mins': 60, + 'secs': 1, + 'milli': 1.0 / 1000, + 'micro': 1.0 / 1000.0 / 1000, + 'nano': 1.0 / 1000.0 / 1000.0 / 1000.0, +} + + +def timeparse(sval): + """Parse a time expression, returning it as a number of seconds. If + possible, the return value will be an `int`; if this is not + possible, the return will be a `float`. Returns `None` if a time + expression cannot be parsed from the given string. + + Arguments: + - `sval`: the string value to parse + + >>> timeparse('1m24s') + 84 + >>> timeparse('1.2 minutes') + 72 + >>> timeparse('1.2 seconds') + 1.2 + """ + match = re.match(r'\s*' + TIMEFORMAT + r'\s*$', sval, re.I) + if not match or not match.group(0).strip(): + return + + mdict = match.groupdict() + return sum( + MULTIPLIERS[k] * cast(v) for (k, v) in mdict.items() if v is not None) + + +def cast(value): + return int(value) if value.isdigit() else float(value) diff --git a/compose/utils.py b/compose/utils.py new file mode 100644 index 00000000000..86af8f8852a --- /dev/null +++ b/compose/utils.py @@ -0,0 +1,191 @@ +import hashlib +import json.decoder +import logging +import ntpath +import random + +from docker.errors import DockerException +from docker.utils import parse_bytes as sdk_parse_bytes + +from .errors import StreamParseError +from .timeparse import MULTIPLIERS +from .timeparse import timeparse + + +json_decoder = json.JSONDecoder() +log = logging.getLogger(__name__) + + +def stream_as_text(stream): + """Given a stream of bytes or text, if any of the items in the stream + are bytes convert them to text. + + This function can be removed once docker-py returns text streams instead + of byte streams. + """ + for data in stream: + if not isinstance(data, str): + data = data.decode('utf-8', 'replace') + yield data + + +def line_splitter(buffer, separator='\n'): + index = buffer.find(str(separator)) + if index == -1: + return None + return buffer[:index + 1], buffer[index + 1:] + + +def split_buffer(stream, splitter=None, decoder=lambda a: a): + """Given a generator which yields strings and a splitter function, + joins all input, splits on the separator and yields each chunk. + + Unlike string.split(), each chunk includes the trailing + separator, except for the last one if none was found on the end + of the input. + """ + splitter = splitter or line_splitter + buffered = '' + + for data in stream_as_text(stream): + buffered += data + while True: + buffer_split = splitter(buffered) + if buffer_split is None: + break + + item, buffered = buffer_split + yield item + + if buffered: + try: + yield decoder(buffered) + except Exception as e: + log.error( + 'Compose tried decoding the following data chunk, but failed:' + '\n%s' % repr(buffered) + ) + raise StreamParseError(e) + + +def json_splitter(buffer): + """Attempt to parse a json object from a buffer. If there is at least one + object, return it and the rest of the buffer, otherwise return None. + """ + buffer = buffer.strip() + try: + obj, index = json_decoder.raw_decode(buffer) + rest = buffer[json.decoder.WHITESPACE.match(buffer, index).end():] + return obj, rest + except ValueError: + return None + + +def json_stream(stream): + """Given a stream of text, return a stream of json objects. + This handles streams which are inconsistently buffered (some entries may + be newline delimited, and others are not). + """ + return split_buffer(stream, json_splitter, json_decoder.decode) + + +def json_hash(obj): + dump = json.dumps(obj, sort_keys=True, separators=(',', ':'), default=lambda x: x.repr()) + h = hashlib.sha256() + h.update(dump.encode('utf8')) + return h.hexdigest() + + +def microseconds_from_time_nano(time_nano): + return int(time_nano % 1000000000 / 1000) + + +def nanoseconds_from_time_seconds(time_seconds): + return int(time_seconds / MULTIPLIERS['nano']) + + +def parse_seconds_float(value): + return timeparse(value or '') + + +def parse_nanoseconds_int(value): + parsed = timeparse(value or '') + if parsed is None: + return None + return nanoseconds_from_time_seconds(parsed) + + +def build_string_dict(source_dict): + return {k: str(v if v is not None else '') for k, v in source_dict.items()} + + +def splitdrive(path): + if len(path) == 0: + return ('', '') + if path[0] in ['.', '\\', '/', '~']: + return ('', path) + return ntpath.splitdrive(path) + + +def parse_bytes(n): + try: + return sdk_parse_bytes(n) + except DockerException: + return None + + +def unquote_path(s): + if not s: + return s + if s[0] == '"' and s[-1] == '"': + return s[1:-1] + return s + + +def generate_random_id(): + while True: + val = hex(random.getrandbits(32 * 8))[2:-1] + try: + int(truncate_id(val)) + continue + except ValueError: + return val + + +def truncate_id(value): + if ':' in value: + value = value[value.index(':') + 1:] + if len(value) > 12: + return value[:12] + return value + + +def unique_everseen(iterable, key=lambda x: x): + "List unique elements, preserving order. Remember all elements ever seen." + seen = set() + for element in iterable: + unique_key = key(element) + if unique_key not in seen: + seen.add(unique_key) + yield element + + +def truncate_string(s, max_chars=35): + if len(s) > max_chars: + return s[:max_chars - 2] + '...' + return s + + +def filter_attached_for_up(items, service_names, attach_dependencies=False, + item_to_service_name=lambda x: x): + """This function contains the logic of choosing which services to + attach when doing docker-compose up. It may be used both with containers + and services, and any other entities that map to service names - + this mapping is provided by item_to_service_name.""" + if attach_dependencies or not service_names: + return items + + return [ + item + for item in items if item_to_service_name(item) in service_names + ] diff --git a/compose/version.py b/compose/version.py new file mode 100644 index 00000000000..c039263acb9 --- /dev/null +++ b/compose/version.py @@ -0,0 +1,7 @@ +from distutils.version import LooseVersion + + +class ComposeVersion(LooseVersion): + """ A hashable version object """ + def __hash__(self): + return hash(self.vstring) diff --git a/compose/volume.py b/compose/volume.py new file mode 100644 index 00000000000..5f36e432ba9 --- /dev/null +++ b/compose/volume.py @@ -0,0 +1,213 @@ +import logging +import re +from itertools import chain + +from docker.errors import NotFound +from docker.utils import version_lt + +from . import __version__ +from .config import ConfigurationError +from .config.types import VolumeSpec +from .const import LABEL_PROJECT +from .const import LABEL_VERSION +from .const import LABEL_VOLUME + + +log = logging.getLogger(__name__) + + +class Volume: + def __init__(self, client, project, name, driver=None, driver_opts=None, + external=False, labels=None, custom_name=False): + self.client = client + self.project = project + self.name = name + self.driver = driver + self.driver_opts = driver_opts + self.external = external + self.labels = labels + self.custom_name = custom_name + self.legacy = None + + def create(self): + return self.client.create_volume( + self.full_name, self.driver, self.driver_opts, labels=self._labels + ) + + def remove(self): + if self.external: + log.info("Volume %s is external, skipping", self.true_name) + return + log.info("Removing volume %s", self.true_name) + return self.client.remove_volume(self.true_name) + + def inspect(self, legacy=None): + if legacy: + return self.client.inspect_volume(self.legacy_full_name) + return self.client.inspect_volume(self.full_name) + + def exists(self): + self._set_legacy_flag() + try: + self.inspect(legacy=self.legacy) + except NotFound: + return False + return True + + @property + def full_name(self): + if self.custom_name: + return self.name + return '{}_{}'.format(self.project.lstrip('-_'), self.name) + + @property + def legacy_full_name(self): + if self.custom_name: + return self.name + return '{}_{}'.format( + re.sub(r'[_-]', '', self.project), self.name + ) + + @property + def true_name(self): + self._set_legacy_flag() + if self.legacy: + return self.legacy_full_name + return self.full_name + + @property + def _labels(self): + if version_lt(self.client._version, '1.23'): + return None + labels = self.labels.copy() if self.labels else {} + labels.update({ + LABEL_PROJECT: self.project, + LABEL_VOLUME: self.name, + LABEL_VERSION: __version__, + }) + return labels + + def _set_legacy_flag(self): + if self.legacy is not None: + return + try: + data = self.inspect(legacy=True) + self.legacy = data is not None + except NotFound: + self.legacy = False + + +class ProjectVolumes: + + def __init__(self, volumes): + self.volumes = volumes + + @classmethod + def from_config(cls, name, config_data, client): + config_volumes = config_data.volumes or {} + volumes = { + vol_name: Volume( + client=client, + project=name, + name=data.get('name', vol_name), + driver=data.get('driver'), + driver_opts=data.get('driver_opts'), + custom_name=data.get('name') is not None, + labels=data.get('labels'), + external=bool(data.get('external', False)) + ) + for vol_name, data in config_volumes.items() + } + return cls(volumes) + + def remove(self): + for volume in self.volumes.values(): + try: + volume.remove() + except NotFound: + log.warning("Volume %s not found.", volume.true_name) + + def initialize(self): + try: + for volume in self.volumes.values(): + volume_exists = volume.exists() + if volume.external: + log.debug( + 'Volume {} declared as external. No new ' + 'volume will be created.'.format(volume.name) + ) + if not volume_exists: + raise ConfigurationError( + 'Volume {name} declared as external, but could' + ' not be found. Please create the volume manually' + ' using `{command}{name}` and try again.'.format( + name=volume.full_name, + command='docker volume create --name=' + ) + ) + continue + + if not volume_exists: + log.info( + 'Creating volume "{}" with {} driver'.format( + volume.full_name, volume.driver or 'default' + ) + ) + volume.create() + else: + check_remote_volume_config(volume.inspect(legacy=volume.legacy), volume) + except NotFound: + raise ConfigurationError( + 'Volume {} specifies nonexistent driver {}'.format(volume.name, volume.driver) + ) + + def namespace_spec(self, volume_spec): + if not volume_spec.is_named_volume: + return volume_spec + + if isinstance(volume_spec, VolumeSpec): + volume = self.volumes[volume_spec.external] + return volume_spec._replace(external=volume.true_name) + else: + volume_spec.source = self.volumes[volume_spec.source].true_name + return volume_spec + + +class VolumeConfigChangedError(ConfigurationError): + def __init__(self, local, property_name, local_value, remote_value): + super().__init__( + 'Configuration for volume {vol_name} specifies {property_name} ' + '{local_value}, but a volume with the same name uses a different ' + '{property_name} ({remote_value}). If you wish to use the new ' + 'configuration, please remove the existing volume "{full_name}" ' + 'first:\n$ docker volume rm {full_name}'.format( + vol_name=local.name, property_name=property_name, + local_value=local_value, remote_value=remote_value, + full_name=local.true_name + ) + ) + + +def check_remote_volume_config(remote, local): + if local.driver and remote.get('Driver') != local.driver: + raise VolumeConfigChangedError(local, 'driver', local.driver, remote.get('Driver')) + local_opts = local.driver_opts or {} + remote_opts = remote.get('Options') or {} + for k in set(chain(remote_opts, local_opts)): + if k.startswith('com.docker.'): # These options are set internally + continue + if remote_opts.get(k) != local_opts.get(k): + raise VolumeConfigChangedError( + local, '"{}" driver_opt'.format(k), local_opts.get(k), remote_opts.get(k), + ) + + local_labels = local.labels or {} + remote_labels = remote.get('Labels') or {} + for k in set(chain(remote_labels, local_labels)): + if k.startswith('com.docker.'): # We are only interested in user-specified labels + continue + if remote_labels.get(k) != local_labels.get(k): + log.warning( + 'Volume {}: label "{}" has changed. It may need to be' + ' recreated.'.format(local.name, k) + ) diff --git a/contrib/completion/bash/docker-compose b/contrib/completion/bash/docker-compose new file mode 100644 index 00000000000..04f1a78ae76 --- /dev/null +++ b/contrib/completion/bash/docker-compose @@ -0,0 +1,666 @@ +#!/bin/bash +# +# bash completion for docker-compose +# +# This work is based on the completion for the docker command. +# +# This script provides completion of: +# - commands and their options +# - service names +# - filepaths +# +# To enable the completions either: +# - place this file in /etc/bash_completion.d +# or +# - copy this file to e.g. ~/.docker-compose-completion.sh and add the line +# below to your .bashrc after bash completion features are loaded +# . ~/.docker-compose-completion.sh + +__docker_compose_previous_extglob_setting=$(shopt -p extglob) +shopt -s extglob + +__docker_compose_q() { + docker-compose 2>/dev/null "${top_level_options[@]}" "$@" +} + +# Transforms a multiline list of strings into a single line string +# with the words separated by "|". +__docker_compose_to_alternatives() { + local parts=( $1 ) + local IFS='|' + echo "${parts[*]}" +} + +# Transforms a multiline list of options into an extglob pattern +# suitable for use in case statements. +__docker_compose_to_extglob() { + local extglob=$( __docker_compose_to_alternatives "$1" ) + echo "@($extglob)" +} + +# Determines whether the option passed as the first argument exist on +# the commandline. The option may be a pattern, e.g. `--force|-f`. +__docker_compose_has_option() { + local pattern="$1" + for (( i=2; i < $cword; ++i)); do + if [[ ${words[$i]} =~ ^($pattern)$ ]] ; then + return 0 + fi + done + return 1 +} + +# Returns `key` if we are currently completing the value of a map option (`key=value`) +# which matches the extglob passed in as an argument. +# This function is needed for key-specific completions. +__docker_compose_map_key_of_current_option() { + local glob="$1" + + local key glob_pos + if [ "$cur" = "=" ] ; then # key= case + key="$prev" + glob_pos=$((cword - 2)) + elif [[ $cur == *=* ]] ; then # key=value case (OSX) + key=${cur%=*} + glob_pos=$((cword - 1)) + elif [ "$prev" = "=" ] ; then + key=${words[$cword - 2]} # key=value case + glob_pos=$((cword - 3)) + else + return + fi + + [ "${words[$glob_pos]}" = "=" ] && ((glob_pos--)) # --option=key=value syntax + + [[ ${words[$glob_pos]} == @($glob) ]] && echo "$key" +} + +# suppress trailing whitespace +__docker_compose_nospace() { + # compopt is not available in ancient bash versions + type compopt &>/dev/null && compopt -o nospace +} + + +# Outputs a list of all defined services, regardless of their running state. +# Arguments for `docker-compose ps` may be passed in order to filter the service list, +# e.g. `status=running`. +__docker_compose_services() { + __docker_compose_q ps --services "$@" +} + +# Applies completion of services based on the current value of `$cur`. +# Arguments for `docker-compose ps` may be passed in order to filter the service list, +# see `__docker_compose_services`. +__docker_compose_complete_services() { + COMPREPLY=( $(compgen -W "$(__docker_compose_services "$@")" -- "$cur") ) +} + +# The services for which at least one running container exists +__docker_compose_complete_running_services() { + local names=$(__docker_compose_services --filter status=running) + COMPREPLY=( $(compgen -W "$names" -- "$cur") ) +} + + +_docker_compose_build() { + case "$prev" in + --build-arg) + COMPREPLY=( $( compgen -e -- "$cur" ) ) + __docker_compose_nospace + return + ;; + --memory|-m) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--build-arg --compress --force-rm --help --memory -m --no-cache --no-rm --pull --parallel -q --quiet" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services --filter source=build + ;; + esac +} + + +_docker_compose_config() { + case "$prev" in + --hash) + if [[ $cur == \\* ]] ; then + COMPREPLY=( '\*' ) + else + COMPREPLY=( $(compgen -W "$(__docker_compose_services) \\\* " -- "$cur") ) + fi + return + ;; + esac + + COMPREPLY=( $( compgen -W "--hash --help --no-interpolate --quiet -q --resolve-image-digests --services --volumes" -- "$cur" ) ) +} + + +_docker_compose_create() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--build --force-recreate --help --no-build --no-recreate" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_docker_compose() { + case "$prev" in + --tlscacert|--tlscert|--tlskey) + _filedir + return + ;; + --file|-f) + _filedir "y?(a)ml" + return + ;; + --log-level) + COMPREPLY=( $( compgen -W "debug info warning error critical" -- "$cur" ) ) + return + ;; + --project-directory) + _filedir -d + return + ;; + --env-file) + _filedir + return + ;; + $(__docker_compose_to_extglob "$daemon_options_with_args") ) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "$daemon_boolean_options $daemon_options_with_args $top_level_options_with_args --help -h --no-ansi --verbose --version -v" -- "$cur" ) ) + ;; + *) + COMPREPLY=( $( compgen -W "${commands[*]}" -- "$cur" ) ) + ;; + esac +} + + +_docker_compose_down() { + case "$prev" in + --rmi) + COMPREPLY=( $( compgen -W "all local" -- "$cur" ) ) + return + ;; + --timeout|-t) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --rmi --timeout -t --volumes -v --remove-orphans" -- "$cur" ) ) + ;; + esac +} + + +_docker_compose_events() { + case "$prev" in + --json) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --json" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_exec() { + case "$prev" in + --index|--user|-u|--workdir|-w) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "-d --detach --help --index --privileged -T --user -u --workdir -w" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_running_services + ;; + esac +} + + +_docker_compose_help() { + COMPREPLY=( $( compgen -W "${commands[*]}" -- "$cur" ) ) +} + +_docker_compose_images() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --quiet -q" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + +_docker_compose_kill() { + case "$prev" in + -s) + COMPREPLY=( $( compgen -W "SIGHUP SIGINT SIGKILL SIGUSR1 SIGUSR2" -- "$(echo $cur | tr '[:lower:]' '[:upper:]')" ) ) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help -s" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_running_services + ;; + esac +} + + +_docker_compose_logs() { + case "$prev" in + --tail) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--follow -f --help --no-color --tail --timestamps -t" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_pause() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_running_services + ;; + esac +} + + +_docker_compose_port() { + case "$prev" in + --protocol) + COMPREPLY=( $( compgen -W "tcp udp" -- "$cur" ) ) + return; + ;; + --index) + return; + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --index --protocol" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_ps() { + local key=$(__docker_compose_map_key_of_current_option '--filter') + case "$key" in + source) + COMPREPLY=( $( compgen -W "build image" -- "${cur##*=}" ) ) + return + ;; + status) + COMPREPLY=( $( compgen -W "paused restarting running stopped" -- "${cur##*=}" ) ) + return + ;; + esac + + case "$prev" in + --filter) + COMPREPLY=( $( compgen -W "source status" -S "=" -- "$cur" ) ) + __docker_compose_nospace + return; + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--all -a --filter --help --quiet -q --services" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_pull() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --ignore-pull-failures --include-deps --no-parallel --quiet -q" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services --filter source=image + ;; + esac +} + + +_docker_compose_push() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --ignore-push-failures" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_restart() { + case "$prev" in + --timeout|-t) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --timeout -t" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_running_services + ;; + esac +} + + +_docker_compose_rm() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--force -f --help --stop -s -v" -- "$cur" ) ) + ;; + *) + if __docker_compose_has_option "--stop|-s" ; then + __docker_compose_complete_services + else + __docker_compose_complete_services --filter status=stopped + fi + ;; + esac +} + + +_docker_compose_run() { + case "$prev" in + -e) + COMPREPLY=( $( compgen -e -- "$cur" ) ) + __docker_compose_nospace + return + ;; + --entrypoint|--label|-l|--name|--user|-u|--volume|-v|--workdir|-w) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--detach -d --entrypoint -e --help --label -l --name --no-deps --publish -p --rm --service-ports -T --use-aliases --user -u --volume -v --workdir -w" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_scale() { + case "$prev" in + =) + COMPREPLY=("$cur") + return + ;; + --timeout|-t) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --timeout -t" -- "$cur" ) ) + ;; + *) + COMPREPLY=( $(compgen -S "=" -W "$(__docker_compose_services)" -- "$cur") ) + __docker_compose_nospace + ;; + esac +} + + +_docker_compose_start() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services --filter status=stopped + ;; + esac +} + + +_docker_compose_stop() { + case "$prev" in + --timeout|-t) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --timeout -t" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_running_services + ;; + esac +} + + +_docker_compose_top() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_running_services + ;; + esac +} + + +_docker_compose_unpause() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services --filter status=paused + ;; + esac +} + + +_docker_compose_up() { + case "$prev" in + =) + COMPREPLY=("$cur") + return + ;; + --exit-code-from) + __docker_compose_complete_services + return + ;; + --scale) + COMPREPLY=( $(compgen -S "=" -W "$(__docker_compose_services)" -- "$cur") ) + __docker_compose_nospace + return + ;; + --timeout|-t) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--abort-on-container-exit --always-recreate-deps --attach-dependencies --build -d --detach --exit-code-from --force-recreate --help --no-build --no-color --no-deps --no-recreate --no-start --renew-anon-volumes -V --remove-orphans --scale --timeout -t" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_version() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--short" -- "$cur" ) ) + ;; + esac +} + + +_docker_compose() { + local previous_extglob_setting=$(shopt -p extglob) + shopt -s extglob + + local commands=( + build + config + create + down + events + exec + help + images + kill + logs + pause + port + ps + pull + push + restart + rm + run + scale + start + stop + top + unpause + up + version + ) + + # Options for the docker daemon that have to be passed to secondary calls to + # docker-compose executed by this script. + local daemon_boolean_options=" + --skip-hostname-check + --tls + --tlsverify + " + local daemon_options_with_args=" + --context -c + --env-file + --file -f + --host -H + --project-directory + --project-name -p + --tlscacert + --tlscert + --tlskey + " + + # These options are require special treatment when searching the command. + local top_level_options_with_args=" + --log-level + " + + COMPREPLY=() + local cur prev words cword + _get_comp_words_by_ref -n : cur prev words cword + + # search subcommand and invoke its handler. + # special treatment of some top-level options + local command='docker_compose' + local top_level_options=() + local counter=1 + + while [ $counter -lt $cword ]; do + case "${words[$counter]}" in + $(__docker_compose_to_extglob "$daemon_boolean_options") ) + local opt=${words[counter]} + top_level_options+=($opt) + ;; + $(__docker_compose_to_extglob "$daemon_options_with_args") ) + local opt=${words[counter]} + local arg=${words[++counter]} + top_level_options+=($opt $arg) + ;; + $(__docker_compose_to_extglob "$top_level_options_with_args") ) + (( counter++ )) + ;; + -*) + ;; + *) + command="${words[$counter]}" + break + ;; + esac + (( counter++ )) + done + + local completions_func=_docker_compose_${command//-/_} + declare -F $completions_func >/dev/null && $completions_func + + eval "$previous_extglob_setting" + return 0 +} + +eval "$__docker_compose_previous_extglob_setting" +unset __docker_compose_previous_extglob_setting + +complete -F _docker_compose docker-compose docker-compose.exe diff --git a/contrib/completion/fish/docker-compose.fish b/contrib/completion/fish/docker-compose.fish new file mode 100644 index 00000000000..0566e16ae88 --- /dev/null +++ b/contrib/completion/fish/docker-compose.fish @@ -0,0 +1,25 @@ +# Tab completion for docker-compose (https://github.com/docker/compose). +# Version: 1.9.0 + +complete -e -c docker-compose + +for line in (docker-compose --help | \ + string match -r '^\s+\w+\s+[^\n]+' | \ + string trim) + set -l doc (string split -m 1 ' ' -- $line) + complete -c docker-compose -n '__fish_use_subcommand' -xa $doc[1] --description $doc[2] +end + +complete -c docker-compose -s f -l file -r -d 'Specify an alternate compose file' +complete -c docker-compose -s p -l project-name -x -d 'Specify an alternate project name' +complete -c docker-compose -l env-file -r -d 'Specify an alternate environment file (default: .env)' +complete -c docker-compose -l verbose -d 'Show more output' +complete -c docker-compose -s H -l host -x -d 'Daemon socket to connect to' +complete -c docker-compose -l tls -d 'Use TLS; implied by --tlsverify' +complete -c docker-compose -l tlscacert -r -d 'Trust certs signed only by this CA' +complete -c docker-compose -l tlscert -r -d 'Path to TLS certificate file' +complete -c docker-compose -l tlskey -r -d 'Path to TLS key file' +complete -c docker-compose -l tlsverify -d 'Use TLS and verify the remote' +complete -c docker-compose -l skip-hostname-check -d "Don't check the daemon's hostname against the name specified in the client certificate (for example if your docker host is an IP address)" +complete -c docker-compose -s h -l help -d 'Print usage' +complete -c docker-compose -s v -l version -d 'Print version and exit' diff --git a/contrib/completion/zsh/_docker-compose b/contrib/completion/zsh/_docker-compose new file mode 100755 index 00000000000..de14149844a --- /dev/null +++ b/contrib/completion/zsh/_docker-compose @@ -0,0 +1,420 @@ +#compdef docker-compose + +# Description +# ----------- +# zsh completion for docker-compose +# ------------------------------------------------------------------------- +# Authors +# ------- +# * Steve Durrheimer <s.durrheimer@gmail.com> +# ------------------------------------------------------------------------- +# Inspiration +# ----------- +# * @albers docker-compose bash completion script +# * @felixr docker zsh completion script : https://github.com/felixr/docker-zsh-completion +# ------------------------------------------------------------------------- + +__docker-compose_q() { + docker-compose 2>/dev/null $compose_options "$@" +} + +# All services defined in docker-compose.yml +__docker-compose_all_services_in_compose_file() { + local already_selected + local -a services + already_selected=$(echo $words | tr " " "|") + __docker-compose_q ps --services "$@" \ + | grep -Ev "^(${already_selected})$" +} + +# All services, even those without an existing container +__docker-compose_services_all() { + [[ $PREFIX = -* ]] && return 1 + integer ret=1 + services=$(__docker-compose_all_services_in_compose_file "$@") + _alternative "args:services:($services)" && ret=0 + + return ret +} + +# All services that are defined by a Dockerfile reference +__docker-compose_services_from_build() { + [[ $PREFIX = -* ]] && return 1 + __docker-compose_services_all --filter source=build +} + +# All services that are defined by an image +__docker-compose_services_from_image() { + [[ $PREFIX = -* ]] && return 1 + __docker-compose_services_all --filter source=image +} + +__docker-compose_pausedservices() { + [[ $PREFIX = -* ]] && return 1 + __docker-compose_services_all --filter status=paused +} + +__docker-compose_stoppedservices() { + [[ $PREFIX = -* ]] && return 1 + __docker-compose_services_all --filter status=stopped +} + +__docker-compose_runningservices() { + [[ $PREFIX = -* ]] && return 1 + __docker-compose_services_all --filter status=running +} + +__docker-compose_services() { + [[ $PREFIX = -* ]] && return 1 + __docker-compose_services_all +} + +__docker-compose_caching_policy() { + oldp=( "$1"(Nmh+1) ) # 1 hour + (( $#oldp )) +} + +__docker-compose_commands() { + local cache_policy + + zstyle -s ":completion:${curcontext}:" cache-policy cache_policy + if [[ -z "$cache_policy" ]]; then + zstyle ":completion:${curcontext}:" cache-policy __docker-compose_caching_policy + fi + + if ( [[ ${+_docker_compose_subcommands} -eq 0 ]] || _cache_invalid docker_compose_subcommands) \ + && ! _retrieve_cache docker_compose_subcommands; + then + local -a lines + lines=(${(f)"$(_call_program commands docker-compose 2>&1)"}) + _docker_compose_subcommands=(${${${lines[$((${lines[(i)Commands:]} + 1)),${lines[(I) *]}]}## #}/ ##/:}) + (( $#_docker_compose_subcommands > 0 )) && _store_cache docker_compose_subcommands _docker_compose_subcommands + fi + _describe -t docker-compose-commands "docker-compose command" _docker_compose_subcommands +} + +__docker-compose_subcommand() { + local opts_help opts_force_recreate opts_no_recreate opts_no_build opts_remove_orphans opts_timeout opts_no_color opts_no_deps + + opts_help='(: -)--help[Print usage]' + opts_force_recreate="(--no-recreate)--force-recreate[Recreate containers even if their configuration and image haven't changed. Incompatible with --no-recreate.]" + opts_no_recreate="(--force-recreate)--no-recreate[If containers already exist, don't recreate them. Incompatible with --force-recreate.]" + opts_no_build="(--build)--no-build[Don't build an image, even if it's missing.]" + opts_remove_orphans="--remove-orphans[Remove containers for services not defined in the Compose file]" + opts_timeout=('(-t --timeout)'{-t,--timeout}"[Specify a shutdown timeout in seconds. (default: 10)]:seconds: ") + opts_no_color='--no-color[Produce monochrome output.]' + opts_no_deps="--no-deps[Don't start linked services.]" + + integer ret=1 + + case "$words[1]" in + (build) + _arguments \ + $opts_help \ + "*--build-arg=[Set build-time variables for one service.]:<varname>=<value>: " \ + '--force-rm[Always remove intermediate containers.]' \ + '(--quiet -q)'{--quiet,-q}'[Curb build output]' \ + '(--memory -m)'{--memory,-m}'[Memory limit for the build container.]' \ + '--no-cache[Do not use cache when building the image.]' \ + '--pull[Always attempt to pull a newer version of the image.]' \ + '--compress[Compress the build context using gzip.]' \ + '--parallel[Build images in parallel.]' \ + '*:services:__docker-compose_services_from_build' && ret=0 + ;; + (config) + _arguments \ + $opts_help \ + '(--quiet -q)'{--quiet,-q}"[Only validate the configuration, don't print anything.]" \ + '--resolve-image-digests[Pin image tags to digests.]' \ + '--services[Print the service names, one per line.]' \ + '--volumes[Print the volume names, one per line.]' \ + '--hash[Print the service config hash, one per line. Set "service1,service2" for a list of specified services.]' \ && ret=0 + ;; + (create) + _arguments \ + $opts_help \ + $opts_force_recreate \ + $opts_no_recreate \ + $opts_no_build \ + "(--no-build)--build[Build images before creating containers.]" \ + '*:services:__docker-compose_services' && ret=0 + ;; + (down) + _arguments \ + $opts_help \ + $opts_timeout \ + "--rmi[Remove images. Type must be one of: 'all': Remove all images used by any service. 'local': Remove only images that don't have a custom tag set by the \`image\` field.]:type:(all local)" \ + '(-v --volumes)'{-v,--volumes}"[Remove named volumes declared in the \`volumes\` section of the Compose file and anonymous volumes attached to containers.]" \ + $opts_remove_orphans && ret=0 + ;; + (events) + _arguments \ + $opts_help \ + '--json[Output events as a stream of json objects]' \ + '*:services:__docker-compose_services' && ret=0 + ;; + (exec) + _arguments \ + $opts_help \ + '-d[Detached mode: Run command in the background.]' \ + '--privileged[Give extended privileges to the process.]' \ + '(-u --user)'{-u,--user=}'[Run the command as this user.]:username:_users' \ + '-T[Disable pseudo-tty allocation. By default `docker-compose exec` allocates a TTY.]' \ + '--index=[Index of the container if there are multiple instances of a service \[default: 1\]]:index: ' \ + '*'{-e,--env}'[KEY=VAL Set an environment variable (can be used multiple times)]:environment variable KEY=VAL: ' \ + '(-w --workdir)'{-w,--workdir=}'[Working directory inside the container]:workdir: ' \ + '(-):running services:__docker-compose_runningservices' \ + '(-):command: _command_names -e' \ + '*::arguments: _normal' && ret=0 + ;; + (help) + _arguments ':subcommand:__docker-compose_commands' && ret=0 + ;; + (images) + _arguments \ + $opts_help \ + '-q[Only display IDs]' \ + '*:services:__docker-compose_services' && ret=0 + ;; + (kill) + _arguments \ + $opts_help \ + '-s[SIGNAL to send to the container. Default signal is SIGKILL.]:signal:_signals' \ + '*:running services:__docker-compose_runningservices' && ret=0 + ;; + (logs) + _arguments \ + $opts_help \ + '(-f --follow)'{-f,--follow}'[Follow log output]' \ + $opts_no_color \ + '--tail=[Number of lines to show from the end of the logs for each container.]:number of lines: ' \ + '(-t --timestamps)'{-t,--timestamps}'[Show timestamps]' \ + '*:services:__docker-compose_services' && ret=0 + ;; + (pause) + _arguments \ + $opts_help \ + '*:running services:__docker-compose_runningservices' && ret=0 + ;; + (port) + _arguments \ + $opts_help \ + '--protocol=[tcp or udp \[default: tcp\]]:protocol:(tcp udp)' \ + '--index=[index of the container if there are multiple instances of a service \[default: 1\]]:index: ' \ + '1:running services:__docker-compose_runningservices' \ + '2:port:_ports' && ret=0 + ;; + (ps) + _arguments \ + $opts_help \ + '-q[Only display IDs]' \ + '--filter KEY=VAL[Filter services by a property]:<filtername>=<value>:' \ + '*:services:__docker-compose_services' && ret=0 + ;; + (pull) + _arguments \ + $opts_help \ + '--ignore-pull-failures[Pull what it can and ignores images with pull failures.]' \ + '--no-parallel[Disable parallel pulling]' \ + '(-q --quiet)'{-q,--quiet}'[Pull without printing progress information]' \ + '--include-deps[Also pull services declared as dependencies]' \ + '*:services:__docker-compose_services_from_image' && ret=0 + ;; + (push) + _arguments \ + $opts_help \ + '--ignore-push-failures[Push what it can and ignores images with push failures.]' \ + '*:services:__docker-compose_services' && ret=0 + ;; + (rm) + _arguments \ + $opts_help \ + '(-f --force)'{-f,--force}"[Don't ask to confirm removal]" \ + '-v[Remove any anonymous volumes attached to containers]' \ + '*:stopped services:__docker-compose_stoppedservices' && ret=0 + ;; + (run) + _arguments \ + $opts_help \ + $opts_no_deps \ + '-d[Detached mode: Run container in the background, print new container name.]' \ + '*-e[KEY=VAL Set an environment variable (can be used multiple times)]:environment variable KEY=VAL: ' \ + '*'{-l,--label}'[KEY=VAL Add or override a label (can be used multiple times)]:label KEY=VAL: ' \ + '--entrypoint[Overwrite the entrypoint of the image.]:entry point: ' \ + '--name=[Assign a name to the container]:name: ' \ + '(-p --publish)'{-p,--publish=}"[Publish a container's port(s) to the host]" \ + '--rm[Remove container after run. Ignored in detached mode.]' \ + "--service-ports[Run command with the service's ports enabled and mapped to the host.]" \ + '-T[Disable pseudo-tty allocation. By default `docker-compose run` allocates a TTY.]' \ + '(-u --user)'{-u,--user=}'[Run as specified username or uid]:username or uid:_users' \ + '(-v --volume)*'{-v,--volume=}'[Bind mount a volume]:volume: ' \ + '(-w --workdir)'{-w,--workdir=}'[Working directory inside the container]:workdir: ' \ + "--use-aliases[Use the services network aliases in the network(s) the container connects to]" \ + '(-):services:__docker-compose_services' \ + '(-):command: _command_names -e' \ + '*::arguments: _normal' && ret=0 + ;; + (scale) + _arguments \ + $opts_help \ + $opts_timeout \ + '*:running services:__docker-compose_runningservices' && ret=0 + ;; + (start) + _arguments \ + $opts_help \ + '*:stopped services:__docker-compose_stoppedservices' && ret=0 + ;; + (stop|restart) + _arguments \ + $opts_help \ + $opts_timeout \ + '*:running services:__docker-compose_runningservices' && ret=0 + ;; + (top) + _arguments \ + $opts_help \ + '*:running services:__docker-compose_runningservices' && ret=0 + ;; + (unpause) + _arguments \ + $opts_help \ + '*:paused services:__docker-compose_pausedservices' && ret=0 + ;; + (up) + _arguments \ + $opts_help \ + '(--abort-on-container-exit)-d[Detached mode: Run containers in the background, print new container names. Incompatible with --abort-on-container-exit and --attach-dependencies.]' \ + $opts_no_color \ + $opts_no_deps \ + $opts_force_recreate \ + $opts_no_recreate \ + $opts_no_build \ + "(--no-build)--build[Build images before starting containers.]" \ + "(-d)--abort-on-container-exit[Stops all containers if any container was stopped. Incompatible with -d.]" \ + "(-d)--attach-dependencies[Attach to dependent containers. Incompatible with -d.]" \ + '(-t --timeout)'{-t,--timeout}"[Use this timeout in seconds for container shutdown when attached or when containers are already running. (default: 10)]:seconds: " \ + '--scale[SERVICE=NUM Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present.]:service scale SERVICE=NUM: ' \ + '--exit-code-from=[Return the exit code of the selected service container. Implies --abort-on-container-exit]:service:__docker-compose_services' \ + $opts_remove_orphans \ + '*:services:__docker-compose_services' && ret=0 + ;; + (version) + _arguments \ + $opts_help \ + "--short[Shows only Compose's version number.]" && ret=0 + ;; + (*) + _message 'Unknown sub command' && ret=1 + ;; + esac + + return ret +} + +_docker-compose() { + # Support for subservices, which allows for `compdef _docker docker-shell=_docker_containers`. + # Based on /usr/share/zsh/functions/Completion/Unix/_git without support for `ret`. + if [[ $service != docker-compose ]]; then + _call_function - _$service + return + fi + + local curcontext="$curcontext" state line + integer ret=1 + typeset -A opt_args + + local file_description + + if [[ -n ${words[(r)-f]} || -n ${words[(r)--file]} ]] ; then + file_description="Specify an override docker-compose file (default: docker-compose.override.yml)" + else + file_description="Specify an alternate docker-compose file (default: docker-compose.yml)" + fi + + _arguments -C \ + '(- :)'{-h,--help}'[Get help]' \ + '*'{-f,--file}"[${file_description}]:file:_files -g '*.yml'" \ + '(-p --project-name)'{-p,--project-name}'[Specify an alternate project name (default: directory name)]:project name:' \ + '--env-file[Specify an alternate environment file (default: .env)]:env-file:_files' \ + "--compatibility[If set, Compose will attempt to convert keys in v3 files to their non-Swarm equivalent]" \ + '(- :)'{-v,--version}'[Print version and exit]' \ + '--verbose[Show more output]' \ + '--log-level=[Set log level]:level:(DEBUG INFO WARNING ERROR CRITICAL)' \ + '--no-ansi[Do not print ANSI control characters]' \ + '(-H --host)'{-H,--host}'[Daemon socket to connect to]:host:' \ + '--tls[Use TLS; implied by --tlsverify]' \ + '--tlscacert=[Trust certs signed only by this CA]:ca path:' \ + '--tlscert=[Path to TLS certificate file]:client cert path:' \ + '--tlskey=[Path to TLS key file]:tls key path:' \ + '--tlsverify[Use TLS and verify the remote]' \ + "--skip-hostname-check[Don't check the daemon's hostname against the name specified in the client certificate (for example if your docker host is an IP address)]" \ + '(-): :->command' \ + '(-)*:: :->option-or-argument' && ret=0 + + local -a relevant_compose_flags relevant_compose_repeatable_flags relevant_docker_flags compose_options docker_options + + relevant_compose_flags=( + "--env-file" + "--file" "-f" + "--host" "-H" + "--project-name" "-p" + "--tls" + "--tlscacert" + "--tlscert" + "--tlskey" + "--tlsverify" + "--skip-hostname-check" + ) + + relevant_compose_repeatable_flags=( + "--file" "-f" + ) + + relevant_docker_flags=( + "--host" "-H" + "--tls" + "--tlscacert" + "--tlscert" + "--tlskey" + "--tlsverify" + ) + + for k in "${(@k)opt_args}"; do + if [[ -n "${relevant_docker_flags[(r)$k]}" ]]; then + docker_options+=$k + if [[ -n "$opt_args[$k]" ]]; then + docker_options+=$opt_args[$k] + fi + fi + if [[ -n "${relevant_compose_flags[(r)$k]}" ]]; then + if [[ -n "${relevant_compose_repeatable_flags[(r)$k]}" ]]; then + values=("${(@s/:/)opt_args[$k]}") + for value in $values + do + compose_options+=$k + compose_options+=$value + done + else + compose_options+=$k + if [[ -n "$opt_args[$k]" ]]; then + compose_options+=$opt_args[$k] + fi + fi + fi + done + + case $state in + (command) + __docker-compose_commands && ret=0 + ;; + (option-or-argument) + curcontext=${curcontext%:*:*}:docker-compose-$words[1]: + __docker-compose_subcommand && ret=0 + ;; + esac + + return ret +} + +_docker-compose "$@" diff --git a/contrib/migration/migrate-compose-file-v1-to-v2.py b/contrib/migration/migrate-compose-file-v1-to-v2.py new file mode 100755 index 00000000000..26511206c5f --- /dev/null +++ b/contrib/migration/migrate-compose-file-v1-to-v2.py @@ -0,0 +1,170 @@ +#!/usr/bin/env python +""" +Migrate a Compose file from the V1 format in Compose 1.5 to the V2 format +supported by Compose 1.6+ +""" +import argparse +import logging +import sys + +import ruamel.yaml + +from compose.config.types import VolumeSpec + + +log = logging.getLogger('migrate') + + +def migrate(content): + data = ruamel.yaml.load(content, ruamel.yaml.RoundTripLoader) + + service_names = data.keys() + + for name, service in data.items(): + warn_for_links(name, service) + warn_for_external_links(name, service) + rewrite_net(service, service_names) + rewrite_build(service) + rewrite_logging(service) + rewrite_volumes_from(service, service_names) + + services = {name: data.pop(name) for name in data.keys()} + + data['version'] = "2" + data['services'] = services + create_volumes_section(data) + + return data + + +def warn_for_links(name, service): + links = service.get('links') + if links: + example_service = links[0].partition(':')[0] + log.warning( + "Service {name} has links, which no longer create environment " + "variables such as {example_service_upper}_PORT. " + "If you are using those in your application code, you should " + "instead connect directly to the hostname, e.g. " + "'{example_service}'." + .format(name=name, example_service=example_service, + example_service_upper=example_service.upper())) + + +def warn_for_external_links(name, service): + external_links = service.get('external_links') + if external_links: + log.warning( + "Service {name} has external_links: {ext}, which now work " + "slightly differently. In particular, two containers must be " + "connected to at least one network in common in order to " + "communicate, even if explicitly linked together.\n\n" + "Either connect the external container to your app's default " + "network, or connect both the external container and your " + "service's containers to a pre-existing network. See " + "https://docs.docker.com/compose/networking/ " + "for more on how to do this." + .format(name=name, ext=external_links)) + + +def rewrite_net(service, service_names): + if 'net' in service: + network_mode = service.pop('net') + + # "container:<service name>" is now "service:<service name>" + if network_mode.startswith('container:'): + name = network_mode.partition(':')[2] + if name in service_names: + network_mode = 'service:{}'.format(name) + + service['network_mode'] = network_mode + + +def rewrite_build(service): + if 'dockerfile' in service: + service['build'] = { + 'context': service.pop('build'), + 'dockerfile': service.pop('dockerfile'), + } + + +def rewrite_logging(service): + if 'log_driver' in service: + service['logging'] = {'driver': service.pop('log_driver')} + if 'log_opt' in service: + service['logging']['options'] = service.pop('log_opt') + + +def rewrite_volumes_from(service, service_names): + for idx, volume_from in enumerate(service.get('volumes_from', [])): + if volume_from.split(':', 1)[0] not in service_names: + service['volumes_from'][idx] = 'container:%s' % volume_from + + +def create_volumes_section(data): + named_volumes = get_named_volumes(data['services']) + if named_volumes: + log.warning( + "Named volumes ({names}) must be explicitly declared. Creating a " + "'volumes' section with declarations.\n\n" + "For backwards-compatibility, they've been declared as external. " + "If you don't mind the volume names being prefixed with the " + "project name, you can remove the 'external' option from each one." + .format(names=', '.join(list(named_volumes)))) + + data['volumes'] = named_volumes + + +def get_named_volumes(services): + volume_specs = [ + VolumeSpec.parse(volume) + for service in services.values() + for volume in service.get('volumes', []) + ] + names = { + spec.external + for spec in volume_specs + if spec.is_named_volume + } + return {name: {'external': True} for name in names} + + +def write(stream, new_format, indent, width): + ruamel.yaml.dump( + new_format, + stream, + Dumper=ruamel.yaml.RoundTripDumper, + indent=indent, + width=width) + + +def parse_opts(args): + parser = argparse.ArgumentParser() + parser.add_argument("filename", help="Compose file filename.") + parser.add_argument("-i", "--in-place", action='store_true') + parser.add_argument( + "--indent", type=int, default=2, + help="Number of spaces used to indent the output yaml.") + parser.add_argument( + "--width", type=int, default=80, + help="Number of spaces used as the output width.") + return parser.parse_args() + + +def main(args): + logging.basicConfig(format='\033[33m%(levelname)s:\033[37m %(message)s\033[0m\n') + + opts = parse_opts(args) + + with open(opts.filename) as fh: + new_format = migrate(fh.read()) + + if opts.in_place: + output = open(opts.filename, 'w') + else: + output = sys.stdout + write(output, new_format, opts.indent, opts.width) + + +if __name__ == "__main__": + main(sys.argv) diff --git a/contrib/update/update-docker-compose.ps1 b/contrib/update/update-docker-compose.ps1 new file mode 100644 index 00000000000..bb033b46467 --- /dev/null +++ b/contrib/update/update-docker-compose.ps1 @@ -0,0 +1,116 @@ +# Self-elevate the script if required +# http://www.expta.com/2017/03/how-to-self-elevate-powershell-script.html +If (-Not ([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] 'Administrator')) { + If ([int](Get-CimInstance -Class Win32_OperatingSystem | Select-Object -ExpandProperty BuildNumber) -ge 6000) { + $CommandLine = "-File `"" + $MyInvocation.MyCommand.Path + "`" " + $MyInvocation.UnboundArguments + Start-Process -FilePath PowerShell.exe -Verb Runas -ArgumentList $CommandLine + Exit + } +} + +$SectionSeparator = "--------------------------------------------------" + +# Update docker-compose if required +Function UpdateDockerCompose() { + Write-Host "Updating docker-compose if required..." + Write-Host $SectionSeparator + + # Find the installed docker-compose.exe location + Try { + $DockerComposePath = Get-Command docker-compose.exe -ErrorAction Stop | ` + Select-Object -First 1 -ExpandProperty Definition + } + Catch { + Write-Host "Error: Could not find path to docker-compose.exe" ` + -ForegroundColor Red + Return $false + } + + # Prefer/enable TLS 1.2 + # https://stackoverflow.com/a/48030563/153079 + [Net.ServicePointManager]::SecurityProtocol = "tls12, tls11, tls" + + # Query for the latest release version + Try { + $URI = "https://api.github.com/repos/docker/compose/releases/latest" + $LatestComposeVersion = [System.Version](Invoke-RestMethod -Method Get -Uri $URI).tag_name + } + Catch { + Write-Host "Error: Query for the latest docker-compose release version failed" ` + -ForegroundColor Red + Return $false + } + + # Check the installed version and compare with latest release + $UpdateDockerCompose = $false + Try { + $InstalledComposeVersion = ` + [System.Version]((docker-compose.exe version --short) | Out-String) + + If ($InstalledComposeVersion -eq $LatestComposeVersion) { + Write-Host ("Installed docker-compose version ({0}) same as latest ({1})." ` + -f $InstalledComposeVersion.ToString(), $LatestComposeVersion.ToString()) + } + ElseIf ($InstalledComposeVersion -lt $LatestComposeVersion) { + Write-Host ("Installed docker-compose version ({0}) older than latest ({1})." ` + -f $InstalledComposeVersion.ToString(), $LatestComposeVersion.ToString()) + $UpdateDockerCompose = $true + } + Else { + Write-Host ("Installed docker-compose version ({0}) newer than latest ({1})." ` + -f $InstalledComposeVersion.ToString(), $LatestComposeVersion.ToString()) ` + -ForegroundColor Yellow + } + } + Catch { + Write-Host ` + "Warning: Couldn't get docker-compose version, assuming an update is required..." ` + -ForegroundColor Yellow + $UpdateDockerCompose = $true + } + + If (-Not $UpdateDockerCompose) { + # Nothing to do! + Return $false + } + + # Download the latest version of docker-compose.exe + Try { + $RemoteFileName = "docker-compose-Windows-x86_64.exe" + $URI = ("https://github.com/docker/compose/releases/download/{0}/{1}" ` + -f $LatestComposeVersion.ToString(), $RemoteFileName) + Invoke-WebRequest -UseBasicParsing -Uri $URI ` + -OutFile $DockerComposePath + Return $true + } + Catch { + Write-Host ("Error: Failed to download the latest version of docker-compose`n{0}" ` + -f $_.Exception.Message) -ForegroundColor Red + Return $false + } + + Return $false +} + +If (UpdateDockerCompose) { + Write-Host "Updated to latest-version of docker-compose, running update again to verify.`n" + If (UpdateDockerCompose) { + Write-Host "Error: Should not have updated twice." -ForegroundColor Red + } +} + +# Assuming elevation popped up a new powershell window, pause so the user can see what happened +# https://stackoverflow.com/a/22362868/153079 +Function Pause ($Message = "Press any key to continue . . . ") { + If ((Test-Path variable:psISE) -and $psISE) { + $Shell = New-Object -ComObject "WScript.Shell" + $Shell.Popup("Click OK to continue.", 0, "Script Paused", 0) + } + Else { + Write-Host "`n$SectionSeparator" + Write-Host -NoNewline $Message + [void][System.Console]::ReadKey($true) + Write-Host + } +} +Pause diff --git a/docker-bake.hcl b/docker-bake.hcl deleted file mode 100644 index 5c6522d3a81..00000000000 --- a/docker-bake.hcl +++ /dev/null @@ -1,148 +0,0 @@ -// Copyright 2022 Docker Compose CLI authors -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -variable "GO_VERSION" { - # default ARG value set in Dockerfile - default = null -} - -variable "BUILD_TAGS" { - default = "e2e" -} - -variable "DOCS_FORMATS" { - default = "md,yaml" -} - -# Defines the output folder to override the default behavior. -# See Makefile for details, this is generally only useful for -# the packaging scripts and care should be taken to not break -# them. -variable "DESTDIR" { - default = "" -} -function "outdir" { - params = [defaultdir] - result = DESTDIR != "" ? DESTDIR : "${defaultdir}" -} - -# Special target: https://github.com/docker/metadata-action#bake-definition -target "meta-helper" {} - -target "_common" { - args = { - GO_VERSION = GO_VERSION - BUILD_TAGS = BUILD_TAGS - BUILDKIT_CONTEXT_KEEP_GIT_DIR = 1 - } -} - -group "default" { - targets = ["binary"] -} - -group "validate" { - targets = ["lint", "vendor-validate", "license-validate"] -} - -target "lint" { - inherits = ["_common"] - target = "lint" - output = ["type=cacheonly"] -} - -target "license-validate" { - target = "license-validate" - output = ["type=cacheonly"] -} - -target "license-update" { - target = "license-update" - output = ["."] -} - -target "vendor-validate" { - inherits = ["_common"] - target = "vendor-validate" - output = ["type=cacheonly"] -} - -target "vendor-update" { - inherits = ["_common"] - target = "vendor-update" - output = ["."] -} - -target "test" { - inherits = ["_common"] - target = "test-coverage" - output = [outdir("./bin/coverage/unit")] -} - -target "binary-with-coverage" { - inherits = ["_common"] - target = "binary" - args = { - BUILD_FLAGS = "-cover -covermode=atomic" - } - output = [outdir("./bin/build")] - platforms = ["local"] -} - -target "binary" { - inherits = ["_common"] - target = "binary" - output = [outdir("./bin/build")] - platforms = ["local"] -} - -target "binary-cross" { - inherits = ["binary"] - platforms = [ - "darwin/amd64", - "darwin/arm64", - "linux/amd64", - "linux/arm/v6", - "linux/arm/v7", - "linux/arm64", - "linux/ppc64le", - "linux/riscv64", - "linux/s390x", - "windows/amd64", - "windows/arm64" - ] -} - -target "release" { - inherits = ["binary-cross"] - target = "release" - output = [outdir("./bin/release")] -} - -target "docs-validate" { - inherits = ["_common"] - target = "docs-validate" - output = ["type=cacheonly"] -} - -target "docs-update" { - inherits = ["_common"] - target = "docs-update" - output = ["./docs"] -} - -target "image-cross" { - inherits = ["meta-helper", "binary-cross"] - output = ["type=image"] -} diff --git a/docker-compose-entrypoint.sh b/docker-compose-entrypoint.sh new file mode 100755 index 00000000000..84436fa0778 --- /dev/null +++ b/docker-compose-entrypoint.sh @@ -0,0 +1,20 @@ +#!/bin/sh +set -e + +# first arg is `-f` or `--some-option` +if [ "${1#-}" != "$1" ]; then + set -- docker-compose "$@" +fi + +# if our command is a valid Docker subcommand, let's invoke it through Docker instead +# (this allows for "docker run docker ps", etc) +if docker-compose help "$1" > /dev/null 2>&1; then + set -- docker-compose "$@" +fi + +# if we have "--link some-docker:docker" and not DOCKER_HOST, let's set DOCKER_HOST automatically +if [ -z "$DOCKER_HOST" -a "$DOCKER_PORT_2375_TCP" ]; then + export DOCKER_HOST='tcp://docker:2375' +fi + +exec "$@" diff --git a/docker-compose.spec b/docker-compose.spec new file mode 100644 index 00000000000..0c2fa3dec9f --- /dev/null +++ b/docker-compose.spec @@ -0,0 +1,42 @@ +# -*- mode: python -*- + +block_cipher = None + +a = Analysis(['bin/docker-compose'], + pathex=['.'], + hiddenimports=[], + hookspath=None, + runtime_hooks=None, + cipher=block_cipher) + +pyz = PYZ(a.pure, cipher=block_cipher) + +exe = EXE(pyz, + a.scripts, + a.binaries, + a.zipfiles, + a.datas, + [ + ( + 'compose/config/config_schema_v1.json', + 'compose/config/config_schema_v1.json', + 'DATA' + ), + ( + 'compose/config/compose_spec.json', + 'compose/config/compose_spec.json', + 'DATA' + ), + ( + 'compose/GITSHA', + 'compose/GITSHA', + 'DATA' + ) + ], + + name='docker-compose', + debug=False, + strip=None, + upx=True, + console=True, + bootloader_ignore_signals=True) diff --git a/docker-compose_darwin.spec b/docker-compose_darwin.spec new file mode 100644 index 00000000000..24889475990 --- /dev/null +++ b/docker-compose_darwin.spec @@ -0,0 +1,48 @@ +# -*- mode: python -*- + +block_cipher = None + +a = Analysis(['bin/docker-compose'], + pathex=['.'], + hiddenimports=[], + hookspath=[], + runtime_hooks=[], + cipher=block_cipher) + +pyz = PYZ(a.pure, a.zipped_data, + cipher=block_cipher) + +exe = EXE(pyz, + a.scripts, + exclude_binaries=True, + name='docker-compose', + debug=False, + strip=False, + upx=True, + console=True, + bootloader_ignore_signals=True) +coll = COLLECT(exe, + a.binaries, + a.zipfiles, + a.datas, + [ + ( + 'compose/config/config_schema_v1.json', + 'compose/config/config_schema_v1.json', + 'DATA' + ), + ( + 'compose/config/compose_spec.json', + 'compose/config/compose_spec.json', + 'DATA' + ), + ( + 'compose/GITSHA', + 'compose/GITSHA', + 'DATA' + ) + ], + strip=False, + upx=True, + upx_exclude=[], + name='docker-compose-Darwin-x86_64') diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000000..accc7c23e42 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,14 @@ +# The docs have been moved! + +The documentation for Compose has been merged into +[the general documentation repo](https://github.com/docker/docker.github.io). + +The docs for Compose are now here: +https://github.com/docker/docker.github.io/tree/master/compose + +Please submit pull requests for unreleased features/changes on the `master` branch (https://github.com/docker/docker.github.io/tree/master), please prefix the PR title with `[WIP]` to indicate that it relates to an unreleased change. + +If you submit a PR to this codebase that has a docs impact, create a second docs PR on `docker.github.io`. Use the docs PR template provided. + +As always, the docs remain open-source and we appreciate your feedback and +pull requests! diff --git a/docs/examples/provider.go b/docs/examples/provider.go deleted file mode 100644 index 79fd3256eed..00000000000 --- a/docs/examples/provider.go +++ /dev/null @@ -1,152 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package main - -import ( - "encoding/json" - "fmt" - "os" - "time" - - "github.com/spf13/cobra" - "github.com/spf13/pflag" -) - -func main() { - cmd := &cobra.Command{ - Short: "Compose Provider Example", - Use: "demo", - } - cmd.AddCommand(composeCommand()) - err := cmd.Execute() - if err != nil { - fmt.Fprintln(os.Stderr, err) - os.Exit(1) - } -} - -type options struct { - db string - size int -} - -func composeCommand() *cobra.Command { - c := &cobra.Command{ - Use: "compose EVENT", - TraverseChildren: true, - } - c.PersistentFlags().String("project-name", "", "compose project name") // unused - - var options options - upCmd := &cobra.Command{ - Use: "up", - Run: func(_ *cobra.Command, args []string) { - up(options, args) - }, - Args: cobra.ExactArgs(1), - } - - upCmd.Flags().StringVar(&options.db, "type", "", "Database type (mysql, postgres, etc.)") - _ = upCmd.MarkFlagRequired("type") - upCmd.Flags().IntVar(&options.size, "size", 10, "Database size in GB") - upCmd.Flags().String("name", "", "Name of the database to be created") - _ = upCmd.MarkFlagRequired("name") - - downCmd := &cobra.Command{ - Use: "down", - Run: down, - Args: cobra.ExactArgs(1), - } - downCmd.Flags().String("name", "", "Name of the database to be deleted") - _ = downCmd.MarkFlagRequired("name") - - c.AddCommand(upCmd, downCmd) - c.AddCommand(metadataCommand(upCmd, downCmd)) - return c -} - -const lineSeparator = "\n" - -func up(options options, args []string) { - servicename := args[0] - fmt.Printf(`{ "type": "debug", "message": "Starting %s" }%s`, servicename, lineSeparator) - - for i := 0; i < options.size; i += 10 { - time.Sleep(1 * time.Second) - fmt.Printf(`{ "type": "info", "message": "Processing ... %d%%" }%s`, i*100/options.size, lineSeparator) - } - fmt.Printf(`{ "type": "setenv", "message": "URL=https://magic.cloud/%s" }%s`, servicename, lineSeparator) -} - -func down(_ *cobra.Command, _ []string) { - fmt.Printf(`{ "type": "error", "message": "Permission error" }%s`, lineSeparator) -} - -func metadataCommand(upCmd, downCmd *cobra.Command) *cobra.Command { - return &cobra.Command{ - Use: "metadata", - Run: func(cmd *cobra.Command, _ []string) { - metadata(upCmd, downCmd) - }, - Args: cobra.NoArgs, - } -} - -func metadata(upCmd, downCmd *cobra.Command) { - metadata := ProviderMetadata{} - metadata.Description = "Manage services on AwesomeCloud" - metadata.Up = commandParameters(upCmd) - metadata.Down = commandParameters(downCmd) - jsonMetadata, err := json.Marshal(metadata) - if err != nil { - panic(err) - } - fmt.Println(string(jsonMetadata)) -} - -func commandParameters(cmd *cobra.Command) CommandMetadata { - cmdMetadata := CommandMetadata{} - cmd.Flags().VisitAll(func(f *pflag.Flag) { - _, isRequired := f.Annotations[cobra.BashCompOneRequiredFlag] - cmdMetadata.Parameters = append(cmdMetadata.Parameters, Metadata{ - Name: f.Name, - Description: f.Usage, - Required: isRequired, - Type: f.Value.Type(), - Default: f.DefValue, - }) - }) - return cmdMetadata -} - -type ProviderMetadata struct { - Description string `json:"description"` - Up CommandMetadata `json:"up"` - Down CommandMetadata `json:"down"` -} - -type CommandMetadata struct { - Parameters []Metadata `json:"parameters"` -} - -type Metadata struct { - Name string `json:"name"` - Description string `json:"description"` - Required bool `json:"required"` - Type string `json:"type"` - Default string `json:"default,omitempty"` -} diff --git a/docs/extension.md b/docs/extension.md deleted file mode 100644 index 3682f15655c..00000000000 --- a/docs/extension.md +++ /dev/null @@ -1,176 +0,0 @@ -# About - -The Compose application model defines `service` as an abstraction for a computing unit managing (a subset of) -application needs, which can interact with other services by relying on network(s). Docker Compose is designed -to use the Docker Engine ("Moby") API to manage services as containers, but the abstraction _could_ also cover -many other runtimes, typically cloud services or services natively provided by host. - -The Compose extensibility model has been designed to extend the `service` support to runtimes accessible through -third-party tooling. - -# Architecture - -Compose extensibility relies on the `provider` attribute to select the actual binary responsible for managing -the resource(s) needed to run a service. - -```yaml - database: - provider: - type: awesomecloud - options: - type: mysql - size: 256 - name: myAwesomeCloudDB -``` - -`provider.type` tells Compose the binary to run, which can be either: -- Another Docker CLI plugin (typically, `model` to run `docker-model`) -- An executable in user's `PATH` - -If `provider.type` doesn't resolve into any of those, Compose will report an error and interrupt the `up` command. - -To be a valid Compose extension, provider command *MUST* accept a `compose` command (which can be hidden) -with subcommands `up` and `down`. - -## Up lifecycle - -To execute an application's `up` lifecycle, Compose executes the provider's `compose up` command, passing -the project name, service name, and additional options. The `provider.options` are translated -into command line flags. For example: -```console -awesomecloud compose --project-name <NAME> up --type=mysql --size=256 "database" -``` - -> __Note:__ `project-name` _should_ be used by the provider to tag resources -> set for project, so that later execution with `down` subcommand releases -> all allocated resources set for the project. - -## Communication with Compose - -Providers can interact with Compose using `stdout` as a channel, sending JSON line delimited messages. -JSON messages MUST include a `type` and a `message` attribute. -```json -{ "type": "info", "message": "preparing mysql ..." } -``` - -`type` can be either: -- `info`: Reports status updates to the user. Compose will render message as the service state in the progress UI -- `error`: Lets the user know something went wrong with details about the error. Compose will render the message as the reason for the service failure. -- `setenv`: Lets the plugin tell Compose how dependent services can access the created resource. See next section for further details. -- `debug`: Those messages could help debugging the provider, but are not rendered to the user by default. They are rendered when Compose is started with `--verbose` flag. - -```mermaid -sequenceDiagram - Shell->>Compose: docker compose up - Compose->>Provider: compose up --project-name=xx --foo=bar "database" - Provider--)Compose: json { "info": "pulling 25%" } - Compose-)Shell: pulling 25% - Provider--)Compose: json { "info": "pulling 50%" } - Compose-)Shell: pulling 50% - Provider--)Compose: json { "info": "pulling 75%" } - Compose-)Shell: pulling 75% - Provider--)Compose: json { "setenv": "URL=http://cloud.com/abcd:1234" } - Compose-)Compose: set DATABASE_URL - Provider-)Compose: EOF (command complete) exit 0 - Compose-)Shell: service started -``` - -## Connection to a service managed by a provider - -A service in the Compose application can declare dependency on a service managed by an external provider: - -```yaml -services: - app: - image: myapp - depends_on: - - database - - database: - provider: - type: awesomecloud -``` - -When the provider command sends a `setenv` JSON message, Compose injects the specified variable into any dependent service, -automatically prefixing it with the service name. For example, if `awesomecloud compose up` returns: -```json -{"type": "setenv", "message": "URL=https://awesomecloud.com/db:1234"} -``` -Then the `app` service, which depends on the service managed by the provider, will receive a `DATABASE_URL` environment variable injected -into its runtime environment. - -> __Note:__ The `compose up` provider command _MUST_ be idempotent. If resource is already running, the command _MUST_ set -> the same environment variables to ensure consistent configuration of dependent services. - -## Down lifecycle - -`down` lifecycle is equivalent to `up` with the `<provider> compose --project-name <NAME> down <SERVICE>` command. -The provider is responsible for releasing all resources associated with the service. - -## Provide metadata about options - -Compose extensions *MAY* optionally implement a `metadata` subcommand to provide information about the parameters accepted by the `up` and `down` commands. - -The `metadata` subcommand takes no parameters and returns a JSON structure on the `stdout` channel that describes the parameters accepted by both the `up` and `down` commands, including whether each parameter is mandatory or optional. - -```console -awesomecloud compose metadata -``` - -The expected JSON output format is: -```json -{ - "description": "Manage services on AwesomeCloud", - "up": { - "parameters": [ - { - "name": "type", - "description": "Database type (mysql, postgres, etc.)", - "required": true, - "type": "string" - }, - { - "name": "size", - "description": "Database size in GB", - "required": false, - "type": "integer", - "default": "10" - }, - { - "name": "name", - "description": "Name of the database to be created", - "required": true, - "type": "string" - } - ] - }, - "down": { - "parameters": [ - { - "name": "name", - "description": "Name of the database to be removed", - "required": true, - "type": "string" - } - ] - } -} -``` -The top elements are: -- `description`: Human-readable description of the provider -- `up`: Object describing the parameters accepted by the `up` command -- `down`: Object describing the parameters accepted by the `down` command - -And for each command parameter, you should include the following properties: -- `name`: The parameter name (without `--` prefix) -- `description`: Human-readable description of the parameter -- `required`: Boolean indicating if the parameter is mandatory -- `type`: Parameter type (`string`, `integer`, `boolean`, etc.) -- `default`: Default value (optional, only for non-required parameters) -- `enum`: List of possible values supported by the parameter separated by `,` (optional, only for parameters with a limited set of values) - -This metadata allows Compose and other tools to understand the provider's interface and provide better user experience, such as validation, auto-completion, and documentation generation. - -## Examples - -See [example](examples/provider.go) for illustration on implementing this API in a command line diff --git a/docs/issue_template.md b/docs/issue_template.md new file mode 100644 index 00000000000..774f27e207f --- /dev/null +++ b/docs/issue_template.md @@ -0,0 +1,50 @@ + + +## Description of the issue + +## Context information (for bug reports) + +``` +Output of "docker-compose version" +``` + +``` +Output of "docker version" +``` + +``` +Output of "docker-compose config" +``` + + +## Steps to reproduce the issue + +1. +2. +3. + +### Observed result + +### Expected result + +### Stacktrace / full error message + +``` +(if applicable) +``` + +## Additional information + +OS version / distribution, `docker-compose` install method, etc. diff --git a/docs/pull_request_template.md b/docs/pull_request_template.md new file mode 100644 index 00000000000..15526af0556 --- /dev/null +++ b/docs/pull_request_template.md @@ -0,0 +1,13 @@ + + + +Resolves # diff --git a/docs/reference/compose.md b/docs/reference/compose.md deleted file mode 100644 index d80bb86ec62..00000000000 --- a/docs/reference/compose.md +++ /dev/null @@ -1,264 +0,0 @@ - -# docker compose - -```text -docker compose [-f <arg>...] [options] [COMMAND] [ARGS...] -``` - - -Define and run multi-container applications with Docker - -### Subcommands - -| Name | Description | -|:--------------------------------|:----------------------------------------------------------------------------------------| -| [`attach`](compose_attach.md) | Attach local standard input, output, and error streams to a service's running container | -| [`bridge`](compose_bridge.md) | Convert compose files into another model | -| [`build`](compose_build.md) | Build or rebuild services | -| [`commit`](compose_commit.md) | Create a new image from a service container's changes | -| [`config`](compose_config.md) | Parse, resolve and render compose file in canonical format | -| [`cp`](compose_cp.md) | Copy files/folders between a service container and the local filesystem | -| [`create`](compose_create.md) | Creates containers for a service | -| [`down`](compose_down.md) | Stop and remove containers, networks | -| [`events`](compose_events.md) | Receive real time events from containers | -| [`exec`](compose_exec.md) | Execute a command in a running container | -| [`export`](compose_export.md) | Export a service container's filesystem as a tar archive | -| [`images`](compose_images.md) | List images used by the created containers | -| [`kill`](compose_kill.md) | Force stop service containers | -| [`logs`](compose_logs.md) | View output from containers | -| [`ls`](compose_ls.md) | List running compose projects | -| [`pause`](compose_pause.md) | Pause services | -| [`port`](compose_port.md) | Print the public port for a port binding | -| [`ps`](compose_ps.md) | List containers | -| [`publish`](compose_publish.md) | Publish compose application | -| [`pull`](compose_pull.md) | Pull service images | -| [`push`](compose_push.md) | Push service images | -| [`restart`](compose_restart.md) | Restart service containers | -| [`rm`](compose_rm.md) | Removes stopped service containers | -| [`run`](compose_run.md) | Run a one-off command on a service | -| [`scale`](compose_scale.md) | Scale services | -| [`start`](compose_start.md) | Start services | -| [`stats`](compose_stats.md) | Display a live stream of container(s) resource usage statistics | -| [`stop`](compose_stop.md) | Stop services | -| [`top`](compose_top.md) | Display the running processes | -| [`unpause`](compose_unpause.md) | Unpause services | -| [`up`](compose_up.md) | Create and start containers | -| [`version`](compose_version.md) | Show the Docker Compose version information | -| [`volumes`](compose_volumes.md) | List volumes | -| [`wait`](compose_wait.md) | Block until containers of all (or specified) services stop. | -| [`watch`](compose_watch.md) | Watch build context for service and rebuild/refresh containers when files are updated | - - -### Options - -| Name | Type | Default | Description | -|:-----------------------|:--------------|:--------|:----------------------------------------------------------------------------------------------------| -| `--all-resources` | `bool` | | Include all resources, even those not used by services | -| `--ansi` | `string` | `auto` | Control when to print ANSI control characters ("never"\|"always"\|"auto") | -| `--compatibility` | `bool` | | Run compose in backward compatibility mode | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--env-file` | `stringArray` | | Specify an alternate environment file | -| `-f`, `--file` | `stringArray` | | Compose configuration files | -| `--parallel` | `int` | `-1` | Control max parallelism, -1 for unlimited | -| `--profile` | `stringArray` | | Specify a profile to enable | -| `--progress` | `string` | | Set type of progress output (auto, tty, plain, json, quiet) | -| `--project-directory` | `string` | | Specify an alternate working directory (default: the path of the, first specified, Compose file) | -| `-p`, `--project-name` | `string` | | Project name | - - - - -## Examples - -### Use `-f` to specify the name and path of one or more Compose files -Use the `-f` flag to specify the location of a Compose [configuration file](/reference/compose-file/). - -#### Specifying multiple Compose files -You can supply multiple `-f` configuration files. When you supply multiple files, Compose combines them into a single -configuration. Compose builds the configuration in the order you supply the files. Subsequent files override and add -to their predecessors. - -For example, consider this command line: - -```console -$ docker compose -f compose.yaml -f compose.admin.yaml run backup_db -``` - -The `compose.yaml` file might specify a `webapp` service. - -```yaml -services: - webapp: - image: examples/web - ports: - - "8000:8000" - volumes: - - "/data" -``` -If the `compose.admin.yaml` also specifies this same service, any matching fields override the previous file. -New values, add to the `webapp` service configuration. - -```yaml -services: - webapp: - build: . - environment: - - DEBUG=1 -``` - -When you use multiple Compose files, all paths in the files are relative to the first configuration file specified -with `-f`. You can use the `--project-directory` option to override this base path. - -Use a `-f` with `-` (dash) as the filename to read the configuration from stdin. When stdin is used all paths in the -configuration are relative to the current working directory. - -The `-f` flag is optional. If you don’t provide this flag on the command line, Compose traverses the working directory -and its parent directories looking for a `compose.yaml` or `docker-compose.yaml` file. - -#### Specifying a path to a single Compose file -You can use the `-f` flag to specify a path to a Compose file that is not located in the current directory, either -from the command line or by setting up a `COMPOSE_FILE` environment variable in your shell or in an environment file. - -For an example of using the `-f` option at the command line, suppose you are running the Compose Rails sample, and -have a `compose.yaml` file in a directory called `sandbox/rails`. You can use a command like `docker compose pull` to -get the postgres image for the db service from anywhere by using the `-f` flag as follows: - -```console -$ docker compose -f ~/sandbox/rails/compose.yaml pull db -``` - -#### Using an OCI published artifact -You can use the `-f` flag with the `oci://` prefix to reference a Compose file that has been published to an OCI registry. -This allows you to distribute and version your Compose configurations as OCI artifacts. - -To use a Compose file from an OCI registry: - -```console -$ docker compose -f oci://registry.example.com/my-compose-project:latest up -``` - -You can also combine OCI artifacts with local files: - -```console -$ docker compose -f oci://registry.example.com/my-compose-project:v1.0 -f compose.override.yaml up -``` - -The OCI artifact must contain a valid Compose file. You can publish Compose files to an OCI registry using the -`docker compose publish` command. - -#### Using a git repository -You can use the `-f` flag to reference a Compose file from a git repository. Compose supports various git URL formats: - -Using HTTPS: -```console -$ docker compose -f https://github.com/user/repo.git up -``` - -Using SSH: -```console -$ docker compose -f git@github.com:user/repo.git up -``` - -You can specify a specific branch, tag, or commit: -```console -$ docker compose -f https://github.com/user/repo.git@main up -$ docker compose -f https://github.com/user/repo.git@v1.0.0 up -$ docker compose -f https://github.com/user/repo.git@abc123 up -``` - -You can also specify a subdirectory within the repository: -```console -$ docker compose -f https://github.com/user/repo.git#main:path/to/compose.yaml up -``` - -When using git resources, Compose will clone the repository and use the specified Compose file. You can combine -git resources with local files: - -```console -$ docker compose -f https://github.com/user/repo.git -f compose.override.yaml up -``` - -### Use `-p` to specify a project name - -Each configuration has a project name. Compose sets the project name using -the following mechanisms, in order of precedence: -- The `-p` command line flag -- The `COMPOSE_PROJECT_NAME` environment variable -- The top level `name:` variable from the config file (or the last `name:` -from a series of config files specified using `-f`) -- The `basename` of the project directory containing the config file (or -containing the first config file specified using `-f`) -- The `basename` of the current directory if no config file is specified -Project names must contain only lowercase letters, decimal digits, dashes, -and underscores, and must begin with a lowercase letter or decimal digit. If -the `basename` of the project directory or current directory violates this -constraint, you must use one of the other mechanisms. - -```console -$ docker compose -p my_project ps -a -NAME SERVICE STATUS PORTS -my_project_demo_1 demo running - -$ docker compose -p my_project logs -demo_1 | PING localhost (127.0.0.1): 56 data bytes -demo_1 | 64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.095 ms -``` - -### Use profiles to enable optional services - -Use `--profile` to specify one or more active profiles -Calling `docker compose --profile frontend up` starts the services with the profile `frontend` and services -without any specified profiles. -You can also enable multiple profiles, e.g. with `docker compose --profile frontend --profile debug up` the profiles `frontend` and `debug` is enabled. - -Profiles can also be set by `COMPOSE_PROFILES` environment variable. - -### Configuring parallelism - -Use `--parallel` to specify the maximum level of parallelism for concurrent engine calls. -Calling `docker compose --parallel 1 pull` pulls the pullable images defined in the Compose file -one at a time. This can also be used to control build concurrency. - -Parallelism can also be set by the `COMPOSE_PARALLEL_LIMIT` environment variable. - -### Set up environment variables - -You can set environment variables for various docker compose options, including the `-f`, `-p` and `--profiles` flags. - -Setting the `COMPOSE_FILE` environment variable is equivalent to passing the `-f` flag, -`COMPOSE_PROJECT_NAME` environment variable does the same as the `-p` flag, -`COMPOSE_PROFILES` environment variable is equivalent to the `--profiles` flag -and `COMPOSE_PARALLEL_LIMIT` does the same as the `--parallel` flag. - -If flags are explicitly set on the command line, the associated environment variable is ignored. - -Setting the `COMPOSE_IGNORE_ORPHANS` environment variable to `true` stops docker compose from detecting orphaned -containers for the project. - -Setting the `COMPOSE_MENU` environment variable to `false` disables the helper menu when running `docker compose up` -in attached mode. Alternatively, you can also run `docker compose up --menu=false` to disable the helper menu. - -### Use Dry Run mode to test your command - -Use `--dry-run` flag to test a command without changing your application stack state. -Dry Run mode shows you all the steps Compose applies when executing a command, for example: -```console -$ docker compose --dry-run up --build -d -[+] Pulling 1/1 - ✔ DRY-RUN MODE - db Pulled 0.9s -[+] Running 10/8 - ✔ DRY-RUN MODE - build service backend 0.0s - ✔ DRY-RUN MODE - ==> ==> writing image dryRun-754a08ddf8bcb1cf22f310f09206dd783d42f7dd 0.0s - ✔ DRY-RUN MODE - ==> ==> naming to nginx-golang-mysql-backend 0.0s - ✔ DRY-RUN MODE - Network nginx-golang-mysql_default Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-db-1 Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-backend-1 Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-proxy-1 Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-db-1 Healthy 0.5s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-backend-1 Started 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-proxy-1 Started Started -``` -From the example above, you can see that the first step is to pull the image defined by `db` service, then build the `backend` service. -Next, the containers are created. The `db` service is started, and the `backend` and `proxy` wait until the `db` service is healthy before starting. - -Dry Run mode works with almost all commands. You cannot use Dry Run mode with a command that doesn't change the state of a Compose stack such as `ps`, `ls`, `logs` for example. diff --git a/docs/reference/compose_alpha.md b/docs/reference/compose_alpha.md deleted file mode 100644 index 34485d7deff..00000000000 --- a/docs/reference/compose_alpha.md +++ /dev/null @@ -1,22 +0,0 @@ -# docker compose alpha - - -Experimental commands - -### Subcommands - -| Name | Description | -|:----------------------------------|:-----------------------------------------------------------------------------------------------------| -| [`viz`](compose_alpha_viz.md) | EXPERIMENTAL - Generate a graphviz graph from your compose file | -| [`watch`](compose_alpha_watch.md) | EXPERIMENTAL - Watch build context for service and rebuild/refresh containers when files are updated | - - -### Options - -| Name | Type | Default | Description | -|:------------|:-----|:--------|:--------------------------------| -| `--dry-run` | | | Execute command in dry run mode | - - - - diff --git a/docs/reference/compose_alpha_dry-run.md b/docs/reference/compose_alpha_dry-run.md deleted file mode 100644 index 7c68d94d66b..00000000000 --- a/docs/reference/compose_alpha_dry-run.md +++ /dev/null @@ -1,8 +0,0 @@ -# docker compose alpha dry-run - - -Dry run command allows you to test a command without applying changes - - - - diff --git a/docs/reference/compose_alpha_generate.md b/docs/reference/compose_alpha_generate.md deleted file mode 100644 index f4054627798..00000000000 --- a/docs/reference/compose_alpha_generate.md +++ /dev/null @@ -1,17 +0,0 @@ -# docker compose alpha generate - - -EXPERIMENTAL - Generate a Compose file from existing containers - -### Options - -| Name | Type | Default | Description | -|:----------------|:---------|:--------|:------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--format` | `string` | `yaml` | Format the output. Values: [yaml \| json] | -| `--name` | `string` | | Project name to set in the Compose file | -| `--project-dir` | `string` | | Directory to use for the project | - - - - diff --git a/docs/reference/compose_alpha_publish.md b/docs/reference/compose_alpha_publish.md deleted file mode 100644 index 6e77d714532..00000000000 --- a/docs/reference/compose_alpha_publish.md +++ /dev/null @@ -1,18 +0,0 @@ -# docker compose alpha publish - - -Publish compose application - -### Options - -| Name | Type | Default | Description | -|:--------------------------|:---------|:--------|:-------------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--oci-version` | `string` | | OCI image/artifact specification version (automatically determined by default) | -| `--resolve-image-digests` | `bool` | | Pin image tags to digests | -| `--with-env` | `bool` | | Include environment variables in the published OCI artifact | -| `-y`, `--yes` | `bool` | | Assume "yes" as answer to all prompts | - - - - diff --git a/docs/reference/compose_alpha_scale.md b/docs/reference/compose_alpha_scale.md deleted file mode 100644 index f783f3335c7..00000000000 --- a/docs/reference/compose_alpha_scale.md +++ /dev/null @@ -1,15 +0,0 @@ -# docker compose alpha scale - - -Scale services - -### Options - -| Name | Type | Default | Description | -|:------------|:-----|:--------|:--------------------------------| -| `--dry-run` | | | Execute command in dry run mode | -| `--no-deps` | | | Don't start linked services | - - - - diff --git a/docs/reference/compose_alpha_viz.md b/docs/reference/compose_alpha_viz.md deleted file mode 100644 index 1a05aaac14d..00000000000 --- a/docs/reference/compose_alpha_viz.md +++ /dev/null @@ -1,19 +0,0 @@ -# docker compose alpha viz - - -EXPERIMENTAL - Generate a graphviz graph from your compose file - -### Options - -| Name | Type | Default | Description | -|:---------------------|:-------|:--------|:---------------------------------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--image` | `bool` | | Include service's image name in output graph | -| `--indentation-size` | `int` | `1` | Number of tabs or spaces to use for indentation | -| `--networks` | `bool` | | Include service's attached networks in output graph | -| `--ports` | `bool` | | Include service's exposed ports in output graph | -| `--spaces` | `bool` | | If given, space character ' ' will be used to indent, otherwise tab character '\t' will be used | - - - - diff --git a/docs/reference/compose_alpha_watch.md b/docs/reference/compose_alpha_watch.md deleted file mode 100644 index aa8130e7a02..00000000000 --- a/docs/reference/compose_alpha_watch.md +++ /dev/null @@ -1,16 +0,0 @@ -# docker compose alpha watch - - -Watch build context for service and rebuild/refresh containers when files are updated - -### Options - -| Name | Type | Default | Description | -|:------------|:-----|:--------|:----------------------------------------------| -| `--dry-run` | | | Execute command in dry run mode | -| `--no-up` | | | Do not build & start services before watching | -| `--quiet` | | | hide build output | - - - - diff --git a/docs/reference/compose_attach.md b/docs/reference/compose_attach.md deleted file mode 100644 index 0b9ede1e01a..00000000000 --- a/docs/reference/compose_attach.md +++ /dev/null @@ -1,17 +0,0 @@ -# docker compose attach - - -Attach local standard input, output, and error streams to a service's running container - -### Options - -| Name | Type | Default | Description | -|:----------------|:---------|:--------|:----------------------------------------------------------| -| `--detach-keys` | `string` | | Override the key sequence for detaching from a container. | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--index` | `int` | `0` | index of the container if service has multiple replicas. | -| `--no-stdin` | `bool` | | Do not attach STDIN | -| `--sig-proxy` | `bool` | `true` | Proxy all received signals to the process | - - - diff --git a/docs/reference/compose_bridge.md b/docs/reference/compose_bridge.md deleted file mode 100644 index 78d3da4934c..00000000000 --- a/docs/reference/compose_bridge.md +++ /dev/null @@ -1,22 +0,0 @@ -# docker compose bridge - - -Convert compose files into another model - -### Subcommands - -| Name | Description | -|:-------------------------------------------------------|:-----------------------------------------------------------------------------| -| [`convert`](compose_bridge_convert.md) | Convert compose files to Kubernetes manifests, Helm charts, or another model | -| [`transformations`](compose_bridge_transformations.md) | Manage transformation images | - - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:--------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | - - - - diff --git a/docs/reference/compose_bridge_convert.md b/docs/reference/compose_bridge_convert.md deleted file mode 100644 index d4b91ba172d..00000000000 --- a/docs/reference/compose_bridge_convert.md +++ /dev/null @@ -1,17 +0,0 @@ -# docker compose bridge convert - - -Convert compose files to Kubernetes manifests, Helm charts, or another model - -### Options - -| Name | Type | Default | Description | -|:-------------------------|:--------------|:--------|:-------------------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-o`, `--output` | `string` | `out` | The output directory for the Kubernetes resources | -| `--templates` | `string` | | Directory containing transformation templates | -| `-t`, `--transformation` | `stringArray` | | Transformation to apply to compose model (default: docker/compose-bridge-kubernetes) | - - - - diff --git a/docs/reference/compose_bridge_transformations.md b/docs/reference/compose_bridge_transformations.md deleted file mode 100644 index 1e1c7be392b..00000000000 --- a/docs/reference/compose_bridge_transformations.md +++ /dev/null @@ -1,22 +0,0 @@ -# docker compose bridge transformations - - -Manage transformation images - -### Subcommands - -| Name | Description | -|:-----------------------------------------------------|:-------------------------------| -| [`create`](compose_bridge_transformations_create.md) | Create a new transformation | -| [`list`](compose_bridge_transformations_list.md) | List available transformations | - - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:--------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | - - - - diff --git a/docs/reference/compose_bridge_transformations_create.md b/docs/reference/compose_bridge_transformations_create.md deleted file mode 100644 index 187e8d9eca3..00000000000 --- a/docs/reference/compose_bridge_transformations_create.md +++ /dev/null @@ -1,15 +0,0 @@ -# docker compose bridge transformations create - - -Create a new transformation - -### Options - -| Name | Type | Default | Description | -|:---------------|:---------|:--------|:----------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-f`, `--from` | `string` | | Existing transformation to copy (default: docker/compose-bridge-kubernetes) | - - - - diff --git a/docs/reference/compose_bridge_transformations_list.md b/docs/reference/compose_bridge_transformations_list.md deleted file mode 100644 index ce0a5e6911a..00000000000 --- a/docs/reference/compose_bridge_transformations_list.md +++ /dev/null @@ -1,20 +0,0 @@ -# docker compose bridge transformations list - - -List available transformations - -### Aliases - -`docker compose bridge transformations list`, `docker compose bridge transformations ls` - -### Options - -| Name | Type | Default | Description | -|:----------------|:---------|:--------|:-------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--format` | `string` | `table` | Format the output. Values: [table \| json] | -| `-q`, `--quiet` | `bool` | | Only display transformer names | - - - - diff --git a/docs/reference/compose_build.md b/docs/reference/compose_build.md deleted file mode 100644 index a715974dfa5..00000000000 --- a/docs/reference/compose_build.md +++ /dev/null @@ -1,46 +0,0 @@ -# docker compose build - - -Services are built once and then tagged, by default as `project-service`. - -If the Compose file specifies an -[image](https://github.com/compose-spec/compose-spec/blob/main/spec.md#image) name, -the image is tagged with that name, substituting any variables beforehand. See -[variable interpolation](https://github.com/compose-spec/compose-spec/blob/main/spec.md#interpolation). - -If you change a service's `Dockerfile` or the contents of its build directory, -run `docker compose build` to rebuild it. - -### Options - -| Name | Type | Default | Description | -|:----------------------|:--------------|:--------|:------------------------------------------------------------------------------------------------------------| -| `--build-arg` | `stringArray` | | Set build-time variables for services | -| `--builder` | `string` | | Set builder to use | -| `--check` | `bool` | | Check build configuration | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-m`, `--memory` | `bytes` | `0` | Set memory limit for the build container. Not supported by BuildKit. | -| `--no-cache` | `bool` | | Do not use cache when building the image | -| `--print` | `bool` | | Print equivalent bake file | -| `--provenance` | `string` | | Add a provenance attestation | -| `--pull` | `bool` | | Always attempt to pull a newer version of the image | -| `--push` | `bool` | | Push service images | -| `-q`, `--quiet` | `bool` | | Suppress the build output | -| `--sbom` | `string` | | Add a SBOM attestation | -| `--ssh` | `string` | | Set SSH authentications used when building service images. (use 'default' for using your default SSH Agent) | -| `--with-dependencies` | `bool` | | Also build dependencies (transitively) | - - - - -## Description - -Services are built once and then tagged, by default as `project-service`. - -If the Compose file specifies an -[image](https://github.com/compose-spec/compose-spec/blob/main/spec.md#image) name, -the image is tagged with that name, substituting any variables beforehand. See -[variable interpolation](https://github.com/compose-spec/compose-spec/blob/main/spec.md#interpolation). - -If you change a service's `Dockerfile` or the contents of its build directory, -run `docker compose build` to rebuild it. diff --git a/docs/reference/compose_commit.md b/docs/reference/compose_commit.md deleted file mode 100644 index 1aad40931f9..00000000000 --- a/docs/reference/compose_commit.md +++ /dev/null @@ -1,19 +0,0 @@ -# docker compose commit - - -Create a new image from a service container's changes - -### Options - -| Name | Type | Default | Description | -|:------------------|:---------|:--------|:-----------------------------------------------------------| -| `-a`, `--author` | `string` | | Author (e.g., "John Hannibal Smith <hannibal@a-team.com>") | -| `-c`, `--change` | `list` | | Apply Dockerfile instruction to the created image | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--index` | `int` | `0` | index of the container if service has multiple replicas. | -| `-m`, `--message` | `string` | | Commit message | -| `-p`, `--pause` | `bool` | `true` | Pause container during commit | - - - - diff --git a/docs/reference/compose_config.md b/docs/reference/compose_config.md deleted file mode 100644 index e2e773feae5..00000000000 --- a/docs/reference/compose_config.md +++ /dev/null @@ -1,40 +0,0 @@ -# docker compose convert - - -`docker compose config` renders the actual data model to be applied on the Docker Engine. -It merges the Compose files set by `-f` flags, resolves variables in the Compose file, and expands short-notation into -the canonical format. - -### Options - -| Name | Type | Default | Description | -|:--------------------------|:---------|:--------|:----------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--environment` | `bool` | | Print environment used for interpolation. | -| `--format` | `string` | | Format the output. Values: [yaml \| json] | -| `--hash` | `string` | | Print the service config hash, one per line. | -| `--images` | `bool` | | Print the image names, one per line. | -| `--lock-image-digests` | `bool` | | Produces an override file with image digests | -| `--models` | `bool` | | Print the model names, one per line. | -| `--networks` | `bool` | | Print the network names, one per line. | -| `--no-consistency` | `bool` | | Don't check model consistency - warning: may produce invalid Compose output | -| `--no-env-resolution` | `bool` | | Don't resolve service env files | -| `--no-interpolate` | `bool` | | Don't interpolate environment variables | -| `--no-normalize` | `bool` | | Don't normalize compose model | -| `--no-path-resolution` | `bool` | | Don't resolve file paths | -| `-o`, `--output` | `string` | | Save to file (default to stdout) | -| `--profiles` | `bool` | | Print the profile names, one per line. | -| `-q`, `--quiet` | `bool` | | Only validate the configuration, don't print anything | -| `--resolve-image-digests` | `bool` | | Pin image tags to digests | -| `--services` | `bool` | | Print the service names, one per line. | -| `--variables` | `bool` | | Print model variables and default values. | -| `--volumes` | `bool` | | Print the volume names, one per line. | - - - - -## Description - -`docker compose config` renders the actual data model to be applied on the Docker Engine. -It merges the Compose files set by `-f` flags, resolves variables in the Compose file, and expands short-notation into -the canonical format. diff --git a/docs/reference/compose_cp.md b/docs/reference/compose_cp.md deleted file mode 100644 index 0886bbd9f94..00000000000 --- a/docs/reference/compose_cp.md +++ /dev/null @@ -1,18 +0,0 @@ -# docker compose cp - - -Copy files/folders between a service container and the local filesystem - -### Options - -| Name | Type | Default | Description | -|:----------------------|:-------|:--------|:--------------------------------------------------------| -| `--all` | `bool` | | Include containers created by the run command | -| `-a`, `--archive` | `bool` | | Archive mode (copy all uid/gid information) | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-L`, `--follow-link` | `bool` | | Always follow symbol link in SRC_PATH | -| `--index` | `int` | `0` | Index of the container if service has multiple replicas | - - - - diff --git a/docs/reference/compose_create.md b/docs/reference/compose_create.md deleted file mode 100644 index 4b0b876da91..00000000000 --- a/docs/reference/compose_create.md +++ /dev/null @@ -1,23 +0,0 @@ -# docker compose create - - -Creates containers for a service - -### Options - -| Name | Type | Default | Description | -|:-------------------|:--------------|:---------|:----------------------------------------------------------------------------------------------| -| `--build` | `bool` | | Build images before starting containers | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--force-recreate` | `bool` | | Recreate containers even if their configuration and image haven't changed | -| `--no-build` | `bool` | | Don't build an image, even if it's policy | -| `--no-recreate` | `bool` | | If containers already exist, don't recreate them. Incompatible with --force-recreate. | -| `--pull` | `string` | `policy` | Pull image before running ("always"\|"missing"\|"never"\|"build") | -| `--quiet-pull` | `bool` | | Pull without printing progress information | -| `--remove-orphans` | `bool` | | Remove containers for services not defined in the Compose file | -| `--scale` | `stringArray` | | Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present. | -| `-y`, `--yes` | `bool` | | Assume "yes" as answer to all prompts and run non-interactively | - - - - diff --git a/docs/reference/compose_down.md b/docs/reference/compose_down.md deleted file mode 100644 index 2ac0bf2da42..00000000000 --- a/docs/reference/compose_down.md +++ /dev/null @@ -1,45 +0,0 @@ -# docker compose down - - -Stops containers and removes containers, networks, volumes, and images created by `up`. - -By default, the only things removed are: - -- Containers for services defined in the Compose file. -- Networks defined in the networks section of the Compose file. -- The default network, if one is used. - -Networks and volumes defined as external are never removed. - -Anonymous volumes are not removed by default. However, as they don’t have a stable name, they are not automatically -mounted by a subsequent `up`. For data that needs to persist between updates, use explicit paths as bind mounts or -named volumes. - -### Options - -| Name | Type | Default | Description | -|:-------------------|:---------|:--------|:------------------------------------------------------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--remove-orphans` | `bool` | | Remove containers for services not defined in the Compose file | -| `--rmi` | `string` | | Remove images used by services. "local" remove only images that don't have a custom tag ("local"\|"all") | -| `-t`, `--timeout` | `int` | `0` | Specify a shutdown timeout in seconds | -| `-v`, `--volumes` | `bool` | | Remove named volumes declared in the "volumes" section of the Compose file and anonymous volumes attached to containers | - - - - -## Description - -Stops containers and removes containers, networks, volumes, and images created by `up`. - -By default, the only things removed are: - -- Containers for services defined in the Compose file. -- Networks defined in the networks section of the Compose file. -- The default network, if one is used. - -Networks and volumes defined as external are never removed. - -Anonymous volumes are not removed by default. However, as they don’t have a stable name, they are not automatically -mounted by a subsequent `up`. For data that needs to persist between updates, use explicit paths as bind mounts or -named volumes. diff --git a/docs/reference/compose_events.md b/docs/reference/compose_events.md deleted file mode 100644 index 066b5cf3831..00000000000 --- a/docs/reference/compose_events.md +++ /dev/null @@ -1,56 +0,0 @@ -# docker compose events - - -Stream container events for every container in the project. - -With the `--json` flag, a json object is printed one per line with the format: - -```json -{ - "time": "2015-11-20T18:01:03.615550", - "type": "container", - "action": "create", - "id": "213cf7...5fc39a", - "service": "web", - "attributes": { - "name": "application_web_1", - "image": "alpine:edge" - } -} -``` - -The events that can be received using this can be seen [here](/reference/cli/docker/system/events/#object-types). - -### Options - -| Name | Type | Default | Description | -|:------------|:---------|:--------|:------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--json` | `bool` | | Output events as a stream of json objects | -| `--since` | `string` | | Show all events created since timestamp | -| `--until` | `string` | | Stream events until this timestamp | - - - - -## Description - -Stream container events for every container in the project. - -With the `--json` flag, a json object is printed one per line with the format: - -```json -{ - "time": "2015-11-20T18:01:03.615550", - "type": "container", - "action": "create", - "id": "213cf7...5fc39a", - "service": "web", - "attributes": { - "name": "application_web_1", - "image": "alpine:edge" - } -} -``` - -The events that can be received using this can be seen [here](https://docs.docker.com/reference/cli/docker/system/events/#object-types). diff --git a/docs/reference/compose_exec.md b/docs/reference/compose_exec.md deleted file mode 100644 index 312219e7316..00000000000 --- a/docs/reference/compose_exec.md +++ /dev/null @@ -1,42 +0,0 @@ -# docker compose exec - - -This is the equivalent of `docker exec` targeting a Compose service. - -With this subcommand, you can run arbitrary commands in your services. Commands allocate a TTY by default, so -you can use a command such as `docker compose exec web sh` to get an interactive prompt. - -By default, Compose will enter container in interactive mode and allocate a TTY, while the equivalent `docker exec` -command requires passing `--interactive --tty` flags to get the same behavior. Compose also support those two flags -to offer a smooth migration between commands, whenever they are no-op by default. Still, `interactive` can be used to -force disabling interactive mode (`--interactive=false`), typically when `docker compose exec` command is used inside -a script. - -### Options - -| Name | Type | Default | Description | -|:------------------|:--------------|:--------|:---------------------------------------------------------------------------------| -| `-d`, `--detach` | `bool` | | Detached mode: Run command in the background | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-e`, `--env` | `stringArray` | | Set environment variables | -| `--index` | `int` | `0` | Index of the container if service has multiple replicas | -| `-T`, `--no-tty` | `bool` | `true` | Disable pseudo-TTY allocation. By default 'docker compose exec' allocates a TTY. | -| `--privileged` | `bool` | | Give extended privileges to the process | -| `-u`, `--user` | `string` | | Run the command as this user | -| `-w`, `--workdir` | `string` | | Path to workdir directory for this command | - - - - -## Description - -This is the equivalent of `docker exec` targeting a Compose service. - -With this subcommand, you can run arbitrary commands in your services. Commands allocate a TTY by default, so -you can use a command such as `docker compose exec web sh` to get an interactive prompt. - -By default, Compose will enter container in interactive mode and allocate a TTY, while the equivalent `docker exec` -command requires passing `--interactive --tty` flags to get the same behavior. Compose also support those two flags -to offer a smooth migration between commands, whenever they are no-op by default. Still, `interactive` can be used to -force disabling interactive mode (`--interactive=false`), typically when `docker compose exec` command is used inside -a script. \ No newline at end of file diff --git a/docs/reference/compose_export.md b/docs/reference/compose_export.md deleted file mode 100644 index 942ea6a347f..00000000000 --- a/docs/reference/compose_export.md +++ /dev/null @@ -1,16 +0,0 @@ -# docker compose export - - -Export a service container's filesystem as a tar archive - -### Options - -| Name | Type | Default | Description | -|:-----------------|:---------|:--------|:---------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--index` | `int` | `0` | index of the container if service has multiple replicas. | -| `-o`, `--output` | `string` | | Write to a file, instead of STDOUT | - - - - diff --git a/docs/reference/compose_images.md b/docs/reference/compose_images.md deleted file mode 100644 index 1e4e0259b1d..00000000000 --- a/docs/reference/compose_images.md +++ /dev/null @@ -1,16 +0,0 @@ -# docker compose images - - -List images used by the created containers - -### Options - -| Name | Type | Default | Description | -|:----------------|:---------|:--------|:-------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--format` | `string` | `table` | Format the output. Values: [table \| json] | -| `-q`, `--quiet` | `bool` | | Only display IDs | - - - - diff --git a/docs/reference/compose_kill.md b/docs/reference/compose_kill.md deleted file mode 100644 index 0b6c1d05f01..00000000000 --- a/docs/reference/compose_kill.md +++ /dev/null @@ -1,27 +0,0 @@ -# docker compose kill - - -Forces running containers to stop by sending a `SIGKILL` signal. Optionally the signal can be passed, for example: - -```console -$ docker compose kill -s SIGINT -``` - -### Options - -| Name | Type | Default | Description | -|:-------------------|:---------|:----------|:---------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--remove-orphans` | `bool` | | Remove containers for services not defined in the Compose file | -| `-s`, `--signal` | `string` | `SIGKILL` | SIGNAL to send to the container | - - - - -## Description - -Forces running containers to stop by sending a `SIGKILL` signal. Optionally the signal can be passed, for example: - -```console -$ docker compose kill -s SIGINT -``` diff --git a/docs/reference/compose_logs.md b/docs/reference/compose_logs.md deleted file mode 100644 index 4c8ba7e3486..00000000000 --- a/docs/reference/compose_logs.md +++ /dev/null @@ -1,25 +0,0 @@ -# docker compose logs - - -Displays log output from services - -### Options - -| Name | Type | Default | Description | -|:---------------------|:---------|:--------|:-----------------------------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-f`, `--follow` | `bool` | | Follow log output | -| `--index` | `int` | `0` | index of the container if service has multiple replicas | -| `--no-color` | `bool` | | Produce monochrome output | -| `--no-log-prefix` | `bool` | | Don't print prefix in logs | -| `--since` | `string` | | Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes) | -| `-n`, `--tail` | `string` | `all` | Number of lines to show from the end of the logs for each container | -| `-t`, `--timestamps` | `bool` | | Show timestamps | -| `--until` | `string` | | Show logs before a timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes) | - - - - -## Description - -Displays log output from services diff --git a/docs/reference/compose_ls.md b/docs/reference/compose_ls.md deleted file mode 100644 index 7719d208609..00000000000 --- a/docs/reference/compose_ls.md +++ /dev/null @@ -1,21 +0,0 @@ -# docker compose ls - - -Lists running Compose projects - -### Options - -| Name | Type | Default | Description | -|:----------------|:---------|:--------|:-------------------------------------------| -| `-a`, `--all` | `bool` | | Show all stopped Compose projects | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--filter` | `filter` | | Filter output based on conditions provided | -| `--format` | `string` | `table` | Format the output. Values: [table \| json] | -| `-q`, `--quiet` | `bool` | | Only display project names | - - - - -## Description - -Lists running Compose projects diff --git a/docs/reference/compose_pause.md b/docs/reference/compose_pause.md deleted file mode 100644 index 4a0d5bdcc03..00000000000 --- a/docs/reference/compose_pause.md +++ /dev/null @@ -1,17 +0,0 @@ -# docker compose pause - - -Pauses running containers of a service. They can be unpaused with `docker compose unpause`. - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:--------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | - - - - -## Description - -Pauses running containers of a service. They can be unpaused with `docker compose unpause`. \ No newline at end of file diff --git a/docs/reference/compose_port.md b/docs/reference/compose_port.md deleted file mode 100644 index bbbfbf15616..00000000000 --- a/docs/reference/compose_port.md +++ /dev/null @@ -1,19 +0,0 @@ -# docker compose port - - -Prints the public port for a port binding - -### Options - -| Name | Type | Default | Description | -|:-------------|:---------|:--------|:--------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--index` | `int` | `0` | Index of the container if service has multiple replicas | -| `--protocol` | `string` | `tcp` | tcp or udp | - - - - -## Description - -Prints the public port for a port binding diff --git a/docs/reference/compose_ps.md b/docs/reference/compose_ps.md deleted file mode 100644 index 3572c530556..00000000000 --- a/docs/reference/compose_ps.md +++ /dev/null @@ -1,138 +0,0 @@ -# docker compose ps - - -Lists containers for a Compose project, with current status and exposed ports. - -```console -$ docker compose ps -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp -``` - -By default, only running containers are shown. `--all` flag can be used to include stopped containers. - -```console -$ docker compose ps --all -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp -example-bar-1 alpine "/entrypoint.…" bar 4 seconds ago exited (0) -``` - -### Options - -| Name | Type | Default | Description | -|:----------------------|:--------------|:--------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `-a`, `--all` | `bool` | | Show all stopped containers (including those created by the run command) | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| [`--filter`](#filter) | `string` | | Filter services by a property (supported filters: status) | -| [`--format`](#format) | `string` | `table` | Format output using a custom template: 'table': Print output in table format with column headers (default) 'table TEMPLATE': Print output in table format using the given Go template 'json': Print in JSON format 'TEMPLATE': Print output using the given Go template. Refer to https://docs.docker.com/go/formatting/ for more information about formatting output with templates | -| `--no-trunc` | `bool` | | Don't truncate output | -| `--orphans` | `bool` | `true` | Include orphaned services (not declared by project) | -| `-q`, `--quiet` | `bool` | | Only display IDs | -| `--services` | `bool` | | Display services | -| [`--status`](#status) | `stringArray` | | Filter services by status. Values: [paused \| restarting \| removing \| running \| dead \| created \| exited] | - - - - -## Description - -Lists containers for a Compose project, with current status and exposed ports. - -```console -$ docker compose ps -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp -``` - -By default, only running containers are shown. `--all` flag can be used to include stopped containers. - -```console -$ docker compose ps --all -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp -example-bar-1 alpine "/entrypoint.…" bar 4 seconds ago exited (0) -``` - -## Examples - -### <a name="format"></a> Format the output (--format) - -By default, the `docker compose ps` command uses a table ("pretty") format to -show the containers. The `--format` flag allows you to specify alternative -presentations for the output. Currently, supported options are `pretty` (default), -and `json`, which outputs information about the containers as a JSON array: - -```console -$ docker compose ps --format json -[{"ID":"1553b0236cf4d2715845f053a4ee97042c4f9a2ef655731ee34f1f7940eaa41a","Name":"example-bar-1","Command":"/docker-entrypoint.sh nginx -g 'daemon off;'","Project":"example","Service":"bar","State":"exited","Health":"","ExitCode":0,"Publishers":null},{"ID":"f02a4efaabb67416e1ff127d51c4b5578634a0ad5743bd65225ff7d1909a3fa0","Name":"example-foo-1","Command":"/docker-entrypoint.sh nginx -g 'daemon off;'","Project":"example","Service":"foo","State":"running","Health":"","ExitCode":0,"Publishers":[{"URL":"0.0.0.0","TargetPort":80,"PublishedPort":8080,"Protocol":"tcp"}]}] -``` - -The JSON output allows you to use the information in other tools for further -processing, for example, using the [`jq` utility](https://stedolan.github.io/jq/) -to pretty-print the JSON: - -```console -$ docker compose ps --format json | jq . -[ - { - "ID": "1553b0236cf4d2715845f053a4ee97042c4f9a2ef655731ee34f1f7940eaa41a", - "Name": "example-bar-1", - "Command": "/docker-entrypoint.sh nginx -g 'daemon off;'", - "Project": "example", - "Service": "bar", - "State": "exited", - "Health": "", - "ExitCode": 0, - "Publishers": null - }, - { - "ID": "f02a4efaabb67416e1ff127d51c4b5578634a0ad5743bd65225ff7d1909a3fa0", - "Name": "example-foo-1", - "Command": "/docker-entrypoint.sh nginx -g 'daemon off;'", - "Project": "example", - "Service": "foo", - "State": "running", - "Health": "", - "ExitCode": 0, - "Publishers": [ - { - "URL": "0.0.0.0", - "TargetPort": 80, - "PublishedPort": 8080, - "Protocol": "tcp" - } - ] - } -] -``` - -### <a name="status"></a> Filter containers by status (--status) - -Use the `--status` flag to filter the list of containers by status. For example, -to show only containers that are running or only containers that have exited: - -```console -$ docker compose ps --status=running -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp - -$ docker compose ps --status=exited -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-bar-1 alpine "/entrypoint.…" bar 4 seconds ago exited (0) -``` - -### <a name="filter"></a> Filter containers by status (--filter) - -The [`--status` flag](#status) is a convenient shorthand for the `--filter status=<status>` -flag. The example below is the equivalent to the example from the previous section, -this time using the `--filter` flag: - -```console -$ docker compose ps --filter status=running -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp -``` - -The `docker compose ps` command currently only supports the `--filter status=<status>` -option, but additional filter options may be added in the future. diff --git a/docs/reference/compose_publish.md b/docs/reference/compose_publish.md deleted file mode 100644 index 9a82fc260a7..00000000000 --- a/docs/reference/compose_publish.md +++ /dev/null @@ -1,19 +0,0 @@ -# docker compose publish - - -Publish compose application - -### Options - -| Name | Type | Default | Description | -|:--------------------------|:---------|:--------|:-------------------------------------------------------------------------------| -| `--app` | `bool` | | Published compose application (includes referenced images) | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--oci-version` | `string` | | OCI image/artifact specification version (automatically determined by default) | -| `--resolve-image-digests` | `bool` | | Pin image tags to digests | -| `--with-env` | `bool` | | Include environment variables in the published OCI artifact | -| `-y`, `--yes` | `bool` | | Assume "yes" as answer to all prompts | - - - - diff --git a/docs/reference/compose_pull.md b/docs/reference/compose_pull.md deleted file mode 100644 index 6a47f9d509f..00000000000 --- a/docs/reference/compose_pull.md +++ /dev/null @@ -1,68 +0,0 @@ -# docker compose pull - - -Pulls an image associated with a service defined in a `compose.yaml` file, but does not start containers based on those images - -### Options - -| Name | Type | Default | Description | -|:-------------------------|:---------|:--------|:-------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--ignore-buildable` | `bool` | | Ignore images that can be built | -| `--ignore-pull-failures` | `bool` | | Pull what it can and ignores images with pull failures | -| `--include-deps` | `bool` | | Also pull services declared as dependencies | -| `--policy` | `string` | | Apply pull policy ("missing"\|"always") | -| `-q`, `--quiet` | `bool` | | Pull without printing progress information | - - - - -## Description - -Pulls an image associated with a service defined in a `compose.yaml` file, but does not start containers based on those images - - -## Examples - -Consider the following `compose.yaml`: - -```yaml -services: - db: - image: postgres - web: - build: . - command: bundle exec rails s -p 3000 -b '0.0.0.0' - volumes: - - .:/myapp - ports: - - "3000:3000" - depends_on: - - db -``` - -If you run `docker compose pull ServiceName` in the same directory as the `compose.yaml` file that defines the service, -Docker pulls the associated image. For example, to call the postgres image configured as the db service in our example, -you would run `docker compose pull db`. - -```console -$ docker compose pull db -[+] Running 1/15 - ⠸ db Pulling 12.4s - ⠿ 45b42c59be33 Already exists 0.0s - ⠹ 40adec129f1a Downloading 3.374MB/4.178MB 9.3s - ⠹ b4c431d00c78 Download complete 9.3s - ⠹ 2696974e2815 Download complete 9.3s - ⠹ 564b77596399 Downloading 5.622MB/7.965MB 9.3s - ⠹ 5044045cf6f2 Downloading 216.7kB/391.1kB 9.3s - ⠹ d736e67e6ac3 Waiting 9.3s - ⠹ 390c1c9a5ae4 Waiting 9.3s - ⠹ c0e62f172284 Waiting 9.3s - ⠹ ebcdc659c5bf Waiting 9.3s - ⠹ 29be22cb3acc Waiting 9.3s - ⠹ f63c47038e66 Waiting 9.3s - ⠹ 77a0c198cde5 Waiting 9.3s - ⠹ c8752d5b785c Waiting 9.3s -``` - -`docker compose pull` tries to pull image for services with a build section. If pull fails, it lets you know this service image must be built. You can skip this by setting `--ignore-buildable` flag. diff --git a/docs/reference/compose_push.md b/docs/reference/compose_push.md deleted file mode 100644 index 0efc48c46d3..00000000000 --- a/docs/reference/compose_push.md +++ /dev/null @@ -1,54 +0,0 @@ -# docker compose push - - -Pushes images for services to their respective registry/repository. - -The following assumptions are made: -- You are pushing an image you have built locally -- You have access to the build key - -Examples - -```yaml -services: - service1: - build: . - image: localhost:5000/yourimage ## goes to local registry - - service2: - build: . - image: your-dockerid/yourimage ## goes to your repository on Docker Hub -``` - -### Options - -| Name | Type | Default | Description | -|:-------------------------|:-------|:--------|:-------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--ignore-push-failures` | `bool` | | Push what it can and ignores images with push failures | -| `--include-deps` | `bool` | | Also push images of services declared as dependencies | -| `-q`, `--quiet` | `bool` | | Push without printing progress information | - - - - -## Description - -Pushes images for services to their respective registry/repository. - -The following assumptions are made: -- You are pushing an image you have built locally -- You have access to the build key - -Examples - -```yaml -services: - service1: - build: . - image: localhost:5000/yourimage ## goes to local registry - - service2: - build: . - image: your-dockerid/yourimage ## goes to your repository on Docker Hub -``` diff --git a/docs/reference/compose_restart.md b/docs/reference/compose_restart.md deleted file mode 100644 index e57f346a81a..00000000000 --- a/docs/reference/compose_restart.md +++ /dev/null @@ -1,37 +0,0 @@ -# docker compose restart - - -Restarts all stopped and running services, or the specified services only. - -If you make changes to your `compose.yml` configuration, these changes are not reflected -after running this command. For example, changes to environment variables (which are added -after a container is built, but before the container's command is executed) are not updated -after restarting. - -If you are looking to configure a service's restart policy, refer to -[restart](https://github.com/compose-spec/compose-spec/blob/main/spec.md#restart) -or [restart_policy](https://github.com/compose-spec/compose-spec/blob/main/deploy.md#restart_policy). - -### Options - -| Name | Type | Default | Description | -|:------------------|:-------|:--------|:--------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--no-deps` | `bool` | | Don't restart dependent services | -| `-t`, `--timeout` | `int` | `0` | Specify a shutdown timeout in seconds | - - - - -## Description - -Restarts all stopped and running services, or the specified services only. - -If you make changes to your `compose.yml` configuration, these changes are not reflected -after running this command. For example, changes to environment variables (which are added -after a container is built, but before the container's command is executed) are not updated -after restarting. - -If you are looking to configure a service's restart policy, refer to -[restart](https://github.com/compose-spec/compose-spec/blob/main/spec.md#restart) -or [restart_policy](https://github.com/compose-spec/compose-spec/blob/main/deploy.md#restart_policy). diff --git a/docs/reference/compose_rm.md b/docs/reference/compose_rm.md deleted file mode 100644 index 5e84930bac3..00000000000 --- a/docs/reference/compose_rm.md +++ /dev/null @@ -1,48 +0,0 @@ -# docker compose rm - - -Removes stopped service containers. - -By default, anonymous volumes attached to containers are not removed. You can override this with `-v`. To list all -volumes, use `docker volume ls`. - -Any data which is not in a volume is lost. - -Running the command with no options also removes one-off containers created by `docker compose run`: - -```console -$ docker compose rm -Going to remove djangoquickstart_web_run_1 -Are you sure? [yN] y -Removing djangoquickstart_web_run_1 ... done -``` - -### Options - -| Name | Type | Default | Description | -|:------------------|:-------|:--------|:----------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-f`, `--force` | `bool` | | Don't ask to confirm removal | -| `-s`, `--stop` | `bool` | | Stop the containers, if required, before removing | -| `-v`, `--volumes` | `bool` | | Remove any anonymous volumes attached to containers | - - - - -## Description - -Removes stopped service containers. - -By default, anonymous volumes attached to containers are not removed. You can override this with `-v`. To list all -volumes, use `docker volume ls`. - -Any data which is not in a volume is lost. - -Running the command with no options also removes one-off containers created by `docker compose run`: - -```console -$ docker compose rm -Going to remove djangoquickstart_web_run_1 -Are you sure? [yN] y -Removing djangoquickstart_web_run_1 ... done -``` diff --git a/docs/reference/compose_run.md b/docs/reference/compose_run.md deleted file mode 100644 index 25b28d1ded8..00000000000 --- a/docs/reference/compose_run.md +++ /dev/null @@ -1,145 +0,0 @@ -# docker compose run - - -Runs a one-time command against a service. - -The following command starts the `web` service and runs `bash` as its command: - -```console -$ docker compose run web bash -``` - -Commands you use with run start in new containers with configuration defined by that of the service, -including volumes, links, and other details. However, there are two important differences: - -First, the command passed by `run` overrides the command defined in the service configuration. For example, if the -`web` service configuration is started with `bash`, then `docker compose run web python app.py` overrides it with -`python app.py`. - -The second difference is that the `docker compose run` command does not create any of the ports specified in the -service configuration. This prevents port collisions with already-open ports. If you do want the service’s ports -to be created and mapped to the host, specify the `--service-ports` - -```console -$ docker compose run --service-ports web python manage.py shell -``` - -Alternatively, manual port mapping can be specified with the `--publish` or `-p` options, just as when using docker run: - -```console -$ docker compose run --publish 8080:80 -p 2022:22 -p 127.0.0.1:2021:21 web python manage.py shell -``` - -If you start a service configured with links, the run command first checks to see if the linked service is running -and starts the service if it is stopped. Once all the linked services are running, the run executes the command you -passed it. For example, you could run: - -```console -$ docker compose run db psql -h db -U docker -``` - -This opens an interactive PostgreSQL shell for the linked `db` container. - -If you do not want the run command to start linked containers, use the `--no-deps` flag: - -```console -$ docker compose run --no-deps web python manage.py shell -``` - -If you want to remove the container after running while overriding the container’s restart policy, use the `--rm` flag: - -```console -$ docker compose run --rm web python manage.py db upgrade -``` - -This runs a database upgrade script, and removes the container when finished running, even if a restart policy is -specified in the service configuration. - -### Options - -| Name | Type | Default | Description | -|:------------------------|:--------------|:---------|:---------------------------------------------------------------------------------| -| `--build` | `bool` | | Build image before starting container | -| `--cap-add` | `list` | | Add Linux capabilities | -| `--cap-drop` | `list` | | Drop Linux capabilities | -| `-d`, `--detach` | `bool` | | Run container in background and print container ID | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--entrypoint` | `string` | | Override the entrypoint of the image | -| `-e`, `--env` | `stringArray` | | Set environment variables | -| `--env-from-file` | `stringArray` | | Set environment variables from file | -| `-i`, `--interactive` | `bool` | `true` | Keep STDIN open even if not attached | -| `-l`, `--label` | `stringArray` | | Add or override a label | -| `--name` | `string` | | Assign a name to the container | -| `-T`, `--no-TTY` | `bool` | `true` | Disable pseudo-TTY allocation (default: auto-detected) | -| `--no-deps` | `bool` | | Don't start linked services | -| `-p`, `--publish` | `stringArray` | | Publish a container's port(s) to the host | -| `--pull` | `string` | `policy` | Pull image before running ("always"\|"missing"\|"never") | -| `-q`, `--quiet` | `bool` | | Don't print anything to STDOUT | -| `--quiet-build` | `bool` | | Suppress progress output from the build process | -| `--quiet-pull` | `bool` | | Pull without printing progress information | -| `--remove-orphans` | `bool` | | Remove containers for services not defined in the Compose file | -| `--rm` | `bool` | | Automatically remove the container when it exits | -| `-P`, `--service-ports` | `bool` | | Run command with all service's ports enabled and mapped to the host | -| `--use-aliases` | `bool` | | Use the service's network useAliases in the network(s) the container connects to | -| `-u`, `--user` | `string` | | Run as specified username or uid | -| `-v`, `--volume` | `stringArray` | | Bind mount a volume | -| `-w`, `--workdir` | `string` | | Working directory inside the container | - - - - -## Description - -Runs a one-time command against a service. - -The following command starts the `web` service and runs `bash` as its command: - -```console -$ docker compose run web bash -``` - -Commands you use with run start in new containers with configuration defined by that of the service, -including volumes, links, and other details. However, there are two important differences: - -First, the command passed by `run` overrides the command defined in the service configuration. For example, if the -`web` service configuration is started with `bash`, then `docker compose run web python app.py` overrides it with -`python app.py`. - -The second difference is that the `docker compose run` command does not create any of the ports specified in the -service configuration. This prevents port collisions with already-open ports. If you do want the service’s ports -to be created and mapped to the host, specify the `--service-ports` - -```console -$ docker compose run --service-ports web python manage.py shell -``` - -Alternatively, manual port mapping can be specified with the `--publish` or `-p` options, just as when using docker run: - -```console -$ docker compose run --publish 8080:80 -p 2022:22 -p 127.0.0.1:2021:21 web python manage.py shell -``` - -If you start a service configured with links, the run command first checks to see if the linked service is running -and starts the service if it is stopped. Once all the linked services are running, the run executes the command you -passed it. For example, you could run: - -```console -$ docker compose run db psql -h db -U docker -``` - -This opens an interactive PostgreSQL shell for the linked `db` container. - -If you do not want the run command to start linked containers, use the `--no-deps` flag: - -```console -$ docker compose run --no-deps web python manage.py shell -``` - -If you want to remove the container after running while overriding the container’s restart policy, use the `--rm` flag: - -```console -$ docker compose run --rm web python manage.py db upgrade -``` - -This runs a database upgrade script, and removes the container when finished running, even if a restart policy is -specified in the service configuration. diff --git a/docs/reference/compose_scale.md b/docs/reference/compose_scale.md deleted file mode 100644 index 3d0dbdb04a2..00000000000 --- a/docs/reference/compose_scale.md +++ /dev/null @@ -1,15 +0,0 @@ -# docker compose scale - - -Scale services - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:--------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--no-deps` | `bool` | | Don't start linked services | - - - - diff --git a/docs/reference/compose_start.md b/docs/reference/compose_start.md deleted file mode 100644 index 06229e5940e..00000000000 --- a/docs/reference/compose_start.md +++ /dev/null @@ -1,19 +0,0 @@ -# docker compose start - - -Starts existing containers for a service - -### Options - -| Name | Type | Default | Description | -|:-----------------|:-------|:--------|:---------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--wait` | `bool` | | Wait for services to be running\|healthy. Implies detached mode. | -| `--wait-timeout` | `int` | `0` | Maximum duration in seconds to wait for the project to be running\|healthy | - - - - -## Description - -Starts existing containers for a service diff --git a/docs/reference/compose_stats.md b/docs/reference/compose_stats.md deleted file mode 100644 index 78d44b89350..00000000000 --- a/docs/reference/compose_stats.md +++ /dev/null @@ -1,18 +0,0 @@ -# docker compose stats - - -Display a live stream of container(s) resource usage statistics - -### Options - -| Name | Type | Default | Description | -|:--------------|:---------|:--------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `-a`, `--all` | `bool` | | Show all containers (default shows just running) | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--format` | `string` | | Format output using a custom template: 'table': Print output in table format with column headers (default) 'table TEMPLATE': Print output in table format using the given Go template 'json': Print in JSON format 'TEMPLATE': Print output using the given Go template. Refer to https://docs.docker.com/engine/cli/formatting/ for more information about formatting output with templates | -| `--no-stream` | `bool` | | Disable streaming stats and only pull the first result | -| `--no-trunc` | `bool` | | Do not truncate output | - - - - diff --git a/docs/reference/compose_stop.md b/docs/reference/compose_stop.md deleted file mode 100644 index fe84f24f8f5..00000000000 --- a/docs/reference/compose_stop.md +++ /dev/null @@ -1,18 +0,0 @@ -# docker compose stop - - -Stops running containers without removing them. They can be started again with `docker compose start`. - -### Options - -| Name | Type | Default | Description | -|:------------------|:-------|:--------|:--------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-t`, `--timeout` | `int` | `0` | Specify a shutdown timeout in seconds | - - - - -## Description - -Stops running containers without removing them. They can be started again with `docker compose start`. diff --git a/docs/reference/compose_top.md b/docs/reference/compose_top.md deleted file mode 100644 index eeacb3866aa..00000000000 --- a/docs/reference/compose_top.md +++ /dev/null @@ -1,26 +0,0 @@ -# docker compose top - - -Displays the running processes - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:--------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | - - - - -## Description - -Displays the running processes - -## Examples - -```console -$ docker compose top -example_foo_1 -UID PID PPID C STIME TTY TIME CMD -root 142353 142331 2 15:33 ? 00:00:00 ping localhost -c 5 -``` diff --git a/docs/reference/compose_unpause.md b/docs/reference/compose_unpause.md deleted file mode 100644 index 92841ceade7..00000000000 --- a/docs/reference/compose_unpause.md +++ /dev/null @@ -1,17 +0,0 @@ -# docker compose unpause - - -Unpauses paused containers of a service - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:--------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | - - - - -## Description - -Unpauses paused containers of a service diff --git a/docs/reference/compose_up.md b/docs/reference/compose_up.md deleted file mode 100644 index b7f17a0fac9..00000000000 --- a/docs/reference/compose_up.md +++ /dev/null @@ -1,82 +0,0 @@ -# docker compose up - - -Builds, (re)creates, starts, and attaches to containers for a service. - -Unless they are already running, this command also starts any linked services. - -The `docker compose up` command aggregates the output of each container (like `docker compose logs --follow` does). -One can optionally select a subset of services to attach to using `--attach` flag, or exclude some services using -`--no-attach` to prevent output to be flooded by some verbose services. - -When the command exits, all containers are stopped. Running `docker compose up --detach` starts the containers in the -background and leaves them running. - -If there are existing containers for a service, and the service’s configuration or image was changed after the -container’s creation, `docker compose up` picks up the changes by stopping and recreating the containers -(preserving mounted volumes). To prevent Compose from picking up changes, use the `--no-recreate` flag. - -If you want to force Compose to stop and recreate all containers, use the `--force-recreate` flag. - -If the process encounters an error, the exit code for this command is `1`. -If the process is interrupted using `SIGINT` (ctrl + C) or `SIGTERM`, the containers are stopped, and the exit code is `0`. - -### Options - -| Name | Type | Default | Description | -|:-------------------------------|:--------------|:---------|:----------------------------------------------------------------------------------------------------------------------------------------------------| -| `--abort-on-container-exit` | `bool` | | Stops all containers if any container was stopped. Incompatible with -d | -| `--abort-on-container-failure` | `bool` | | Stops all containers if any container exited with failure. Incompatible with -d | -| `--always-recreate-deps` | `bool` | | Recreate dependent containers. Incompatible with --no-recreate. | -| `--attach` | `stringArray` | | Restrict attaching to the specified services. Incompatible with --attach-dependencies. | -| `--attach-dependencies` | `bool` | | Automatically attach to log output of dependent services | -| `--build` | `bool` | | Build images before starting containers | -| `-d`, `--detach` | `bool` | | Detached mode: Run containers in the background | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--exit-code-from` | `string` | | Return the exit code of the selected service container. Implies --abort-on-container-exit | -| `--force-recreate` | `bool` | | Recreate containers even if their configuration and image haven't changed | -| `--menu` | `bool` | | Enable interactive shortcuts when running attached. Incompatible with --detach. Can also be enable/disable by setting COMPOSE_MENU environment var. | -| `--no-attach` | `stringArray` | | Do not attach (stream logs) to the specified services | -| `--no-build` | `bool` | | Don't build an image, even if it's policy | -| `--no-color` | `bool` | | Produce monochrome output | -| `--no-deps` | `bool` | | Don't start linked services | -| `--no-log-prefix` | `bool` | | Don't print prefix in logs | -| `--no-recreate` | `bool` | | If containers already exist, don't recreate them. Incompatible with --force-recreate. | -| `--no-start` | `bool` | | Don't start the services after creating them | -| `--pull` | `string` | `policy` | Pull image before running ("always"\|"missing"\|"never") | -| `--quiet-build` | `bool` | | Suppress the build output | -| `--quiet-pull` | `bool` | | Pull without printing progress information | -| `--remove-orphans` | `bool` | | Remove containers for services not defined in the Compose file | -| `-V`, `--renew-anon-volumes` | `bool` | | Recreate anonymous volumes instead of retrieving data from the previous containers | -| `--scale` | `stringArray` | | Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present. | -| `-t`, `--timeout` | `int` | `0` | Use this timeout in seconds for container shutdown when attached or when containers are already running | -| `--timestamps` | `bool` | | Show timestamps | -| `--wait` | `bool` | | Wait for services to be running\|healthy. Implies detached mode. | -| `--wait-timeout` | `int` | `0` | Maximum duration in seconds to wait for the project to be running\|healthy | -| `-w`, `--watch` | `bool` | | Watch source code and rebuild/refresh containers when files are updated. | -| `-y`, `--yes` | `bool` | | Assume "yes" as answer to all prompts and run non-interactively | - - - - -## Description - -Builds, (re)creates, starts, and attaches to containers for a service. - -Unless they are already running, this command also starts any linked services. - -The `docker compose up` command aggregates the output of each container (like `docker compose logs --follow` does). -One can optionally select a subset of services to attach to using `--attach` flag, or exclude some services using -`--no-attach` to prevent output to be flooded by some verbose services. - -When the command exits, all containers are stopped. Running `docker compose up --detach` starts the containers in the -background and leaves them running. - -If there are existing containers for a service, and the service’s configuration or image was changed after the -container’s creation, `docker compose up` picks up the changes by stopping and recreating the containers -(preserving mounted volumes). To prevent Compose from picking up changes, use the `--no-recreate` flag. - -If you want to force Compose to stop and recreate all containers, use the `--force-recreate` flag. - -If the process encounters an error, the exit code for this command is `1`. -If the process is interrupted using `SIGINT` (ctrl + C) or `SIGTERM`, the containers are stopped, and the exit code is `0`. diff --git a/docs/reference/compose_version.md b/docs/reference/compose_version.md deleted file mode 100644 index 3a6329dadb4..00000000000 --- a/docs/reference/compose_version.md +++ /dev/null @@ -1,15 +0,0 @@ -# docker compose version - - -Show the Docker Compose version information - -### Options - -| Name | Type | Default | Description | -|:-----------------|:---------|:--------|:---------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-f`, `--format` | `string` | | Format the output. Values: [pretty \| json]. (Default: pretty) | -| `--short` | `bool` | | Shows only Compose's version number | - - - diff --git a/docs/reference/compose_volumes.md b/docs/reference/compose_volumes.md deleted file mode 100644 index 6bad874f187..00000000000 --- a/docs/reference/compose_volumes.md +++ /dev/null @@ -1,16 +0,0 @@ -# docker compose volumes - - -List volumes - -### Options - -| Name | Type | Default | Description | -|:----------------|:---------|:--------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--format` | `string` | `table` | Format output using a custom template: 'table': Print output in table format with column headers (default) 'table TEMPLATE': Print output in table format using the given Go template 'json': Print in JSON format 'TEMPLATE': Print output using the given Go template. Refer to https://docs.docker.com/go/formatting/ for more information about formatting output with templates | -| `-q`, `--quiet` | `bool` | | Only display volume names | - - - - diff --git a/docs/reference/compose_wait.md b/docs/reference/compose_wait.md deleted file mode 100644 index 59474c9b509..00000000000 --- a/docs/reference/compose_wait.md +++ /dev/null @@ -1,15 +0,0 @@ -# docker compose wait - - -Block until containers of all (or specified) services stop. - -### Options - -| Name | Type | Default | Description | -|:-----------------|:-------|:--------|:---------------------------------------------| -| `--down-project` | `bool` | | Drops project when the first container stops | -| `--dry-run` | `bool` | | Execute command in dry run mode | - - - - diff --git a/docs/reference/compose_watch.md b/docs/reference/compose_watch.md deleted file mode 100644 index f6040c9094f..00000000000 --- a/docs/reference/compose_watch.md +++ /dev/null @@ -1,17 +0,0 @@ -# docker compose watch - - -Watch build context for service and rebuild/refresh containers when files are updated - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:----------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--no-up` | `bool` | | Do not build & start services before watching | -| `--prune` | `bool` | `true` | Prune dangling images on rebuild | -| `--quiet` | `bool` | | hide build output | - - - - diff --git a/docs/reference/docker_compose.yaml b/docs/reference/docker_compose.yaml deleted file mode 100644 index c5fdb937510..00000000000 --- a/docs/reference/docker_compose.yaml +++ /dev/null @@ -1,445 +0,0 @@ -command: docker compose -short: Docker Compose -long: Define and run multi-container applications with Docker -usage: docker compose -pname: docker -plink: docker.yaml -cname: - - docker compose attach - - docker compose bridge - - docker compose build - - docker compose commit - - docker compose config - - docker compose cp - - docker compose create - - docker compose down - - docker compose events - - docker compose exec - - docker compose export - - docker compose images - - docker compose kill - - docker compose logs - - docker compose ls - - docker compose pause - - docker compose port - - docker compose ps - - docker compose publish - - docker compose pull - - docker compose push - - docker compose restart - - docker compose rm - - docker compose run - - docker compose scale - - docker compose start - - docker compose stats - - docker compose stop - - docker compose top - - docker compose unpause - - docker compose up - - docker compose version - - docker compose volumes - - docker compose wait - - docker compose watch -clink: - - docker_compose_attach.yaml - - docker_compose_bridge.yaml - - docker_compose_build.yaml - - docker_compose_commit.yaml - - docker_compose_config.yaml - - docker_compose_cp.yaml - - docker_compose_create.yaml - - docker_compose_down.yaml - - docker_compose_events.yaml - - docker_compose_exec.yaml - - docker_compose_export.yaml - - docker_compose_images.yaml - - docker_compose_kill.yaml - - docker_compose_logs.yaml - - docker_compose_ls.yaml - - docker_compose_pause.yaml - - docker_compose_port.yaml - - docker_compose_ps.yaml - - docker_compose_publish.yaml - - docker_compose_pull.yaml - - docker_compose_push.yaml - - docker_compose_restart.yaml - - docker_compose_rm.yaml - - docker_compose_run.yaml - - docker_compose_scale.yaml - - docker_compose_start.yaml - - docker_compose_stats.yaml - - docker_compose_stop.yaml - - docker_compose_top.yaml - - docker_compose_unpause.yaml - - docker_compose_up.yaml - - docker_compose_version.yaml - - docker_compose_volumes.yaml - - docker_compose_wait.yaml - - docker_compose_watch.yaml -options: - - option: all-resources - value_type: bool - default_value: "false" - description: Include all resources, even those not used by services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: ansi - value_type: string - default_value: auto - description: | - Control when to print ANSI control characters ("never"|"always"|"auto") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: compatibility - value_type: bool - default_value: "false" - description: Run compose in backward compatibility mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: env-file - value_type: stringArray - default_value: '[]' - description: Specify an alternate environment file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: file - shorthand: f - value_type: stringArray - default_value: '[]' - description: Compose configuration files - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: insecure-registry - value_type: stringArray - default_value: '[]' - description: | - Use insecure registry to pull Compose OCI artifacts. Doesn't apply to images - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-ansi - value_type: bool - default_value: "false" - description: Do not print ANSI control characters (DEPRECATED) - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: parallel - value_type: int - default_value: "-1" - description: Control max parallelism, -1 for unlimited - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: profile - value_type: stringArray - default_value: '[]' - description: Specify a profile to enable - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: progress - value_type: string - description: Set type of progress output (auto, tty, plain, json, quiet) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: project-directory - value_type: string - description: |- - Specify an alternate working directory - (default: the path of the, first specified, Compose file) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: project-name - shorthand: p - value_type: string - description: Project name - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: verbose - value_type: bool - default_value: "false" - description: Show more output - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: version - shorthand: v - value_type: bool - default_value: "false" - description: Show the Docker Compose version information - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: workdir - value_type: string - description: |- - DEPRECATED! USE --project-directory INSTEAD. - Specify an alternate working directory - (default: the path of the, first specified, Compose file) - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -examples: |- - ### Use `-f` to specify the name and path of one or more Compose files - Use the `-f` flag to specify the location of a Compose [configuration file](/reference/compose-file/). - - #### Specifying multiple Compose files - You can supply multiple `-f` configuration files. When you supply multiple files, Compose combines them into a single - configuration. Compose builds the configuration in the order you supply the files. Subsequent files override and add - to their predecessors. - - For example, consider this command line: - - ```console - $ docker compose -f compose.yaml -f compose.admin.yaml run backup_db - ``` - - The `compose.yaml` file might specify a `webapp` service. - - ```yaml - services: - webapp: - image: examples/web - ports: - - "8000:8000" - volumes: - - "/data" - ``` - If the `compose.admin.yaml` also specifies this same service, any matching fields override the previous file. - New values, add to the `webapp` service configuration. - - ```yaml - services: - webapp: - build: . - environment: - - DEBUG=1 - ``` - - When you use multiple Compose files, all paths in the files are relative to the first configuration file specified - with `-f`. You can use the `--project-directory` option to override this base path. - - Use a `-f` with `-` (dash) as the filename to read the configuration from stdin. When stdin is used all paths in the - configuration are relative to the current working directory. - - The `-f` flag is optional. If you don’t provide this flag on the command line, Compose traverses the working directory - and its parent directories looking for a `compose.yaml` or `docker-compose.yaml` file. - - #### Specifying a path to a single Compose file - You can use the `-f` flag to specify a path to a Compose file that is not located in the current directory, either - from the command line or by setting up a `COMPOSE_FILE` environment variable in your shell or in an environment file. - - For an example of using the `-f` option at the command line, suppose you are running the Compose Rails sample, and - have a `compose.yaml` file in a directory called `sandbox/rails`. You can use a command like `docker compose pull` to - get the postgres image for the db service from anywhere by using the `-f` flag as follows: - - ```console - $ docker compose -f ~/sandbox/rails/compose.yaml pull db - ``` - - #### Using an OCI published artifact - You can use the `-f` flag with the `oci://` prefix to reference a Compose file that has been published to an OCI registry. - This allows you to distribute and version your Compose configurations as OCI artifacts. - - To use a Compose file from an OCI registry: - - ```console - $ docker compose -f oci://registry.example.com/my-compose-project:latest up - ``` - - You can also combine OCI artifacts with local files: - - ```console - $ docker compose -f oci://registry.example.com/my-compose-project:v1.0 -f compose.override.yaml up - ``` - - The OCI artifact must contain a valid Compose file. You can publish Compose files to an OCI registry using the - `docker compose publish` command. - - #### Using a git repository - You can use the `-f` flag to reference a Compose file from a git repository. Compose supports various git URL formats: - - Using HTTPS: - ```console - $ docker compose -f https://github.com/user/repo.git up - ``` - - Using SSH: - ```console - $ docker compose -f git@github.com:user/repo.git up - ``` - - You can specify a specific branch, tag, or commit: - ```console - $ docker compose -f https://github.com/user/repo.git@main up - $ docker compose -f https://github.com/user/repo.git@v1.0.0 up - $ docker compose -f https://github.com/user/repo.git@abc123 up - ``` - - You can also specify a subdirectory within the repository: - ```console - $ docker compose -f https://github.com/user/repo.git#main:path/to/compose.yaml up - ``` - - When using git resources, Compose will clone the repository and use the specified Compose file. You can combine - git resources with local files: - - ```console - $ docker compose -f https://github.com/user/repo.git -f compose.override.yaml up - ``` - - ### Use `-p` to specify a project name - - Each configuration has a project name. Compose sets the project name using - the following mechanisms, in order of precedence: - - The `-p` command line flag - - The `COMPOSE_PROJECT_NAME` environment variable - - The top level `name:` variable from the config file (or the last `name:` - from a series of config files specified using `-f`) - - The `basename` of the project directory containing the config file (or - containing the first config file specified using `-f`) - - The `basename` of the current directory if no config file is specified - Project names must contain only lowercase letters, decimal digits, dashes, - and underscores, and must begin with a lowercase letter or decimal digit. If - the `basename` of the project directory or current directory violates this - constraint, you must use one of the other mechanisms. - - ```console - $ docker compose -p my_project ps -a - NAME SERVICE STATUS PORTS - my_project_demo_1 demo running - - $ docker compose -p my_project logs - demo_1 | PING localhost (127.0.0.1): 56 data bytes - demo_1 | 64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.095 ms - ``` - - ### Use profiles to enable optional services - - Use `--profile` to specify one or more active profiles - Calling `docker compose --profile frontend up` starts the services with the profile `frontend` and services - without any specified profiles. - You can also enable multiple profiles, e.g. with `docker compose --profile frontend --profile debug up` the profiles `frontend` and `debug` is enabled. - - Profiles can also be set by `COMPOSE_PROFILES` environment variable. - - ### Configuring parallelism - - Use `--parallel` to specify the maximum level of parallelism for concurrent engine calls. - Calling `docker compose --parallel 1 pull` pulls the pullable images defined in the Compose file - one at a time. This can also be used to control build concurrency. - - Parallelism can also be set by the `COMPOSE_PARALLEL_LIMIT` environment variable. - - ### Set up environment variables - - You can set environment variables for various docker compose options, including the `-f`, `-p` and `--profiles` flags. - - Setting the `COMPOSE_FILE` environment variable is equivalent to passing the `-f` flag, - `COMPOSE_PROJECT_NAME` environment variable does the same as the `-p` flag, - `COMPOSE_PROFILES` environment variable is equivalent to the `--profiles` flag - and `COMPOSE_PARALLEL_LIMIT` does the same as the `--parallel` flag. - - If flags are explicitly set on the command line, the associated environment variable is ignored. - - Setting the `COMPOSE_IGNORE_ORPHANS` environment variable to `true` stops docker compose from detecting orphaned - containers for the project. - - Setting the `COMPOSE_MENU` environment variable to `false` disables the helper menu when running `docker compose up` - in attached mode. Alternatively, you can also run `docker compose up --menu=false` to disable the helper menu. - - ### Use Dry Run mode to test your command - - Use `--dry-run` flag to test a command without changing your application stack state. - Dry Run mode shows you all the steps Compose applies when executing a command, for example: - ```console - $ docker compose --dry-run up --build -d - [+] Pulling 1/1 - ✔ DRY-RUN MODE - db Pulled 0.9s - [+] Running 10/8 - ✔ DRY-RUN MODE - build service backend 0.0s - ✔ DRY-RUN MODE - ==> ==> writing image dryRun-754a08ddf8bcb1cf22f310f09206dd783d42f7dd 0.0s - ✔ DRY-RUN MODE - ==> ==> naming to nginx-golang-mysql-backend 0.0s - ✔ DRY-RUN MODE - Network nginx-golang-mysql_default Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-db-1 Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-backend-1 Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-proxy-1 Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-db-1 Healthy 0.5s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-backend-1 Started 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-proxy-1 Started Started - ``` - From the example above, you can see that the first step is to pull the image defined by `db` service, then build the `backend` service. - Next, the containers are created. The `db` service is started, and the `backend` and `proxy` wait until the `db` service is healthy before starting. - - Dry Run mode works with almost all commands. You cannot use Dry Run mode with a command that doesn't change the state of a Compose stack such as `ps`, `ls`, `logs` for example. -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha.yaml b/docs/reference/docker_compose_alpha.yaml deleted file mode 100644 index e6b6b6e6b6f..00000000000 --- a/docs/reference/docker_compose_alpha.yaml +++ /dev/null @@ -1,31 +0,0 @@ -command: docker compose alpha -short: Experimental commands -long: Experimental commands -pname: docker compose -plink: docker_compose.yaml -cname: - - docker compose alpha generate - - docker compose alpha publish - - docker compose alpha viz -clink: - - docker_compose_alpha_generate.yaml - - docker_compose_alpha_publish.yaml - - docker_compose_alpha_viz.yaml -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: true -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha_dry-run.yaml b/docs/reference/docker_compose_alpha_dry-run.yaml deleted file mode 100644 index d489d39aeba..00000000000 --- a/docs/reference/docker_compose_alpha_dry-run.yaml +++ /dev/null @@ -1,14 +0,0 @@ -command: docker compose alpha dry-run -short: | - EXPERIMENTAL - Dry run command allow you to test a command without applying changes -long: | - EXPERIMENTAL - Dry run command allow you to test a command without applying changes -usage: docker compose alpha dry-run -- [COMMAND...] -pname: docker compose alpha -plink: docker_compose_alpha.yaml -deprecated: false -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha_generate.yaml b/docs/reference/docker_compose_alpha_generate.yaml deleted file mode 100644 index f31429c2d72..00000000000 --- a/docs/reference/docker_compose_alpha_generate.yaml +++ /dev/null @@ -1,53 +0,0 @@ -command: docker compose alpha generate -short: EXPERIMENTAL - Generate a Compose file from existing containers -long: EXPERIMENTAL - Generate a Compose file from existing containers -usage: docker compose alpha generate [OPTIONS] [CONTAINERS...] -pname: docker compose alpha -plink: docker_compose_alpha.yaml -options: - - option: format - value_type: string - default_value: yaml - description: 'Format the output. Values: [yaml | json]' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: name - value_type: string - description: Project name to set in the Compose file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: project-dir - value_type: string - description: Directory to use for the project - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: true -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha_publish.yaml b/docs/reference/docker_compose_alpha_publish.yaml deleted file mode 100644 index 9059cbf4869..00000000000 --- a/docs/reference/docker_compose_alpha_publish.yaml +++ /dev/null @@ -1,86 +0,0 @@ -command: docker compose alpha publish -short: Publish compose application -long: Publish compose application -usage: docker compose alpha publish [OPTIONS] REPOSITORY[:TAG] -pname: docker compose alpha -plink: docker_compose_alpha.yaml -options: - - option: app - value_type: bool - default_value: "false" - description: Published compose application (includes referenced images) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: insecure-registry - value_type: bool - default_value: "false" - description: Use insecure registry - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: oci-version - value_type: string - description: | - OCI image/artifact specification version (automatically determined by default) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: resolve-image-digests - value_type: bool - default_value: "false" - description: Pin image tags to digests - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: with-env - value_type: bool - default_value: "false" - description: Include environment variables in the published OCI artifact - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: "yes" - shorthand: "y" - value_type: bool - default_value: "false" - description: Assume "yes" as answer to all prompts - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: true -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha_scale.yaml b/docs/reference/docker_compose_alpha_scale.yaml deleted file mode 100644 index cc381493fa3..00000000000 --- a/docs/reference/docker_compose_alpha_scale.yaml +++ /dev/null @@ -1,35 +0,0 @@ -command: docker compose alpha scale -short: Scale services -long: Scale services -usage: docker compose alpha scale [SERVICE=REPLICAS...] -pname: docker compose alpha -plink: docker_compose_alpha.yaml -options: - - option: no-deps - value_type: bool - default_value: "false" - description: Don't start linked services. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha_viz.yaml b/docs/reference/docker_compose_alpha_viz.yaml deleted file mode 100644 index c07475caac8..00000000000 --- a/docs/reference/docker_compose_alpha_viz.yaml +++ /dev/null @@ -1,77 +0,0 @@ -command: docker compose alpha viz -short: EXPERIMENTAL - Generate a graphviz graph from your compose file -long: EXPERIMENTAL - Generate a graphviz graph from your compose file -usage: docker compose alpha viz [OPTIONS] -pname: docker compose alpha -plink: docker_compose_alpha.yaml -options: - - option: image - value_type: bool - default_value: "false" - description: Include service's image name in output graph - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: indentation-size - value_type: int - default_value: "1" - description: Number of tabs or spaces to use for indentation - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: networks - value_type: bool - default_value: "false" - description: Include service's attached networks in output graph - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: ports - value_type: bool - default_value: "false" - description: Include service's exposed ports in output graph - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: spaces - value_type: bool - default_value: "false" - description: |- - If given, space character ' ' will be used to indent, - otherwise tab character '\t' will be used - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: true -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha_watch.yaml b/docs/reference/docker_compose_alpha_watch.yaml deleted file mode 100644 index e8a7d9845ab..00000000000 --- a/docs/reference/docker_compose_alpha_watch.yaml +++ /dev/null @@ -1,47 +0,0 @@ -command: docker compose alpha watch -short: | - Watch build context for service and rebuild/refresh containers when files are updated -long: | - Watch build context for service and rebuild/refresh containers when files are updated -usage: docker compose alpha watch [SERVICE...] -pname: docker compose alpha -plink: docker_compose_alpha.yaml -options: - - option: no-up - value_type: bool - default_value: "false" - description: Do not build & start services before watching - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - value_type: bool - default_value: "false" - description: hide build output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_attach.yaml b/docs/reference/docker_compose_attach.yaml deleted file mode 100644 index 8fd6957ca18..00000000000 --- a/docs/reference/docker_compose_attach.yaml +++ /dev/null @@ -1,66 +0,0 @@ -command: docker compose attach -short: | - Attach local standard input, output, and error streams to a service's running container -long: | - Attach local standard input, output, and error streams to a service's running container -usage: docker compose attach [OPTIONS] SERVICE -pname: docker compose -plink: docker_compose.yaml -options: - - option: detach-keys - value_type: string - description: Override the key sequence for detaching from a container. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: index - value_type: int - default_value: "0" - description: index of the container if service has multiple replicas. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-stdin - value_type: bool - default_value: "false" - description: Do not attach STDIN - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: sig-proxy - value_type: bool - default_value: "true" - description: Proxy all received signals to the process - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_bridge.yaml b/docs/reference/docker_compose_bridge.yaml deleted file mode 100644 index 5ef9ebf5585..00000000000 --- a/docs/reference/docker_compose_bridge.yaml +++ /dev/null @@ -1,29 +0,0 @@ -command: docker compose bridge -short: Convert compose files into another model -long: Convert compose files into another model -pname: docker compose -plink: docker_compose.yaml -cname: - - docker compose bridge convert - - docker compose bridge transformations -clink: - - docker_compose_bridge_convert.yaml - - docker_compose_bridge_transformations.yaml -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_bridge_convert.yaml b/docs/reference/docker_compose_bridge_convert.yaml deleted file mode 100644 index f55f0b233c3..00000000000 --- a/docs/reference/docker_compose_bridge_convert.yaml +++ /dev/null @@ -1,59 +0,0 @@ -command: docker compose bridge convert -short: | - Convert compose files to Kubernetes manifests, Helm charts, or another model -long: | - Convert compose files to Kubernetes manifests, Helm charts, or another model -usage: docker compose bridge convert -pname: docker compose bridge -plink: docker_compose_bridge.yaml -options: - - option: output - shorthand: o - value_type: string - default_value: out - description: The output directory for the Kubernetes resources - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: templates - value_type: string - description: Directory containing transformation templates - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: transformation - shorthand: t - value_type: stringArray - default_value: '[]' - description: | - Transformation to apply to compose model (default: docker/compose-bridge-kubernetes) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_bridge_transformations.yaml b/docs/reference/docker_compose_bridge_transformations.yaml deleted file mode 100644 index 2ab5661f0b2..00000000000 --- a/docs/reference/docker_compose_bridge_transformations.yaml +++ /dev/null @@ -1,29 +0,0 @@ -command: docker compose bridge transformations -short: Manage transformation images -long: Manage transformation images -pname: docker compose bridge -plink: docker_compose_bridge.yaml -cname: - - docker compose bridge transformations create - - docker compose bridge transformations list -clink: - - docker_compose_bridge_transformations_create.yaml - - docker_compose_bridge_transformations_list.yaml -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_bridge_transformations_create.yaml b/docs/reference/docker_compose_bridge_transformations_create.yaml deleted file mode 100644 index e8dd9e58a51..00000000000 --- a/docs/reference/docker_compose_bridge_transformations_create.yaml +++ /dev/null @@ -1,36 +0,0 @@ -command: docker compose bridge transformations create -short: Create a new transformation -long: Create a new transformation -usage: docker compose bridge transformations create [OPTION] PATH -pname: docker compose bridge transformations -plink: docker_compose_bridge_transformations.yaml -options: - - option: from - shorthand: f - value_type: string - description: | - Existing transformation to copy (default: docker/compose-bridge-kubernetes) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_bridge_transformations_list.yaml b/docs/reference/docker_compose_bridge_transformations_list.yaml deleted file mode 100644 index 3afd3a84b8e..00000000000 --- a/docs/reference/docker_compose_bridge_transformations_list.yaml +++ /dev/null @@ -1,47 +0,0 @@ -command: docker compose bridge transformations list -aliases: docker compose bridge transformations list, docker compose bridge transformations ls -short: List available transformations -long: List available transformations -usage: docker compose bridge transformations list -pname: docker compose bridge transformations -plink: docker_compose_bridge_transformations.yaml -options: - - option: format - value_type: string - default_value: table - description: 'Format the output. Values: [table | json]' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only display transformer names - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_build.yaml b/docs/reference/docker_compose_build.yaml deleted file mode 100644 index e645a40aac2..00000000000 --- a/docs/reference/docker_compose_build.yaml +++ /dev/null @@ -1,214 +0,0 @@ -command: docker compose build -short: Build or rebuild services -long: |- - Services are built once and then tagged, by default as `project-service`. - - If the Compose file specifies an - [image](https://github.com/compose-spec/compose-spec/blob/main/spec.md#image) name, - the image is tagged with that name, substituting any variables beforehand. See - [variable interpolation](https://github.com/compose-spec/compose-spec/blob/main/spec.md#interpolation). - - If you change a service's `Dockerfile` or the contents of its build directory, - run `docker compose build` to rebuild it. -usage: docker compose build [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: build-arg - value_type: stringArray - default_value: '[]' - description: Set build-time variables for services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: builder - value_type: string - description: Set builder to use - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: check - value_type: bool - default_value: "false" - description: Check build configuration - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: compress - value_type: bool - default_value: "true" - description: Compress the build context using gzip. DEPRECATED - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: force-rm - value_type: bool - default_value: "true" - description: Always remove intermediate containers. DEPRECATED - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: memory - shorthand: m - value_type: bytes - default_value: "0" - description: | - Set memory limit for the build container. Not supported by BuildKit. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-cache - value_type: bool - default_value: "false" - description: Do not use cache when building the image - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-rm - value_type: bool - default_value: "false" - description: | - Do not remove intermediate containers after a successful build. DEPRECATED - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: parallel - value_type: bool - default_value: "true" - description: Build images in parallel. DEPRECATED - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: print - value_type: bool - default_value: "false" - description: Print equivalent bake file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: progress - value_type: string - description: Set type of ui output (auto, tty, plain, json, quiet) - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: provenance - value_type: string - description: Add a provenance attestation - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: pull - value_type: bool - default_value: "false" - description: Always attempt to pull a newer version of the image - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: push - value_type: bool - default_value: "false" - description: Push service images - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Suppress the build output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: sbom - value_type: string - description: Add a SBOM attestation - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: ssh - value_type: string - description: | - Set SSH authentications used when building service images. (use 'default' for using your default SSH Agent) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: with-dependencies - value_type: bool - default_value: "false" - description: Also build dependencies (transitively) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_commit.yaml b/docs/reference/docker_compose_commit.yaml deleted file mode 100644 index 95f4834a97b..00000000000 --- a/docs/reference/docker_compose_commit.yaml +++ /dev/null @@ -1,76 +0,0 @@ -command: docker compose commit -short: Create a new image from a service container's changes -long: Create a new image from a service container's changes -usage: docker compose commit [OPTIONS] SERVICE [REPOSITORY[:TAG]] -pname: docker compose -plink: docker_compose.yaml -options: - - option: author - shorthand: a - value_type: string - description: Author (e.g., "John Hannibal Smith <hannibal@a-team.com>") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: change - shorthand: c - value_type: list - description: Apply Dockerfile instruction to the created image - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: index - value_type: int - default_value: "0" - description: index of the container if service has multiple replicas. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: message - shorthand: m - value_type: string - description: Commit message - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: pause - shorthand: p - value_type: bool - default_value: "true" - description: Pause container during commit - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_config.yaml b/docs/reference/docker_compose_config.yaml deleted file mode 100644 index 3efc922b219..00000000000 --- a/docs/reference/docker_compose_config.yaml +++ /dev/null @@ -1,218 +0,0 @@ -command: docker compose config -short: Parse, resolve and render compose file in canonical format -long: |- - `docker compose config` renders the actual data model to be applied on the Docker Engine. - It merges the Compose files set by `-f` flags, resolves variables in the Compose file, and expands short-notation into - the canonical format. -usage: docker compose config [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: environment - value_type: bool - default_value: "false" - description: Print environment used for interpolation. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: format - value_type: string - description: 'Format the output. Values: [yaml | json]' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: hash - value_type: string - description: Print the service config hash, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: images - value_type: bool - default_value: "false" - description: Print the image names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: lock-image-digests - value_type: bool - default_value: "false" - description: Produces an override file with image digests - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: models - value_type: bool - default_value: "false" - description: Print the model names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: networks - value_type: bool - default_value: "false" - description: Print the network names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-consistency - value_type: bool - default_value: "false" - description: | - Don't check model consistency - warning: may produce invalid Compose output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-env-resolution - value_type: bool - default_value: "false" - description: Don't resolve service env files - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-interpolate - value_type: bool - default_value: "false" - description: Don't interpolate environment variables - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-normalize - value_type: bool - default_value: "false" - description: Don't normalize compose model - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-path-resolution - value_type: bool - default_value: "false" - description: Don't resolve file paths - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: output - shorthand: o - value_type: string - description: Save to file (default to stdout) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: profiles - value_type: bool - default_value: "false" - description: Print the profile names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only validate the configuration, don't print anything - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: resolve-image-digests - value_type: bool - default_value: "false" - description: Pin image tags to digests - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: services - value_type: bool - default_value: "false" - description: Print the service names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: variables - value_type: bool - default_value: "false" - description: Print model variables and default values. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: volumes - value_type: bool - default_value: "false" - description: Print the volume names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_convert.yaml b/docs/reference/docker_compose_convert.yaml deleted file mode 100644 index d1913221968..00000000000 --- a/docs/reference/docker_compose_convert.yaml +++ /dev/null @@ -1,140 +0,0 @@ -command: docker compose convert -aliases: docker compose convert, docker compose config -short: Converts the compose file to platform's canonical format -long: |- - `docker compose convert` renders the actual data model to be applied on the target platform. When used with the Docker engine, - it merges the Compose files set by `-f` flags, resolves variables in the Compose file, and expands short-notation into - the canonical format. - - To allow smooth migration from docker-compose, this subcommand declares alias `docker compose config` -usage: docker compose convert [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: format - value_type: string - default_value: yaml - description: 'Format the output. Values: [yaml | json]' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: hash - value_type: string - description: Print the service config hash, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: images - value_type: bool - default_value: "false" - description: Print the image names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-consistency - value_type: bool - default_value: "false" - description: | - Don't check model consistency - warning: may produce invalid Compose output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-interpolate - value_type: bool - default_value: "false" - description: Don't interpolate environment variables. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-normalize - value_type: bool - default_value: "false" - description: Don't normalize compose model. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: output - shorthand: o - value_type: string - description: Save to file (default to stdout) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: profiles - value_type: bool - default_value: "false" - description: Print the profile names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only validate the configuration, don't print anything. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: resolve-image-digests - value_type: bool - default_value: "false" - description: Pin image tags to digests. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: services - value_type: bool - default_value: "false" - description: Print the service names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: volumes - value_type: bool - default_value: "false" - description: Print the volume names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_cp.yaml b/docs/reference/docker_compose_cp.yaml deleted file mode 100644 index 24f6aec87f9..00000000000 --- a/docs/reference/docker_compose_cp.yaml +++ /dev/null @@ -1,69 +0,0 @@ -command: docker compose cp -short: Copy files/folders between a service container and the local filesystem -long: Copy files/folders between a service container and the local filesystem -usage: |- - docker compose cp [OPTIONS] SERVICE:SRC_PATH DEST_PATH|- - docker compose cp [OPTIONS] SRC_PATH|- SERVICE:DEST_PATH -pname: docker compose -plink: docker_compose.yaml -options: - - option: all - value_type: bool - default_value: "false" - description: Include containers created by the run command - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: archive - shorthand: a - value_type: bool - default_value: "false" - description: Archive mode (copy all uid/gid information) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: follow-link - shorthand: L - value_type: bool - default_value: "false" - description: Always follow symbol link in SRC_PATH - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: index - value_type: int - default_value: "0" - description: Index of the container if service has multiple replicas - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_create.yaml b/docs/reference/docker_compose_create.yaml deleted file mode 100644 index f6ab1b86824..00000000000 --- a/docs/reference/docker_compose_create.yaml +++ /dev/null @@ -1,119 +0,0 @@ -command: docker compose create -short: Creates containers for a service -long: Creates containers for a service -usage: docker compose create [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: build - value_type: bool - default_value: "false" - description: Build images before starting containers - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: force-recreate - value_type: bool - default_value: "false" - description: | - Recreate containers even if their configuration and image haven't changed - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-build - value_type: bool - default_value: "false" - description: Don't build an image, even if it's policy - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-recreate - value_type: bool - default_value: "false" - description: | - If containers already exist, don't recreate them. Incompatible with --force-recreate. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: pull - value_type: string - default_value: policy - description: Pull image before running ("always"|"missing"|"never"|"build") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet-pull - value_type: bool - default_value: "false" - description: Pull without printing progress information - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: remove-orphans - value_type: bool - default_value: "false" - description: Remove containers for services not defined in the Compose file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: scale - value_type: stringArray - default_value: '[]' - description: | - Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: "yes" - shorthand: "y" - value_type: bool - default_value: "false" - description: Assume "yes" as answer to all prompts and run non-interactively - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_down.yaml b/docs/reference/docker_compose_down.yaml deleted file mode 100644 index 77bf526289b..00000000000 --- a/docs/reference/docker_compose_down.yaml +++ /dev/null @@ -1,81 +0,0 @@ -command: docker compose down -short: Stop and remove containers, networks -long: |- - Stops containers and removes containers, networks, volumes, and images created by `up`. - - By default, the only things removed are: - - - Containers for services defined in the Compose file. - - Networks defined in the networks section of the Compose file. - - The default network, if one is used. - - Networks and volumes defined as external are never removed. - - Anonymous volumes are not removed by default. However, as they don’t have a stable name, they are not automatically - mounted by a subsequent `up`. For data that needs to persist between updates, use explicit paths as bind mounts or - named volumes. -usage: docker compose down [OPTIONS] [SERVICES] -pname: docker compose -plink: docker_compose.yaml -options: - - option: remove-orphans - value_type: bool - default_value: "false" - description: Remove containers for services not defined in the Compose file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: rmi - value_type: string - description: | - Remove images used by services. "local" remove only images that don't have a custom tag ("local"|"all") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: timeout - shorthand: t - value_type: int - default_value: "0" - description: Specify a shutdown timeout in seconds - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: volumes - shorthand: v - value_type: bool - default_value: "false" - description: | - Remove named volumes declared in the "volumes" section of the Compose file and anonymous volumes attached to containers - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_events.yaml b/docs/reference/docker_compose_events.yaml deleted file mode 100644 index 7c4cb4297f9..00000000000 --- a/docs/reference/docker_compose_events.yaml +++ /dev/null @@ -1,72 +0,0 @@ -command: docker compose events -short: Receive real time events from containers -long: |- - Stream container events for every container in the project. - - With the `--json` flag, a json object is printed one per line with the format: - - ```json - { - "time": "2015-11-20T18:01:03.615550", - "type": "container", - "action": "create", - "id": "213cf7...5fc39a", - "service": "web", - "attributes": { - "name": "application_web_1", - "image": "alpine:edge" - } - } - ``` - - The events that can be received using this can be seen [here](/reference/cli/docker/system/events/#object-types). -usage: docker compose events [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: json - value_type: bool - default_value: "false" - description: Output events as a stream of json objects - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: since - value_type: string - description: Show all events created since timestamp - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: until - value_type: string - description: Stream events until this timestamp - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_exec.yaml b/docs/reference/docker_compose_exec.yaml deleted file mode 100644 index 66ecfddab8d..00000000000 --- a/docs/reference/docker_compose_exec.yaml +++ /dev/null @@ -1,131 +0,0 @@ -command: docker compose exec -short: Execute a command in a running container -long: |- - This is the equivalent of `docker exec` targeting a Compose service. - - With this subcommand, you can run arbitrary commands in your services. Commands allocate a TTY by default, so - you can use a command such as `docker compose exec web sh` to get an interactive prompt. - - By default, Compose will enter container in interactive mode and allocate a TTY, while the equivalent `docker exec` - command requires passing `--interactive --tty` flags to get the same behavior. Compose also support those two flags - to offer a smooth migration between commands, whenever they are no-op by default. Still, `interactive` can be used to - force disabling interactive mode (`--interactive=false`), typically when `docker compose exec` command is used inside - a script. -usage: docker compose exec [OPTIONS] SERVICE COMMAND [ARGS...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: detach - shorthand: d - value_type: bool - default_value: "false" - description: 'Detached mode: Run command in the background' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: env - shorthand: e - value_type: stringArray - default_value: '[]' - description: Set environment variables - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: index - value_type: int - default_value: "0" - description: Index of the container if service has multiple replicas - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: interactive - shorthand: i - value_type: bool - default_value: "true" - description: Keep STDIN open even if not attached - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-tty - shorthand: T - value_type: bool - default_value: "true" - description: | - Disable pseudo-TTY allocation. By default 'docker compose exec' allocates a TTY. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: privileged - value_type: bool - default_value: "false" - description: Give extended privileges to the process - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: tty - shorthand: t - value_type: bool - default_value: "true" - description: Allocate a pseudo-TTY - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: user - shorthand: u - value_type: string - description: Run the command as this user - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: workdir - shorthand: w - value_type: string - description: Path to workdir directory for this command - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_export.yaml b/docs/reference/docker_compose_export.yaml deleted file mode 100644 index 5dfb3be0a47..00000000000 --- a/docs/reference/docker_compose_export.yaml +++ /dev/null @@ -1,45 +0,0 @@ -command: docker compose export -short: Export a service container's filesystem as a tar archive -long: Export a service container's filesystem as a tar archive -usage: docker compose export [OPTIONS] SERVICE -pname: docker compose -plink: docker_compose.yaml -options: - - option: index - value_type: int - default_value: "0" - description: index of the container if service has multiple replicas. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: output - shorthand: o - value_type: string - description: Write to a file, instead of STDOUT - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_images.yaml b/docs/reference/docker_compose_images.yaml deleted file mode 100644 index 33187df42d6..00000000000 --- a/docs/reference/docker_compose_images.yaml +++ /dev/null @@ -1,46 +0,0 @@ -command: docker compose images -short: List images used by the created containers -long: List images used by the created containers -usage: docker compose images [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: format - value_type: string - default_value: table - description: 'Format the output. Values: [table | json]' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only display IDs - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_kill.yaml b/docs/reference/docker_compose_kill.yaml deleted file mode 100644 index fbffcc9cb89..00000000000 --- a/docs/reference/docker_compose_kill.yaml +++ /dev/null @@ -1,51 +0,0 @@ -command: docker compose kill -short: Force stop service containers -long: |- - Forces running containers to stop by sending a `SIGKILL` signal. Optionally the signal can be passed, for example: - - ```console - $ docker compose kill -s SIGINT - ``` -usage: docker compose kill [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: remove-orphans - value_type: bool - default_value: "false" - description: Remove containers for services not defined in the Compose file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: signal - shorthand: s - value_type: string - default_value: SIGKILL - description: SIGNAL to send to the container - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_logs.yaml b/docs/reference/docker_compose_logs.yaml deleted file mode 100644 index 92d94dd108c..00000000000 --- a/docs/reference/docker_compose_logs.yaml +++ /dev/null @@ -1,109 +0,0 @@ -command: docker compose logs -short: View output from containers -long: Displays log output from services -usage: docker compose logs [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: follow - shorthand: f - value_type: bool - default_value: "false" - description: Follow log output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: index - value_type: int - default_value: "0" - description: index of the container if service has multiple replicas - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-color - value_type: bool - default_value: "false" - description: Produce monochrome output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-log-prefix - value_type: bool - default_value: "false" - description: Don't print prefix in logs - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: since - value_type: string - description: | - Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: tail - shorthand: "n" - value_type: string - default_value: all - description: | - Number of lines to show from the end of the logs for each container - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: timestamps - shorthand: t - value_type: bool - default_value: "false" - description: Show timestamps - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: until - value_type: string - description: | - Show logs before a timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_ls.yaml b/docs/reference/docker_compose_ls.yaml deleted file mode 100644 index dd6418c652f..00000000000 --- a/docs/reference/docker_compose_ls.yaml +++ /dev/null @@ -1,66 +0,0 @@ -command: docker compose ls -short: List running compose projects -long: Lists running Compose projects -usage: docker compose ls [OPTIONS] -pname: docker compose -plink: docker_compose.yaml -options: - - option: all - shorthand: a - value_type: bool - default_value: "false" - description: Show all stopped Compose projects - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: filter - value_type: filter - description: Filter output based on conditions provided - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: format - value_type: string - default_value: table - description: 'Format the output. Values: [table | json]' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only display project names - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_pause.yaml b/docs/reference/docker_compose_pause.yaml deleted file mode 100644 index 2ae1c402792..00000000000 --- a/docs/reference/docker_compose_pause.yaml +++ /dev/null @@ -1,25 +0,0 @@ -command: docker compose pause -short: Pause services -long: | - Pauses running containers of a service. They can be unpaused with `docker compose unpause`. -usage: docker compose pause [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_port.yaml b/docs/reference/docker_compose_port.yaml deleted file mode 100644 index 8a07f31ea50..00000000000 --- a/docs/reference/docker_compose_port.yaml +++ /dev/null @@ -1,45 +0,0 @@ -command: docker compose port -short: Print the public port for a port binding -long: Prints the public port for a port binding -usage: docker compose port [OPTIONS] SERVICE PRIVATE_PORT -pname: docker compose -plink: docker_compose.yaml -options: - - option: index - value_type: int - default_value: "0" - description: Index of the container if service has multiple replicas - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: protocol - value_type: string - default_value: tcp - description: tcp or udp - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_ps.yaml b/docs/reference/docker_compose_ps.yaml deleted file mode 100644 index d0370275695..00000000000 --- a/docs/reference/docker_compose_ps.yaml +++ /dev/null @@ -1,214 +0,0 @@ -command: docker compose ps -short: List containers -long: |- - Lists containers for a Compose project, with current status and exposed ports. - - ```console - $ docker compose ps - NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS - example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp - ``` - - By default, only running containers are shown. `--all` flag can be used to include stopped containers. - - ```console - $ docker compose ps --all - NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS - example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp - example-bar-1 alpine "/entrypoint.…" bar 4 seconds ago exited (0) - ``` -usage: docker compose ps [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: all - shorthand: a - value_type: bool - default_value: "false" - description: | - Show all stopped containers (including those created by the run command) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: filter - value_type: string - description: 'Filter services by a property (supported filters: status)' - details_url: '#filter' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: format - value_type: string - default_value: table - description: |- - Format output using a custom template: - 'table': Print output in table format with column headers (default) - 'table TEMPLATE': Print output in table format using the given Go template - 'json': Print in JSON format - 'TEMPLATE': Print output using the given Go template. - Refer to https://docs.docker.com/go/formatting/ for more information about formatting output with templates - details_url: '#format' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-trunc - value_type: bool - default_value: "false" - description: Don't truncate output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: orphans - value_type: bool - default_value: "true" - description: Include orphaned services (not declared by project) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only display IDs - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: services - value_type: bool - default_value: "false" - description: Display services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: status - value_type: stringArray - default_value: '[]' - description: | - Filter services by status. Values: [paused | restarting | removing | running | dead | created | exited] - details_url: '#status' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -examples: |- - ### Format the output (--format) {#format} - - By default, the `docker compose ps` command uses a table ("pretty") format to - show the containers. The `--format` flag allows you to specify alternative - presentations for the output. Currently, supported options are `pretty` (default), - and `json`, which outputs information about the containers as a JSON array: - - ```console - $ docker compose ps --format json - [{"ID":"1553b0236cf4d2715845f053a4ee97042c4f9a2ef655731ee34f1f7940eaa41a","Name":"example-bar-1","Command":"/docker-entrypoint.sh nginx -g 'daemon off;'","Project":"example","Service":"bar","State":"exited","Health":"","ExitCode":0,"Publishers":null},{"ID":"f02a4efaabb67416e1ff127d51c4b5578634a0ad5743bd65225ff7d1909a3fa0","Name":"example-foo-1","Command":"/docker-entrypoint.sh nginx -g 'daemon off;'","Project":"example","Service":"foo","State":"running","Health":"","ExitCode":0,"Publishers":[{"URL":"0.0.0.0","TargetPort":80,"PublishedPort":8080,"Protocol":"tcp"}]}] - ``` - - The JSON output allows you to use the information in other tools for further - processing, for example, using the [`jq` utility](https://stedolan.github.io/jq/) - to pretty-print the JSON: - - ```console - $ docker compose ps --format json | jq . - [ - { - "ID": "1553b0236cf4d2715845f053a4ee97042c4f9a2ef655731ee34f1f7940eaa41a", - "Name": "example-bar-1", - "Command": "/docker-entrypoint.sh nginx -g 'daemon off;'", - "Project": "example", - "Service": "bar", - "State": "exited", - "Health": "", - "ExitCode": 0, - "Publishers": null - }, - { - "ID": "f02a4efaabb67416e1ff127d51c4b5578634a0ad5743bd65225ff7d1909a3fa0", - "Name": "example-foo-1", - "Command": "/docker-entrypoint.sh nginx -g 'daemon off;'", - "Project": "example", - "Service": "foo", - "State": "running", - "Health": "", - "ExitCode": 0, - "Publishers": [ - { - "URL": "0.0.0.0", - "TargetPort": 80, - "PublishedPort": 8080, - "Protocol": "tcp" - } - ] - } - ] - ``` - - ### Filter containers by status (--status) {#status} - - Use the `--status` flag to filter the list of containers by status. For example, - to show only containers that are running or only containers that have exited: - - ```console - $ docker compose ps --status=running - NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS - example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp - - $ docker compose ps --status=exited - NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS - example-bar-1 alpine "/entrypoint.…" bar 4 seconds ago exited (0) - ``` - - ### Filter containers by status (--filter) {#filter} - - The [`--status` flag](#status) is a convenient shorthand for the `--filter status=<status>` - flag. The example below is the equivalent to the example from the previous section, - this time using the `--filter` flag: - - ```console - $ docker compose ps --filter status=running - NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS - example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp - ``` - - The `docker compose ps` command currently only supports the `--filter status=<status>` - option, but additional filter options may be added in the future. -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_publish.yaml b/docs/reference/docker_compose_publish.yaml deleted file mode 100644 index c3189d89c57..00000000000 --- a/docs/reference/docker_compose_publish.yaml +++ /dev/null @@ -1,86 +0,0 @@ -command: docker compose publish -short: Publish compose application -long: Publish compose application -usage: docker compose publish [OPTIONS] REPOSITORY[:TAG] -pname: docker compose -plink: docker_compose.yaml -options: - - option: app - value_type: bool - default_value: "false" - description: Published compose application (includes referenced images) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: insecure-registry - value_type: bool - default_value: "false" - description: Use insecure registry - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: oci-version - value_type: string - description: | - OCI image/artifact specification version (automatically determined by default) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: resolve-image-digests - value_type: bool - default_value: "false" - description: Pin image tags to digests - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: with-env - value_type: bool - default_value: "false" - description: Include environment variables in the published OCI artifact - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: "yes" - shorthand: "y" - value_type: bool - default_value: "false" - description: Assume "yes" as answer to all prompts - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_pull.yaml b/docs/reference/docker_compose_pull.yaml deleted file mode 100644 index 5b1316df132..00000000000 --- a/docs/reference/docker_compose_pull.yaml +++ /dev/null @@ -1,139 +0,0 @@ -command: docker compose pull -short: Pull service images -long: | - Pulls an image associated with a service defined in a `compose.yaml` file, but does not start containers based on those images -usage: docker compose pull [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: ignore-buildable - value_type: bool - default_value: "false" - description: Ignore images that can be built - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: ignore-pull-failures - value_type: bool - default_value: "false" - description: Pull what it can and ignores images with pull failures - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: include-deps - value_type: bool - default_value: "false" - description: Also pull services declared as dependencies - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-parallel - value_type: bool - default_value: "true" - description: DEPRECATED disable parallel pulling - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: parallel - value_type: bool - default_value: "true" - description: DEPRECATED pull multiple images in parallel - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: policy - value_type: string - description: Apply pull policy ("missing"|"always") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Pull without printing progress information - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -examples: |- - Consider the following `compose.yaml`: - - ```yaml - services: - db: - image: postgres - web: - build: . - command: bundle exec rails s -p 3000 -b '0.0.0.0' - volumes: - - .:/myapp - ports: - - "3000:3000" - depends_on: - - db - ``` - - If you run `docker compose pull ServiceName` in the same directory as the `compose.yaml` file that defines the service, - Docker pulls the associated image. For example, to call the postgres image configured as the db service in our example, - you would run `docker compose pull db`. - - ```console - $ docker compose pull db - [+] Running 1/15 - ⠸ db Pulling 12.4s - ⠿ 45b42c59be33 Already exists 0.0s - ⠹ 40adec129f1a Downloading 3.374MB/4.178MB 9.3s - ⠹ b4c431d00c78 Download complete 9.3s - ⠹ 2696974e2815 Download complete 9.3s - ⠹ 564b77596399 Downloading 5.622MB/7.965MB 9.3s - ⠹ 5044045cf6f2 Downloading 216.7kB/391.1kB 9.3s - ⠹ d736e67e6ac3 Waiting 9.3s - ⠹ 390c1c9a5ae4 Waiting 9.3s - ⠹ c0e62f172284 Waiting 9.3s - ⠹ ebcdc659c5bf Waiting 9.3s - ⠹ 29be22cb3acc Waiting 9.3s - ⠹ f63c47038e66 Waiting 9.3s - ⠹ 77a0c198cde5 Waiting 9.3s - ⠹ c8752d5b785c Waiting 9.3s - ``` - - `docker compose pull` tries to pull image for services with a build section. If pull fails, it lets you know this service image must be built. You can skip this by setting `--ignore-buildable` flag. -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_push.yaml b/docs/reference/docker_compose_push.yaml deleted file mode 100644 index be7c116065f..00000000000 --- a/docs/reference/docker_compose_push.yaml +++ /dev/null @@ -1,74 +0,0 @@ -command: docker compose push -short: Push service images -long: |- - Pushes images for services to their respective registry/repository. - - The following assumptions are made: - - You are pushing an image you have built locally - - You have access to the build key - - Examples - - ```yaml - services: - service1: - build: . - image: localhost:5000/yourimage ## goes to local registry - - service2: - build: . - image: your-dockerid/yourimage ## goes to your repository on Docker Hub - ``` -usage: docker compose push [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: ignore-push-failures - value_type: bool - default_value: "false" - description: Push what it can and ignores images with push failures - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: include-deps - value_type: bool - default_value: "false" - description: Also push images of services declared as dependencies - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Push without printing progress information - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_restart.yaml b/docs/reference/docker_compose_restart.yaml deleted file mode 100644 index 3bc0a3ad83a..00000000000 --- a/docs/reference/docker_compose_restart.yaml +++ /dev/null @@ -1,56 +0,0 @@ -command: docker compose restart -short: Restart service containers -long: |- - Restarts all stopped and running services, or the specified services only. - - If you make changes to your `compose.yml` configuration, these changes are not reflected - after running this command. For example, changes to environment variables (which are added - after a container is built, but before the container's command is executed) are not updated - after restarting. - - If you are looking to configure a service's restart policy, refer to - [restart](https://github.com/compose-spec/compose-spec/blob/main/spec.md#restart) - or [restart_policy](https://github.com/compose-spec/compose-spec/blob/main/deploy.md#restart_policy). -usage: docker compose restart [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: no-deps - value_type: bool - default_value: "false" - description: Don't restart dependent services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: timeout - shorthand: t - value_type: int - default_value: "0" - description: Specify a shutdown timeout in seconds - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_rm.yaml b/docs/reference/docker_compose_rm.yaml deleted file mode 100644 index 7ddafae4809..00000000000 --- a/docs/reference/docker_compose_rm.yaml +++ /dev/null @@ -1,84 +0,0 @@ -command: docker compose rm -short: Removes stopped service containers -long: |- - Removes stopped service containers. - - By default, anonymous volumes attached to containers are not removed. You can override this with `-v`. To list all - volumes, use `docker volume ls`. - - Any data which is not in a volume is lost. - - Running the command with no options also removes one-off containers created by `docker compose run`: - - ```console - $ docker compose rm - Going to remove djangoquickstart_web_run_1 - Are you sure? [yN] y - Removing djangoquickstart_web_run_1 ... done - ``` -usage: docker compose rm [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: all - shorthand: a - value_type: bool - default_value: "false" - description: Deprecated - no effect - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: force - shorthand: f - value_type: bool - default_value: "false" - description: Don't ask to confirm removal - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: stop - shorthand: s - value_type: bool - default_value: "false" - description: Stop the containers, if required, before removing - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: volumes - shorthand: v - value_type: bool - default_value: "false" - description: Remove any anonymous volumes attached to containers - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_run.yaml b/docs/reference/docker_compose_run.yaml deleted file mode 100644 index 61c7ca0e8cb..00000000000 --- a/docs/reference/docker_compose_run.yaml +++ /dev/null @@ -1,336 +0,0 @@ -command: docker compose run -short: Run a one-off command on a service -long: |- - Runs a one-time command against a service. - - The following command starts the `web` service and runs `bash` as its command: - - ```console - $ docker compose run web bash - ``` - - Commands you use with run start in new containers with configuration defined by that of the service, - including volumes, links, and other details. However, there are two important differences: - - First, the command passed by `run` overrides the command defined in the service configuration. For example, if the - `web` service configuration is started with `bash`, then `docker compose run web python app.py` overrides it with - `python app.py`. - - The second difference is that the `docker compose run` command does not create any of the ports specified in the - service configuration. This prevents port collisions with already-open ports. If you do want the service’s ports - to be created and mapped to the host, specify the `--service-ports` - - ```console - $ docker compose run --service-ports web python manage.py shell - ``` - - Alternatively, manual port mapping can be specified with the `--publish` or `-p` options, just as when using docker run: - - ```console - $ docker compose run --publish 8080:80 -p 2022:22 -p 127.0.0.1:2021:21 web python manage.py shell - ``` - - If you start a service configured with links, the run command first checks to see if the linked service is running - and starts the service if it is stopped. Once all the linked services are running, the run executes the command you - passed it. For example, you could run: - - ```console - $ docker compose run db psql -h db -U docker - ``` - - This opens an interactive PostgreSQL shell for the linked `db` container. - - If you do not want the run command to start linked containers, use the `--no-deps` flag: - - ```console - $ docker compose run --no-deps web python manage.py shell - ``` - - If you want to remove the container after running while overriding the container’s restart policy, use the `--rm` flag: - - ```console - $ docker compose run --rm web python manage.py db upgrade - ``` - - This runs a database upgrade script, and removes the container when finished running, even if a restart policy is - specified in the service configuration. -usage: docker compose run [OPTIONS] SERVICE [COMMAND] [ARGS...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: build - value_type: bool - default_value: "false" - description: Build image before starting container - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: cap-add - value_type: list - description: Add Linux capabilities - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: cap-drop - value_type: list - description: Drop Linux capabilities - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: detach - shorthand: d - value_type: bool - default_value: "false" - description: Run container in background and print container ID - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: entrypoint - value_type: string - description: Override the entrypoint of the image - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: env - shorthand: e - value_type: stringArray - default_value: '[]' - description: Set environment variables - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: env-from-file - value_type: stringArray - default_value: '[]' - description: Set environment variables from file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: interactive - shorthand: i - value_type: bool - default_value: "true" - description: Keep STDIN open even if not attached - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: label - shorthand: l - value_type: stringArray - default_value: '[]' - description: Add or override a label - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: name - value_type: string - description: Assign a name to the container - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-TTY - shorthand: T - value_type: bool - default_value: "true" - description: 'Disable pseudo-TTY allocation (default: auto-detected)' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-deps - value_type: bool - default_value: "false" - description: Don't start linked services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: publish - shorthand: p - value_type: stringArray - default_value: '[]' - description: Publish a container's port(s) to the host - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: pull - value_type: string - default_value: policy - description: Pull image before running ("always"|"missing"|"never") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Don't print anything to STDOUT - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet-build - value_type: bool - default_value: "false" - description: Suppress progress output from the build process - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet-pull - value_type: bool - default_value: "false" - description: Pull without printing progress information - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: remove-orphans - value_type: bool - default_value: "false" - description: Remove containers for services not defined in the Compose file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: rm - value_type: bool - default_value: "false" - description: Automatically remove the container when it exits - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: service-ports - shorthand: P - value_type: bool - default_value: "false" - description: | - Run command with all service's ports enabled and mapped to the host - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: tty - shorthand: t - value_type: bool - default_value: "true" - description: Allocate a pseudo-TTY - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: use-aliases - value_type: bool - default_value: "false" - description: | - Use the service's network useAliases in the network(s) the container connects to - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: user - shorthand: u - value_type: string - description: Run as specified username or uid - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: volume - shorthand: v - value_type: stringArray - default_value: '[]' - description: Bind mount a volume - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: workdir - shorthand: w - value_type: string - description: Working directory inside the container - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_scale.yaml b/docs/reference/docker_compose_scale.yaml deleted file mode 100644 index f840a51b4e1..00000000000 --- a/docs/reference/docker_compose_scale.yaml +++ /dev/null @@ -1,35 +0,0 @@ -command: docker compose scale -short: Scale services -long: Scale services -usage: docker compose scale [SERVICE=REPLICAS...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: no-deps - value_type: bool - default_value: "false" - description: Don't start linked services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_start.yaml b/docs/reference/docker_compose_start.yaml deleted file mode 100644 index 56f9bcdff7c..00000000000 --- a/docs/reference/docker_compose_start.yaml +++ /dev/null @@ -1,46 +0,0 @@ -command: docker compose start -short: Start services -long: Starts existing containers for a service -usage: docker compose start [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: wait - value_type: bool - default_value: "false" - description: Wait for services to be running|healthy. Implies detached mode. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: wait-timeout - value_type: int - default_value: "0" - description: | - Maximum duration in seconds to wait for the project to be running|healthy - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_stats.yaml b/docs/reference/docker_compose_stats.yaml deleted file mode 100644 index e6854b05a25..00000000000 --- a/docs/reference/docker_compose_stats.yaml +++ /dev/null @@ -1,71 +0,0 @@ -command: docker compose stats -short: Display a live stream of container(s) resource usage statistics -long: Display a live stream of container(s) resource usage statistics -usage: docker compose stats [OPTIONS] [SERVICE] -pname: docker compose -plink: docker_compose.yaml -options: - - option: all - shorthand: a - value_type: bool - default_value: "false" - description: Show all containers (default shows just running) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: format - value_type: string - description: |- - Format output using a custom template: - 'table': Print output in table format with column headers (default) - 'table TEMPLATE': Print output in table format using the given Go template - 'json': Print in JSON format - 'TEMPLATE': Print output using the given Go template. - Refer to https://docs.docker.com/engine/cli/formatting/ for more information about formatting output with templates - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-stream - value_type: bool - default_value: "false" - description: Disable streaming stats and only pull the first result - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-trunc - value_type: bool - default_value: "false" - description: Do not truncate output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_stop.yaml b/docs/reference/docker_compose_stop.yaml deleted file mode 100644 index f2ec34ccb3d..00000000000 --- a/docs/reference/docker_compose_stop.yaml +++ /dev/null @@ -1,37 +0,0 @@ -command: docker compose stop -short: Stop services -long: | - Stops running containers without removing them. They can be started again with `docker compose start`. -usage: docker compose stop [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: timeout - shorthand: t - value_type: int - default_value: "0" - description: Specify a shutdown timeout in seconds - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_top.yaml b/docs/reference/docker_compose_top.yaml deleted file mode 100644 index 17cdf7e3818..00000000000 --- a/docs/reference/docker_compose_top.yaml +++ /dev/null @@ -1,31 +0,0 @@ -command: docker compose top -short: Display the running processes -long: Displays the running processes -usage: docker compose top [SERVICES...] -pname: docker compose -plink: docker_compose.yaml -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -examples: |- - ```console - $ docker compose top - example_foo_1 - UID PID PPID C STIME TTY TIME CMD - root 142353 142331 2 15:33 ? 00:00:00 ping localhost -c 5 - ``` -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_unpause.yaml b/docs/reference/docker_compose_unpause.yaml deleted file mode 100644 index e2047720b8f..00000000000 --- a/docs/reference/docker_compose_unpause.yaml +++ /dev/null @@ -1,24 +0,0 @@ -command: docker compose unpause -short: Unpause services -long: Unpauses paused containers of a service -usage: docker compose unpause [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_up.yaml b/docs/reference/docker_compose_up.yaml deleted file mode 100644 index 8c78a8fa683..00000000000 --- a/docs/reference/docker_compose_up.yaml +++ /dev/null @@ -1,350 +0,0 @@ -command: docker compose up -short: Create and start containers -long: |- - Builds, (re)creates, starts, and attaches to containers for a service. - - Unless they are already running, this command also starts any linked services. - - The `docker compose up` command aggregates the output of each container (like `docker compose logs --follow` does). - One can optionally select a subset of services to attach to using `--attach` flag, or exclude some services using - `--no-attach` to prevent output to be flooded by some verbose services. - - When the command exits, all containers are stopped. Running `docker compose up --detach` starts the containers in the - background and leaves them running. - - If there are existing containers for a service, and the service’s configuration or image was changed after the - container’s creation, `docker compose up` picks up the changes by stopping and recreating the containers - (preserving mounted volumes). To prevent Compose from picking up changes, use the `--no-recreate` flag. - - If you want to force Compose to stop and recreate all containers, use the `--force-recreate` flag. - - If the process encounters an error, the exit code for this command is `1`. - If the process is interrupted using `SIGINT` (ctrl + C) or `SIGTERM`, the containers are stopped, and the exit code is `0`. -usage: docker compose up [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: abort-on-container-exit - value_type: bool - default_value: "false" - description: | - Stops all containers if any container was stopped. Incompatible with -d - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: abort-on-container-failure - value_type: bool - default_value: "false" - description: | - Stops all containers if any container exited with failure. Incompatible with -d - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: always-recreate-deps - value_type: bool - default_value: "false" - description: Recreate dependent containers. Incompatible with --no-recreate. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: attach - value_type: stringArray - default_value: '[]' - description: | - Restrict attaching to the specified services. Incompatible with --attach-dependencies. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: attach-dependencies - value_type: bool - default_value: "false" - description: Automatically attach to log output of dependent services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: build - value_type: bool - default_value: "false" - description: Build images before starting containers - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: detach - shorthand: d - value_type: bool - default_value: "false" - description: 'Detached mode: Run containers in the background' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: exit-code-from - value_type: string - description: | - Return the exit code of the selected service container. Implies --abort-on-container-exit - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: force-recreate - value_type: bool - default_value: "false" - description: | - Recreate containers even if their configuration and image haven't changed - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: menu - value_type: bool - default_value: "false" - description: | - Enable interactive shortcuts when running attached. Incompatible with --detach. Can also be enable/disable by setting COMPOSE_MENU environment var. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-attach - value_type: stringArray - default_value: '[]' - description: Do not attach (stream logs) to the specified services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-build - value_type: bool - default_value: "false" - description: Don't build an image, even if it's policy - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-color - value_type: bool - default_value: "false" - description: Produce monochrome output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-deps - value_type: bool - default_value: "false" - description: Don't start linked services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-log-prefix - value_type: bool - default_value: "false" - description: Don't print prefix in logs - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-recreate - value_type: bool - default_value: "false" - description: | - If containers already exist, don't recreate them. Incompatible with --force-recreate. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-start - value_type: bool - default_value: "false" - description: Don't start the services after creating them - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: pull - value_type: string - default_value: policy - description: Pull image before running ("always"|"missing"|"never") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet-build - value_type: bool - default_value: "false" - description: Suppress the build output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet-pull - value_type: bool - default_value: "false" - description: Pull without printing progress information - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: remove-orphans - value_type: bool - default_value: "false" - description: Remove containers for services not defined in the Compose file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: renew-anon-volumes - shorthand: V - value_type: bool - default_value: "false" - description: | - Recreate anonymous volumes instead of retrieving data from the previous containers - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: scale - value_type: stringArray - default_value: '[]' - description: | - Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: timeout - shorthand: t - value_type: int - default_value: "0" - description: | - Use this timeout in seconds for container shutdown when attached or when containers are already running - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: timestamps - value_type: bool - default_value: "false" - description: Show timestamps - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: wait - value_type: bool - default_value: "false" - description: Wait for services to be running|healthy. Implies detached mode. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: wait-timeout - value_type: int - default_value: "0" - description: | - Maximum duration in seconds to wait for the project to be running|healthy - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: watch - shorthand: w - value_type: bool - default_value: "false" - description: | - Watch source code and rebuild/refresh containers when files are updated. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: "yes" - shorthand: "y" - value_type: bool - default_value: "false" - description: Assume "yes" as answer to all prompts and run non-interactively - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_version.yaml b/docs/reference/docker_compose_version.yaml deleted file mode 100644 index 789e94818ea..00000000000 --- a/docs/reference/docker_compose_version.yaml +++ /dev/null @@ -1,45 +0,0 @@ -command: docker compose version -short: Show the Docker Compose version information -long: Show the Docker Compose version information -usage: docker compose version [OPTIONS] -pname: docker compose -plink: docker_compose.yaml -options: - - option: format - shorthand: f - value_type: string - description: 'Format the output. Values: [pretty | json]. (Default: pretty)' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: short - value_type: bool - default_value: "false" - description: Shows only Compose's version number - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_volumes.yaml b/docs/reference/docker_compose_volumes.yaml deleted file mode 100644 index 20516db7f13..00000000000 --- a/docs/reference/docker_compose_volumes.yaml +++ /dev/null @@ -1,52 +0,0 @@ -command: docker compose volumes -short: List volumes -long: List volumes -usage: docker compose volumes [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: format - value_type: string - default_value: table - description: |- - Format output using a custom template: - 'table': Print output in table format with column headers (default) - 'table TEMPLATE': Print output in table format using the given Go template - 'json': Print in JSON format - 'TEMPLATE': Print output using the given Go template. - Refer to https://docs.docker.com/go/formatting/ for more information about formatting output with templates - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only display volume names - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_wait.yaml b/docs/reference/docker_compose_wait.yaml deleted file mode 100644 index 5d8f3013cc0..00000000000 --- a/docs/reference/docker_compose_wait.yaml +++ /dev/null @@ -1,35 +0,0 @@ -command: docker compose wait -short: Block until containers of all (or specified) services stop. -long: Block until containers of all (or specified) services stop. -usage: docker compose wait SERVICE [SERVICE...] [OPTIONS] -pname: docker compose -plink: docker_compose.yaml -options: - - option: down-project - value_type: bool - default_value: "false" - description: Drops project when the first container stops - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_watch.yaml b/docs/reference/docker_compose_watch.yaml deleted file mode 100644 index a3e3e802201..00000000000 --- a/docs/reference/docker_compose_watch.yaml +++ /dev/null @@ -1,57 +0,0 @@ -command: docker compose watch -short: | - Watch build context for service and rebuild/refresh containers when files are updated -long: | - Watch build context for service and rebuild/refresh containers when files are updated -usage: docker compose watch [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: no-up - value_type: bool - default_value: "false" - description: Do not build & start services before watching - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: prune - value_type: bool - default_value: "true" - description: Prune dangling images on rebuild - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - value_type: bool - default_value: "false" - description: hide build output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/sdk.md b/docs/sdk.md deleted file mode 100644 index 3a03c7fdc1d..00000000000 --- a/docs/sdk.md +++ /dev/null @@ -1,157 +0,0 @@ -# Using the `docker/compose` SDK - -The `docker/compose` package can be used as a Go library by third-party applications to programmatically manage -containerized applications defined in Compose files. This SDK provides a comprehensive API that lets you -integrate Compose functionality directly into your applications, allowing you to load, validate, and manage -multi-container environments without relying on the Compose CLI. - -Whether you need to orchestrate containers as part of -a deployment pipeline, build custom management tools, or embed container orchestration into your application, the -Compose SDK offers the same powerful capabilities that drive the Docker Compose command-line tool. - -## Set up the SDK - -To get started, create an SDK instance using the `NewComposeService()` function, which initializes a service with the -necessary configuration to interact with the Docker daemon and manage Compose projects. This service instance provides -methods for all core Compose operations including creating, starting, stopping, and removing containers, as well as -loading and validating Compose files. The service handles the underlying Docker API interactions and resource -management, allowing you to focus on your application logic. - -## Example usage - -Here's a basic example demonstrating how to load a Compose project and start the services: - -```go -package main - -import ( - "context" - "log" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/flags" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -func main() { - ctx := context.Background() - - dockerCLI, err := command.NewDockerCli() - if err != nil { - log.Fatalf("Failed to create docker CLI: %v", err) - } - err = dockerCLI.Initialize(&flags.ClientOptions{}) - if err != nil { - log.Fatalf("Failed to initialize docker CLI: %v", err) - } - - // Create a new Compose service instance - service, err := compose.NewComposeService(dockerCLI) - if err != nil { - log.Fatalf("Failed to create compose service: %v", err) - } - - // Load the Compose project from a compose file - project, err := service.LoadProject(ctx, api.ProjectLoadOptions{ - ConfigPaths: []string{"compose.yaml"}, - ProjectName: "my-app", - }) - if err != nil { - log.Fatalf("Failed to load project: %v", err) - } - - // Start the services defined in the Compose file - err = service.Up(ctx, project, api.UpOptions{ - Create: api.CreateOptions{}, - Start: api.StartOptions{}, - }) - if err != nil { - log.Fatalf("Failed to start services: %v", err) - } - - log.Printf("Successfully started project: %s", project.Name) -} -``` - -This example demonstrates the core workflow - creating a service instance, loading a project from a Compose file, and -starting the services. The SDK provides many additional operations for managing the lifecycle of your containerized -application. - -## Customizing the SDK - -The `NewComposeService()` function accepts optional `compose.Option` parameters to customize the SDK behavior. These -options allow you to configure I/O streams, concurrency limits, dry-run mode, and other advanced features. - -```go - // Create a custom output buffer to capture logs - var outputBuffer bytes.Buffer - - // Create a compose service with custom options - service, err := compose.NewComposeService(dockerCLI, - compose.WithOutputStream(&outputBuffer), // Redirect output to custom writer - compose.WithErrorStream(os.Stderr), // Use stderr for errors - compose.WithMaxConcurrency(4), // Limit concurrent operations - compose.WithPrompt(compose.AlwaysOkPrompt()), // Auto-confirm all prompts - ) -``` - -### Available options - -- `WithOutputStream(io.Writer)` - Redirect standard output to a custom writer -- `WithErrorStream(io.Writer)` - Redirect error output to a custom writer -- `WithInputStream(io.Reader)` - Provide a custom input stream for interactive prompts -- `WithStreams(out, err, in)` - Set all I/O streams at once -- `WithMaxConcurrency(int)` - Limit the number of concurrent operations against the Docker API -- `WithPrompt(Prompt)` - Customize user confirmation behavior (use `AlwaysOkPrompt()` for non-interactive mode) -- `WithDryRun` - Run operations in dry-run mode without actually applying changes -- `WithContextInfo(api.ContextInfo)` - Set custom Docker context information -- `WithProxyConfig(map[string]string)` - Configure HTTP proxy settings for builds -- `WithEventProcessor(progress.EventProcessor)` - Receive progress events and operation notifications - -These options provide fine-grained control over the SDK's behavior, making it suitable for various integration -scenarios including CLI tools, web services, automation scripts, and testing environments. - -## Tracking operations with `EventProcessor` - -The `EventProcessor` interface allows you to monitor Compose operations in real-time by receiving events about changes -applied to Docker resources such as images, containers, volumes, and networks. This is particularly useful for building -user interfaces, logging systems, or monitoring tools that need to track the progress of Compose operations. - -### Understanding `EventProcessor` - -A Compose operation, such as `up`, `down`, `build`, performs a series of changes to Docker resources. The -`EventProcessor` receives notifications about these changes through three key methods: - -- `Start(ctx, operation)` - Called when a Compose operation begins, for example `up` -- `On(events...)` - Called with progress events for individual resource changes, for example, container starting, image - being pulled -- `Done(operation, success)` - Called when the operation completes, indicating success or failure - -Each event contains information about the resource being modified, its current status, and progress indicators when -applicable (such as download progress for image pulls). - -### Event status types - -Events report resource changes with the following status types: - -- Working - Operation is in progress, for example, creating, starting, pulling -- Done - Operation completed successfully -- Warning - Operation completed with warnings -- Error - Operation failed - -Common status text values include: `Creating`, `Created`, `Starting`, `Started`, `Running`, `Stopping`, `Stopped`, -`Removing`, `Removed`, `Building`, `Built`, `Pulling`, `Pulled`, and more. - -### Built-in `EventProcessor` implementations - -The SDK provides three ready-to-use `EventProcessor` implementations: - -- `progress.NewTTYWriter(io.Writer)` - Renders an interactive terminal UI with progress bars and task lists - (similar to the Docker Compose CLI output) -- `progress.NewPlainWriter(io.Writer)` - Outputs simple text-based progress messages suitable for non-interactive - environments or log files -- `progress.NewJSONWriter()` - Render events as JSON objects -- `progress.NewQuietWriter()` - (Default) Silently processes events without producing any output - -Using `EventProcessor`, a custom UI can be plugged into `docker/compose`. diff --git a/docs/yaml/main/generate.go b/docs/yaml/main/generate.go deleted file mode 100644 index 335efe9cc3b..00000000000 --- a/docs/yaml/main/generate.go +++ /dev/null @@ -1,104 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package main - -import ( - "fmt" - "os" - "path/filepath" - - clidocstool "github.com/docker/cli-docs-tool" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/compose" -) - -func generateDocs(opts *options) error { - dockerCLI, err := command.NewDockerCli() - if err != nil { - return err - } - cmd := &cobra.Command{ - Use: "docker", - DisableAutoGenTag: true, - } - cmd.AddCommand(compose.RootCommand(dockerCLI, nil)) - disableFlagsInUseLine(cmd) - - tool, err := clidocstool.New(clidocstool.Options{ - Root: cmd, - SourceDir: opts.source, - TargetDir: opts.target, - Plugin: true, - }) - if err != nil { - return err - } - for _, format := range opts.formats { - switch format { - case "yaml": - if err := tool.GenYamlTree(cmd); err != nil { - return err - } - case "md": - if err := tool.GenMarkdownTree(cmd); err != nil { - return err - } - default: - return fmt.Errorf("unknown format %q", format) - } - } - return nil -} - -func disableFlagsInUseLine(cmd *cobra.Command) { - visitAll(cmd, func(ccmd *cobra.Command) { - // do not add a `[flags]` to the end of the usage line. - ccmd.DisableFlagsInUseLine = true - }) -} - -// visitAll will traverse all commands from the root. -// This is different from the VisitAll of cobra.Command where only parents -// are checked. -func visitAll(root *cobra.Command, fn func(*cobra.Command)) { - for _, cmd := range root.Commands() { - visitAll(cmd, fn) - } - fn(root) -} - -type options struct { - source string - target string - formats []string -} - -func main() { - cwd, _ := os.Getwd() - opts := &options{ - source: filepath.Join(cwd, "docs", "reference"), - target: filepath.Join(cwd, "docs", "reference"), - formats: []string{"yaml", "md"}, - } - fmt.Printf("Project root: %s\n", opts.source) - fmt.Printf("Generating yaml files into %s\n", opts.target) - if err := generateDocs(opts); err != nil { - _, _ = fmt.Fprintf(os.Stderr, "Failed to generate documentation: %s\n", err.Error()) - } -} diff --git a/experimental/compose_swarm_networking.md b/experimental/compose_swarm_networking.md new file mode 100644 index 00000000000..905f52f8007 --- /dev/null +++ b/experimental/compose_swarm_networking.md @@ -0,0 +1,5 @@ +# Experimental: Compose, Swarm and Multi-Host Networking + +Compose now supports multi-host networking as standard. Read more here: + +https://docs.docker.com/compose/networking diff --git a/go.mod b/go.mod deleted file mode 100644 index 466b4e5999c..00000000000 --- a/go.mod +++ /dev/null @@ -1,157 +0,0 @@ -module github.com/docker/compose/v5 - -go 1.24.3 - -require ( - github.com/AlecAivazis/survey/v2 v2.3.7 - github.com/DefangLabs/secret-detector v0.0.0-20250403165618-22662109213e - github.com/Microsoft/go-winio v0.6.2 - github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d - github.com/buger/goterm v1.0.4 - github.com/compose-spec/compose-go/v2 v2.10.1 - github.com/containerd/console v1.0.5 - github.com/containerd/containerd/v2 v2.2.1 - github.com/containerd/errdefs v1.0.0 - github.com/containerd/platforms v1.0.0-rc.2 - github.com/distribution/reference v0.6.0 - github.com/docker/buildx v0.30.1 - github.com/docker/cli v28.5.2+incompatible - github.com/docker/cli-docs-tool v0.11.0 - github.com/docker/docker v28.5.2+incompatible - github.com/docker/go-connections v0.6.0 - github.com/docker/go-units v0.5.0 - github.com/eiannone/keyboard v0.0.0-20220611211555-0d226195f203 - github.com/fsnotify/fsevents v0.2.0 - github.com/go-viper/mapstructure/v2 v2.5.0 - github.com/google/go-cmp v0.7.0 - github.com/google/uuid v1.6.0 - github.com/hashicorp/go-version v1.8.0 - github.com/jonboulle/clockwork v0.5.0 - github.com/mattn/go-shellwords v1.0.12 - github.com/mitchellh/go-ps v1.0.0 - github.com/moby/buildkit v0.26.3 - github.com/moby/go-archive v0.1.0 - github.com/moby/patternmatcher v0.6.0 - github.com/moby/sys/atomicwriter v0.1.0 - github.com/morikuni/aec v1.1.0 - github.com/opencontainers/go-digest v1.0.0 - github.com/opencontainers/image-spec v1.1.1 - github.com/otiai10/copy v1.14.1 - github.com/sirupsen/logrus v1.9.4 - github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 - github.com/spf13/cobra v1.10.2 - github.com/spf13/pflag v1.0.10 - github.com/stretchr/testify v1.11.1 - github.com/tilt-dev/fsnotify v1.4.8-0.20220602155310-fff9c274a375 - go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 - go.opentelemetry.io/otel v1.38.0 - go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0 - go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.38.0 - go.opentelemetry.io/otel/metric v1.38.0 - go.opentelemetry.io/otel/sdk v1.38.0 - go.opentelemetry.io/otel/trace v1.38.0 - go.uber.org/goleak v1.3.0 - go.uber.org/mock v0.6.0 - go.yaml.in/yaml/v4 v4.0.0-rc.4 - golang.org/x/sync v0.19.0 - golang.org/x/sys v0.41.0 - google.golang.org/grpc v1.78.0 - gotest.tools/v3 v3.5.2 - tags.cncf.io/container-device-interface v1.1.0 -) - -require ( - github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect - github.com/beorn7/perks v1.0.1 // indirect - github.com/cenkalti/backoff/v5 v5.0.3 // indirect - github.com/cespare/xxhash/v2 v2.3.0 // indirect - github.com/containerd/containerd/api v1.10.0 // indirect - github.com/containerd/continuity v0.4.5 // indirect - github.com/containerd/errdefs/pkg v0.3.0 // indirect - github.com/containerd/log v0.1.0 // indirect - github.com/containerd/ttrpc v1.2.7 // indirect - github.com/containerd/typeurl/v2 v2.2.3 // indirect - github.com/cpuguy83/go-md2man/v2 v2.0.7 // indirect - github.com/davecgh/go-spew v1.1.1 // indirect - github.com/docker/distribution v2.8.3+incompatible // indirect - github.com/docker/docker-credential-helpers v0.9.3 // indirect - github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c // indirect - github.com/docker/go-metrics v0.0.1 // indirect - github.com/felixge/httpsnoop v1.0.4 // indirect - github.com/fvbommel/sortorder v1.1.0 // indirect - github.com/go-logr/logr v1.4.3 // indirect - github.com/go-logr/stdr v1.2.2 // indirect - github.com/gofrs/flock v0.13.0 // indirect - github.com/gogo/protobuf v1.3.2 // indirect - github.com/golang-jwt/jwt/v5 v5.3.0 // indirect - github.com/golang/protobuf v1.5.4 // indirect - github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect - github.com/gorilla/mux v1.8.1 // indirect - github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2 // indirect - github.com/hashicorp/errwrap v1.1.0 // indirect - github.com/hashicorp/go-multierror v1.1.1 // indirect - github.com/in-toto/in-toto-golang v0.9.0 // indirect - github.com/inconshreveable/mousetrap v1.1.0 // indirect - github.com/inhies/go-bytesize v0.0.0-20220417184213-4913239db9cf // indirect - github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect - github.com/klauspost/compress v1.18.2 // indirect - github.com/magiconair/properties v1.8.9 // indirect - github.com/mattn/go-colorable v0.1.14 // indirect - github.com/mattn/go-isatty v0.0.20 // indirect - github.com/mattn/go-runewidth v0.0.16 // indirect - github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b // indirect - github.com/miekg/pkcs11 v1.1.1 // indirect - github.com/mitchellh/hashstructure/v2 v2.0.2 // indirect - github.com/mitchellh/mapstructure v1.5.0 // indirect - github.com/moby/docker-image-spec v1.3.1 // indirect - github.com/moby/locker v1.0.1 // indirect - github.com/moby/sys/capability v0.4.0 // indirect - github.com/moby/sys/sequential v0.6.0 // indirect - github.com/moby/sys/signal v0.7.1 // indirect - github.com/moby/sys/symlink v0.3.0 // indirect - github.com/moby/sys/user v0.4.0 // indirect - github.com/moby/sys/userns v0.1.0 // indirect - github.com/moby/term v0.5.2 // indirect - github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect - github.com/otiai10/mint v1.6.3 // indirect - github.com/pelletier/go-toml v1.9.5 // indirect - github.com/pkg/errors v0.9.1 // indirect - github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect - github.com/pmezard/go-difflib v1.0.0 // indirect - github.com/prometheus/client_golang v1.23.2 // indirect - github.com/prometheus/client_model v0.6.2 // indirect - github.com/prometheus/common v0.66.1 // indirect - github.com/prometheus/procfs v0.16.1 // indirect - github.com/rivo/uniseg v0.2.0 // indirect - github.com/russross/blackfriday/v2 v2.1.0 // indirect - github.com/santhosh-tekuri/jsonschema/v6 v6.0.1 // indirect - github.com/secure-systems-lab/go-securesystemslib v0.9.1 // indirect - github.com/shibumi/go-pathspec v1.3.0 // indirect - github.com/theupdateframework/notary v0.7.0 // indirect - github.com/tonistiigi/dchapes-mode v0.0.0-20250318174251-73d941a28323 // indirect - github.com/tonistiigi/fsutil v0.0.0-20250605211040-586307ad452f // indirect - github.com/tonistiigi/go-csvvalue v0.0.0-20240814133006-030d3b2625d0 // indirect - github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea // indirect - github.com/tonistiigi/vt100 v0.0.0-20240514184818-90bafcd6abab // indirect - github.com/xhit/go-str2duration/v2 v2.1.0 // indirect - go.opentelemetry.io/auto/sdk v1.2.1 // indirect - go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0 // indirect - go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.63.0 // indirect - go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.38.0 // indirect - go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.38.0 // indirect - go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0 // indirect - go.opentelemetry.io/otel/sdk/metric v1.38.0 // indirect - go.opentelemetry.io/proto/otlp v1.7.1 // indirect - go.yaml.in/yaml/v2 v2.4.2 // indirect - go.yaml.in/yaml/v3 v3.0.4 // indirect - golang.org/x/crypto v0.45.0 // indirect - golang.org/x/net v0.47.0 // indirect - golang.org/x/term v0.37.0 // indirect - golang.org/x/text v0.31.0 // indirect - golang.org/x/time v0.14.0 // indirect - google.golang.org/genproto/googleapis/api v0.0.0-20251029180050-ab9386a59fda // indirect - google.golang.org/genproto/googleapis/rpc v0.0.0-20251029180050-ab9386a59fda // indirect - google.golang.org/protobuf v1.36.10 // indirect - gopkg.in/ini.v1 v1.67.0 // indirect - gopkg.in/yaml.v3 v3.0.1 // indirect -) diff --git a/go.sum b/go.sum deleted file mode 100644 index 75f074fb4e2..00000000000 --- a/go.sum +++ /dev/null @@ -1,568 +0,0 @@ -github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 h1:He8afgbRMd7mFxO99hRNu+6tazq8nFF9lIwo9JFroBk= -github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8= -github.com/AlecAivazis/survey/v2 v2.3.7 h1:6I/u8FvytdGsgonrYsVn2t8t4QiRnh6QSTqkkhIiSjQ= -github.com/AlecAivazis/survey/v2 v2.3.7/go.mod h1:xUTIdE4KCOIjsBAE1JYsUPoCqYdZ1reCfTwbto0Fduo= -github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg= -github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= -github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ= -github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= -github.com/DefangLabs/secret-detector v0.0.0-20250403165618-22662109213e h1:rd4bOvKmDIx0WeTv9Qz+hghsgyjikFiPrseXHlKepO0= -github.com/DefangLabs/secret-detector v0.0.0-20250403165618-22662109213e/go.mod h1:blbwPQh4DTlCZEfk1BLU4oMIhLda2U+A840Uag9DsZw= -github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= -github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= -github.com/Microsoft/hcsshim v0.14.0-rc.1 h1:qAPXKwGOkVn8LlqgBN8GS0bxZ83hOJpcjxzmlQKxKsQ= -github.com/Microsoft/hcsshim v0.14.0-rc.1/go.mod h1:hTKFGbnDtQb1wHiOWv4v0eN+7boSWAHyK/tNAaYZL0c= -github.com/Netflix/go-expect v0.0.0-20220104043353-73e0943537d2 h1:+vx7roKuyA63nhn5WAunQHLTznkw5W8b1Xc0dNjp83s= -github.com/Netflix/go-expect v0.0.0-20220104043353-73e0943537d2/go.mod h1:HBCaDeC1lPdgDeDbhX8XFpy1jqjK0IBG8W5K+xYqA0w= -github.com/Shopify/logrus-bugsnag v0.0.0-20170309145241-6dbc35f2c30d h1:hi6J4K6DKrR4/ljxn6SF6nURyu785wKMuQcjt7H3VCQ= -github.com/Shopify/logrus-bugsnag v0.0.0-20170309145241-6dbc35f2c30d/go.mod h1:HI8ITrYtUY+O+ZhtlqUnD8+KwNPOyugEhfP9fdUIaEQ= -github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d h1:licZJFw2RwpHMqeKTCYkitsPqHNxTmd4SNR5r94FGM8= -github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d/go.mod h1:asat636LX7Bqt5lYEZ27JNDcqxfjdBQuJ/MM4CN/Lzo= -github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= -github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= -github.com/anchore/go-struct-converter v0.0.0-20221118182256-c68fdcfa2092 h1:aM1rlcoLz8y5B2r4tTLMiVTrMtpfY0O8EScKJxaSaEc= -github.com/anchore/go-struct-converter v0.0.0-20221118182256-c68fdcfa2092/go.mod h1:rYqSE9HbjzpHTI74vwPvae4ZVYZd1lue2ta6xHPdblA= -github.com/beorn7/perks v0.0.0-20150223135152-b965b613227f/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= -github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= -github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= -github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= -github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= -github.com/bitly/go-hostpool v0.1.0/go.mod h1:4gOCgp6+NZnVqlKyZ/iBZFTAJKembaVENUpMkpg42fw= -github.com/bitly/go-simplejson v0.5.0/go.mod h1:cXHtHw4XUPsvGaxgjIAn8PhEWG9NfngEKAMDJEczWVA= -github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869/go.mod h1:Ekp36dRnpXw/yCqJaO+ZrUyxD+3VXMFFr56k5XYrpB4= -github.com/buger/goterm v1.0.4 h1:Z9YvGmOih81P0FbVtEYTFF6YsSgxSUKEhf/f9bTMXbY= -github.com/buger/goterm v1.0.4/go.mod h1:HiFWV3xnkolgrBV3mY8m0X0Pumt4zg4QhbdOzQtB8tE= -github.com/bugsnag/bugsnag-go v1.0.5-0.20150529004307-13fd6b8acda0 h1:s7+5BfS4WFJoVF9pnB8kBk03S7pZXRdKamnV0FOl5Sc= -github.com/bugsnag/bugsnag-go v1.0.5-0.20150529004307-13fd6b8acda0/go.mod h1:2oa8nejYd4cQ/b0hMIopN0lCRxU0bueqREvZLWFrtK8= -github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b h1:otBG+dV+YK+Soembjv71DPz3uX/V/6MMlSyD9JBQ6kQ= -github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b/go.mod h1:obH5gd0BsqsP2LwDJ9aOkm/6J86V6lyAXCoQWGw3K50= -github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 h1:nvj0OLI3YqYXer/kZD8Ri1aaunCxIEsOst1BVJswV0o= -github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0/go.mod h1:D/8v3kj0zr8ZAKg1AQ6crr+5VwKN5eIywRkfhyM/+dE= -github.com/cenkalti/backoff/v5 v5.0.3 h1:ZN+IMa753KfX5hd8vVaMixjnqRZ3y8CuJKRKj1xcsSM= -github.com/cenkalti/backoff/v5 v5.0.3/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw= -github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= -github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/cloudflare/cfssl v0.0.0-20180223231731-4e2dcbde5004 h1:lkAMpLVBDaj17e85keuznYcH5rqI438v41pKcBl4ZxQ= -github.com/cloudflare/cfssl v0.0.0-20180223231731-4e2dcbde5004/go.mod h1:yMWuSON2oQp+43nFtAV/uvKQIFpSPerB57DCt9t8sSA= -github.com/codahale/rfc6979 v0.0.0-20141003034818-6a90f24967eb h1:EDmT6Q9Zs+SbUoc7Ik9EfrFqcylYqgPZ9ANSbTAntnE= -github.com/codahale/rfc6979 v0.0.0-20141003034818-6a90f24967eb/go.mod h1:ZjrT6AXHbDs86ZSdt/osfBi5qfexBrKUdONk989Wnk4= -github.com/compose-spec/compose-go/v2 v2.10.1 h1:mFbXobojGRFIVi1UknrvaDAZ+PkJfyjqkA1yseh+vAU= -github.com/compose-spec/compose-go/v2 v2.10.1/go.mod h1:Ohac1SzhO/4fXXrzWIztIVB6ckmKBv1Nt5Z5mGVESUg= -github.com/containerd/cgroups/v3 v3.1.2 h1:OSosXMtkhI6Qove637tg1XgK4q+DhR0mX8Wi8EhrHa4= -github.com/containerd/cgroups/v3 v3.1.2/go.mod h1:PKZ2AcWmSBsY/tJUVhtS/rluX0b1uq1GmPO1ElCmbOw= -github.com/containerd/console v1.0.5 h1:R0ymNeydRqH2DmakFNdmjR2k0t7UPuiOV/N/27/qqsc= -github.com/containerd/console v1.0.5/go.mod h1:YynlIjWYF8myEu6sdkwKIvGQq+cOckRm6So2avqoYAk= -github.com/containerd/containerd/api v1.10.0 h1:5n0oHYVBwN4VhoX9fFykCV9dF1/BvAXeg2F8W6UYq1o= -github.com/containerd/containerd/api v1.10.0/go.mod h1:NBm1OAk8ZL+LG8R0ceObGxT5hbUYj7CzTmR3xh0DlMM= -github.com/containerd/containerd/v2 v2.2.1 h1:TpyxcY4AL5A+07dxETevunVS5zxqzuq7ZqJXknM11yk= -github.com/containerd/containerd/v2 v2.2.1/go.mod h1:NR70yW1iDxe84F2iFWbR9xfAN0N2F0NcjTi1OVth4nU= -github.com/containerd/continuity v0.4.5 h1:ZRoN1sXq9u7V6QoHMcVWGhOwDFqZ4B9i5H6un1Wh0x4= -github.com/containerd/continuity v0.4.5/go.mod h1:/lNJvtJKUQStBzpVQ1+rasXO1LAWtUQssk28EZvJ3nE= -github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI= -github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M= -github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE= -github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk= -github.com/containerd/fifo v1.1.0 h1:4I2mbh5stb1u6ycIABlBw9zgtlK8viPI9QkQNRQEEmY= -github.com/containerd/fifo v1.1.0/go.mod h1:bmC4NWMbXlt2EZ0Hc7Fx7QzTFxgPID13eH0Qu+MAb2o= -github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I= -github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo= -github.com/containerd/nydus-snapshotter v0.15.4 h1:l59kGRVMtwMLDLh322HsWhEsBCkRKMkGWYV5vBeLYCE= -github.com/containerd/nydus-snapshotter v0.15.4/go.mod h1:eRJqnxQDr48HNop15kZdLZpFF5B6vf6Q11Aq1K0E4Ms= -github.com/containerd/platforms v1.0.0-rc.2 h1:0SPgaNZPVWGEi4grZdV8VRYQn78y+nm6acgLGv/QzE4= -github.com/containerd/platforms v1.0.0-rc.2/go.mod h1:J71L7B+aiM5SdIEqmd9wp6THLVRzJGXfNuWCZCllLA4= -github.com/containerd/plugin v1.0.0 h1:c8Kf1TNl6+e2TtMHZt+39yAPDbouRH9WAToRjex483Y= -github.com/containerd/plugin v1.0.0/go.mod h1:hQfJe5nmWfImiqT1q8Si3jLv3ynMUIBB47bQ+KexvO8= -github.com/containerd/stargz-snapshotter v0.17.0 h1:djNS4KU8ztFhLdEDZ1bsfzOiYuVHT6TgSU5qwRk+cNc= -github.com/containerd/stargz-snapshotter/estargz v0.17.0 h1:+TyQIsR/zSFI1Rm31EQBwpAA1ovYgIKHy7kctL3sLcE= -github.com/containerd/stargz-snapshotter/estargz v0.17.0/go.mod h1:s06tWAiJcXQo9/8AReBCIo/QxcXFZ2n4qfsRnpl71SM= -github.com/containerd/ttrpc v1.2.7 h1:qIrroQvuOL9HQ1X6KHe2ohc7p+HP/0VE6XPU7elJRqQ= -github.com/containerd/ttrpc v1.2.7/go.mod h1:YCXHsb32f+Sq5/72xHubdiJRQY9inL4a4ZQrAbN1q9o= -github.com/containerd/typeurl/v2 v2.2.3 h1:yNA/94zxWdvYACdYO8zofhrTVuQY73fFU1y++dYSw40= -github.com/containerd/typeurl/v2 v2.2.3/go.mod h1:95ljDnPfD3bAbDJRugOiShd/DlAAsxGtUBhJxIn7SCk= -github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= -github.com/cpuguy83/go-md2man/v2 v2.0.7 h1:zbFlGlXEAKlwXpmvle3d8Oe3YnkKIK4xSRTd3sHPnBo= -github.com/cpuguy83/go-md2man/v2 v2.0.7/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= -github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= -github.com/creack/pty v1.1.17/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4= -github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s= -github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE= -github.com/cyphar/filepath-securejoin v0.5.1 h1:eYgfMq5yryL4fbWfkLpFFy2ukSELzaJOTaUTuh+oF48= -github.com/cyphar/filepath-securejoin v0.5.1/go.mod h1:Sdj7gXlvMcPZsbhwhQ33GguGLDGQL7h7bg04C/+u9jI= -github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= -github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/denisenkom/go-mssqldb v0.0.0-20191128021309-1d7a30a10f73/go.mod h1:xbL0rPBG9cCiLr28tMa8zpbdarY27NDyej4t/EjAShU= -github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk= -github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= -github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI= -github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8= -github.com/docker/buildx v0.30.1 h1:3vthfaTQOLt5QfN2nl7rKuPLUvx69nL5ZikFIXp//c8= -github.com/docker/buildx v0.30.1/go.mod h1:8nwT0V6UNYNo9rXq6WO/BQd9KrJ0JYcY/QX6x0y1Oro= -github.com/docker/cli v28.5.2+incompatible h1:XmG99IHcBmIAoC1PPg9eLBZPlTrNUAijsHLm8PjhBlg= -github.com/docker/cli v28.5.2+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= -github.com/docker/cli-docs-tool v0.11.0 h1:7d8QARFb7QEobizqxmEM7fOteZEHwH/zWgHQtHZEcfE= -github.com/docker/cli-docs-tool v0.11.0/go.mod h1:ma8BKiisUo8D6W05XEYIh3oa1UbgrZhi1nowyKFJa8Q= -github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= -github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk= -github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= -github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaftjlSWt4AFexzM= -github.com/docker/docker v28.5.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= -github.com/docker/docker-credential-helpers v0.9.3 h1:gAm/VtF9wgqJMoxzT3Gj5p4AqIjCBS4wrsOh9yRqcz8= -github.com/docker/docker-credential-helpers v0.9.3/go.mod h1:x+4Gbw9aGmChi3qTLZj8Dfn0TD20M/fuWy0E5+WDeCo= -github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c h1:lzqkGL9b3znc+ZUgi7FlLnqjQhcXxkNM/quxIjBVMD0= -github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c/go.mod h1:CADgU4DSXK5QUlFslkQu2yW2TKzFZcXq/leZfM0UH5Q= -github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec= -github.com/docker/go-connections v0.6.0 h1:LlMG9azAe1TqfR7sO+NJttz1gy6KO7VJBh+pMmjSD94= -github.com/docker/go-connections v0.6.0/go.mod h1:AahvXYshr6JgfUJGdDCs2b5EZG/vmaMAntpSFH5BFKE= -github.com/docker/go-metrics v0.0.0-20180209012529-399ea8c73916/go.mod h1:/u0gXw0Gay3ceNrsHubL3BtdOL2fHf93USgMTe0W5dI= -github.com/docker/go-metrics v0.0.1 h1:AgB/0SvBxihN0X8OR4SjsblXkbMvalQ8cjmtKQ2rQV8= -github.com/docker/go-metrics v0.0.1/go.mod h1:cG1hvH2utMXtqgqqYE9plW6lDxS3/5ayHzueweSI3Vw= -github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= -github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= -github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7 h1:UhxFibDNY/bfvqU5CAUmr9zpesgbU6SWc8/B4mflAE4= -github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7/go.mod h1:cyGadeNEkKy96OOhEzfZl+yxihPEzKnqJwvfuSUqbZE= -github.com/dvsekhvalnov/jose2go v0.0.0-20170216131308-f21a8cedbbae/go.mod h1:7BvyPhdbLxMXIYTFPLsyJRFMsKmOZnQmzh6Gb+uquuM= -github.com/eiannone/keyboard v0.0.0-20220611211555-0d226195f203 h1:XBBHcIb256gUJtLmY22n99HaZTz+r2Z51xUPi01m3wg= -github.com/eiannone/keyboard v0.0.0-20220611211555-0d226195f203/go.mod h1:E1jcSv8FaEny+OP/5k9UxZVw9YFWGj7eI4KR/iOBqCg= -github.com/erikstmartin/go-testdb v0.0.0-20160219214506-8d10e4a1bae5/go.mod h1:a2zkGnVExMxdzMo3M0Hi/3sEU+cWnZpSni0O6/Yb/P0= -github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= -github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= -github.com/fsnotify/fsevents v0.2.0 h1:BRlvlqjvNTfogHfeBOFvSC9N0Ddy+wzQCQukyoD7o/c= -github.com/fsnotify/fsevents v0.2.0/go.mod h1:B3eEk39i4hz8y1zaWS/wPrAP4O6wkIl7HQwKBr1qH/w= -github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= -github.com/fvbommel/sortorder v1.1.0 h1:fUmoe+HLsBTctBDoaBwpQo5N+nrCp8g/BjKb/6ZQmYw= -github.com/fvbommel/sortorder v1.1.0/go.mod h1:uk88iVf1ovNn1iLfgUVU2F9o5eO30ui720w+kxuqRs0= -github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= -github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= -github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= -github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= -github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= -github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= -github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= -github.com/go-sql-driver/mysql v1.3.0 h1:pgwjLi/dvffoP9aabwkT3AKpXQM93QARkjFhDDqC1UE= -github.com/go-sql-driver/mysql v1.3.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w= -github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= -github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro= -github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= -github.com/gofrs/flock v0.13.0 h1:95JolYOvGMqeH31+FC7D2+uULf6mG61mEZ/A8dRYMzw= -github.com/gofrs/flock v0.13.0/go.mod h1:jxeyy9R1auM5S6JYDBhDt+E2TCo7DkratH4Pgi8P+Z0= -github.com/gogo/protobuf v1.0.0/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= -github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= -github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= -github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= -github.com/golang-jwt/jwt/v5 v5.3.0 h1:pv4AsKCKKZuqlgs5sUmn4x8UlGa0kEVt/puTpKx9vvo= -github.com/golang-jwt/jwt/v5 v5.3.0/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE= -github.com/golang-sql/civil v0.0.0-20190719163853-cb61b32ac6fe/go.mod h1:8vg3r2VgvsThLBIFL93Qb5yWzgyZWhEmBwUJWevAkK0= -github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 h1:f+oWsMOmNPc8JmEHVZIycC7hBoQxHH9pNKQORJNozsQ= -github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8/go.mod h1:wcDNUvekVysuuOpQKo3191zZyTpiI6se1N1ULghS0sw= -github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= -github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek= -github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= -github.com/google/certificate-transparency-go v1.0.10-0.20180222191210-5ab67e519c93 h1:jc2UWq7CbdszqeH6qu1ougXMIUBfSy8Pbh/anURYbGI= -github.com/google/certificate-transparency-go v1.0.10-0.20180222191210-5ab67e519c93/go.mod h1:QeJfpSbVSfYc7RgB3gJFj9cbuQMMchQxrWXz8Ruopmg= -github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= -github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= -github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= -github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= -github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4= -github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ= -github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= -github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= -github.com/gorilla/mux v1.7.0/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs= -github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY= -github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ= -github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2 h1:8Tjv8EJ+pM1xP8mK6egEbD1OgnVTyacbefKhmbLhIhU= -github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2/go.mod h1:pkJQ2tZHJ0aFOVEEot6oZmaVEZcRme73eIFmhiVuRWs= -github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed h1:5upAirOpQc1Q53c0bnx2ufif5kANL7bfZWcc6VJWJd8= -github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed/go.mod h1:tMWxXQ9wFIaZeTI9F+hmhFiGpFmhOHzyShyFUhRm0H4= -github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= -github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I= -github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= -github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo= -github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM= -github.com/hashicorp/go-version v1.8.0 h1:KAkNb1HAiZd1ukkxDFGmokVZe1Xy9HG6NUp+bPle2i4= -github.com/hashicorp/go-version v1.8.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA= -github.com/hinshun/vt10x v0.0.0-20220119200601-820417d04eec h1:qv2VnGeEQHchGaZ/u7lxST/RaJw+cv273q79D81Xbog= -github.com/hinshun/vt10x v0.0.0-20220119200601-820417d04eec/go.mod h1:Q48J4R4DvxnHolD5P8pOtXigYlRuPLGl6moFx3ulM68= -github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= -github.com/in-toto/in-toto-golang v0.9.0 h1:tHny7ac4KgtsfrG6ybU8gVOZux2H8jN05AXJ9EBM1XU= -github.com/in-toto/in-toto-golang v0.9.0/go.mod h1:xsBVrVsHNsB61++S6Dy2vWosKhuA3lUTQd+eF9HdeMo= -github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= -github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= -github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= -github.com/inhies/go-bytesize v0.0.0-20220417184213-4913239db9cf h1:FtEj8sfIcaaBfAKrE1Cwb61YDtYq9JxChK1c7AKce7s= -github.com/inhies/go-bytesize v0.0.0-20220417184213-4913239db9cf/go.mod h1:yrqSXGoD/4EKfF26AOGzscPOgTTJcyAwM2rpixWT+t4= -github.com/jinzhu/gorm v0.0.0-20170222002820-5409931a1bb8 h1:CZkYfurY6KGhVtlalI4QwQ6T0Cu6iuY3e0x5RLu96WE= -github.com/jinzhu/gorm v0.0.0-20170222002820-5409931a1bb8/go.mod h1:Vla75njaFJ8clLU1W44h34PjIkijhjHIYnZxMqCdxqo= -github.com/jinzhu/inflection v0.0.0-20170102125226-1c35d901db3d h1:jRQLvyVGL+iVtDElaEIDdKwpPqUIZJfzkNLV34htpEc= -github.com/jinzhu/inflection v0.0.0-20170102125226-1c35d901db3d/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc= -github.com/jinzhu/now v1.1.1/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8= -github.com/jonboulle/clockwork v0.5.0 h1:Hyh9A8u51kptdkR+cqRpT1EebBwTn1oK9YfGYbdFz6I= -github.com/jonboulle/clockwork v0.5.0/go.mod h1:3mZlmanh0g2NDKO5TWZVJAfofYk64M7XN3SzBPjZF60= -github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= -github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= -github.com/juju/loggo v0.0.0-20190526231331-6e530bcce5d8/go.mod h1:vgyd7OREkbtVEN/8IXZe5Ooef3LQePvuBm9UWj6ZL8U= -github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= -github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs= -github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8= -github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= -github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= -github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk= -github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4= -github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= -github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= -github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= -github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= -github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= -github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= -github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= -github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= -github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= -github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= -github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= -github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= -github.com/lib/pq v0.0.0-20150723085316-0dad96c0b94f/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo= -github.com/magiconair/properties v1.5.3/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= -github.com/magiconair/properties v1.8.9 h1:nWcCbLq1N2v/cpNsy5WvQ37Fb+YElfq20WJ/a8RkpQM= -github.com/magiconair/properties v1.8.9/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0= -github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= -github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE= -github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8= -github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= -github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= -github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= -github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc= -github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= -github.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk= -github.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y= -github.com/mattn/go-sqlite3 v1.6.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc= -github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= -github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b h1:j7+1HpAFS1zy5+Q4qx1fWh90gTKwiN4QCGoY9TWyyO4= -github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE= -github.com/miekg/pkcs11 v1.0.2/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs= -github.com/miekg/pkcs11 v1.1.1 h1:Ugu9pdy6vAYku5DEpVWVFPYnzV+bxB+iRdbuFSu7TvU= -github.com/miekg/pkcs11 v1.1.1/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs= -github.com/mitchellh/go-ps v1.0.0 h1:i6ampVEEF4wQFF+bkYfwYgY+F/uYJDktmvLPf7qIgjc= -github.com/mitchellh/go-ps v1.0.0/go.mod h1:J4lOc8z8yJs6vUwklHw2XEIiT4z4C40KtWVN3nvg8Pg= -github.com/mitchellh/hashstructure/v2 v2.0.2 h1:vGKWl0YJqUNxE8d+h8f6NJLcCJrgbhC4NcD46KavDd4= -github.com/mitchellh/hashstructure/v2 v2.0.2/go.mod h1:MG3aRVU/N29oo/V/IhBX8GR/zz4kQkprJgF2EVszyDE= -github.com/mitchellh/mapstructure v0.0.0-20150613213606-2caf8efc9366/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= -github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY= -github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= -github.com/moby/buildkit v0.26.3 h1:D+ruZVAk/3ipRq5XRxBH9/DIFpRjSlTtMbghT5gQP9g= -github.com/moby/buildkit v0.26.3/go.mod h1:4T4wJzQS4kYWIfFRjsbJry4QoxDBjK+UGOEOs1izL7w= -github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0= -github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo= -github.com/moby/go-archive v0.1.0 h1:Kk/5rdW/g+H8NHdJW2gsXyZ7UnzvJNOy6VKJqueWdcQ= -github.com/moby/go-archive v0.1.0/go.mod h1:G9B+YoujNohJmrIYFBpSd54GTUB4lt9S+xVQvsJyFuo= -github.com/moby/locker v1.0.1 h1:fOXqR41zeveg4fFODix+1Ch4mj/gT0NE1XJbp/epuBg= -github.com/moby/locker v1.0.1/go.mod h1:S7SDdo5zpBK84bzzVlKr2V0hz+7x9hWbYC/kq7oQppc= -github.com/moby/patternmatcher v0.6.0 h1:GmP9lR19aU5GqSSFko+5pRqHi+Ohk1O69aFiKkVGiPk= -github.com/moby/patternmatcher v0.6.0/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc= -github.com/moby/sys/atomicwriter v0.1.0 h1:kw5D/EqkBwsBFi0ss9v1VG3wIkVhzGvLklJ+w3A14Sw= -github.com/moby/sys/atomicwriter v0.1.0/go.mod h1:Ul8oqv2ZMNHOceF643P6FKPXeCmYtlQMvpizfsSoaWs= -github.com/moby/sys/capability v0.4.0 h1:4D4mI6KlNtWMCM1Z/K0i7RV1FkX+DBDHKVJpCndZoHk= -github.com/moby/sys/capability v0.4.0/go.mod h1:4g9IK291rVkms3LKCDOoYlnV8xKwoDTpIrNEE35Wq0I= -github.com/moby/sys/mountinfo v0.7.2 h1:1shs6aH5s4o5H2zQLn796ADW1wMrIwHsyJ2v9KouLrg= -github.com/moby/sys/mountinfo v0.7.2/go.mod h1:1YOa8w8Ih7uW0wALDUgT1dTTSBrZ+HiBLGws92L2RU4= -github.com/moby/sys/sequential v0.6.0 h1:qrx7XFUd/5DxtqcoH1h438hF5TmOvzC/lspjy7zgvCU= -github.com/moby/sys/sequential v0.6.0/go.mod h1:uyv8EUTrca5PnDsdMGXhZe6CCe8U/UiTWd+lL+7b/Ko= -github.com/moby/sys/signal v0.7.1 h1:PrQxdvxcGijdo6UXXo/lU/TvHUWyPhj7UOpSo8tuvk0= -github.com/moby/sys/signal v0.7.1/go.mod h1:Se1VGehYokAkrSQwL4tDzHvETwUZlnY7S5XtQ50mQp8= -github.com/moby/sys/symlink v0.3.0 h1:GZX89mEZ9u53f97npBy4Rc3vJKj7JBDj/PN2I22GrNU= -github.com/moby/sys/symlink v0.3.0/go.mod h1:3eNdhduHmYPcgsJtZXW1W4XUJdZGBIkttZ8xKqPUJq0= -github.com/moby/sys/user v0.4.0 h1:jhcMKit7SA80hivmFJcbB1vqmw//wU61Zdui2eQXuMs= -github.com/moby/sys/user v0.4.0/go.mod h1:bG+tYYYJgaMtRKgEmuueC0hJEAZWwtIbZTB+85uoHjs= -github.com/moby/sys/userns v0.1.0 h1:tVLXkFOxVu9A64/yh59slHVv9ahO9UIev4JZusOLG/g= -github.com/moby/sys/userns v0.1.0/go.mod h1:IHUYgu/kao6N8YZlp9Cf444ySSvCmDlmzUcYfDHOl28= -github.com/moby/term v0.5.2 h1:6qk3FJAFDs6i/q3W/pQ97SX192qKfZgGjCQqfCJkgzQ= -github.com/moby/term v0.5.2/go.mod h1:d3djjFCrjnB+fl8NJux+EJzu0msscUP+f8it8hPkFLc= -github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= -github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= -github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= -github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= -github.com/morikuni/aec v1.1.0 h1:vBBl0pUnvi/Je71dsRrhMBtreIqNMYErSAbEeb8jrXQ= -github.com/morikuni/aec v1.1.0/go.mod h1:xDRgiq/iw5l+zkao76YTKzKttOp2cwPEne25HDkJnBw= -github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= -github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= -github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= -github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= -github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= -github.com/onsi/ginkgo v1.12.0/go.mod h1:oUhWkIvk5aDxtKvDDuw8gItl8pKl42LzjC9KZE0HfGg= -github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= -github.com/onsi/gomega v1.9.0/go.mod h1:Ho0h+IUsWyvy1OpqCwxlQ/21gkhVunqlU8fDGcoTdcA= -github.com/opencontainers/go-digest v0.0.0-20170106003457-a6d0ee40d420/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s= -github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= -github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= -github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0= -github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040= -github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M= -github.com/opencontainers/runtime-spec v1.3.0 h1:YZupQUdctfhpZy3TM39nN9Ika5CBWT5diQ8ibYCRkxg= -github.com/opencontainers/runtime-spec v1.3.0/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= -github.com/opencontainers/selinux v1.13.1 h1:A8nNeceYngH9Ow++M+VVEwJVpdFmrlxsN22F+ISDCJE= -github.com/opencontainers/selinux v1.13.1/go.mod h1:S10WXZ/osk2kWOYKy1x2f/eXF5ZHJoUs8UU/2caNRbg= -github.com/opentracing/opentracing-go v1.1.0 h1:pWlfV3Bxv7k65HYwkikxat0+s3pV4bsqf19k25Ur8rU= -github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= -github.com/otiai10/copy v1.14.1 h1:5/7E6qsUMBaH5AnQ0sSLzzTg1oTECmcCmT6lvF45Na8= -github.com/otiai10/copy v1.14.1/go.mod h1:oQwrEDDOci3IM8dJF0d8+jnbfPDllW6vUjNc3DoZm9I= -github.com/otiai10/mint v1.6.3 h1:87qsV/aw1F5as1eH1zS/yqHY85ANKVMgkDrf9rcxbQs= -github.com/otiai10/mint v1.6.3/go.mod h1:MJm72SBthJjz8qhefc4z1PYEieWmy8Bku7CjcAqyUSM= -github.com/pelletier/go-toml v1.9.5 h1:4yBQzkHv+7BHq2PQUZF3Mx0IYxG7LsP222s7Agd3ve8= -github.com/pelletier/go-toml v1.9.5/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c= -github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= -github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 h1:GFCKgmp0tecUJ0sJuv4pzYCqS9+RGSn52M3FUwPs+uo= -github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10/go.mod h1:t/avpk3KcrXxUnYOhZhMXJlSEyie6gQbtLq5NM3loB8= -github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= -github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_golang v0.9.0-pre1.0.20180209125602-c332b6f63c06/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= -github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= -github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= -github.com/prometheus/client_golang v1.1.0/go.mod h1:I1FGZT9+L76gKKOs5djB6ezCbFQP1xR9D75/vuwEF3g= -github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= -github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= -github.com/prometheus/client_model v0.0.0-20171117100541-99fa1f4be8e5/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= -github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= -github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= -github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= -github.com/prometheus/common v0.0.0-20180110214958-89604d197083/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= -github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= -github.com/prometheus/common v0.6.0/go.mod h1:eBmuwkDJBwy6iBfxCBob6t6dR6ENT/y+J+Zk0j9GMYc= -github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= -github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= -github.com/prometheus/procfs v0.0.0-20180125133057-cb4147076ac7/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= -github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= -github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= -github.com/prometheus/procfs v0.0.3/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDaekg4FpcdQ= -github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= -github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= -github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY= -github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc= -github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= -github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= -github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk= -github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= -github.com/santhosh-tekuri/jsonschema/v6 v6.0.1 h1:PKK9DyHxif4LZo+uQSgXNqs0jj5+xZwwfKHgph2lxBw= -github.com/santhosh-tekuri/jsonschema/v6 v6.0.1/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU= -github.com/secure-systems-lab/go-securesystemslib v0.9.1 h1:nZZaNz4DiERIQguNy0cL5qTdn9lR8XKHf4RUyG1Sx3g= -github.com/secure-systems-lab/go-securesystemslib v0.9.1/go.mod h1:np53YzT0zXGMv6x4iEWc9Z59uR+x+ndLwCLqPYpLXVU= -github.com/shibumi/go-pathspec v1.3.0 h1:QUyMZhFo0Md5B8zV8x2tesohbb5kfbpTi9rBnKh5dkI= -github.com/shibumi/go-pathspec v1.3.0/go.mod h1:Xutfslp817l2I1cZvgcfeMQJG5QnU2lh5tVaaMCl3jE= -github.com/sirupsen/logrus v1.0.6/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc= -github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= -github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q= -github.com/sirupsen/logrus v1.9.4 h1:TsZE7l11zFCLZnZ+teH4Umoq5BhEIfIzfRDZ1Uzql2w= -github.com/sirupsen/logrus v1.9.4/go.mod h1:ftWc9WdOfJ0a92nsE2jF5u5ZwH8Bv2zdeOC42RjbV2g= -github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 h1:JIAuq3EEf9cgbU6AtGPK4CTG3Zf6CKMNqf0MHTggAUA= -github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966/go.mod h1:sUM3LWHvSMaG192sy56D9F7CNvL7jUJVXoqM1QKLnog= -github.com/spdx/tools-golang v0.5.5 h1:61c0KLfAcNqAjlg6UNMdkwpMernhw3zVRwDZ2x9XOmk= -github.com/spdx/tools-golang v0.5.5/go.mod h1:MVIsXx8ZZzaRWNQpUDhC4Dud34edUYJYecciXgrw5vE= -github.com/spf13/cast v0.0.0-20150508191742-4d07383ffe94 h1:JmfC365KywYwHB946TTiQWEb8kqPY+pybPLoGE9GgVk= -github.com/spf13/cast v0.0.0-20150508191742-4d07383ffe94/go.mod h1:r2rcYCSwa1IExKTDiTfzaxqT2FNHs8hODu4LnUfgKEg= -github.com/spf13/cobra v0.0.1/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= -github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU= -github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4= -github.com/spf13/jwalterweatherman v0.0.0-20141219030609-3d60171a6431 h1:XTHrT015sxHyJ5FnQ0AeemSspZWaDq7DoTRW0EVsDCE= -github.com/spf13/jwalterweatherman v0.0.0-20141219030609-3d60171a6431/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= -github.com/spf13/pflag v1.0.0/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= -github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= -github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk= -github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= -github.com/spf13/viper v0.0.0-20150530192845-be5ff3e4840c h1:2EejZtjFjKJGk71ANb+wtFK5EjUzUkEM3R0xnp559xg= -github.com/spf13/viper v0.0.0-20150530192845-be5ff3e4840c/go.mod h1:A8kyI5cUJhb8N+3pkfONlcEcZbueH6nhAm0Fq7SrnBM= -github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE= -github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= -github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= -github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= -github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= -github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= -github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= -github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= -github.com/theupdateframework/notary v0.7.0 h1:QyagRZ7wlSpjT5N2qQAh/pN+DVqgekv4DzbAiAiEL3c= -github.com/theupdateframework/notary v0.7.0/go.mod h1:c9DRxcmhHmVLDay4/2fUYdISnHqbFDGRSlXPO0AhYWw= -github.com/tilt-dev/fsnotify v1.4.8-0.20220602155310-fff9c274a375 h1:QB54BJwA6x8QU9nHY3xJSZR2kX9bgpZekRKGkLTmEXA= -github.com/tilt-dev/fsnotify v1.4.8-0.20220602155310-fff9c274a375/go.mod h1:xRroudyp5iVtxKqZCrA6n2TLFRBf8bmnjr1UD4x+z7g= -github.com/tonistiigi/dchapes-mode v0.0.0-20250318174251-73d941a28323 h1:r0p7fK56l8WPequOaR3i9LBqfPtEdXIQbUTzT55iqT4= -github.com/tonistiigi/dchapes-mode v0.0.0-20250318174251-73d941a28323/go.mod h1:3Iuxbr0P7D3zUzBMAZB+ois3h/et0shEz0qApgHYGpY= -github.com/tonistiigi/fsutil v0.0.0-20250605211040-586307ad452f h1:MoxeMfHAe5Qj/ySSBfL8A7l1V+hxuluj8owsIEEZipI= -github.com/tonistiigi/fsutil v0.0.0-20250605211040-586307ad452f/go.mod h1:BKdcez7BiVtBvIcef90ZPc6ebqIWr4JWD7+EvLm6J98= -github.com/tonistiigi/go-csvvalue v0.0.0-20240814133006-030d3b2625d0 h1:2f304B10LaZdB8kkVEaoXvAMVan2tl9AiK4G0odjQtE= -github.com/tonistiigi/go-csvvalue v0.0.0-20240814133006-030d3b2625d0/go.mod h1:278M4p8WsNh3n4a1eqiFcV2FGk7wE5fwUpUom9mK9lE= -github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea h1:SXhTLE6pb6eld/v/cCndK0AMpt1wiVFb/YYmqB3/QG0= -github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea/go.mod h1:WPnis/6cRcDZSUvVmezrxJPkiO87ThFYsoUiMwWNDJk= -github.com/tonistiigi/vt100 v0.0.0-20240514184818-90bafcd6abab h1:H6aJ0yKQ0gF49Qb2z5hI1UHxSQt4JMyxebFR15KnApw= -github.com/tonistiigi/vt100 v0.0.0-20240514184818-90bafcd6abab/go.mod h1:ulncasL3N9uLrVann0m+CDlJKWsIAP34MPcOJF6VRvc= -github.com/vbatts/tar-split v0.12.2 h1:w/Y6tjxpeiFMR47yzZPlPj/FcPLpXbTUi/9H7d3CPa4= -github.com/vbatts/tar-split v0.12.2/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA= -github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8Ydu2Bstc= -github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= -github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= -go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0= -go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo= -go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64= -go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= -go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0 h1:YH4g8lQroajqUwWbq/tr2QX1JFmEXaDLgG+ew9bLMWo= -go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0/go.mod h1:fvPi2qXDqFs8M4B4fmJhE92TyQs9Ydjlg3RvfUp+NbQ= -go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.63.0 h1:2pn7OzMewmYRiNtv1doZnLo3gONcnMHlFnmOR8Vgt+8= -go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.63.0/go.mod h1:rjbQTDEPQymPE0YnRQp9/NuPwwtL0sesz/fnqRW/v84= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 h1:RbKq8BG0FI8OiXhBfcRtqqHcZcka+gU3cskNuf05R18= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0/go.mod h1:h06DGIukJOevXaj/xrNjhi/2098RZzcLTbc0jDAUbsg= -go.opentelemetry.io/otel v1.38.0 h1:RkfdswUDRimDg0m2Az18RKOsnI8UDzppJAtj01/Ymk8= -go.opentelemetry.io/otel v1.38.0/go.mod h1:zcmtmQ1+YmQM9wrNsTGV/q/uyusom3P8RxwExxkZhjM= -go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.38.0 h1:vl9obrcoWVKp/lwl8tRE33853I8Xru9HFbw/skNeLs8= -go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.38.0/go.mod h1:GAXRxmLJcVM3u22IjTg74zWBrRCKq8BnOqUVLodpcpw= -go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.38.0 h1:Oe2z/BCg5q7k4iXC3cqJxKYg0ieRiOqF0cecFYdPTwk= -go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.38.0/go.mod h1:ZQM5lAJpOsKnYagGg/zV2krVqTtaVdYdDkhMoX6Oalg= -go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0 h1:GqRJVj7UmLjCVyVJ3ZFLdPRmhDUp2zFmQe3RHIOsw24= -go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0/go.mod h1:ri3aaHSmCTVYu2AWv44YMauwAQc0aqI9gHKIcSbI1pU= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.38.0 h1:lwI4Dc5leUqENgGuQImwLo4WnuXFPetmPpkLi2IrX54= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.38.0/go.mod h1:Kz/oCE7z5wuyhPxsXDuaPteSWqjSBD5YaSdbxZYGbGk= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0 h1:aTL7F04bJHUlztTsNGJ2l+6he8c+y/b//eR0jjjemT4= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0/go.mod h1:kldtb7jDTeol0l3ewcmd8SDvx3EmIE7lyvqbasU3QC4= -go.opentelemetry.io/otel/metric v1.38.0 h1:Kl6lzIYGAh5M159u9NgiRkmoMKjvbsKtYRwgfrA6WpA= -go.opentelemetry.io/otel/metric v1.38.0/go.mod h1:kB5n/QoRM8YwmUahxvI3bO34eVtQf2i4utNVLr9gEmI= -go.opentelemetry.io/otel/sdk v1.38.0 h1:l48sr5YbNf2hpCUj/FoGhW9yDkl+Ma+LrVl8qaM5b+E= -go.opentelemetry.io/otel/sdk v1.38.0/go.mod h1:ghmNdGlVemJI3+ZB5iDEuk4bWA3GkTpW+DOoZMYBVVg= -go.opentelemetry.io/otel/sdk/metric v1.38.0 h1:aSH66iL0aZqo//xXzQLYozmWrXxyFkBJ6qT5wthqPoM= -go.opentelemetry.io/otel/sdk/metric v1.38.0/go.mod h1:dg9PBnW9XdQ1Hd6ZnRz689CbtrUp0wMMs9iPcgT9EZA= -go.opentelemetry.io/otel/trace v1.38.0 h1:Fxk5bKrDZJUH+AMyyIXGcFAPah0oRcT+LuNtJrmcNLE= -go.opentelemetry.io/otel/trace v1.38.0/go.mod h1:j1P9ivuFsTceSWe1oY+EeW3sc+Pp42sO++GHkg4wwhs= -go.opentelemetry.io/proto/otlp v1.7.1 h1:gTOMpGDb0WTBOP8JaO72iL3auEZhVmAQg4ipjOVAtj4= -go.opentelemetry.io/proto/otlp v1.7.1/go.mod h1:b2rVh6rfI/s2pHWNlB7ILJcRALpcNDzKhACevjI+ZnE= -go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= -go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= -go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y= -go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU= -go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= -go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= -go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= -go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= -go.yaml.in/yaml/v4 v4.0.0-rc.4 h1:UP4+v6fFrBIb1l934bDl//mmnoIZEDK0idg1+AIvX5U= -go.yaml.in/yaml/v4 v4.0.0-rc.4/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0= -golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= -golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= -golang.org/x/crypto v0.0.0-20190325154230-a5d413f7728c/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= -golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= -golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20201117144127-c1f2f97bffc9/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I= -golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= -golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q= -golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4= -golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= -golang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA= -golang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w= -golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= -golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= -golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= -golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY= -golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU= -golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= -golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= -golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190801041406-cbf593c0f2f3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210331175145-43e1dd70ce54/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k= -golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= -golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw= -golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= -golang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU= -golang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254= -golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= -golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= -golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= -golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM= -golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM= -golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI= -golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= -golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= -golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= -golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk= -gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E= -google.golang.org/genproto/googleapis/api v0.0.0-20251029180050-ab9386a59fda h1:+2XxjfsAu6vqFxwGBRcHiMaDCuZiqXGDUDVWVtrFAnE= -google.golang.org/genproto/googleapis/api v0.0.0-20251029180050-ab9386a59fda/go.mod h1:fDMmzKV90WSg1NbozdqrE64fkuTv6mlq2zxo9ad+3yo= -google.golang.org/genproto/googleapis/rpc v0.0.0-20251029180050-ab9386a59fda h1:i/Q+bfisr7gq6feoJnS/DlpdwEL4ihp41fvRiM3Ork0= -google.golang.org/genproto/googleapis/rpc v0.0.0-20251029180050-ab9386a59fda/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk= -google.golang.org/grpc v1.0.5/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw= -google.golang.org/grpc v1.78.0 h1:K1XZG/yGDJnzMdd/uZHAkVqJE+xIDOcmdSFZkBUicNc= -google.golang.org/grpc v1.78.0/go.mod h1:I47qjTo4OKbMkjA/aOOwxDIiPSBofUtQUI5EfpWvW7U= -google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= -google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= -gopkg.in/airbrake/gobrake.v2 v2.0.9/go.mod h1:/h5ZAUhDkGaJfjzjKLSjv6zCL6O0LLBxU4K+aSYdM/U= -gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= -gopkg.in/cenkalti/backoff.v2 v2.2.1 h1:eJ9UAg01/HIHG987TwxvnzK2MgxXq97YY6rYDpY9aII= -gopkg.in/cenkalti/backoff.v2 v2.2.1/go.mod h1:S0QdOvT2AlerfSBkp0O+dk+bbIMaNbEmVk876gPCthU= -gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= -gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= -gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= -gopkg.in/gemnasium/logrus-airbrake-hook.v2 v2.1.2/go.mod h1:Xk6kEKp8OKb+X14hQBKWaSkCsqBpgog8nAV2xsGOxlo= -gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA= -gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= -gopkg.in/rethinkdb/rethinkdb-go.v6 v6.2.1 h1:d4KQkxAaAiRY2h5Zqis161Pv91A37uZyJOx73duwUwM= -gopkg.in/rethinkdb/rethinkdb-go.v6 v6.2.1/go.mod h1:WbjuEoo1oadwzQ4apSDU+JTvmllEHtsNHS6y7vFc7iw= -gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= -gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10= -gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= -gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -gotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q= -gotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA= -tags.cncf.io/container-device-interface v1.1.0 h1:RnxNhxF1JOu6CJUVpetTYvrXHdxw9j9jFYgZpI+anSY= -tags.cncf.io/container-device-interface v1.1.0/go.mod h1:76Oj0Yqp9FwTx/pySDc8Bxjpg+VqXfDb50cKAXVJ34Q= diff --git a/internal/desktop/client.go b/internal/desktop/client.go deleted file mode 100644 index 1e28899370d..00000000000 --- a/internal/desktop/client.go +++ /dev/null @@ -1,146 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package desktop - -import ( - "context" - "encoding/json" - "fmt" - "io" - "net" - "net/http" - "strings" - - "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp" - - "github.com/docker/compose/v5/internal" - "github.com/docker/compose/v5/internal/memnet" -) - -// identify this client in the logs -var userAgent = "compose/" + internal.Version - -// Client for integration with Docker Desktop features. -type Client struct { - apiEndpoint string - client *http.Client -} - -// NewClient creates a Desktop integration client for the provided in-memory -// socket address (AF_UNIX or named pipe). -func NewClient(apiEndpoint string) *Client { - var transport http.RoundTripper = &http.Transport{ - DisableCompression: true, - DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) { - return memnet.DialEndpoint(ctx, apiEndpoint) - }, - } - transport = otelhttp.NewTransport(transport) - - return &Client{ - apiEndpoint: apiEndpoint, - client: &http.Client{Transport: transport}, - } -} - -func (c *Client) Endpoint() string { - return c.apiEndpoint -} - -// Close releases any open connections. -func (c *Client) Close() error { - c.client.CloseIdleConnections() - return nil -} - -type PingResponse struct { - ServerTime int64 `json:"serverTime"` -} - -// Ping is a minimal API used to ensure that the server is available. -func (c *Client) Ping(ctx context.Context) (*PingResponse, error) { - req, err := c.newRequest(ctx, http.MethodGet, "/ping", http.NoBody) - if err != nil { - return nil, err - } - - resp, err := c.client.Do(req) - if err != nil { - return nil, err - } - defer func() { - _ = resp.Body.Close() - }() - - if resp.StatusCode != http.StatusOK { - return nil, fmt.Errorf("unexpected status code: %d", resp.StatusCode) - } - - var ret PingResponse - if err := json.NewDecoder(resp.Body).Decode(&ret); err != nil { - return nil, err - } - return &ret, nil -} - -type FeatureFlagResponse map[string]FeatureFlagValue - -type FeatureFlagValue struct { - Enabled bool `json:"enabled"` -} - -func (c *Client) FeatureFlags(ctx context.Context) (FeatureFlagResponse, error) { - req, err := c.newRequest(ctx, http.MethodGet, "/features", http.NoBody) - if err != nil { - return nil, err - } - - resp, err := c.client.Do(req) - if err != nil { - return nil, err - } - defer func() { - _ = resp.Body.Close() - }() - - if resp.StatusCode != http.StatusOK { - return nil, fmt.Errorf("unexpected status code: %d", resp.StatusCode) - } - - var ret FeatureFlagResponse - if err := json.NewDecoder(resp.Body).Decode(&ret); err != nil { - return nil, err - } - return ret, nil -} - -func (c *Client) newRequest(ctx context.Context, method, path string, body io.Reader) (*http.Request, error) { - req, err := http.NewRequestWithContext(ctx, method, backendURL(path), body) - if err != nil { - return nil, err - } - req.Header.Set("User-Agent", userAgent) - return req, nil -} - -// backendURL generates a URL for the given API path. -// -// NOTE: Custom transport handles communication. The host is to create a valid -// URL for the Go http.Client that is also descriptive in error/logs. -func backendURL(path string) string { - return "http://docker-desktop/" + strings.TrimPrefix(path, "/") -} diff --git a/internal/desktop/client_test.go b/internal/desktop/client_test.go deleted file mode 100644 index abc4f33122b..00000000000 --- a/internal/desktop/client_test.go +++ /dev/null @@ -1,48 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package desktop - -import ( - "os" - "testing" - "time" - - "github.com/stretchr/testify/require" -) - -func TestClientPing(t *testing.T) { - if testing.Short() { - t.Skip("Skipped in short mode - test connects to Docker Desktop") - } - desktopEndpoint := os.Getenv("COMPOSE_TEST_DESKTOP_ENDPOINT") - if desktopEndpoint == "" { - t.Skip("Skipping - COMPOSE_TEST_DESKTOP_ENDPOINT not defined") - } - - client := NewClient(desktopEndpoint) - t.Cleanup(func() { - _ = client.Close() - }) - - now := time.Now() - - ret, err := client.Ping(t.Context()) - require.NoError(t, err) - - serverTime := time.Unix(0, ret.ServerTime) - require.True(t, now.Before(serverTime)) -} diff --git a/internal/experimental/experimental.go b/internal/experimental/experimental.go deleted file mode 100644 index d6378024307..00000000000 --- a/internal/experimental/experimental.go +++ /dev/null @@ -1,68 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package experimental - -import ( - "context" - "os" - "strconv" - - "github.com/docker/compose/v5/internal/desktop" -) - -// envComposeExperimentalGlobal can be set to a falsy value (e.g. 0, false) to -// globally opt-out of any experimental features in Compose. -const envComposeExperimentalGlobal = "COMPOSE_EXPERIMENTAL" - -// State of experiments (enabled/disabled) based on environment and local config. -type State struct { - // active is false if experiments have been opted-out of globally. - active bool - desktopValues desktop.FeatureFlagResponse -} - -func NewState() *State { - // experimental features have individual controls, but users can opt out - // of ALL experiments easily if desired - experimentsActive := true - if v := os.Getenv(envComposeExperimentalGlobal); v != "" { - experimentsActive, _ = strconv.ParseBool(v) - } - return &State{ - active: experimentsActive, - } -} - -func (s *State) Load(ctx context.Context, client *desktop.Client) error { - if !s.active { - // user opted out of experiments globally, no need to load state from - // Desktop - return nil - } - - if client == nil { - // not running under Docker Desktop - return nil - } - - desktopValues, err := client.FeatureFlags(ctx) - if err != nil { - return err - } - s.desktopValues = desktopValues - return nil -} diff --git a/internal/locker/pidfile.go b/internal/locker/pidfile.go deleted file mode 100644 index 08dcea1f3a5..00000000000 --- a/internal/locker/pidfile.go +++ /dev/null @@ -1,35 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "fmt" - "path/filepath" -) - -type Pidfile struct { - path string -} - -func NewPidfile(projectName string) (*Pidfile, error) { - run, err := runDir() - if err != nil { - return nil, err - } - path := filepath.Join(run, fmt.Sprintf("%s.pid", projectName)) - return &Pidfile{path: path}, nil -} diff --git a/internal/locker/pidfile_unix.go b/internal/locker/pidfile_unix.go deleted file mode 100644 index 484b65d8250..00000000000 --- a/internal/locker/pidfile_unix.go +++ /dev/null @@ -1,29 +0,0 @@ -//go:build !windows - -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "os" - - "github.com/docker/docker/pkg/pidfile" -) - -func (f *Pidfile) Lock() error { - return pidfile.Write(f.path, os.Getpid()) -} diff --git a/internal/locker/pidfile_windows.go b/internal/locker/pidfile_windows.go deleted file mode 100644 index adc151827dc..00000000000 --- a/internal/locker/pidfile_windows.go +++ /dev/null @@ -1,48 +0,0 @@ -//go:build windows - -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "os" - - "github.com/docker/docker/pkg/pidfile" - "github.com/mitchellh/go-ps" -) - -func (f *Pidfile) Lock() error { - newPID := os.Getpid() - err := pidfile.Write(f.path, newPID) - if err != nil { - // Get PID registered in the file - pid, errPid := pidfile.Read(f.path) - if errPid != nil { - return err - } - // Some users faced issues on Windows where the process written in the pidfile was identified as still existing - // So we used a 2nd process library to verify if this not a false positive feedback - // Check if the process exists - process, errPid := ps.FindProcess(pid) - if process == nil && errPid == nil { - // If the process does not exist, remove the pidfile and try to lock again - _ = os.Remove(f.path) - return pidfile.Write(f.path, newPID) - } - } - return err -} diff --git a/internal/locker/runtime.go b/internal/locker/runtime.go deleted file mode 100644 index e60db5cc15e..00000000000 --- a/internal/locker/runtime.go +++ /dev/null @@ -1,35 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "os" -) - -func runDir() (string, error) { - run, ok := os.LookupEnv("XDG_RUNTIME_DIR") - if ok { - return run, nil - } - - path, err := osDependentRunDir() - if err != nil { - return "", err - } - err = os.MkdirAll(path, 0o700) - return path, err -} diff --git a/internal/locker/runtime_darwin.go b/internal/locker/runtime_darwin.go deleted file mode 100644 index 127a5fb048d..00000000000 --- a/internal/locker/runtime_darwin.go +++ /dev/null @@ -1,34 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "os" - "path/filepath" -) - -// Based on https://github.com/adrg/xdg -// Licensed under MIT License (MIT) -// Copyright (c) 2014 Adrian-George Bostan <adrg@epistack.com> - -func osDependentRunDir() (string, error) { - home, err := os.UserHomeDir() - if err != nil { - return "", err - } - return filepath.Join(home, "Library", "Application Support", "com.docker.compose"), nil -} diff --git a/internal/locker/runtime_unix.go b/internal/locker/runtime_unix.go deleted file mode 100644 index 5daf61aac52..00000000000 --- a/internal/locker/runtime_unix.go +++ /dev/null @@ -1,43 +0,0 @@ -//go:build linux || openbsd || freebsd - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "os" - "path/filepath" - "strconv" -) - -// Based on https://github.com/adrg/xdg -// Licensed under MIT License (MIT) -// Copyright (c) 2014 Adrian-George Bostan <adrg@epistack.com> - -func osDependentRunDir() (string, error) { - run := filepath.Join("run", "user", strconv.Itoa(os.Getuid())) - if _, err := os.Stat(run); err == nil { - return run, nil - } - - // /run/user/$uid is set by pam_systemd, but might not be present, especially in containerized environments - home, err := os.UserHomeDir() - if err != nil { - return "", err - } - return filepath.Join(home, ".docker", "docker-compose"), nil -} diff --git a/internal/locker/runtime_windows.go b/internal/locker/runtime_windows.go deleted file mode 100644 index 4d4451b61c6..00000000000 --- a/internal/locker/runtime_windows.go +++ /dev/null @@ -1,49 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "os" - "path/filepath" - - "golang.org/x/sys/windows" -) - -// Based on https://github.com/adrg/xdg -// Licensed under MIT License (MIT) -// Copyright (c) 2014 Adrian-George Bostan <adrg@epistack.com> - -func osDependentRunDir() (string, error) { - flags := []uint32{windows.KF_FLAG_DEFAULT, windows.KF_FLAG_DEFAULT_PATH} - for _, flag := range flags { - p, _ := windows.KnownFolderPath(windows.FOLDERID_LocalAppData, flag|windows.KF_FLAG_DONT_VERIFY) - if p != "" { - return filepath.Join(p, "docker-compose"), nil - } - } - - appData, ok := os.LookupEnv("LOCALAPPDATA") - if ok { - return filepath.Join(appData, "docker-compose"), nil - } - - home, err := os.UserHomeDir() - if err != nil { - return "", err - } - return filepath.Join(home, "AppData", "Local", "docker-compose"), nil -} diff --git a/internal/memnet/conn.go b/internal/memnet/conn.go deleted file mode 100644 index 224bec78830..00000000000 --- a/internal/memnet/conn.go +++ /dev/null @@ -1,50 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package memnet - -import ( - "context" - "fmt" - "net" - "strings" -) - -func DialEndpoint(ctx context.Context, endpoint string) (net.Conn, error) { - if addr, ok := strings.CutPrefix(endpoint, "unix://"); ok { - return Dial(ctx, "unix", addr) - } - if addr, ok := strings.CutPrefix(endpoint, "npipe://"); ok { - return Dial(ctx, "npipe", addr) - } - return nil, fmt.Errorf("unsupported protocol for address: %s", endpoint) -} - -func Dial(ctx context.Context, network, addr string) (net.Conn, error) { - var d net.Dialer - switch network { - case "unix": - if err := validateSocketPath(addr); err != nil { - return nil, err - } - return d.DialContext(ctx, "unix", addr) - case "npipe": - // N.B. this will return an error on non-Windows - return dialNamedPipe(ctx, addr) - default: - return nil, fmt.Errorf("unsupported network: %s", network) - } -} diff --git a/internal/memnet/conn_unix.go b/internal/memnet/conn_unix.go deleted file mode 100644 index e151984848a..00000000000 --- a/internal/memnet/conn_unix.go +++ /dev/null @@ -1,39 +0,0 @@ -//go:build !windows - -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package memnet - -import ( - "context" - "fmt" - "net" - "syscall" -) - -const maxUnixSocketPathSize = len(syscall.RawSockaddrUnix{}.Path) - -func dialNamedPipe(_ context.Context, _ string) (net.Conn, error) { - return nil, fmt.Errorf("named pipes are only available on Windows") -} - -func validateSocketPath(addr string) error { - if len(addr) > maxUnixSocketPathSize { - return fmt.Errorf("socket address is too long: %s", addr) - } - return nil -} diff --git a/internal/memnet/conn_windows.go b/internal/memnet/conn_windows.go deleted file mode 100644 index b7f7d9ea8fa..00000000000 --- a/internal/memnet/conn_windows.go +++ /dev/null @@ -1,33 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package memnet - -import ( - "context" - "net" - - "github.com/Microsoft/go-winio" -) - -func dialNamedPipe(ctx context.Context, addr string) (net.Conn, error) { - return winio.DialPipeContext(ctx, addr) -} - -func validateSocketPath(addr string) error { - // AF_UNIX sockets do not have strict path limits on Windows - return nil -} diff --git a/internal/oci/push.go b/internal/oci/push.go deleted file mode 100644 index 1e2d0f2e95b..00000000000 --- a/internal/oci/push.go +++ /dev/null @@ -1,223 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package oci - -import ( - "context" - "encoding/json" - "errors" - "fmt" - "net/http" - "path/filepath" - "slices" - "time" - - "github.com/containerd/containerd/v2/core/remotes" - pusherrors "github.com/containerd/containerd/v2/core/remotes/errors" - "github.com/distribution/reference" - "github.com/opencontainers/go-digest" - "github.com/opencontainers/image-spec/specs-go" - v1 "github.com/opencontainers/image-spec/specs-go/v1" - - "github.com/docker/compose/v5/pkg/api" -) - -const ( - // ComposeProjectArtifactType is the OCI 1.1-compliant artifact type value - // for the generated image manifest. - ComposeProjectArtifactType = "application/vnd.docker.compose.project" - // ComposeYAMLMediaType is the media type for each layer (Compose file) - // in the image manifest. - ComposeYAMLMediaType = "application/vnd.docker.compose.file+yaml" - // ComposeEmptyConfigMediaType is a media type used for the config descriptor - // when doing OCI 1.0-style pushes. - // - // The content is always `{}`, the same as a normal empty descriptor, but - // the specific media type allows clients to fall back to the config media - // type to recognize the manifest as a Compose project since the artifact - // type field is not available in OCI 1.0. - // - // This is based on guidance from the OCI 1.1 spec: - // > Implementers note: artifacts have historically been created without - // > an artifactType field, and tooling to work with artifacts should - // > fallback to the config.mediaType value. - ComposeEmptyConfigMediaType = "application/vnd.docker.compose.config.empty.v1+json" - // ComposeEnvFileMediaType is the media type for each Env File layer in the image manifest. - ComposeEnvFileMediaType = "application/vnd.docker.compose.envfile" -) - -// clientAuthStatusCodes are client (4xx) errors that are authentication -// related. -var clientAuthStatusCodes = []int{ - http.StatusUnauthorized, - http.StatusForbidden, - http.StatusProxyAuthRequired, -} - -func DescriptorForComposeFile(path string, content []byte) v1.Descriptor { - return v1.Descriptor{ - MediaType: ComposeYAMLMediaType, - Digest: digest.FromString(string(content)), - Size: int64(len(content)), - Annotations: map[string]string{ - "com.docker.compose.version": api.ComposeVersion, - "com.docker.compose.file": filepath.Base(path), - }, - Data: content, - } -} - -func DescriptorForEnvFile(path string, content []byte) v1.Descriptor { - return v1.Descriptor{ - MediaType: ComposeEnvFileMediaType, - Digest: digest.FromString(string(content)), - Size: int64(len(content)), - Annotations: map[string]string{ - "com.docker.compose.version": api.ComposeVersion, - "com.docker.compose.envfile": filepath.Base(path), - }, - Data: content, - } -} - -func PushManifest(ctx context.Context, resolver remotes.Resolver, named reference.Named, layers []v1.Descriptor, ociVersion api.OCIVersion) (v1.Descriptor, error) { - // Check if we need an extra empty layer for the manifest config - if ociVersion == api.OCIVersion1_1 || ociVersion == "" { - err := push(ctx, resolver, named, v1.DescriptorEmptyJSON) - if err != nil { - return v1.Descriptor{}, err - } - } - // prepare to push the manifest by pushing the layers - layerDescriptors := make([]v1.Descriptor, len(layers)) - for i := range layers { - layerDescriptors[i] = layers[i] - if err := push(ctx, resolver, named, layers[i]); err != nil { - return v1.Descriptor{}, err - } - } - - if ociVersion != "" { - // if a version was explicitly specified, use it - return createAndPushManifest(ctx, resolver, named, layerDescriptors, ociVersion) - } - - // try to push in the OCI 1.1 format but fallback to OCI 1.0 on 4xx errors - // (other than auth) since it's most likely the result of the registry not - // having support - descriptor, err := createAndPushManifest(ctx, resolver, named, layerDescriptors, api.OCIVersion1_1) - var pushErr pusherrors.ErrUnexpectedStatus - if errors.As(err, &pushErr) && isNonAuthClientError(pushErr.StatusCode) { - // TODO(milas): show a warning here (won't work with logrus) - return createAndPushManifest(ctx, resolver, named, layerDescriptors, api.OCIVersion1_0) - } - return descriptor, err -} - -func push(ctx context.Context, resolver remotes.Resolver, ref reference.Named, descriptor v1.Descriptor) error { - fullRef, err := reference.WithDigest(reference.TagNameOnly(ref), descriptor.Digest) - if err != nil { - return err - } - - return Push(ctx, resolver, fullRef, descriptor) -} - -func createAndPushManifest(ctx context.Context, resolver remotes.Resolver, named reference.Named, layers []v1.Descriptor, ociVersion api.OCIVersion) (v1.Descriptor, error) { - descriptor, toPush, err := generateManifest(layers, ociVersion) - if err != nil { - return v1.Descriptor{}, err - } - for _, p := range toPush { - err = push(ctx, resolver, named, p) - if err != nil { - return v1.Descriptor{}, err - } - } - return descriptor, nil -} - -func isNonAuthClientError(statusCode int) bool { - if statusCode < 400 || statusCode >= 500 { - // not a client error - return false - } - return !slices.Contains(clientAuthStatusCodes, statusCode) -} - -func generateManifest(layers []v1.Descriptor, ociCompat api.OCIVersion) (v1.Descriptor, []v1.Descriptor, error) { - var toPush []v1.Descriptor - var config v1.Descriptor - var artifactType string - switch ociCompat { - case api.OCIVersion1_0: - // "Content other than OCI container images MAY be packaged using the image manifest. - // When this is done, the config.mediaType value MUST be set to a value specific to - // the artifact type or the empty value." - // Source: https://github.com/opencontainers/image-spec/blob/main/manifest.md#guidelines-for-artifact-usage - // - // The `ComposeEmptyConfigMediaType` is used specifically for this purpose: - // there is no config, and an empty descriptor is used for OCI 1.1 in - // conjunction with the `ArtifactType`, but for OCI 1.0 compatibility, - // tooling falls back to the config media type, so this is used to - // indicate that it's not a container image but custom content. - configData := []byte("{}") - config = v1.Descriptor{ - MediaType: ComposeEmptyConfigMediaType, - Digest: digest.FromBytes(configData), - Size: int64(len(configData)), - Data: configData, - } - // N.B. OCI 1.0 does NOT support specifying the artifact type, so it's - // left as an empty string to omit it from the marshaled JSON - artifactType = "" - toPush = append(toPush, config) - case api.OCIVersion1_1: - config = v1.DescriptorEmptyJSON - artifactType = ComposeProjectArtifactType - toPush = append(toPush, config) - default: - return v1.Descriptor{}, nil, fmt.Errorf("unsupported OCI version: %s", ociCompat) - } - - manifest, err := json.Marshal(v1.Manifest{ - Versioned: specs.Versioned{SchemaVersion: 2}, - MediaType: v1.MediaTypeImageManifest, - ArtifactType: artifactType, - Config: config, - Layers: layers, - Annotations: map[string]string{ - "org.opencontainers.image.created": time.Now().Format(time.RFC3339), - }, - }) - if err != nil { - return v1.Descriptor{}, nil, err - } - - manifestDescriptor := v1.Descriptor{ - MediaType: v1.MediaTypeImageManifest, - Digest: digest.FromString(string(manifest)), - Size: int64(len(manifest)), - Annotations: map[string]string{ - "com.docker.compose.version": api.ComposeVersion, - }, - ArtifactType: artifactType, - Data: manifest, - } - toPush = append(toPush, manifestDescriptor) - return manifestDescriptor, toPush, nil -} diff --git a/internal/oci/resolver.go b/internal/oci/resolver.go deleted file mode 100644 index f18c474de57..00000000000 --- a/internal/oci/resolver.go +++ /dev/null @@ -1,143 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package oci - -import ( - "context" - "io" - "net/url" - "slices" - "strings" - - "github.com/containerd/containerd/v2/core/remotes" - "github.com/containerd/containerd/v2/core/remotes/docker" - "github.com/containerd/containerd/v2/pkg/labels" - "github.com/containerd/errdefs" - "github.com/distribution/reference" - "github.com/docker/cli/cli/config/configfile" - "github.com/moby/buildkit/util/contentutil" - spec "github.com/opencontainers/image-spec/specs-go/v1" - - "github.com/docker/compose/v5/internal/registry" -) - -// NewResolver setup an OCI Resolver based on docker/cli config to provide registry credentials -func NewResolver(config *configfile.ConfigFile, insecureRegistries ...string) remotes.Resolver { - return docker.NewResolver(docker.ResolverOptions{ - Hosts: docker.ConfigureDefaultRegistries( - docker.WithAuthorizer(docker.NewDockerAuthorizer( - docker.WithAuthCreds(func(host string) (string, string, error) { - host = registry.GetAuthConfigKey(host) - auth, err := config.GetAuthConfig(host) - if err != nil { - return "", "", err - } - if auth.IdentityToken != "" { - return "", auth.IdentityToken, nil - } - return auth.Username, auth.Password, nil - }), - )), - docker.WithPlainHTTP(func(domain string) (bool, error) { - // Should be used for testing **only** - return slices.Contains(insecureRegistries, domain), nil - }), - ), - }) -} - -// Get retrieves a Named OCI resource and returns OCI Descriptor and Manifest -func Get(ctx context.Context, resolver remotes.Resolver, ref reference.Named) (spec.Descriptor, []byte, error) { - _, descriptor, err := resolver.Resolve(ctx, ref.String()) - if err != nil { - return spec.Descriptor{}, nil, err - } - - fetcher, err := resolver.Fetcher(ctx, ref.String()) - if err != nil { - return spec.Descriptor{}, nil, err - } - fetch, err := fetcher.Fetch(ctx, descriptor) - if err != nil { - return spec.Descriptor{}, nil, err - } - content, err := io.ReadAll(fetch) - if err != nil { - return spec.Descriptor{}, nil, err - } - return descriptor, content, nil -} - -func Copy(ctx context.Context, resolver remotes.Resolver, image reference.Named, named reference.Named) (spec.Descriptor, error) { - src, desc, err := resolver.Resolve(ctx, image.String()) - if err != nil { - return spec.Descriptor{}, err - } - if desc.Annotations == nil { - desc.Annotations = make(map[string]string) - } - // set LabelDistributionSource so push will actually use a registry mount - refspec := reference.TrimNamed(image).String() - u, err := url.Parse("dummy://" + refspec) - if err != nil { - return spec.Descriptor{}, err - } - source, repo := u.Hostname(), strings.TrimPrefix(u.Path, "/") - desc.Annotations[labels.LabelDistributionSource+"."+source] = repo - - p, err := resolver.Pusher(ctx, named.Name()) - if err != nil { - return spec.Descriptor{}, err - } - f, err := resolver.Fetcher(ctx, src) - if err != nil { - return spec.Descriptor{}, err - } - - err = contentutil.CopyChain(ctx, - contentutil.FromPusher(p), - contentutil.FromFetcher(f), desc) - return desc, err -} - -func Push(ctx context.Context, resolver remotes.Resolver, ref reference.Named, descriptor spec.Descriptor) error { - pusher, err := resolver.Pusher(ctx, ref.String()) - if err != nil { - return err - } - ctx = remotes.WithMediaTypeKeyPrefix(ctx, ComposeYAMLMediaType, "artifact-") - ctx = remotes.WithMediaTypeKeyPrefix(ctx, ComposeEnvFileMediaType, "artifact-") - ctx = remotes.WithMediaTypeKeyPrefix(ctx, ComposeEmptyConfigMediaType, "config-") - ctx = remotes.WithMediaTypeKeyPrefix(ctx, spec.MediaTypeEmptyJSON, "config-") - - push, err := pusher.Push(ctx, descriptor) - if errdefs.IsAlreadyExists(err) { - return nil - } - if err != nil { - return err - } - - _, err = push.Write(descriptor.Data) - if err != nil { - // Close the writer on error since Commit won't be called - _ = push.Close() - return err - } - // Commit will close the writer - return push.Commit(ctx, int64(len(descriptor.Data)), descriptor.Digest) -} diff --git a/internal/paths/paths.go b/internal/paths/paths.go deleted file mode 100644 index 4e4c01b8cc4..00000000000 --- a/internal/paths/paths.go +++ /dev/null @@ -1,120 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package paths - -import ( - "os" - "path/filepath" - "strings" -) - -func IsChild(dir string, file string) bool { - if dir == "" { - return false - } - - dir = filepath.Clean(dir) - current := filepath.Clean(file) - child := "." - for { - if strings.EqualFold(dir, current) { - // If the two paths are exactly equal, then they must be the same. - if dir == current { - return true - } - - // If the two paths are equal under case-folding, but not exactly equal, - // then the only way to check if they're truly "equal" is to check - // to see if we're on a case-insensitive file system. - // - // This is a notoriously tricky problem. See how dep solves it here: - // https://github.com/golang/dep/blob/v0.5.4/internal/fs/fs.go#L33 - // - // because you can mount case-sensitive filesystems onto case-insensitive - // file-systems, and vice versa :scream: - // - // We want to do as much of this check as possible with strings-only - // (to avoid a file system read and error handling), so we only - // do this check if we have no other choice. - dirInfo, err := os.Stat(dir) - if err != nil { - return false - } - - currentInfo, err := os.Stat(current) - if err != nil { - return false - } - - if !os.SameFile(dirInfo, currentInfo) { - return false - } - return true - } - - if len(current) <= len(dir) || current == "." { - return false - } - - cDir := filepath.Dir(current) - cBase := filepath.Base(current) - child = filepath.Join(cBase, child) - current = cDir - } -} - -// EncompassingPaths returns the minimal set of paths that root all paths -// from the original collection. -// -// For example, ["/foo", "/foo/bar", "/foo", "/baz"] -> ["/foo", "/baz]. -func EncompassingPaths(paths []string) []string { - result := []string{} - for _, current := range paths { - isCovered := false - hasRemovals := false - - for i, existing := range result { - if IsChild(existing, current) { - // The path is already covered, so there's no need to include it - isCovered = true - break - } - - if IsChild(current, existing) { - // Mark the element empty for removal. - result[i] = "" - hasRemovals = true - } - } - - if !isCovered { - result = append(result, current) - } - - if hasRemovals { - // Remove all the empties - newResult := []string{} - for _, r := range result { - if r != "" { - newResult = append(newResult, r) - } - } - result = newResult - } - } - return result -} diff --git a/internal/registry/registry.go b/internal/registry/registry.go deleted file mode 100644 index 0ee73883070..00000000000 --- a/internal/registry/registry.go +++ /dev/null @@ -1,44 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package registry - -const ( - // DefaultNamespace is the default namespace - DefaultNamespace = "docker.io" - // DefaultRegistryHost is the hostname for the default (Docker Hub) registry - // used for pushing and pulling images. This hostname is hard-coded to handle - // the conversion from image references without registry name (e.g. "ubuntu", - // or "ubuntu:latest"), as well as references using the "docker.io" domain - // name, which is used as canonical reference for images on Docker Hub, but - // does not match the domain-name of Docker Hub's registry. - DefaultRegistryHost = "registry-1.docker.io" - // IndexHostname is the index hostname, used for authentication and image search. - IndexHostname = "index.docker.io" - // IndexServer is used for user auth and image search - IndexServer = "https://" + IndexHostname + "/v1/" - // IndexName is the name of the index - IndexName = "docker.io" -) - -// GetAuthConfigKey special-cases using the full index address of the official -// index as the AuthConfig key, and uses the (host)name[:port] for private indexes. -func GetAuthConfigKey(indexName string) string { - if indexName == IndexName || indexName == IndexHostname || indexName == DefaultRegistryHost { - return IndexServer - } - return indexName -} diff --git a/internal/sync/shared.go b/internal/sync/shared.go deleted file mode 100644 index 4fd9df37719..00000000000 --- a/internal/sync/shared.go +++ /dev/null @@ -1,40 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package sync - -import ( - "context" -) - -// PathMapping contains the Compose service and modified host system path. -type PathMapping struct { - // HostPath that was created/modified/deleted outside the container. - // - // This is the path as seen from the user's perspective, e.g. - // - C:\Users\moby\Documents\hello-world\main.go (file on Windows) - // - /Users/moby/Documents/hello-world (directory on macOS) - HostPath string - // ContainerPath for the target file inside the container (only populated - // for sync events, not rebuild). - // - // This is the path as used in Docker CLI commands, e.g. - // - /workdir/main.go - // - /workdir/subdir - ContainerPath string -} - -type Syncer interface { - Sync(ctx context.Context, service string, paths []*PathMapping) error -} diff --git a/internal/sync/tar.go b/internal/sync/tar.go deleted file mode 100644 index 4250b6afc4a..00000000000 --- a/internal/sync/tar.go +++ /dev/null @@ -1,352 +0,0 @@ -/* - Copyright 2018 The Tilt Dev Authors - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package sync - -import ( - "archive/tar" - "bytes" - "context" - "errors" - "fmt" - "io" - "io/fs" - "os" - "path" - "path/filepath" - "strings" - "sync" - - "github.com/docker/docker/api/types/container" - "github.com/moby/go-archive" - "golang.org/x/sync/errgroup" -) - -type archiveEntry struct { - path string - info os.FileInfo - header *tar.Header -} - -type LowLevelClient interface { - ContainersForService(ctx context.Context, projectName string, serviceName string) ([]container.Summary, error) - - Exec(ctx context.Context, containerID string, cmd []string, in io.Reader) error - Untar(ctx context.Context, id string, reader io.ReadCloser) error -} - -type Tar struct { - client LowLevelClient - - projectName string -} - -var _ Syncer = &Tar{} - -func NewTar(projectName string, client LowLevelClient) *Tar { - return &Tar{ - projectName: projectName, - client: client, - } -} - -func (t *Tar) Sync(ctx context.Context, service string, paths []*PathMapping) error { - containers, err := t.client.ContainersForService(ctx, t.projectName, service) - if err != nil { - return err - } - - var pathsToCopy []PathMapping - var pathsToDelete []string - for _, p := range paths { - if _, err := os.Stat(p.HostPath); err != nil && errors.Is(err, fs.ErrNotExist) { - pathsToDelete = append(pathsToDelete, p.ContainerPath) - } else { - pathsToCopy = append(pathsToCopy, *p) - } - } - - var deleteCmd []string - if len(pathsToDelete) != 0 { - deleteCmd = append([]string{"rm", "-rf"}, pathsToDelete...) - } - - var ( - eg errgroup.Group - errMu sync.Mutex - errs = make([]error, 0, len(containers)*2) // max 2 errs per container - ) - - eg.SetLimit(16) // arbitrary limit, adjust to taste :D - for i := range containers { - containerID := containers[i].ID - tarReader := tarArchive(pathsToCopy) - - eg.Go(func() error { - if len(deleteCmd) != 0 { - if err := t.client.Exec(ctx, containerID, deleteCmd, nil); err != nil { - errMu.Lock() - errs = append(errs, fmt.Errorf("deleting paths in %s: %w", containerID, err)) - errMu.Unlock() - } - } - - if err := t.client.Untar(ctx, containerID, tarReader); err != nil { - errMu.Lock() - errs = append(errs, fmt.Errorf("copying files to %s: %w", containerID, err)) - errMu.Unlock() - } - return nil // don't fail-fast; collect all errors - }) - } - - _ = eg.Wait() - return errors.Join(errs...) -} - -type ArchiveBuilder struct { - tw *tar.Writer - // A shared I/O buffer to help with file copying. - copyBuf *bytes.Buffer -} - -func NewArchiveBuilder(writer io.Writer) *ArchiveBuilder { - tw := tar.NewWriter(writer) - return &ArchiveBuilder{ - tw: tw, - copyBuf: &bytes.Buffer{}, - } -} - -func (a *ArchiveBuilder) Close() error { - return a.tw.Close() -} - -// ArchivePathsIfExist creates a tar archive of all local files in `paths`. It quietly skips any paths that don't exist. -func (a *ArchiveBuilder) ArchivePathsIfExist(paths []PathMapping) error { - // In order to handle overlapping syncs, we - // 1) collect all the entries, - // 2) de-dupe them, with last-one-wins semantics - // 3) write all the entries - // - // It's not obvious that this is the correct behavior. A better approach - // (that's more in-line with how syncs work) might ignore files in earlier - // path mappings when we know they're going to be "synced" over. - // There's a bunch of subtle product decisions about how overlapping path - // mappings work that we're not sure about. - var entries []archiveEntry - for _, p := range paths { - newEntries, err := a.entriesForPath(p.HostPath, p.ContainerPath) - if err != nil { - return fmt.Errorf("inspecting %q: %w", p.HostPath, err) - } - - entries = append(entries, newEntries...) - } - - entries = dedupeEntries(entries) - for _, entry := range entries { - err := a.writeEntry(entry) - if err != nil { - return fmt.Errorf("archiving %q: %w", entry.path, err) - } - } - return nil -} - -func (a *ArchiveBuilder) writeEntry(entry archiveEntry) error { - pathInTar := entry.path - header := entry.header - - if header.Typeflag != tar.TypeReg { - // anything other than a regular file (e.g. dir, symlink) just needs the header - if err := a.tw.WriteHeader(header); err != nil { - return fmt.Errorf("writing %q header: %w", pathInTar, err) - } - return nil - } - - file, err := os.Open(pathInTar) - if err != nil { - // In case the file has been deleted since we last looked at it. - if os.IsNotExist(err) { - return nil - } - return err - } - - defer func() { - _ = file.Close() - }() - - // The size header must match the number of contents bytes. - // - // There is room for a race condition here if something writes to the file - // after we've read the file size. - // - // For small files, we avoid this by first copying the file into a buffer, - // and using the size of the buffer to populate the header. - // - // For larger files, we don't want to copy the whole thing into a buffer, - // because that would blow up heap size. There is some danger that this - // will lead to a spurious error when the tar writer validates the sizes. - // That error will be disruptive but will be handled as best as we - // can downstream. - useBuf := header.Size < 5000000 - if useBuf { - a.copyBuf.Reset() - _, err = io.Copy(a.copyBuf, file) - if err != nil && !errors.Is(err, io.EOF) { - return fmt.Errorf("copying %q: %w", pathInTar, err) - } - header.Size = int64(len(a.copyBuf.Bytes())) - } - - // wait to write the header until _after_ the file is successfully opened - // to avoid generating an invalid tar entry that has a header but no contents - // in the case the file has been deleted - err = a.tw.WriteHeader(header) - if err != nil { - return fmt.Errorf("writing %q header: %w", pathInTar, err) - } - - if useBuf { - _, err = io.Copy(a.tw, a.copyBuf) - } else { - _, err = io.Copy(a.tw, file) - } - - if err != nil && !errors.Is(err, io.EOF) { - return fmt.Errorf("copying %q: %w", pathInTar, err) - } - - // explicitly flush so that if the entry is invalid we will detect it now and - // provide a more meaningful error - if err := a.tw.Flush(); err != nil { - return fmt.Errorf("finalizing %q: %w", pathInTar, err) - } - return nil -} - -// entriesForPath writes the given source path into tarWriter at the given dest (recursively for directories). -// e.g. tarring my_dir --> dest d: d/file_a, d/file_b -// If source path does not exist, quietly skips it and returns no err -func (a *ArchiveBuilder) entriesForPath(localPath, containerPath string) ([]archiveEntry, error) { - localInfo, err := os.Stat(localPath) - if err != nil { - if os.IsNotExist(err) { - return nil, nil - } - return nil, err - } - - localPathIsDir := localInfo.IsDir() - if localPathIsDir { - // Make sure we can trim this off filenames to get valid relative filepaths - if !strings.HasSuffix(localPath, string(filepath.Separator)) { - localPath += string(filepath.Separator) - } - } - - containerPath = strings.TrimPrefix(containerPath, "/") - - result := make([]archiveEntry, 0) - err = filepath.Walk(localPath, func(curLocalPath string, info os.FileInfo, err error) error { - if err != nil { - return fmt.Errorf("walking %q: %w", curLocalPath, err) - } - - linkname := "" - if info.Mode()&os.ModeSymlink != 0 { - var err error - linkname, err = os.Readlink(curLocalPath) - if err != nil { - return err - } - } - - var name string - //nolint:gocritic - if localPathIsDir { - // Name of file in tar should be relative to source directory... - tmp, err := filepath.Rel(localPath, curLocalPath) - if err != nil { - return fmt.Errorf("making %q relative to %q: %w", curLocalPath, localPath, err) - } - // ...and live inside `dest` - name = path.Join(containerPath, filepath.ToSlash(tmp)) - } else if strings.HasSuffix(containerPath, "/") { - name = containerPath + filepath.Base(curLocalPath) - } else { - name = containerPath - } - - header, err := archive.FileInfoHeader(name, info, linkname) - if err != nil { - // Not all types of files are allowed in a tarball. That's OK. - // Mimic the Docker behavior and just skip the file. - return nil - } - - result = append(result, archiveEntry{ - path: curLocalPath, - info: info, - header: header, - }) - - return nil - }) - if err != nil { - return nil, err - } - return result, nil -} - -func tarArchive(ops []PathMapping) io.ReadCloser { - pr, pw := io.Pipe() - go func() { - ab := NewArchiveBuilder(pw) - err := ab.ArchivePathsIfExist(ops) - if err != nil { - _ = pw.CloseWithError(fmt.Errorf("adding files to tar: %w", err)) - } else { - // propagate errors from the TarWriter::Close() because it performs a final - // Flush() and any errors mean the tar is invalid - if err := ab.Close(); err != nil { - _ = pw.CloseWithError(fmt.Errorf("closing tar: %w", err)) - } else { - _ = pw.Close() - } - } - }() - return pr -} - -// Dedupe the entries with last-entry-wins semantics. -func dedupeEntries(entries []archiveEntry) []archiveEntry { - seenIndex := make(map[string]int, len(entries)) - result := make([]archiveEntry, 0, len(entries)) - for i, entry := range entries { - seenIndex[entry.header.Name] = i - } - for i, entry := range entries { - if seenIndex[entry.header.Name] == i { - result = append(result, entry) - } - } - return result -} diff --git a/internal/tracing/attributes.go b/internal/tracing/attributes.go deleted file mode 100644 index 2c8779bc86c..00000000000 --- a/internal/tracing/attributes.go +++ /dev/null @@ -1,196 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "context" - "crypto/sha256" - "encoding/json" - "fmt" - "strings" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/trace" -) - -// SpanOptions is a small helper type to make it easy to share the options helpers between -// downstream functions that accept slices of trace.SpanStartOption and trace.EventOption. -type SpanOptions []trace.SpanStartEventOption - -type MetricsKey struct{} - -type Metrics struct { - CountExtends int - CountIncludesLocal int - CountIncludesRemote int -} - -func (s SpanOptions) SpanStartOptions() []trace.SpanStartOption { - out := make([]trace.SpanStartOption, len(s)) - for i := range s { - out[i] = s[i] - } - return out -} - -func (s SpanOptions) EventOptions() []trace.EventOption { - out := make([]trace.EventOption, len(s)) - for i := range s { - out[i] = s[i] - } - return out -} - -// ProjectOptions returns common attributes from a Compose project. -// -// For convenience, it's returned as a SpanOptions object to allow it to be -// passed directly to the wrapping helper methods in this package such as -// SpanWrapFunc. -func ProjectOptions(ctx context.Context, proj *types.Project) SpanOptions { - if proj == nil { - return nil - } - capabilities, gpu, tpu := proj.ServicesWithCapabilities() - attrs := []attribute.KeyValue{ - attribute.String("project.name", proj.Name), - attribute.String("project.dir", proj.WorkingDir), - attribute.StringSlice("project.compose_files", proj.ComposeFiles), - attribute.StringSlice("project.profiles", proj.Profiles), - attribute.StringSlice("project.volumes", proj.VolumeNames()), - attribute.StringSlice("project.networks", proj.NetworkNames()), - attribute.StringSlice("project.secrets", proj.SecretNames()), - attribute.StringSlice("project.configs", proj.ConfigNames()), - attribute.StringSlice("project.models", proj.ModelNames()), - attribute.StringSlice("project.extensions", keys(proj.Extensions)), - attribute.StringSlice("project.services.active", proj.ServiceNames()), - attribute.StringSlice("project.services.disabled", proj.DisabledServiceNames()), - attribute.StringSlice("project.services.build", proj.ServicesWithBuild()), - attribute.StringSlice("project.services.depends_on", proj.ServicesWithDependsOn()), - attribute.StringSlice("project.services.models", proj.ServicesWithModels()), - attribute.StringSlice("project.services.capabilities", capabilities), - attribute.StringSlice("project.services.capabilities.gpu", gpu), - attribute.StringSlice("project.services.capabilities.tpu", tpu), - } - if metrics, ok := ctx.Value(MetricsKey{}).(Metrics); ok { - attrs = append(attrs, attribute.Int("project.services.extends", metrics.CountExtends)) - attrs = append(attrs, attribute.Int("project.includes.local", metrics.CountIncludesLocal)) - attrs = append(attrs, attribute.Int("project.includes.remote", metrics.CountIncludesRemote)) - } - - if projHash, ok := projectHash(proj); ok { - attrs = append(attrs, attribute.String("project.hash", projHash)) - } - return []trace.SpanStartEventOption{ - trace.WithAttributes(attrs...), - } -} - -// ServiceOptions returns common attributes from a Compose service. -// -// For convenience, it's returned as a SpanOptions object to allow it to be -// passed directly to the wrapping helper methods in this package such as -// SpanWrapFunc. -func ServiceOptions(service types.ServiceConfig) SpanOptions { - attrs := []attribute.KeyValue{ - attribute.String("service.name", service.Name), - attribute.String("service.image", service.Image), - attribute.StringSlice("service.networks", keys(service.Networks)), - attribute.StringSlice("service.models", keys(service.Models)), - } - - configNames := make([]string, len(service.Configs)) - for i := range service.Configs { - configNames[i] = service.Configs[i].Source - } - attrs = append(attrs, attribute.StringSlice("service.configs", configNames)) - - secretNames := make([]string, len(service.Secrets)) - for i := range service.Secrets { - secretNames[i] = service.Secrets[i].Source - } - attrs = append(attrs, attribute.StringSlice("service.secrets", secretNames)) - - volNames := make([]string, len(service.Volumes)) - for i := range service.Volumes { - volNames[i] = service.Volumes[i].Source - } - attrs = append(attrs, attribute.StringSlice("service.volumes", volNames)) - - return []trace.SpanStartEventOption{ - trace.WithAttributes(attrs...), - } -} - -// ContainerOptions returns common attributes from a Moby container. -// -// For convenience, it's returned as a SpanOptions object to allow it to be -// passed directly to the wrapping helper methods in this package such as -// SpanWrapFunc. -func ContainerOptions(ctr container.Summary) SpanOptions { - attrs := []attribute.KeyValue{ - attribute.String("container.id", ctr.ID), - attribute.String("container.image", ctr.Image), - unixTimeAttr("container.created_at", ctr.Created), - } - - if len(ctr.Names) != 0 { - attrs = append(attrs, attribute.String("container.name", strings.TrimPrefix(ctr.Names[0], "/"))) - } - - return []trace.SpanStartEventOption{ - trace.WithAttributes(attrs...), - } -} - -func keys[T any](m map[string]T) []string { - out := make([]string, 0, len(m)) - for k := range m { - out = append(out, k) - } - return out -} - -func timeAttr(key string, value time.Time) attribute.KeyValue { - return attribute.String(key, value.Format(time.RFC3339)) -} - -func unixTimeAttr(key string, value int64) attribute.KeyValue { - return timeAttr(key, time.Unix(value, 0).UTC()) -} - -// projectHash returns a checksum from the JSON encoding of the project. -func projectHash(p *types.Project) (string, bool) { - if p == nil { - return "", false - } - // disabled services aren't included in the output, so make a copy with - // all the services active for hashing - var err error - p, err = p.WithServicesEnabled(append(p.ServiceNames(), p.DisabledServiceNames()...)...) - if err != nil { - return "", false - } - projData, err := json.Marshal(p) - if err != nil { - return "", false - } - d := sha256.Sum256(projData) - return fmt.Sprintf("%x", d), true -} diff --git a/internal/tracing/attributes_test.go b/internal/tracing/attributes_test.go deleted file mode 100644 index d4277a940ab..00000000000 --- a/internal/tracing/attributes_test.go +++ /dev/null @@ -1,67 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/stretchr/testify/require" -) - -func TestProjectHash(t *testing.T) { - projA := &types.Project{ - Name: "fake-proj", - WorkingDir: "/tmp", - Services: map[string]types.ServiceConfig{ - "foo": {Image: "fake-image"}, - }, - DisabledServices: map[string]types.ServiceConfig{ - "bar": {Image: "diff-image"}, - }, - } - projB := &types.Project{ - Name: "fake-proj", - WorkingDir: "/tmp", - Services: map[string]types.ServiceConfig{ - "foo": {Image: "fake-image"}, - "bar": {Image: "diff-image"}, - }, - } - projC := &types.Project{ - Name: "fake-proj", - WorkingDir: "/tmp", - Services: map[string]types.ServiceConfig{ - "foo": {Image: "fake-image"}, - "bar": {Image: "diff-image"}, - "baz": {Image: "yet-another-image"}, - }, - } - - hashA, ok := projectHash(projA) - require.True(t, ok) - require.NotEmpty(t, hashA) - hashB, ok := projectHash(projB) - require.True(t, ok) - require.NotEmpty(t, hashB) - require.Equal(t, hashA, hashB) - - hashC, ok := projectHash(projC) - require.True(t, ok) - require.NotEmpty(t, hashC) - require.NotEqual(t, hashC, hashA) -} diff --git a/internal/tracing/docker_context.go b/internal/tracing/docker_context.go deleted file mode 100644 index 5d5367b0231..00000000000 --- a/internal/tracing/docker_context.go +++ /dev/null @@ -1,123 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "fmt" - "os" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/context/store" - "go.opentelemetry.io/otel/exporters/otlp/otlptrace" - "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc" - "google.golang.org/grpc" - "google.golang.org/grpc/credentials/insecure" - - "github.com/docker/compose/v5/internal/memnet" -) - -const otelConfigFieldName = "otel" - -// traceClientFromDockerContext creates a gRPC OTLP client based on metadata -// from the active Docker CLI context. -func traceClientFromDockerContext(dockerCli command.Cli, otelEnv envMap) (otlptrace.Client, error) { - // attempt to extract an OTEL config from the Docker context to enable - // automatic integration with Docker Desktop; - cfg, err := ConfigFromDockerContext(dockerCli.ContextStore(), dockerCli.CurrentContext()) - if err != nil { - return nil, fmt.Errorf("loading otel config from docker context metadata: %w", err) - } - - if cfg.Endpoint == "" { - return nil, nil - } - - // HACK: unfortunately _all_ public OTEL initialization functions - // implicitly read from the OS env, so temporarily unset them all and - // restore afterwards - defer func() { - for k, v := range otelEnv { - if err := os.Setenv(k, v); err != nil { - panic(fmt.Errorf("restoring env for %q: %w", k, err)) - } - } - }() - for k := range otelEnv { - if err := os.Unsetenv(k); err != nil { - return nil, fmt.Errorf("stashing env for %q: %w", k, err) - } - } - - conn, err := grpc.NewClient(cfg.Endpoint, - grpc.WithContextDialer(memnet.DialEndpoint), - // this dial is restricted to using a local Unix socket / named pipe, - // so there is no need for TLS - grpc.WithTransportCredentials(insecure.NewCredentials()), - ) - if err != nil { - return nil, fmt.Errorf("initializing otel connection from docker context metadata: %w", err) - } - - client := otlptracegrpc.NewClient(otlptracegrpc.WithGRPCConn(conn)) - return client, nil -} - -// ConfigFromDockerContext inspects extra metadata included as part of the -// specified Docker context to try and extract a valid OTLP client configuration. -func ConfigFromDockerContext(st store.Store, name string) (OTLPConfig, error) { - meta, err := st.GetMetadata(name) - if err != nil { - return OTLPConfig{}, err - } - - var otelCfg any - switch m := meta.Metadata.(type) { - case command.DockerContext: - otelCfg = m.AdditionalFields[otelConfigFieldName] - case map[string]any: - otelCfg = m[otelConfigFieldName] - } - if otelCfg == nil { - return OTLPConfig{}, nil - } - - otelMap, ok := otelCfg.(map[string]any) - if !ok { - return OTLPConfig{}, fmt.Errorf( - "unexpected type for field %q: %T (expected: %T)", - otelConfigFieldName, - otelCfg, - otelMap, - ) - } - - // keys from https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/ - cfg := OTLPConfig{ - Endpoint: valueOrDefault[string](otelMap, "OTEL_EXPORTER_OTLP_ENDPOINT"), - } - return cfg, nil -} - -// valueOrDefault returns the type-cast value at the specified key in the map -// if present and the correct type; otherwise, it returns the default value for -// T. -func valueOrDefault[T any](m map[string]any, key string) T { - if v, ok := m[key].(T); ok { - return v - } - return *new(T) -} diff --git a/internal/tracing/errors.go b/internal/tracing/errors.go deleted file mode 100644 index 9fa615054c0..00000000000 --- a/internal/tracing/errors.go +++ /dev/null @@ -1,29 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "go.opentelemetry.io/otel" -) - -// skipErrors is a no-op otel.ErrorHandler. -type skipErrors struct{} - -// Handle does nothing, ignoring any errors passed to it. -func (skipErrors) Handle(_ error) {} - -var _ otel.ErrorHandler = skipErrors{} diff --git a/internal/tracing/keyboard_metrics.go b/internal/tracing/keyboard_metrics.go deleted file mode 100644 index 2e5120fbea7..00000000000 --- a/internal/tracing/keyboard_metrics.go +++ /dev/null @@ -1,35 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "context" - - "go.opentelemetry.io/otel/attribute" -) - -func KeyboardMetrics(ctx context.Context, enabled, isDockerDesktopActive bool) { - commandAvailable := []string{} - if isDockerDesktopActive { - commandAvailable = append(commandAvailable, "gui") - commandAvailable = append(commandAvailable, "gui/composeview") - } - - AddAttributeToSpan(ctx, - attribute.Bool("navmenu.enabled", enabled), - attribute.StringSlice("navmenu.command_available", commandAvailable)) -} diff --git a/internal/tracing/mux.go b/internal/tracing/mux.go deleted file mode 100644 index 1a09ef1d96a..00000000000 --- a/internal/tracing/mux.go +++ /dev/null @@ -1,73 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "context" - "errors" - "sync" - - sdktrace "go.opentelemetry.io/otel/sdk/trace" -) - -type MuxExporter struct { - exporters []sdktrace.SpanExporter -} - -func (m MuxExporter) ExportSpans(ctx context.Context, spans []sdktrace.ReadOnlySpan) error { - var ( - wg sync.WaitGroup - errMu sync.Mutex - errs = make([]error, 0, len(m.exporters)) - ) - - for _, exporter := range m.exporters { - wg.Add(1) - go func() { - defer wg.Done() - if err := exporter.ExportSpans(ctx, spans); err != nil { - errMu.Lock() - errs = append(errs, err) - errMu.Unlock() - } - }() - } - wg.Wait() - return errors.Join(errs...) -} - -func (m MuxExporter) Shutdown(ctx context.Context) error { - var ( - wg sync.WaitGroup - errMu sync.Mutex - errs = make([]error, 0, len(m.exporters)) - ) - - for _, exporter := range m.exporters { - wg.Add(1) - go func() { - defer wg.Done() - if err := exporter.Shutdown(ctx); err != nil { - errMu.Lock() - errs = append(errs, err) - errMu.Unlock() - } - }() - } - wg.Wait() - return errors.Join(errs...) -} diff --git a/internal/tracing/tracing.go b/internal/tracing/tracing.go deleted file mode 100644 index ec09f61e8bd..00000000000 --- a/internal/tracing/tracing.go +++ /dev/null @@ -1,145 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "context" - "errors" - "fmt" - "os" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/moby/buildkit/util/tracing/detect" - _ "github.com/moby/buildkit/util/tracing/env" //nolint:blank-imports - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/exporters/otlp/otlptrace" - "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc" - "go.opentelemetry.io/otel/propagation" - "go.opentelemetry.io/otel/sdk/resource" - sdktrace "go.opentelemetry.io/otel/sdk/trace" - semconv "go.opentelemetry.io/otel/semconv/v1.21.0" - - "github.com/docker/compose/v5/internal" -) - -func init() { - detect.ServiceName = "compose" - // do not log tracing errors to stdio - otel.SetErrorHandler(skipErrors{}) -} - -// OTLPConfig contains the necessary values to initialize an OTLP client -// manually. -// -// This supports a minimal set of options based on what is necessary for -// automatic OTEL configuration from Docker context metadata. -type OTLPConfig struct { - Endpoint string -} - -// ShutdownFunc flushes and stops an OTEL exporter. -type ShutdownFunc func(ctx context.Context) error - -// envMap is a convenience type for OS environment variables. -type envMap map[string]string - -func InitTracing(dockerCli command.Cli) (ShutdownFunc, error) { - // set global propagator to tracecontext (the default is no-op). - otel.SetTextMapPropagator(propagation.TraceContext{}) - return InitProvider(dockerCli) -} - -func InitProvider(dockerCli command.Cli) (ShutdownFunc, error) { - ctx := context.Background() - - var errs []error - var exporters []sdktrace.SpanExporter - - envClient, otelEnv := traceClientFromEnv() - if envClient != nil { - if envExporter, err := otlptrace.New(ctx, envClient); err != nil { - errs = append(errs, err) - } else if envExporter != nil { - exporters = append(exporters, envExporter) - } - } - - if dcClient, err := traceClientFromDockerContext(dockerCli, otelEnv); err != nil { - errs = append(errs, err) - } else if dcClient != nil { - if dcExporter, err := otlptrace.New(ctx, dcClient); err != nil { - errs = append(errs, err) - } else if dcExporter != nil { - exporters = append(exporters, dcExporter) - } - } - if len(errs) != 0 { - return nil, errors.Join(errs...) - } - - res, err := resource.New( - ctx, - resource.WithAttributes( - semconv.ServiceName("compose"), - semconv.ServiceVersion(internal.Version), - attribute.String("docker.context", dockerCli.CurrentContext()), - ), - ) - if err != nil { - return nil, fmt.Errorf("failed to create resource: %w", err) - } - - muxExporter := MuxExporter{exporters: exporters} - tracerProvider := sdktrace.NewTracerProvider( - sdktrace.WithResource(res), - sdktrace.WithBatcher(muxExporter), - ) - otel.SetTracerProvider(tracerProvider) - - // Shutdown will flush any remaining spans and shut down the exporter. - return tracerProvider.Shutdown, nil -} - -// traceClientFromEnv creates a GRPC OTLP client based on OS environment -// variables. -// -// https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/ -func traceClientFromEnv() (otlptrace.Client, envMap) { - hasOtelEndpointInEnv := false - otelEnv := make(map[string]string) - for _, kv := range os.Environ() { - k, v, ok := strings.Cut(kv, "=") - if !ok { - continue - } - if strings.HasPrefix(k, "OTEL_") { - otelEnv[k] = v - if strings.HasSuffix(k, "ENDPOINT") { - hasOtelEndpointInEnv = true - } - } - } - - if !hasOtelEndpointInEnv { - return nil, nil - } - - client := otlptracegrpc.NewClient() - return client, otelEnv -} diff --git a/internal/tracing/tracing_test.go b/internal/tracing/tracing_test.go deleted file mode 100644 index 3d262631921..00000000000 --- a/internal/tracing/tracing_test.go +++ /dev/null @@ -1,60 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing_test - -import ( - "testing" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/context/store" - "github.com/stretchr/testify/require" - - "github.com/docker/compose/v5/internal/tracing" -) - -var testStoreCfg = store.NewConfig( - func() any { - return &map[string]any{} - }, -) - -func TestExtractOtelFromContext(t *testing.T) { - if testing.Short() { - t.Skip("Requires filesystem access") - } - - dir := t.TempDir() - - st := store.New(dir, testStoreCfg) - err := st.CreateOrUpdate(store.Metadata{ - Name: "test", - Metadata: command.DockerContext{ - Description: t.Name(), - AdditionalFields: map[string]any{ - "otel": map[string]any{ - "OTEL_EXPORTER_OTLP_ENDPOINT": "localhost:1234", - }, - }, - }, - Endpoints: make(map[string]any), - }) - require.NoError(t, err) - - cfg, err := tracing.ConfigFromDockerContext(st, "test") - require.NoError(t, err) - require.Equal(t, "localhost:1234", cfg.Endpoint) -} diff --git a/internal/tracing/wrap.go b/internal/tracing/wrap.go deleted file mode 100644 index e525b738b42..00000000000 --- a/internal/tracing/wrap.go +++ /dev/null @@ -1,98 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "context" - - "github.com/acarl005/stripansi" - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/codes" - semconv "go.opentelemetry.io/otel/semconv/v1.19.0" - "go.opentelemetry.io/otel/trace" -) - -// SpanWrapFunc wraps a function that takes a context with a trace.Span, marking the status as codes.Error if the -// wrapped function returns an error. -// -// The context passed to the function is created from the span to ensure correct propagation. -// -// NOTE: This function is nearly identical to SpanWrapFuncForErrGroup, except the latter is designed specially for -// convenience with errgroup.Group due to its prevalence throughout the codebase. The code is duplicated to avoid -// adding even more levels of function wrapping/indirection. -func SpanWrapFunc(spanName string, opts SpanOptions, fn func(ctx context.Context) error) func(context.Context) error { - return func(ctx context.Context) error { - ctx, span := otel.Tracer("").Start(ctx, spanName, opts.SpanStartOptions()...) - defer span.End() - - if err := fn(ctx); err != nil { - span.SetStatus(codes.Error, err.Error()) - return err - } - - span.SetStatus(codes.Ok, "") - return nil - } -} - -// SpanWrapFuncForErrGroup wraps a function that takes a context with a trace.Span, marking the status as codes.Error -// if the wrapped function returns an error. -// -// The context passed to the function is created from the span to ensure correct propagation. -// -// NOTE: This function is nearly identical to SpanWrapFunc, except this function is designed specially for -// convenience with errgroup.Group due to its prevalence throughout the codebase. The code is duplicated to avoid -// adding even more levels of function wrapping/indirection. -func SpanWrapFuncForErrGroup(ctx context.Context, spanName string, opts SpanOptions, fn func(ctx context.Context) error) func() error { - return func() error { - ctx, span := otel.Tracer("").Start(ctx, spanName, opts.SpanStartOptions()...) - defer span.End() - - if err := fn(ctx); err != nil { - span.SetStatus(codes.Error, err.Error()) - return err - } - - span.SetStatus(codes.Ok, "") - return nil - } -} - -// EventWrapFuncForErrGroup invokes a function and records an event, optionally including the returned -// error as the "exception message" on the event. -// -// This is intended for lightweight usage to wrap errgroup.Group calls where a full span is not desired. -func EventWrapFuncForErrGroup(ctx context.Context, eventName string, opts SpanOptions, fn func(ctx context.Context) error) func() error { - return func() error { - span := trace.SpanFromContext(ctx) - eventOpts := opts.EventOptions() - - err := fn(ctx) - if err != nil { - eventOpts = append(eventOpts, trace.WithAttributes(semconv.ExceptionMessage(stripansi.Strip(err.Error())))) - } - span.AddEvent(eventName, eventOpts...) - - return err - } -} - -func AddAttributeToSpan(ctx context.Context, attr ...attribute.KeyValue) { - span := trace.SpanFromContext(ctx) - span.SetAttributes(attr...) -} diff --git a/internal/variables.go b/internal/variables.go deleted file mode 100644 index 28b701cc2db..00000000000 --- a/internal/variables.go +++ /dev/null @@ -1,20 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package internal - -// Version is the version of the CLI injected in compilation time -var Version = "dev" diff --git a/pkg/api/api.go b/pkg/api/api.go deleted file mode 100644 index aed77af1523..00000000000 --- a/pkg/api/api.go +++ /dev/null @@ -1,759 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "context" - "fmt" - "io" - "slices" - "strings" - "time" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/platforms" - "github.com/docker/cli/opts" - "github.com/docker/docker/api/types/volume" -) - -// LoadListener receives events during project loading. -// Events include: -// - "extends": when a service extends another (metadata: service info) -// - "include": when including external compose files (metadata: {"path": StringList}) -// -// Multiple listeners can be registered, and all will be notified of events. -type LoadListener func(event string, metadata map[string]any) - -// ProjectLoadOptions configures how a Compose project should be loaded -type ProjectLoadOptions struct { - // ProjectName to use, or empty to infer from directory - ProjectName string - // ConfigPaths are paths to compose files - ConfigPaths []string - // WorkingDir is the project directory - WorkingDir string - // EnvFiles are paths to .env files - EnvFiles []string - // Profiles to activate - Profiles []string - // Services to select (empty = all) - Services []string - // Offline mode disables remote resource loading - Offline bool - // All includes all resources (not just those used by services) - All bool - // Compatibility enables v1 compatibility mode - Compatibility bool - - // ProjectOptionsFns are compose-go project options to apply. - // Use cli.WithInterpolation(false), cli.WithNormalization(false), etc. - // This is optional - pass nil or empty slice to use defaults. - ProjectOptionsFns []cli.ProjectOptionsFn - - // LoadListeners receive events during project loading. - // All registered listeners will be notified of events. - // This is optional - pass nil or empty slice if not needed. - LoadListeners []LoadListener - - OCI OCIOptions -} - -type OCIOptions struct { - InsecureRegistries []string -} - -// Compose is the API interface one can use to programmatically use docker/compose in a third-party software -// Use [compose.NewComposeService] to get an actual instance -type Compose interface { - // Build executes the equivalent to a `compose build` - Build(ctx context.Context, project *types.Project, options BuildOptions) error - // Push executes the equivalent to a `compose push` - Push(ctx context.Context, project *types.Project, options PushOptions) error - // Pull executes the equivalent of a `compose pull` - Pull(ctx context.Context, project *types.Project, options PullOptions) error - // Create executes the equivalent to a `compose create` - Create(ctx context.Context, project *types.Project, options CreateOptions) error - // Start executes the equivalent to a `compose start` - Start(ctx context.Context, projectName string, options StartOptions) error - // Restart restarts containers - Restart(ctx context.Context, projectName string, options RestartOptions) error - // Stop executes the equivalent to a `compose stop` - Stop(ctx context.Context, projectName string, options StopOptions) error - // Up executes the equivalent to a `compose up` - Up(ctx context.Context, project *types.Project, options UpOptions) error - // Down executes the equivalent to a `compose down` - Down(ctx context.Context, projectName string, options DownOptions) error - // Logs executes the equivalent to a `compose logs` - Logs(ctx context.Context, projectName string, consumer LogConsumer, options LogOptions) error - // Ps executes the equivalent to a `compose ps` - Ps(ctx context.Context, projectName string, options PsOptions) ([]ContainerSummary, error) - // List executes the equivalent to a `docker stack ls` - List(ctx context.Context, options ListOptions) ([]Stack, error) - // Kill executes the equivalent to a `compose kill` - Kill(ctx context.Context, projectName string, options KillOptions) error - // RunOneOffContainer creates a service oneoff container and starts its dependencies - RunOneOffContainer(ctx context.Context, project *types.Project, opts RunOptions) (int, error) - // Remove executes the equivalent to a `compose rm` - Remove(ctx context.Context, projectName string, options RemoveOptions) error - // Exec executes a command in a running service container - Exec(ctx context.Context, projectName string, options RunOptions) (int, error) - // Attach STDIN,STDOUT,STDERR to a running service container - Attach(ctx context.Context, projectName string, options AttachOptions) error - // Copy copies a file/folder between a service container and the local filesystem - Copy(ctx context.Context, projectName string, options CopyOptions) error - // Pause executes the equivalent to a `compose pause` - Pause(ctx context.Context, projectName string, options PauseOptions) error - // UnPause executes the equivalent to a `compose unpause` - UnPause(ctx context.Context, projectName string, options PauseOptions) error - // Top executes the equivalent to a `compose top` - Top(ctx context.Context, projectName string, services []string) ([]ContainerProcSummary, error) - // Events executes the equivalent to a `compose events` - Events(ctx context.Context, projectName string, options EventsOptions) error - // Port executes the equivalent to a `compose port` - Port(ctx context.Context, projectName string, service string, port uint16, options PortOptions) (string, int, error) - // Publish executes the equivalent to a `compose publish` - Publish(ctx context.Context, project *types.Project, repository string, options PublishOptions) error - // Images executes the equivalent of a `compose images` - Images(ctx context.Context, projectName string, options ImagesOptions) (map[string]ImageSummary, error) - // Watch services' development context and sync/notify/rebuild/restart on changes - Watch(ctx context.Context, project *types.Project, options WatchOptions) error - // Viz generates a graphviz graph of the project services - Viz(ctx context.Context, project *types.Project, options VizOptions) (string, error) - // Wait blocks until at least one of the services' container exits - Wait(ctx context.Context, projectName string, options WaitOptions) (int64, error) - // Scale manages numbers of container instances running per service - Scale(ctx context.Context, project *types.Project, options ScaleOptions) error - // Export a service container's filesystem as a tar archive - Export(ctx context.Context, projectName string, options ExportOptions) error - // Create a new image from a service container's changes - Commit(ctx context.Context, projectName string, options CommitOptions) error - // Generate generates a Compose Project from existing containers - Generate(ctx context.Context, options GenerateOptions) (*types.Project, error) - // Volumes executes the equivalent to a `docker volume ls` - Volumes(ctx context.Context, project string, options VolumesOptions) ([]VolumesSummary, error) - // LoadProject loads and validates a Compose project from configuration files. - LoadProject(ctx context.Context, options ProjectLoadOptions) (*types.Project, error) -} - -type VolumesOptions struct { - Services []string -} - -type VolumesSummary = *volume.Volume - -type ScaleOptions struct { - Services []string -} - -type WaitOptions struct { - // Services passed in the command line to be waited - Services []string - // Executes a down when a container exits - DownProjectOnContainerExit bool -} - -type VizOptions struct { - // IncludeNetworks if true, network names a container is attached to should appear in the graph node - IncludeNetworks bool - // IncludePorts if true, ports a container exposes should appear in the graph node - IncludePorts bool - // IncludeImageName if true, name of the image used to create a container should appear in the graph node - IncludeImageName bool - // Indentation string to be used to indent graphviz code, e.g. "\t", " " - Indentation string -} - -// WatchLogger is a reserved name to log watch events -const WatchLogger = "#watch" - -// WatchOptions group options of the Watch API -type WatchOptions struct { - Build *BuildOptions - LogTo LogConsumer - Prune bool - Services []string -} - -// BuildOptions group options of the Build API -type BuildOptions struct { - // Pull always attempt to pull a newer version of the image - Pull bool - // Push pushes service images - Push bool - // Progress set type of progress output ("auto", "plain", "tty") - Progress string - // Args set build-time args - Args types.MappingWithEquals - // NoCache disables cache use - NoCache bool - // Quiet make the build process not output to the console - Quiet bool - // Services passed in the command line to be built - Services []string - // Deps also build selected services dependencies - Deps bool - // Ssh authentications passed in the command line - SSHs []types.SSHKey - // Memory limit for the build container - Memory int64 - // Builder name passed in the command line - Builder string - // Print don't actually run builder but print equivalent build config - Print bool - // Check let builder validate build configuration - Check bool - // Attestations allows to enable attestations generation - Attestations bool - // Provenance generate a provenance attestation - Provenance string - // SBOM generate a SBOM attestation - SBOM string - // Out is the stream to write build progress - Out io.Writer -} - -// Apply mutates project according to build options -func (o BuildOptions) Apply(project *types.Project) error { - platform := project.Environment["DOCKER_DEFAULT_PLATFORM"] - for name, service := range project.Services { - if service.Provider == nil && service.Image == "" && service.Build == nil { - return fmt.Errorf("invalid service %q. Must specify either image or build", name) - } - - if service.Build == nil { - continue - } - if platform != "" { - if len(service.Build.Platforms) > 0 && !slices.Contains(service.Build.Platforms, platform) { - return fmt.Errorf("service %q build.platforms does not support value set by DOCKER_DEFAULT_PLATFORM: %s", name, platform) - } - service.Platform = platform - } - if service.Platform != "" { - if len(service.Build.Platforms) > 0 && !slices.Contains(service.Build.Platforms, service.Platform) { - return fmt.Errorf("service %q build configuration does not support platform: %s", name, service.Platform) - } - } - - service.Build.Pull = service.Build.Pull || o.Pull - service.Build.NoCache = service.Build.NoCache || o.NoCache - - project.Services[name] = service - } - return nil -} - -// CreateOptions group options of the Create API -type CreateOptions struct { - Build *BuildOptions - // Services defines the services user interacts with - Services []string - // Remove legacy containers for services that are not defined in the project - RemoveOrphans bool - // Ignore legacy containers for services that are not defined in the project - IgnoreOrphans bool - // Recreate define the strategy to apply on existing containers - Recreate string - // RecreateDependencies define the strategy to apply on dependencies services - RecreateDependencies string - // Inherit reuse anonymous volumes from previous container - Inherit bool - // Timeout set delay to wait for container to gracefully stop before sending SIGKILL - Timeout *time.Duration - // QuietPull makes the pulling process quiet - QuietPull bool -} - -// StartOptions group options of the Start API -type StartOptions struct { - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project - // Attach to container and forward logs if not nil - Attach LogConsumer - // AttachTo set the services to attach to - AttachTo []string - // OnExit defines behavior when a container stops - OnExit Cascade - // ExitCodeFrom return exit code from specified service - ExitCodeFrom string - // Wait won't return until containers reached the running|healthy state - Wait bool - WaitTimeout time.Duration - // Services passed in the command line to be started - Services []string - Watch bool - NavigationMenu bool -} - -type Cascade int - -const ( - CascadeIgnore Cascade = iota - CascadeStop Cascade = iota - CascadeFail Cascade = iota -) - -// RestartOptions group options of the Restart API -type RestartOptions struct { - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project - // Timeout override container restart timeout - Timeout *time.Duration - // Services passed in the command line to be restarted - Services []string - // NoDeps ignores services dependencies - NoDeps bool -} - -// StopOptions group options of the Stop API -type StopOptions struct { - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project - // Timeout override container stop timeout - Timeout *time.Duration - // Services passed in the command line to be stopped - Services []string -} - -// UpOptions group options of the Up API -type UpOptions struct { - Create CreateOptions - Start StartOptions -} - -// DownOptions group options of the Down API -type DownOptions struct { - // RemoveOrphans will cleanup containers that are not declared on the compose model but own the same labels - RemoveOrphans bool - // Project is the compose project used to define this app. Might be nil if user ran `down` just with project name - Project *types.Project - // Timeout override container stop timeout - Timeout *time.Duration - // Images remove image used by services. 'all': Remove all images. 'local': Remove only images that don't have a tag - Images string - // Volumes remove volumes, both declared in the `volumes` section and anonymous ones - Volumes bool - // Services passed in the command line to be stopped - Services []string -} - -// ConfigOptions group options of the Config API -type ConfigOptions struct { - // Format define the output format used to dump converted application model (json|yaml) - Format string - // Output defines the path to save the application model - Output string - // Resolve image reference to digests - ResolveImageDigests bool -} - -// PushOptions group options of the Push API -type PushOptions struct { - Quiet bool - IgnoreFailures bool - ImageMandatory bool -} - -// PullOptions group options of the Pull API -type PullOptions struct { - Quiet bool - IgnoreFailures bool - IgnoreBuildable bool -} - -// ImagesOptions group options of the Images API -type ImagesOptions struct { - Services []string -} - -// KillOptions group options of the Kill API -type KillOptions struct { - // RemoveOrphans will cleanup containers that are not declared on the compose model but own the same labels - RemoveOrphans bool - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project - // Services passed in the command line to be killed - Services []string - // Signal to send to containers - Signal string - // All can be set to true to try to kill all found containers, independently of their state - All bool -} - -// RemoveOptions group options of the Remove API -type RemoveOptions struct { - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project - // Stop option passed in the command line - Stop bool - // Volumes remove anonymous volumes - Volumes bool - // Force don't ask to confirm removal - Force bool - // Services passed in the command line to be removed - Services []string -} - -// RunOptions group options of the Run API -type RunOptions struct { - CreateOptions - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project - Name string - Service string - Command []string - Entrypoint []string - Detach bool - AutoRemove bool - Tty bool - Interactive bool - WorkingDir string - User string - Environment []string - CapAdd []string - CapDrop []string - Labels types.Labels - Privileged bool - UseNetworkAliases bool - NoDeps bool - // used by exec - Index int -} - -// AttachOptions group options of the Attach API -type AttachOptions struct { - Project *types.Project - Service string - Index int - DetachKeys string - NoStdin bool - Proxy bool -} - -// EventsOptions group options of the Events API -type EventsOptions struct { - Services []string - Consumer func(event Event) error - Since string - Until string -} - -// Event is a container runtime event served by Events API -type Event struct { - Timestamp time.Time - Service string - Container string - Status string - Attributes map[string]string -} - -// PortOptions group options of the Port API -type PortOptions struct { - Protocol string - Index int -} - -// OCIVersion controls manifest generation to ensure compatibility -// with different registries. -// -// Currently, this is not exposed as an option to the user – Compose uses -// OCI 1.0 mode automatically for ECR registries based on domain and OCI 1.1 -// for all other registries. -// -// There are likely other popular registries that do not support the OCI 1.1 -// format, so it might make sense to expose this as a CLI flag or see if -// there's a way to generically probe the registry for support level. -type OCIVersion string - -const ( - OCIVersion1_0 OCIVersion = "1.0" - OCIVersion1_1 OCIVersion = "1.1" -) - -// PublishOptions group options of the Publish API -type PublishOptions struct { - ResolveImageDigests bool - Application bool - WithEnvironment bool - OCIVersion OCIVersion - // Use plain HTTP to access registry. Should only be used for testing purpose - InsecureRegistry bool -} - -func (e Event) String() string { - t := e.Timestamp.Format("2006-01-02 15:04:05.000000") - var attr []string - for k, v := range e.Attributes { - attr = append(attr, fmt.Sprintf("%s=%s", k, v)) - } - return fmt.Sprintf("%s container %s %s (%s)\n", t, e.Status, e.Container, strings.Join(attr, ", ")) -} - -// ListOptions group options of the ls API -type ListOptions struct { - All bool -} - -// PsOptions group options of the Ps API -type PsOptions struct { - Project *types.Project - All bool - Services []string -} - -// CopyOptions group options of the cp API -type CopyOptions struct { - Source string - Destination string - All bool - Index int - FollowLink bool - CopyUIDGID bool -} - -// PortPublisher hold status about published port -type PortPublisher struct { - URL string - TargetPort int - PublishedPort int - Protocol string -} - -// ContainerSummary hold high-level description of a container -type ContainerSummary struct { - ID string - Name string - Names []string - Image string - Command string - Project string - Service string - Created int64 - State string - Status string - Health string - ExitCode int - Publishers PortPublishers - Labels map[string]string - SizeRw int64 `json:",omitempty"` - SizeRootFs int64 `json:",omitempty"` - Mounts []string - Networks []string - LocalVolumes int -} - -// PortPublishers is a slice of PortPublisher -type PortPublishers []PortPublisher - -// Len implements sort.Interface -func (p PortPublishers) Len() int { - return len(p) -} - -// Less implements sort.Interface -func (p PortPublishers) Less(i, j int) bool { - left := p[i] - right := p[j] - if left.URL != right.URL { - return left.URL < right.URL - } - if left.TargetPort != right.TargetPort { - return left.TargetPort < right.TargetPort - } - if left.PublishedPort != right.PublishedPort { - return left.PublishedPort < right.PublishedPort - } - return left.Protocol < right.Protocol -} - -// Swap implements sort.Interface -func (p PortPublishers) Swap(i, j int) { - p[i], p[j] = p[j], p[i] -} - -// ContainerProcSummary holds container processes top data -type ContainerProcSummary struct { - ID string - Name string - Processes [][]string - Titles []string - Service string - Replica string -} - -// ImageSummary holds container image description -type ImageSummary struct { - ID string - Repository string - Tag string - Platform platforms.Platform - Size int64 - Created *time.Time - LastTagTime time.Time -} - -// ServiceStatus hold status about a service -type ServiceStatus struct { - ID string - Name string - Replicas int - Desired int - Ports []string - Publishers []PortPublisher -} - -// LogOptions defines optional parameters for the `Log` API -type LogOptions struct { - Project *types.Project - Index int - Services []string - Tail string - Since string - Until string - Follow bool - Timestamps bool -} - -// PauseOptions group options of the Pause API -type PauseOptions struct { - // Services passed in the command line to be started - Services []string - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project -} - -// ExportOptions group options of the Export API -type ExportOptions struct { - Service string - Index int - Output string -} - -// CommitOptions group options of the Commit API -type CommitOptions struct { - Service string - Reference string - - Pause bool - Comment string - Author string - Changes opts.ListOpts - - Index int -} - -type GenerateOptions struct { - // ProjectName to set in the Compose file - ProjectName string - // Containers passed in the command line to be used as reference for service definition - Containers []string -} - -const ( - // STARTING indicates that stack is being deployed - STARTING string = "Starting" - // RUNNING indicates that stack is deployed and services are running - RUNNING string = "Running" - // UPDATING indicates that some stack resources are being recreated - UPDATING string = "Updating" - // REMOVING indicates that stack is being deleted - REMOVING string = "Removing" - // UNKNOWN indicates unknown stack state - UNKNOWN string = "Unknown" - // FAILED indicates that stack deployment failed - FAILED string = "Failed" -) - -const ( - // RecreateDiverged to recreate services which configuration diverges from compose model - RecreateDiverged = "diverged" - // RecreateForce to force service container being recreated - RecreateForce = "force" - // RecreateNever to never recreate existing service containers - RecreateNever = "never" -) - -// Stack holds the name and state of a compose application/stack -type Stack struct { - ID string - Name string - Status string - ConfigFiles string - Reason string -} - -// LogConsumer is a callback to process log messages from services -type LogConsumer interface { - Log(containerName, message string) - Err(containerName, message string) - Status(container, msg string) -} - -// ContainerEventListener is a callback to process ContainerEvent from services -type ContainerEventListener func(event ContainerEvent) - -// ContainerEvent notify an event has been collected on source container implementing Service -type ContainerEvent struct { - Type int - Time int64 - Container *ContainerSummary - // Source is the name of the container _without the project prefix_. - // - // This is only suitable for display purposes within Compose, as it's - // not guaranteed to be unique across services. - Source string - ID string - Service string - Line string - // ExitCode is only set on ContainerEventExited events - ExitCode int - Restarting bool -} - -const ( - // ContainerEventLog is a ContainerEvent of type log on stdout. Line is set - ContainerEventLog = iota - // ContainerEventErr is a ContainerEvent of type log on stderr. Line is set - ContainerEventErr - // ContainerEventStarted let consumer know a container has been started - ContainerEventStarted - // ContainerEventRestarted let consumer know a container has been restarted - ContainerEventRestarted - // ContainerEventStopped is a ContainerEvent of type stopped. - ContainerEventStopped - // ContainerEventCreated let consumer know a new container has been created - ContainerEventCreated - // ContainerEventRecreated let consumer know container stopped but his being replaced - ContainerEventRecreated - // ContainerEventExited is a ContainerEvent of type exit. ExitCode is set - ContainerEventExited - // UserCancel user canceled compose up, we are stopping containers - HookEventLog -) - -// Separator is used for naming components -var Separator = "-" - -// GetImageNameOrDefault computes the default image name for a service, used to tag built images -func GetImageNameOrDefault(service types.ServiceConfig, projectName string) string { - imageName := service.Image - if imageName == "" { - imageName = projectName + Separator + service.Name - } - return imageName -} diff --git a/pkg/api/api_test.go b/pkg/api/api_test.go deleted file mode 100644 index fc44abe7f1a..00000000000 --- a/pkg/api/api_test.go +++ /dev/null @@ -1,38 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "gotest.tools/v3/assert" -) - -func TestRunOptionsEnvironmentMap(t *testing.T) { - opts := RunOptions{ - Environment: []string{ - "FOO=BAR", - "ZOT=", - "QIX", - }, - } - env := types.NewMappingWithEquals(opts.Environment) - assert.Equal(t, *env["FOO"], "BAR") - assert.Equal(t, *env["ZOT"], "") - assert.Check(t, env["QIX"] == nil) -} diff --git a/pkg/api/context.go b/pkg/api/context.go deleted file mode 100644 index af49c5d2433..00000000000 --- a/pkg/api/context.go +++ /dev/null @@ -1,32 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -// ContextInfo provides Docker context information for advanced scenarios -type ContextInfo interface { - // CurrentContext returns the name of the current Docker context - // Returns "default" for simple clients without context support - CurrentContext() string - - // ServerOSType returns the Docker daemon's operating system (linux/windows/darwin) - // Used for OS-specific compatibility checks - ServerOSType() string - - // BuildKitEnabled determines whether BuildKit should be used for builds - // Checks DOCKER_BUILDKIT env var, config, and daemon capabilities - BuildKitEnabled() (bool, error) -} diff --git a/pkg/api/env.go b/pkg/api/env.go deleted file mode 100644 index 46319342417..00000000000 --- a/pkg/api/env.go +++ /dev/null @@ -1,20 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -// ComposeCompatibility try to mimic compose v1 as much as possible -const ComposeCompatibility = "COMPOSE_COMPATIBILITY" diff --git a/pkg/api/errors.go b/pkg/api/errors.go deleted file mode 100644 index 7bf4d4a02c3..00000000000 --- a/pkg/api/errors.go +++ /dev/null @@ -1,88 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "errors" -) - -const ( - // ExitCodeLoginRequired exit code when command cannot execute because it requires cloud login - // This will be used by VSCode to detect when creating context if the user needs to login first - ExitCodeLoginRequired = 5 -) - -var ( - // ErrNotFound is returned when an object is not found - ErrNotFound = errors.New("not found") - // ErrAlreadyExists is returned when an object already exists - ErrAlreadyExists = errors.New("already exists") - // ErrForbidden is returned when an operation is not permitted - ErrForbidden = errors.New("forbidden") - // ErrUnknown is returned when the error type is unmapped - ErrUnknown = errors.New("unknown") - // ErrNotImplemented is returned when a backend doesn't implement an action - ErrNotImplemented = errors.New("not implemented") - // ErrUnsupportedFlag is returned when a backend doesn't support a flag - ErrUnsupportedFlag = errors.New("unsupported flag") - // ErrCanceled is returned when the command was canceled by user - ErrCanceled = errors.New("canceled") - // ErrParsingFailed is returned when a string cannot be parsed - ErrParsingFailed = errors.New("parsing failed") - // ErrNoResources is returned when operation didn't selected any resource - ErrNoResources = errors.New("no resources") -) - -// IsNotFoundError returns true if the unwrapped error is ErrNotFound -func IsNotFoundError(err error) bool { - return errors.Is(err, ErrNotFound) -} - -// IsAlreadyExistsError returns true if the unwrapped error is ErrAlreadyExists -func IsAlreadyExistsError(err error) bool { - return errors.Is(err, ErrAlreadyExists) -} - -// IsForbiddenError returns true if the unwrapped error is ErrForbidden -func IsForbiddenError(err error) bool { - return errors.Is(err, ErrForbidden) -} - -// IsUnknownError returns true if the unwrapped error is ErrUnknown -func IsUnknownError(err error) bool { - return errors.Is(err, ErrUnknown) -} - -// IsErrUnsupportedFlag returns true if the unwrapped error is ErrUnsupportedFlag -func IsErrUnsupportedFlag(err error) bool { - return errors.Is(err, ErrUnsupportedFlag) -} - -// IsErrNotImplemented returns true if the unwrapped error is ErrNotImplemented -func IsErrNotImplemented(err error) bool { - return errors.Is(err, ErrNotImplemented) -} - -// IsErrParsingFailed returns true if the unwrapped error is ErrParsingFailed -func IsErrParsingFailed(err error) bool { - return errors.Is(err, ErrParsingFailed) -} - -// IsErrCanceled returns true if the unwrapped error is ErrCanceled -func IsErrCanceled(err error) bool { - return errors.Is(err, ErrCanceled) -} diff --git a/pkg/api/errors_test.go b/pkg/api/errors_test.go deleted file mode 100644 index d221db3fdec..00000000000 --- a/pkg/api/errors_test.go +++ /dev/null @@ -1,53 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "errors" - "fmt" - "testing" - - "gotest.tools/v3/assert" -) - -func TestIsNotFound(t *testing.T) { - err := fmt.Errorf(`object "name": %w`, ErrNotFound) - assert.Assert(t, IsNotFoundError(err)) - - assert.Assert(t, !IsNotFoundError(errors.New("another error"))) -} - -func TestIsAlreadyExists(t *testing.T) { - err := fmt.Errorf(`object "name": %w`, ErrAlreadyExists) - assert.Assert(t, IsAlreadyExistsError(err)) - - assert.Assert(t, !IsAlreadyExistsError(errors.New("another error"))) -} - -func TestIsForbidden(t *testing.T) { - err := fmt.Errorf(`object "name": %w`, ErrForbidden) - assert.Assert(t, IsForbiddenError(err)) - - assert.Assert(t, !IsForbiddenError(errors.New("another error"))) -} - -func TestIsUnknown(t *testing.T) { - err := fmt.Errorf(`object "name": %w`, ErrUnknown) - assert.Assert(t, IsUnknownError(err)) - - assert.Assert(t, !IsUnknownError(errors.New("another error"))) -} diff --git a/pkg/api/event.go b/pkg/api/event.go deleted file mode 100644 index 7ccb1138210..00000000000 --- a/pkg/api/event.go +++ /dev/null @@ -1,107 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "context" -) - -// EventStatus indicates the status of an action -type EventStatus int - -const ( - // Working means that the current task is working - Working EventStatus = iota - // Done means that the current task is done - Done - // Warning means that the current task has warning - Warning - // Error means that the current task has errored - Error -) - -// ResourceCompose is a special resource ID used when event applies to all resources in the application -const ResourceCompose = "Compose" - -const ( - StatusError = "Error" - StatusCreating = "Creating" - StatusStarting = "Starting" - StatusStarted = "Started" - StatusWaiting = "Waiting" - StatusHealthy = "Healthy" - StatusExited = "Exited" - StatusRestarting = "Restarting" - StatusRestarted = "Restarted" - StatusRunning = "Running" - StatusCreated = "Created" - StatusStopping = "Stopping" - StatusStopped = "Stopped" - StatusKilling = "Killing" - StatusKilled = "Killed" - StatusRemoving = "Removing" - StatusRemoved = "Removed" - StatusBuilding = "Building" - StatusBuilt = "Built" - StatusPulling = "Pulling" - StatusPulled = "Pulled" - StatusCommitting = "Committing" - StatusCommitted = "Committed" - StatusCopying = "Copying" - StatusCopied = "Copied" - StatusExporting = "Exporting" - StatusExported = "Exported" - StatusDownloading = "Downloading" - StatusDownloadComplete = "Download complete" - StatusConfiguring = "Configuring" - StatusConfigured = "Configured" -) - -// Resource represents status change and progress for a compose resource. -type Resource struct { - ID string - ParentID string - Text string - Details string - Status EventStatus - Current int64 - Percent int - Total int64 -} - -func (e *Resource) StatusText() string { - switch e.Status { - case Working: - return "Working" - case Warning: - return "Warning" - case Done: - return "Done" - default: - return "Error" - } -} - -// EventProcessor is notified about Compose operations and tasks -type EventProcessor interface { - // Start is triggered as a Compose operation is starting with context - Start(ctx context.Context, operation string) - // On notify about (sub)task and progress processing operation - On(events ...Resource) - // Done is triggered as a Compose operation completed - Done(operation string, success bool) -} diff --git a/pkg/api/labels.go b/pkg/api/labels.go deleted file mode 100644 index 3a0f684b98b..00000000000 --- a/pkg/api/labels.go +++ /dev/null @@ -1,68 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "github.com/hashicorp/go-version" - - "github.com/docker/compose/v5/internal" -) - -const ( - // ProjectLabel allow to track resource related to a compose project - ProjectLabel = "com.docker.compose.project" - // ServiceLabel allow to track resource related to a compose service - ServiceLabel = "com.docker.compose.service" - // ConfigHashLabel stores configuration hash for a compose service - ConfigHashLabel = "com.docker.compose.config-hash" - // ContainerNumberLabel stores the container index of a replicated service - ContainerNumberLabel = "com.docker.compose.container-number" - // VolumeLabel allow to track resource related to a compose volume - VolumeLabel = "com.docker.compose.volume" - // NetworkLabel allow to track resource related to a compose network - NetworkLabel = "com.docker.compose.network" - // WorkingDirLabel stores absolute path to compose project working directory - WorkingDirLabel = "com.docker.compose.project.working_dir" - // ConfigFilesLabel stores absolute path to compose project configuration files - ConfigFilesLabel = "com.docker.compose.project.config_files" - // EnvironmentFileLabel stores absolute path to compose project env file set by `--env-file` - EnvironmentFileLabel = "com.docker.compose.project.environment_file" - // OneoffLabel stores value 'True' for one-off containers created by `compose run` - OneoffLabel = "com.docker.compose.oneoff" - // SlugLabel stores unique slug used for one-off container identity - SlugLabel = "com.docker.compose.slug" - // ImageDigestLabel stores digest of the container image used to run service - ImageDigestLabel = "com.docker.compose.image" - // DependenciesLabel stores service dependencies - DependenciesLabel = "com.docker.compose.depends_on" - // VersionLabel stores the compose tool version used to build/run application - VersionLabel = "com.docker.compose.version" - // ImageBuilderLabel stores the builder (classic or BuildKit) used to produce the image. - ImageBuilderLabel = "com.docker.compose.image.builder" - // ContainerReplaceLabel is set when container is created to replace another container (recreated) - ContainerReplaceLabel = "com.docker.compose.replace" -) - -// ComposeVersion is the compose tool version as declared by label VersionLabel -var ComposeVersion string - -func init() { - v, err := version.NewVersion(internal.Version) - if err == nil { - ComposeVersion = v.Core().String() - } -} diff --git a/pkg/api/labels_test.go b/pkg/api/labels_test.go deleted file mode 100644 index c4af2003e9f..00000000000 --- a/pkg/api/labels_test.go +++ /dev/null @@ -1,36 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "testing" - - "github.com/hashicorp/go-version" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/internal" -) - -func TestComposeVersionInitialization(t *testing.T) { - v, err := version.NewVersion(internal.Version) - if err != nil { - assert.Equal(t, "", ComposeVersion, "ComposeVersion should be empty for a non-semver internal version (e.g. 'devel')") - } else { - expected := v.Core().String() - assert.Equal(t, expected, ComposeVersion, "ComposeVersion should be the core of internal.Version") - } -} diff --git a/pkg/bridge/convert.go b/pkg/bridge/convert.go deleted file mode 100644 index cceb99fdd4a..00000000000 --- a/pkg/bridge/convert.go +++ /dev/null @@ -1,236 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package bridge - -import ( - "context" - "fmt" - "io" - "os" - "os/user" - "path/filepath" - "runtime" - "strconv" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/docker/cli/cli/command" - cli "github.com/docker/cli/cli/command/container" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/pkg/jsonmessage" - "github.com/docker/go-connections/nat" - "github.com/sirupsen/logrus" - "go.yaml.in/yaml/v4" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -type ConvertOptions struct { - Output string - Templates string - Transformations []string -} - -func Convert(ctx context.Context, dockerCli command.Cli, project *types.Project, opts ConvertOptions) error { - if len(opts.Transformations) == 0 { - opts.Transformations = []string{DefaultTransformerImage} - } - // Load image references, secrets and configs, also expose ports - project, err := LoadAdditionalResources(ctx, dockerCli, project) - if err != nil { - return err - } - // for user to rely on compose.yaml attribute names, not go struct ones, we marshall back into YAML - raw, err := project.MarshalYAML(types.WithSecretContent) - // Marshall to YAML - if err != nil { - return fmt.Errorf("cannot render project into yaml: %w", err) - } - var model map[string]any - err = yaml.Unmarshal(raw, &model) - if err != nil { - return fmt.Errorf("cannot render project into yaml: %w", err) - } - - if opts.Output != "" { - _ = os.RemoveAll(opts.Output) - err := os.MkdirAll(opts.Output, 0o744) - if err != nil && !os.IsExist(err) { - return fmt.Errorf("cannot create output folder: %w", err) - } - } - // Run Transformers images - return convert(ctx, dockerCli, model, opts) -} - -func convert(ctx context.Context, dockerCli command.Cli, model map[string]any, opts ConvertOptions) error { - raw, err := yaml.Marshal(model) - if err != nil { - return err - } - - dir, err := os.MkdirTemp("", "compose-convert-*") - if err != nil { - return err - } - defer func() { - err := os.RemoveAll(dir) - if err != nil { - logrus.Warnf("failed to remove temp dir %s: %v", dir, err) - } - }() - - composeYaml := filepath.Join(dir, "compose.yaml") - err = os.WriteFile(composeYaml, raw, 0o600) - if err != nil { - return err - } - - out, err := filepath.Abs(opts.Output) - if err != nil { - return err - } - binds := []string{ - fmt.Sprintf("%s:%s", dir, "/in"), - fmt.Sprintf("%s:%s", out, "/out"), - } - if opts.Templates != "" { - templateDir, err := filepath.Abs(opts.Templates) - if err != nil { - return err - } - binds = append(binds, fmt.Sprintf("%s:%s", templateDir, "/templates")) - } - - for _, transformation := range opts.Transformations { - _, err = inspectWithPull(ctx, dockerCli, transformation) - if err != nil { - return err - } - - containerConfig := &container.Config{ - Image: transformation, - Env: []string{"LICENSE_AGREEMENT=true"}, - } - // On POSIX systems, this is a decimal number representing the uid. - // On Windows, this is a security identifier (SID) in a string format and the engine isn't able to manage it - if runtime.GOOS != "windows" { - usr, err := user.Current() - if err != nil { - return err - } - containerConfig.User = usr.Uid - } - created, err := dockerCli.Client().ContainerCreate(ctx, containerConfig, &container.HostConfig{ - AutoRemove: true, - Binds: binds, - }, &network.NetworkingConfig{}, nil, "") - if err != nil { - return err - } - - err = cli.RunStart(ctx, dockerCli, &cli.StartOptions{ - Attach: true, - Containers: []string{created.ID}, - }) - if err != nil { - return err - } - } - return nil -} - -// LoadAdditionalResources loads additional resources from the project, such as image references, secrets, configs and exposed ports -func LoadAdditionalResources(ctx context.Context, dockerCLI command.Cli, project *types.Project) (*types.Project, error) { - for name, service := range project.Services { - imageName := api.GetImageNameOrDefault(service, project.Name) - - inspect, err := inspectWithPull(ctx, dockerCLI, imageName) - if err != nil { - return nil, err - } - service.Image = imageName - exposed := utils.Set[string]{} - exposed.AddAll(service.Expose...) - for port := range inspect.Config.ExposedPorts { - exposed.Add(nat.Port(port).Port()) - } - for _, port := range service.Ports { - exposed.Add(strconv.Itoa(int(port.Target))) - } - service.Expose = exposed.Elements() - project.Services[name] = service - } - - for name, secret := range project.Secrets { - f, err := loadFileObject(types.FileObjectConfig(secret)) - if err != nil { - return nil, err - } - project.Secrets[name] = types.SecretConfig(f) - } - - for name, config := range project.Configs { - f, err := loadFileObject(types.FileObjectConfig(config)) - if err != nil { - return nil, err - } - project.Configs[name] = types.ConfigObjConfig(f) - } - - return project, nil -} - -func loadFileObject(conf types.FileObjectConfig) (types.FileObjectConfig, error) { - if !conf.External { - switch { - case conf.Environment != "": - conf.Content = os.Getenv(conf.Environment) - case conf.File != "": - bytes, err := os.ReadFile(conf.File) - if err != nil { - return conf, err - } - conf.Content = string(bytes) - } - } - return conf, nil -} - -func inspectWithPull(ctx context.Context, dockerCli command.Cli, imageName string) (image.InspectResponse, error) { - inspect, err := dockerCli.Client().ImageInspect(ctx, imageName) - if errdefs.IsNotFound(err) { - var stream io.ReadCloser - stream, err = dockerCli.Client().ImagePull(ctx, imageName, image.PullOptions{}) - if err != nil { - return image.InspectResponse{}, err - } - defer func() { _ = stream.Close() }() - - err = jsonmessage.DisplayJSONMessagesToStream(stream, dockerCli.Out(), nil) - if err != nil { - return image.InspectResponse{}, err - } - if inspect, err = dockerCli.Client().ImageInspect(ctx, imageName); err != nil { - return image.InspectResponse{}, err - } - } - return inspect, err -} diff --git a/pkg/bridge/transformers.go b/pkg/bridge/transformers.go deleted file mode 100644 index dbf4fc6d9dc..00000000000 --- a/pkg/bridge/transformers.go +++ /dev/null @@ -1,120 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package bridge - -import ( - "context" - "fmt" - "os" - "path/filepath" - - "github.com/docker/cli/cli/command" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/api/types/network" - "github.com/moby/go-archive" -) - -const ( - TransformerLabel = "com.docker.compose.bridge" - DefaultTransformerImage = "docker/compose-bridge-kubernetes" - - templatesPath = "/templates" -) - -type CreateTransformerOptions struct { - Dest string - From string -} - -func CreateTransformer(ctx context.Context, dockerCli command.Cli, options CreateTransformerOptions) error { - if options.From == "" { - options.From = DefaultTransformerImage - } - out, err := filepath.Abs(options.Dest) - if err != nil { - return err - } - - if _, err := os.Stat(out); err == nil { - return fmt.Errorf("output folder %s already exists", out) - } - - tmpl := filepath.Join(out, "templates") - err = os.MkdirAll(tmpl, 0o744) - if err != nil && !os.IsExist(err) { - return fmt.Errorf("cannot create output folder: %w", err) - } - - if err := command.ValidateOutputPath(out); err != nil { - return err - } - - created, err := dockerCli.Client().ContainerCreate(ctx, &container.Config{ - Image: options.From, - }, &container.HostConfig{}, &network.NetworkingConfig{}, nil, "") - defer func() { - _ = dockerCli.Client().ContainerRemove(context.Background(), created.ID, container.RemoveOptions{Force: true}) - }() - - if err != nil { - return err - } - content, stat, err := dockerCli.Client().CopyFromContainer(ctx, created.ID, templatesPath) - if err != nil { - return err - } - defer func() { - _ = content.Close() - }() - - srcInfo := archive.CopyInfo{ - Path: templatesPath, - Exists: true, - IsDir: stat.Mode.IsDir(), - } - - preArchive := content - if srcInfo.RebaseName != "" { - _, srcBase := archive.SplitPathDirEntry(srcInfo.Path) - preArchive = archive.RebaseArchiveEntries(content, srcBase, srcInfo.RebaseName) - } - - if err := archive.CopyTo(preArchive, srcInfo, out); err != nil { - return err - } - - dockerfile := `FROM docker/compose-bridge-transformer -LABEL com.docker.compose.bridge=transformation -COPY templates /templates -` - if err := os.WriteFile(filepath.Join(out, "Dockerfile"), []byte(dockerfile), 0o700); err != nil { - return err - } - _, err = fmt.Fprintf(dockerCli.Out(), "Transformer created in %q\n", out) - return err -} - -func ListTransformers(ctx context.Context, dockerCli command.Cli) ([]image.Summary, error) { - api := dockerCli.Client() - return api.ImageList(ctx, image.ListOptions{ - Filters: filters.NewArgs( - filters.Arg("label", fmt.Sprintf("%s=%s", TransformerLabel, "transformation")), - ), - }) -} diff --git a/pkg/compose/apiSocket.go b/pkg/compose/apiSocket.go deleted file mode 100644 index ddc7a029030..00000000000 --- a/pkg/compose/apiSocket.go +++ /dev/null @@ -1,92 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "errors" - "fmt" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/config/configfile" -) - -// --use-api-socket is not actually supported by the Docker Engine -// but is a client-side hack (see https://github.com/docker/cli/blob/master/cli/command/container/create.go#L246) -// we replicate here by transforming the project model - -func (s *composeService) useAPISocket(project *types.Project) (*types.Project, error) { - useAPISocket := false - for _, service := range project.Services { - if service.UseAPISocket { - useAPISocket = true - break - } - } - if !useAPISocket { - return project, nil - } - - if s.getContextInfo().ServerOSType() == "windows" { - return nil, errors.New("use_api_socket can't be used with a Windows Docker Engine") - } - - creds, err := s.configFile().GetAllCredentials() - if err != nil { - return nil, fmt.Errorf("resolving credentials failed: %w", err) - } - - newConfig := &configfile.ConfigFile{ - AuthConfigs: creds, - } - var configBuf bytes.Buffer - if err := newConfig.SaveToWriter(&configBuf); err != nil { - return nil, fmt.Errorf("saving creds for API socket: %w", err) - } - - project.Configs["#apisocket"] = types.ConfigObjConfig{ - Content: configBuf.String(), - } - - for name, service := range project.Services { - if !service.UseAPISocket { - continue - } - service.Volumes = append(service.Volumes, types.ServiceVolumeConfig{ - Type: types.VolumeTypeBind, - Source: "/var/run/docker.sock", - Target: "/var/run/docker.sock", - }) - - _, envvarPresent := service.Environment["DOCKER_CONFIG"] - - // If the DOCKER_CONFIG env var is already present, we assume the client knows - // what they're doing and don't inject the creds. - if !envvarPresent { - // Set our special little location for the config file. - path := "/run/secrets/docker" - service.Environment["DOCKER_CONFIG"] = &path - } - - service.Configs = append(service.Configs, types.ServiceConfigObjConfig{ - Source: "#apisocket", - Target: "/run/secrets/docker/config.json", - }) - project.Services[name] = service - } - return project, nil -} diff --git a/pkg/compose/api_versions.go b/pkg/compose/api_versions.go deleted file mode 100644 index 2caa520dbdd..00000000000 --- a/pkg/compose/api_versions.go +++ /dev/null @@ -1,73 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -// Docker Engine API version constants. -// These versions correspond to specific Docker Engine releases and their features. -const ( - // APIVersion144 represents Docker Engine API version 1.44 (Engine v25.0). - // - // New features in this version: - // - Endpoint-specific MAC address configuration - // - Multiple networks can be connected during container creation - // - healthcheck.start_interval parameter support - // - // Before this version: - // - MAC address was container-wide only - // - Extra networks required post-creation NetworkConnect calls - // - healthcheck.start_interval was not available - APIVersion144 = "1.44" - - // APIVersion148 represents Docker Engine API version 1.48 (Engine v28.0). - // - // New features in this version: - // - Volume mounts with type=image support - // - // Before this version: - // - Only bind, volume, and tmpfs mount types were supported - APIVersion148 = "1.48" - - // APIVersion149 represents Docker Engine API version 1.49 (Engine v28.1). - // - // New features in this version: - // - Network interface_name configuration - // - Platform parameter in ImageList API - // - // Before this version: - // - interface_name was not configurable - // - ImageList didn't support platform filtering - APIVersion149 = "1.49" -) - -// Docker Engine version strings for user-facing error messages. -// These should be used in error messages to provide clear version requirements. -const ( - // DockerEngineV25 is the major version string for Docker Engine 25.x - DockerEngineV25 = "v25" - - // DockerEngineV28 is the major version string for Docker Engine 28.x - DockerEngineV28 = "v28" - - // DockerEngineV28_1 is the specific version string for Docker Engine 28.1 - DockerEngineV28_1 = "v28.1" -) - -// Build tool version constants -const ( - // BuildxMinVersion is the minimum required version of buildx for compose build - BuildxMinVersion = "0.17.0" -) diff --git a/pkg/compose/attach.go b/pkg/compose/attach.go deleted file mode 100644 index 2ab07b1e995..00000000000 --- a/pkg/compose/attach.go +++ /dev/null @@ -1,161 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "io" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - containerType "github.com/docker/docker/api/types/container" - "github.com/docker/docker/pkg/stdcopy" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -func (s *composeService) attach(ctx context.Context, project *types.Project, listener api.ContainerEventListener, selectedServices []string) (Containers, error) { - containers, err := s.getContainers(ctx, project.Name, oneOffExclude, true, selectedServices...) - if err != nil { - return nil, err - } - if len(containers) == 0 { - return containers, nil - } - - containers.sorted() // This enforces predictable colors assignment - - var names []string - for _, c := range containers { - names = append(names, getContainerNameWithoutProject(c)) - } - - _, err = fmt.Fprintf(s.stdout(), "Attaching to %s\n", strings.Join(names, ", ")) - if err != nil { - logrus.Debugf("failed to write attach message: %v", err) - } - - for _, ctr := range containers { - err := s.attachContainer(ctx, ctr, listener) - if err != nil { - return nil, err - } - } - return containers, nil -} - -func (s *composeService) attachContainer(ctx context.Context, container containerType.Summary, listener api.ContainerEventListener) error { - service := container.Labels[api.ServiceLabel] - name := getContainerNameWithoutProject(container) - return s.doAttachContainer(ctx, service, container.ID, name, listener) -} - -func (s *composeService) doAttachContainer(ctx context.Context, service, id, name string, listener api.ContainerEventListener) error { - inspect, err := s.apiClient().ContainerInspect(ctx, id) - if err != nil { - return err - } - - wOut := utils.GetWriter(func(line string) { - listener(api.ContainerEvent{ - Type: api.ContainerEventLog, - Source: name, - ID: id, - Service: service, - Line: line, - }) - }) - wErr := utils.GetWriter(func(line string) { - listener(api.ContainerEvent{ - Type: api.ContainerEventErr, - Source: name, - ID: id, - Service: service, - Line: line, - }) - }) - - err = s.attachContainerStreams(ctx, id, inspect.Config.Tty, wOut, wErr) - if err != nil { - return err - } - - return nil -} - -func (s *composeService) attachContainerStreams(ctx context.Context, container string, tty bool, stdout, stderr io.WriteCloser) error { - streamOut, err := s.getContainerStreams(ctx, container) - if err != nil { - return err - } - - if stdout != nil { - go func() { - defer func() { - if err := stdout.Close(); err != nil { - logrus.Debugf("failed to close stdout: %v", err) - } - if err := stderr.Close(); err != nil { - logrus.Debugf("failed to close stderr: %v", err) - } - if err := streamOut.Close(); err != nil { - logrus.Debugf("failed to close stream output: %v", err) - } - }() - - var err error - if tty { - _, err = io.Copy(stdout, streamOut) - } else { - _, err = stdcopy.StdCopy(stdout, stderr, streamOut) - } - if err != nil && !errors.Is(err, io.EOF) { - logrus.Debugf("stream copy error for container %s: %v", container, err) - } - }() - } - return nil -} - -func (s *composeService) getContainerStreams(ctx context.Context, container string) (io.ReadCloser, error) { - cnx, err := s.apiClient().ContainerAttach(ctx, container, containerType.AttachOptions{ - Stream: true, - Stdin: false, - Stdout: true, - Stderr: true, - Logs: false, - }) - if err == nil { - stdout := ContainerStdout{HijackedResponse: cnx} - return stdout, nil - } - - // Fallback to logs API - logs, err := s.apiClient().ContainerLogs(ctx, container, containerType.LogsOptions{ - ShowStdout: true, - ShowStderr: true, - Follow: true, - }) - if err != nil { - return nil, err - } - return logs, nil -} diff --git a/pkg/compose/attach_service.go b/pkg/compose/attach_service.go deleted file mode 100644 index c6209e02244..00000000000 --- a/pkg/compose/attach_service.go +++ /dev/null @@ -1,45 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strings" - - "github.com/docker/cli/cli/command/container" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Attach(ctx context.Context, projectName string, options api.AttachOptions) error { - projectName = strings.ToLower(projectName) - target, err := s.getSpecifiedContainer(ctx, projectName, oneOffInclude, false, options.Service, options.Index) - if err != nil { - return err - } - - detachKeys := options.DetachKeys - if detachKeys == "" { - detachKeys = s.configFile().DetachKeys - } - - var attach container.AttachOptions - attach.DetachKeys = detachKeys - attach.NoStdin = options.NoStdin - attach.Proxy = options.Proxy - return container.RunAttach(ctx, s.dockerCli, target.ID, &attach) -} diff --git a/pkg/compose/build.go b/pkg/compose/build.go deleted file mode 100644 index 7b9cc76f96c..00000000000 --- a/pkg/compose/build.go +++ /dev/null @@ -1,303 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "strings" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/platforms" - specs "github.com/opencontainers/image-spec/specs-go/v1" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -func (s *composeService) Build(ctx context.Context, project *types.Project, options api.BuildOptions) error { - err := options.Apply(project) - if err != nil { - return err - } - return Run(ctx, func(ctx context.Context) error { - return tracing.SpanWrapFunc("project/build", tracing.ProjectOptions(ctx, project), - func(ctx context.Context) error { - builtImages, err := s.build(ctx, project, options, nil) - if err == nil && len(builtImages) == 0 { - logrus.Warn("No services to build") - } - return err - })(ctx) - }, "build", s.events) -} - -func (s *composeService) build(ctx context.Context, project *types.Project, options api.BuildOptions, localImages map[string]api.ImageSummary) (map[string]string, error) { - imageIDs := map[string]string{} - serviceToBuild := types.Services{} - - var policy types.DependencyOption = types.IgnoreDependencies - if options.Deps { - policy = types.IncludeDependencies - } - - if len(options.Services) == 0 { - options.Services = project.ServiceNames() - } - - // also include services used as additional_contexts with service: prefix - options.Services = addBuildDependencies(options.Services, project) - - // Some build dependencies we just introduced may not be enabled - var err error - project, err = project.WithServicesEnabled(options.Services...) - if err != nil { - return nil, err - } - - project, err = project.WithSelectedServices(options.Services) - if err != nil { - return nil, err - } - - err = project.ForEachService(options.Services, func(serviceName string, service *types.ServiceConfig) error { - if service.Build == nil { - return nil - } - image := api.GetImageNameOrDefault(*service, project.Name) - _, localImagePresent := localImages[image] - if localImagePresent && service.PullPolicy != types.PullPolicyBuild { - return nil - } - serviceToBuild[serviceName] = *service - return nil - }, policy) - if err != nil { - return imageIDs, err - } - - if len(serviceToBuild) == 0 { - return imageIDs, nil - } - - bake, err := buildWithBake(s.dockerCli) - if err != nil { - return nil, err - } - if bake { - return s.doBuildBake(ctx, project, serviceToBuild, options) - } - return s.doBuildClassic(ctx, project, serviceToBuild, options) -} - -func (s *composeService) ensureImagesExists(ctx context.Context, project *types.Project, buildOpts *api.BuildOptions, quietPull bool) error { - for name, service := range project.Services { - if service.Provider == nil && service.Image == "" && service.Build == nil { - return fmt.Errorf("invalid service %q. Must specify either image or build", name) - } - } - - images, err := s.getLocalImagesDigests(ctx, project) - if err != nil { - return err - } - - err = tracing.SpanWrapFunc("project/pull", tracing.ProjectOptions(ctx, project), - func(ctx context.Context) error { - return s.pullRequiredImages(ctx, project, images, quietPull) - }, - )(ctx) - if err != nil { - return err - } - - if buildOpts != nil { - err = tracing.SpanWrapFunc("project/build", tracing.ProjectOptions(ctx, project), - func(ctx context.Context) error { - builtImages, err := s.build(ctx, project, *buildOpts, images) - if err != nil { - return err - } - - for name, digest := range builtImages { - images[name] = api.ImageSummary{ - Repository: name, - ID: digest, - LastTagTime: time.Now(), - } - } - return nil - }, - )(ctx) - if err != nil { - return err - } - } - - // set digest as com.docker.compose.image label so we can detect outdated containers - for name, service := range project.Services { - image := api.GetImageNameOrDefault(service, project.Name) - img, ok := images[image] - if ok { - service.CustomLabels.Add(api.ImageDigestLabel, img.ID) - } - - resolveImageVolumes(&service, images, project.Name) - - project.Services[name] = service - } - return nil -} - -func resolveImageVolumes(service *types.ServiceConfig, images map[string]api.ImageSummary, projectName string) { - for i, vol := range service.Volumes { - if vol.Type == types.VolumeTypeImage { - imgName := vol.Source - if _, ok := images[vol.Source]; !ok { - // check if source is another service in the project - imgName = api.GetImageNameOrDefault(types.ServiceConfig{Name: vol.Source}, projectName) - // If we still can't find it, it might be an external image that wasn't pulled yet or doesn't exist - if _, ok := images[imgName]; !ok { - continue - } - } - if img, ok := images[imgName]; ok { - // Use Image ID directly as source. - // Using name@digest format (via reference.WithDigest) fails for local-only images - // that don't have RepoDigests (e.g. built locally in CI). - // Image ID (sha256:...) is always valid and ensures ServiceHash changes on rebuild. - service.Volumes[i].Source = img.ID - } - } - } -} - -func (s *composeService) getLocalImagesDigests(ctx context.Context, project *types.Project) (map[string]api.ImageSummary, error) { - imageNames := utils.Set[string]{} - for _, s := range project.Services { - imageNames.Add(api.GetImageNameOrDefault(s, project.Name)) - for _, volume := range s.Volumes { - if volume.Type == types.VolumeTypeImage { - imageNames.Add(volume.Source) - } - } - } - imgs, err := s.getImageSummaries(ctx, imageNames.Elements()) - if err != nil { - return nil, err - } - - for i, service := range project.Services { - imgName := api.GetImageNameOrDefault(service, project.Name) - img, ok := imgs[imgName] - if !ok { - continue - } - if service.Platform != "" { - platform, err := platforms.Parse(service.Platform) - if err != nil { - return nil, err - } - inspect, err := s.apiClient().ImageInspect(ctx, img.ID) - if err != nil { - return nil, err - } - actual := specs.Platform{ - Architecture: inspect.Architecture, - OS: inspect.Os, - Variant: inspect.Variant, - } - if !platforms.NewMatcher(platform).Match(actual) { - logrus.Debugf("local image %s doesn't match expected platform %s", service.Image, service.Platform) - // there is a local image, but it's for the wrong platform, so - // pretend it doesn't exist so that we can pull/build an image - // for the correct platform instead - delete(imgs, imgName) - } - } - - project.Services[i].CustomLabels.Add(api.ImageDigestLabel, img.ID) - - } - - return imgs, nil -} - -// resolveAndMergeBuildArgs returns the final set of build arguments to use for the service image build. -// -// First, args directly defined via `build.args` in YAML are considered. -// Then, any explicitly passed args in opts (e.g. via `--build-arg` on the CLI) are merged, overwriting any -// keys that already exist. -// Next, any keys without a value are resolved using the project environment. -// -// Finally, standard proxy variables based on the Docker client configuration are added, but will not overwrite -// any values if already present. -func resolveAndMergeBuildArgs(proxyConfig map[string]string, project *types.Project, service types.ServiceConfig, opts api.BuildOptions) types.MappingWithEquals { - result := make(types.MappingWithEquals). - OverrideBy(service.Build.Args). - OverrideBy(opts.Args). - Resolve(envResolver(project.Environment)) - - // proxy arguments do NOT override and should NOT have env resolution applied, - // so they're handled last - for k, v := range proxyConfig { - if _, ok := result[k]; !ok { - v := v - result[k] = &v - } - } - return result -} - -func getImageBuildLabels(project *types.Project, service types.ServiceConfig) types.Labels { - ret := make(types.Labels) - if service.Build != nil { - for k, v := range service.Build.Labels { - ret.Add(k, v) - } - } - - ret.Add(api.VersionLabel, api.ComposeVersion) - ret.Add(api.ProjectLabel, project.Name) - ret.Add(api.ServiceLabel, service.Name) - return ret -} - -func addBuildDependencies(services []string, project *types.Project) []string { - servicesWithDependencies := utils.NewSet(services...) - for _, service := range services { - s, ok := project.Services[service] - if !ok { - s = project.DisabledServices[service] - } - b := s.Build - if b != nil { - for _, target := range b.AdditionalContexts { - if s, found := strings.CutPrefix(target, types.ServicePrefix); found { - servicesWithDependencies.Add(s) - } - } - } - } - if len(servicesWithDependencies) > len(services) { - return addBuildDependencies(servicesWithDependencies.Elements(), project) - } - return servicesWithDependencies.Elements() -} diff --git a/pkg/compose/build_bake.go b/pkg/compose/build_bake.go deleted file mode 100644 index 90bd0d5a353..00000000000 --- a/pkg/compose/build_bake.go +++ /dev/null @@ -1,591 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bufio" - "bytes" - "context" - "crypto/sha1" - "encoding/json" - "errors" - "fmt" - "io" - "io/fs" - "os" - "os/exec" - "path/filepath" - "slices" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/console" - "github.com/containerd/errdefs" - "github.com/docker/cli/cli-plugins/manager" - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/command/image/build" - "github.com/docker/cli/cli/streams" - "github.com/docker/docker/api/types/versions" - "github.com/google/uuid" - "github.com/moby/buildkit/client" - gitutil "github.com/moby/buildkit/frontend/dockerfile/dfgitutil" - "github.com/moby/buildkit/util/progress/progressui" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func buildWithBake(dockerCli command.Cli) (bool, error) { - enabled, err := dockerCli.BuildKitEnabled() - if err != nil { - return false, err - } - if !enabled { - return false, nil - } - - _, err = manager.GetPlugin("buildx", dockerCli, &cobra.Command{}) - if err != nil { - if errdefs.IsNotFound(err) { - logrus.Warnf("Docker Compose requires buildx plugin to be installed") - return false, nil - } - return false, err - } - return true, err -} - -// We _could_ use bake.* types from github.com/docker/buildx but long term plan is to remove buildx as a dependency -type bakeConfig struct { - Groups map[string]bakeGroup `json:"group"` - Targets map[string]bakeTarget `json:"target"` -} - -type bakeGroup struct { - Targets []string `json:"targets"` -} - -type bakeTarget struct { - Context string `json:"context,omitempty"` - Contexts map[string]string `json:"contexts,omitempty"` - Dockerfile string `json:"dockerfile,omitempty"` - DockerfileInline string `json:"dockerfile-inline,omitempty"` - Args map[string]string `json:"args,omitempty"` - Labels map[string]string `json:"labels,omitempty"` - Tags []string `json:"tags,omitempty"` - CacheFrom []string `json:"cache-from,omitempty"` - CacheTo []string `json:"cache-to,omitempty"` - Target string `json:"target,omitempty"` - Secrets []string `json:"secret,omitempty"` - SSH []string `json:"ssh,omitempty"` - Platforms []string `json:"platforms,omitempty"` - Pull bool `json:"pull,omitempty"` - NoCache bool `json:"no-cache,omitempty"` - NetworkMode string `json:"network,omitempty"` - NoCacheFilter []string `json:"no-cache-filter,omitempty"` - ShmSize types.UnitBytes `json:"shm-size,omitempty"` - Ulimits []string `json:"ulimits,omitempty"` - Call string `json:"call,omitempty"` - Entitlements []string `json:"entitlements,omitempty"` - ExtraHosts map[string]string `json:"extra-hosts,omitempty"` - Outputs []string `json:"output,omitempty"` - Attest []string `json:"attest,omitempty"` -} - -type bakeMetadata map[string]buildStatus - -type buildStatus struct { - Digest string `json:"containerimage.digest"` - Image string `json:"image.name"` -} - -func (s *composeService) doBuildBake(ctx context.Context, project *types.Project, serviceToBeBuild types.Services, options api.BuildOptions) (map[string]string, error) { //nolint:gocyclo - eg := errgroup.Group{} - ch := make(chan *client.SolveStatus) - displayMode := progressui.DisplayMode(options.Progress) - if p, ok := os.LookupEnv("BUILDKIT_PROGRESS"); ok && displayMode == progressui.AutoMode { - displayMode = progressui.DisplayMode(p) - } - out := options.Out - if out == nil { - out = s.stdout() - } - display, err := progressui.NewDisplay(makeConsole(out), displayMode) - if err != nil { - return nil, err - } - eg.Go(func() error { - _, err := display.UpdateFrom(ctx, ch) - return err - }) - - cfg := bakeConfig{ - Groups: map[string]bakeGroup{}, - Targets: map[string]bakeTarget{}, - } - var ( - group bakeGroup - privileged bool - read []string - expectedImages = make(map[string]string, len(serviceToBeBuild)) // service name -> expected image - targets = make(map[string]string, len(serviceToBeBuild)) // service name -> build target - ) - - // produce a unique ID for service used as bake target - for serviceName := range project.Services { - t := strings.ReplaceAll(serviceName, ".", "_") - for { - if _, ok := targets[serviceName]; !ok { - targets[serviceName] = t - break - } - t += "_" - } - } - - var secretsEnv []string - for serviceName, service := range project.Services { - if service.Build == nil { - continue - } - buildConfig := *service.Build - labels := getImageBuildLabels(project, service) - - args := resolveAndMergeBuildArgs(s.getProxyConfig(), project, service, options).ToMapping() - for k, v := range args { - args[k] = strings.ReplaceAll(v, "${", "$${") - } - - entitlements := buildConfig.Entitlements - if slices.Contains(buildConfig.Entitlements, "security.insecure") { - privileged = true - } - if buildConfig.Privileged { - entitlements = append(entitlements, "security.insecure") - privileged = true - } - - var outputs []string - var call string - push := options.Push && service.Image != "" - switch { - case options.Check: - call = "lint" - case len(service.Build.Platforms) > 1: - outputs = []string{fmt.Sprintf("type=image,push=%t", push)} - default: - if push { - outputs = []string{"type=registry"} - } else { - outputs = []string{"type=docker"} - } - } - - read = append(read, buildConfig.Context) - for _, path := range buildConfig.AdditionalContexts { - _, _, err := gitutil.ParseGitRef(path) - if !strings.Contains(path, "://") && err != nil { - read = append(read, path) - } - } - - image := api.GetImageNameOrDefault(service, project.Name) - s.events.On(buildingEvent(image)) - - expectedImages[serviceName] = image - - pull := service.Build.Pull || options.Pull - noCache := service.Build.NoCache || options.NoCache - - target := targets[serviceName] - - secrets, env := toBakeSecrets(project, buildConfig.Secrets) - secretsEnv = append(secretsEnv, env...) - - cfg.Targets[target] = bakeTarget{ - Context: buildConfig.Context, - Contexts: additionalContexts(buildConfig.AdditionalContexts, targets), - Dockerfile: dockerFilePath(buildConfig.Context, buildConfig.Dockerfile), - DockerfileInline: strings.ReplaceAll(buildConfig.DockerfileInline, "${", "$${"), - Args: args, - Labels: labels, - Tags: append(buildConfig.Tags, image), - - CacheFrom: buildConfig.CacheFrom, - CacheTo: buildConfig.CacheTo, - NetworkMode: buildConfig.Network, - NoCacheFilter: buildConfig.NoCacheFilter, - Platforms: buildConfig.Platforms, - Target: buildConfig.Target, - Secrets: secrets, - SSH: toBakeSSH(append(buildConfig.SSH, options.SSHs...)), - Pull: pull, - NoCache: noCache, - ShmSize: buildConfig.ShmSize, - Ulimits: toBakeUlimits(buildConfig.Ulimits), - Entitlements: entitlements, - ExtraHosts: toBakeExtraHosts(buildConfig.ExtraHosts), - - Outputs: outputs, - Call: call, - Attest: toBakeAttest(buildConfig), - } - } - - // create a bake group with targets for services to build - for serviceName, service := range serviceToBeBuild { - if service.Build == nil { - continue - } - group.Targets = append(group.Targets, targets[serviceName]) - } - - cfg.Groups["default"] = group - - b, err := json.MarshalIndent(cfg, "", " ") - if err != nil { - return nil, err - } - - if options.Print { - _, err = fmt.Fprintln(s.stdout(), string(b)) - return nil, err - } - logrus.Debugf("bake build config:\n%s", string(b)) - - tmpdir := os.TempDir() - var metadataFile string - for { - // we don't use os.CreateTemp here as we need a temporary file name, but don't want it actually created - // as bake relies on atomicwriter and this creates conflict during rename - metadataFile = filepath.Join(tmpdir, fmt.Sprintf("compose-build-metadataFile-%s.json", uuid.New().String())) - if _, err = os.Stat(metadataFile); err != nil { - if os.IsNotExist(err) { - break - } - var pathError *fs.PathError - if errors.As(err, &pathError) { - return nil, fmt.Errorf("can't access os.tempDir %s: %w", tmpdir, pathError.Err) - } - } - } - defer func() { - _ = os.Remove(metadataFile) - }() - - buildx, err := s.getBuildxPlugin() - if err != nil { - return nil, err - } - - args := []string{"bake", "--file", "-", "--progress", "rawjson", "--metadata-file", metadataFile} - // FIXME we should prompt user about this, but this is a breaking change in UX - for _, path := range read { - args = append(args, "--allow", "fs.read="+path) - } - if privileged { - args = append(args, "--allow", "security.insecure") - } - if options.SBOM != "" { - args = append(args, "--sbom="+options.SBOM) - } - if options.Provenance != "" { - args = append(args, "--provenance="+options.Provenance) - } - - if options.Builder != "" { - args = append(args, "--builder", options.Builder) - } - if options.Quiet { - args = append(args, "--progress=quiet") - } - - logrus.Debugf("Executing bake with args: %v", args) - - if s.dryRun { - return s.dryRunBake(cfg), nil - } - cmd := exec.CommandContext(ctx, buildx.Path, args...) - - err = s.prepareShellOut(ctx, types.NewMapping(os.Environ()), cmd) - if err != nil { - return nil, err - } - endpoint, cleanup, err := s.propagateDockerEndpoint() - if err != nil { - return nil, err - } - cmd.Env = append(cmd.Env, endpoint...) - cmd.Env = append(cmd.Env, secretsEnv...) - defer cleanup() - - cmd.Stdout = s.stdout() - cmd.Stdin = bytes.NewBuffer(b) - pipe, err := cmd.StderrPipe() - if err != nil { - return nil, err - } - - var errMessage []string - reader := bufio.NewReader(pipe) - - err = cmd.Start() - if err != nil { - return nil, err - } - eg.Go(cmd.Wait) - for { - line, readErr := reader.ReadString('\n') - if readErr != nil { - if readErr == io.EOF { - break - } - if errors.Is(readErr, os.ErrClosed) { - logrus.Debugf("bake stopped") - break - } - return nil, fmt.Errorf("failed to execute bake: %w", readErr) - } - decoder := json.NewDecoder(strings.NewReader(line)) - var status client.SolveStatus - err := decoder.Decode(&status) - if err != nil { - if strings.HasPrefix(line, "ERROR: ") { - errMessage = append(errMessage, line[7:]) - } else { - errMessage = append(errMessage, line) - } - continue - } - ch <- &status - } - close(ch) // stop build progress UI - - err = eg.Wait() - if err != nil { - if len(errMessage) > 0 { - return nil, errors.New(strings.Join(errMessage, "\n")) - } - return nil, fmt.Errorf("failed to execute bake: %w", err) - } - - b, err = os.ReadFile(metadataFile) - if err != nil { - return nil, err - } - - var md bakeMetadata - err = json.Unmarshal(b, &md) - if err != nil { - return nil, err - } - - results := map[string]string{} - for name := range serviceToBeBuild { - image := expectedImages[name] - target := targets[name] - built, ok := md[target] - if !ok { - return nil, fmt.Errorf("build result not found in Bake metadata for service %s", name) - } - results[image] = built.Digest - s.events.On(builtEvent(image)) - } - return results, nil -} - -func (s *composeService) getBuildxPlugin() (*manager.Plugin, error) { - buildx, err := manager.GetPlugin("buildx", s.dockerCli, &cobra.Command{}) - if err != nil { - return nil, err - } - - if buildx.Err != nil { - return nil, buildx.Err - } - - if buildx.Version == "" { - return nil, fmt.Errorf("failed to get version of buildx") - } - - if versions.LessThan(buildx.Version[1:], BuildxMinVersion) { - return nil, fmt.Errorf("compose build requires buildx %s or later", BuildxMinVersion) - } - - return buildx, nil -} - -// makeConsole wraps the provided writer to match [containerd.File] interface if it is of type *streams.Out. -// buildkit's NewDisplay doesn't actually require a [io.Reader], it only uses the [containerd.Console] type to -// benefits from ANSI capabilities, but only does writes. -func makeConsole(out io.Writer) io.Writer { - if s, ok := out.(*streams.Out); ok { - return &_console{s} - } - return out -} - -var _ console.File = &_console{} - -type _console struct { - *streams.Out -} - -func (c _console) Read(p []byte) (n int, err error) { - return 0, errors.New("not implemented") -} - -func (c _console) Close() error { - return nil -} - -func (c _console) Fd() uintptr { - return c.FD() -} - -func (c _console) Name() string { - return "compose" -} - -func toBakeExtraHosts(hosts types.HostsList) map[string]string { - m := make(map[string]string) - for k, v := range hosts { - m[k] = strings.Join(v, ",") - } - return m -} - -func additionalContexts(contexts types.Mapping, targets map[string]string) map[string]string { - ac := map[string]string{} - for k, v := range contexts { - if target, found := strings.CutPrefix(v, types.ServicePrefix); found { - v = "target:" + targets[target] - } - ac[k] = v - } - return ac -} - -func toBakeUlimits(ulimits map[string]*types.UlimitsConfig) []string { - s := []string{} - for u, l := range ulimits { - if l.Single > 0 { - s = append(s, fmt.Sprintf("%s=%d", u, l.Single)) - } else { - s = append(s, fmt.Sprintf("%s=%d:%d", u, l.Soft, l.Hard)) - } - } - return s -} - -func toBakeSSH(ssh types.SSHConfig) []string { - var s []string - for _, key := range ssh { - s = append(s, fmt.Sprintf("%s=%s", key.ID, key.Path)) - } - return s -} - -func toBakeSecrets(project *types.Project, secrets []types.ServiceSecretConfig) ([]string, []string) { - var s []string - var env []string - for _, ref := range secrets { - def := project.Secrets[ref.Source] - target := ref.Target - if target == "" { - target = ref.Source - } - switch { - case def.Environment != "": - env = append(env, fmt.Sprintf("%s=%s", def.Environment, project.Environment[def.Environment])) - s = append(s, fmt.Sprintf("id=%s,type=env,env=%s", target, def.Environment)) - case def.File != "": - s = append(s, fmt.Sprintf("id=%s,type=file,src=%s", target, def.File)) - } - } - return s, env -} - -func toBakeAttest(buildConfig types.BuildConfig) []string { - var attests []string - - // Handle per-service provenance configuration (only from build config, not global options) - if buildConfig.Provenance != "" { - if buildConfig.Provenance == "true" { - attests = append(attests, "type=provenance") - } else if buildConfig.Provenance != "false" { - attests = append(attests, fmt.Sprintf("type=provenance,%s", buildConfig.Provenance)) - } - } - - // Handle per-service SBOM configuration (only from build config, not global options) - if buildConfig.SBOM != "" { - if buildConfig.SBOM == "true" { - attests = append(attests, "type=sbom") - } else if buildConfig.SBOM != "false" { - attests = append(attests, fmt.Sprintf("type=sbom,%s", buildConfig.SBOM)) - } - } - - return attests -} - -func dockerFilePath(ctxName string, dockerfile string) string { - if dockerfile == "" { - return "" - } - if contextType, _ := build.DetectContextType(ctxName); contextType == build.ContextTypeGit { - return dockerfile - } - if !filepath.IsAbs(dockerfile) { - dockerfile = filepath.Join(ctxName, dockerfile) - } - dir := filepath.Dir(dockerfile) - symlinks, err := filepath.EvalSymlinks(dir) - if err == nil { - return filepath.Join(symlinks, filepath.Base(dockerfile)) - } - return dockerfile -} - -func (s composeService) dryRunBake(cfg bakeConfig) map[string]string { - bakeResponse := map[string]string{} - for name, target := range cfg.Targets { - dryRunUUID := fmt.Sprintf("dryRun-%x", sha1.Sum([]byte(name))) - s.displayDryRunBuildEvent(name, dryRunUUID, target.Tags[0]) - bakeResponse[name] = dryRunUUID - } - for name := range bakeResponse { - s.events.On(builtEvent(name)) - } - return bakeResponse -} - -func (s composeService) displayDryRunBuildEvent(name, dryRunUUID, tag string) { - s.events.On(api.Resource{ - ID: name + " ==>", - Status: api.Done, - Text: fmt.Sprintf("==> writing image %s", dryRunUUID), - }) - s.events.On(api.Resource{ - ID: name + " ==> ==>", - Status: api.Done, - Text: fmt.Sprintf(`naming to %s`, tag), - }) -} diff --git a/pkg/compose/build_classic.go b/pkg/compose/build_classic.go deleted file mode 100644 index 1cfa2575322..00000000000 --- a/pkg/compose/build_classic.go +++ /dev/null @@ -1,309 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "encoding/json" - "errors" - "fmt" - "io" - "os" - "path/filepath" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command/image/build" - buildtypes "github.com/docker/docker/api/types/build" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/registry" - "github.com/docker/docker/pkg/jsonmessage" - "github.com/docker/docker/pkg/progress" - "github.com/docker/docker/pkg/streamformatter" - "github.com/moby/go-archive" - "github.com/sirupsen/logrus" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/trace" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) doBuildClassic(ctx context.Context, project *types.Project, serviceToBuild types.Services, options api.BuildOptions) (map[string]string, error) { - imageIDs := map[string]string{} - - // Not using bake, additional_context: service:xx is implemented by building images in dependency order - project, err := project.WithServicesTransform(func(serviceName string, service types.ServiceConfig) (types.ServiceConfig, error) { - if service.Build != nil { - for _, c := range service.Build.AdditionalContexts { - if t, found := strings.CutPrefix(c, types.ServicePrefix); found { - if service.DependsOn == nil { - service.DependsOn = map[string]types.ServiceDependency{} - } - service.DependsOn[t] = types.ServiceDependency{ - Condition: "build", // non-canonical, but will force dependency graph ordering - } - } - } - } - return service, nil - }) - if err != nil { - return imageIDs, err - } - - // we use a pre-allocated []string to collect build digest by service index while running concurrent goroutines - builtDigests := make([]string, len(project.Services)) - names := project.ServiceNames() - getServiceIndex := func(name string) int { - for idx, n := range names { - if n == name { - return idx - } - } - return -1 - } - - err = InDependencyOrder(ctx, project, func(ctx context.Context, name string) error { - trace.SpanFromContext(ctx).SetAttributes(attribute.String("builder", "classic")) - service, ok := serviceToBuild[name] - if !ok { - return nil - } - - image := api.GetImageNameOrDefault(service, project.Name) - s.events.On(buildingEvent(image)) - id, err := s.doBuildImage(ctx, project, service, options) - if err != nil { - return err - } - s.events.On(builtEvent(image)) - builtDigests[getServiceIndex(name)] = id - - if options.Push { - return s.push(ctx, project, api.PushOptions{}) - } - return nil - }, func(traversal *graphTraversal) { - traversal.maxConcurrency = s.maxConcurrency - }) - if err != nil { - return nil, err - } - - for i, imageDigest := range builtDigests { - if imageDigest != "" { - service := project.Services[names[i]] - imageRef := api.GetImageNameOrDefault(service, project.Name) - imageIDs[imageRef] = imageDigest - } - } - return imageIDs, err -} - -//nolint:gocyclo -func (s *composeService) doBuildImage(ctx context.Context, project *types.Project, service types.ServiceConfig, options api.BuildOptions) (string, error) { - var ( - buildCtx io.ReadCloser - dockerfileCtx io.ReadCloser - contextDir string - relDockerfile string - ) - - if len(service.Build.Platforms) > 1 { - return "", fmt.Errorf("the classic builder doesn't support multi-arch build, set DOCKER_BUILDKIT=1 to use BuildKit") - } - if service.Build.Privileged { - return "", fmt.Errorf("the classic builder doesn't support privileged mode, set DOCKER_BUILDKIT=1 to use BuildKit") - } - if len(service.Build.AdditionalContexts) > 0 { - return "", fmt.Errorf("the classic builder doesn't support additional contexts, set DOCKER_BUILDKIT=1 to use BuildKit") - } - if len(service.Build.SSH) > 0 { - return "", fmt.Errorf("the classic builder doesn't support SSH keys, set DOCKER_BUILDKIT=1 to use BuildKit") - } - if len(service.Build.Secrets) > 0 { - return "", fmt.Errorf("the classic builder doesn't support secrets, set DOCKER_BUILDKIT=1 to use BuildKit") - } - - if service.Build.Labels == nil { - service.Build.Labels = make(map[string]string) - } - service.Build.Labels[api.ImageBuilderLabel] = "classic" - - dockerfileName := dockerFilePath(service.Build.Context, service.Build.Dockerfile) - specifiedContext := service.Build.Context - progBuff := s.stdout() - buildBuff := s.stdout() - - contextType, err := build.DetectContextType(specifiedContext) - if err != nil { - return "", err - } - - switch contextType { - case build.ContextTypeStdin: - return "", fmt.Errorf("building from STDIN is not supported") - case build.ContextTypeLocal: - contextDir, relDockerfile, err = build.GetContextFromLocalDir(specifiedContext, dockerfileName) - if err != nil { - return "", fmt.Errorf("unable to prepare context: %w", err) - } - if strings.HasPrefix(relDockerfile, ".."+string(filepath.Separator)) { - // Dockerfile is outside build-context; read the Dockerfile and pass it as dockerfileCtx - dockerfileCtx, err = os.Open(dockerfileName) - if err != nil { - return "", fmt.Errorf("unable to open Dockerfile: %w", err) - } - defer dockerfileCtx.Close() //nolint:errcheck - } - case build.ContextTypeGit: - var tempDir string - tempDir, relDockerfile, err = build.GetContextFromGitURL(specifiedContext, dockerfileName) - if err != nil { - return "", fmt.Errorf("unable to prepare context: %w", err) - } - defer func() { - _ = os.RemoveAll(tempDir) - }() - contextDir = tempDir - case build.ContextTypeRemote: - buildCtx, relDockerfile, err = build.GetContextFromURL(progBuff, specifiedContext, dockerfileName) - if err != nil { - return "", fmt.Errorf("unable to prepare context: %w", err) - } - default: - return "", fmt.Errorf("unable to prepare context: path %q not found", specifiedContext) - } - - // read from a directory into tar archive - if buildCtx == nil { - excludes, err := build.ReadDockerignore(contextDir) - if err != nil { - return "", err - } - - if err := build.ValidateContextDirectory(contextDir, excludes); err != nil { - return "", fmt.Errorf("checking context: %w", err) - } - - // And canonicalize dockerfile name to a platform-independent one - relDockerfile = filepath.ToSlash(relDockerfile) - - excludes = build.TrimBuildFilesFromExcludes(excludes, relDockerfile, false) - buildCtx, err = archive.TarWithOptions(contextDir, &archive.TarOptions{ - ExcludePatterns: excludes, - ChownOpts: &archive.ChownOpts{UID: 0, GID: 0}, - }) - if err != nil { - return "", err - } - } - - // replace Dockerfile if it was added from stdin or a file outside the build-context, and there is archive context - if dockerfileCtx != nil && buildCtx != nil { - buildCtx, relDockerfile, err = build.AddDockerfileToBuildContext(dockerfileCtx, buildCtx) - if err != nil { - return "", err - } - } - - buildCtx, err = build.Compress(buildCtx) - if err != nil { - return "", err - } - - // Setup an upload progress bar - progressOutput := streamformatter.NewProgressOutput(progBuff) - body := progress.NewProgressReader(buildCtx, progressOutput, 0, "", "Sending build context to Docker daemon") - - configFile := s.configFile() - creds, err := configFile.GetAllCredentials() - if err != nil { - return "", err - } - authConfigs := make(map[string]registry.AuthConfig, len(creds)) - for k, authConfig := range creds { - authConfigs[k] = registry.AuthConfig{ - Username: authConfig.Username, - Password: authConfig.Password, - ServerAddress: authConfig.ServerAddress, - - // TODO(thaJeztah): Are these expected to be included? See https://github.com/docker/cli/pull/6516#discussion_r2387586472 - Auth: authConfig.Auth, - IdentityToken: authConfig.IdentityToken, - RegistryToken: authConfig.RegistryToken, - } - } - buildOpts := imageBuildOptions(s.getProxyConfig(), project, service, options) - imageName := api.GetImageNameOrDefault(service, project.Name) - buildOpts.Tags = append(buildOpts.Tags, imageName) - buildOpts.Dockerfile = relDockerfile - buildOpts.AuthConfigs = authConfigs - buildOpts.Memory = options.Memory - - ctx, cancel := context.WithCancel(ctx) - defer cancel() - s.events.On(buildingEvent(imageName)) - response, err := s.apiClient().ImageBuild(ctx, body, buildOpts) - if err != nil { - return "", err - } - defer response.Body.Close() //nolint:errcheck - - imageID := "" - aux := func(msg jsonmessage.JSONMessage) { - var result buildtypes.Result - if err := json.Unmarshal(*msg.Aux, &result); err != nil { - logrus.Errorf("Failed to parse aux message: %s", err) - } else { - imageID = result.ID - } - } - - err = jsonmessage.DisplayJSONMessagesStream(response.Body, buildBuff, progBuff.FD(), true, aux) - if err != nil { - var jerr *jsonmessage.JSONError - if errors.As(err, &jerr) { - // If no error code is set, default to 1 - if jerr.Code == 0 { - jerr.Code = 1 - } - return "", cli.StatusError{Status: jerr.Message, StatusCode: jerr.Code} - } - return "", err - } - s.events.On(builtEvent(imageName)) - return imageID, nil -} - -func imageBuildOptions(proxyConfigs map[string]string, project *types.Project, service types.ServiceConfig, options api.BuildOptions) buildtypes.ImageBuildOptions { - config := service.Build - return buildtypes.ImageBuildOptions{ - Version: buildtypes.BuilderV1, - Tags: config.Tags, - NoCache: config.NoCache, - Remove: true, - PullParent: config.Pull, - BuildArgs: resolveAndMergeBuildArgs(proxyConfigs, project, service, options), - Labels: config.Labels, - NetworkMode: config.Network, - ExtraHosts: config.ExtraHosts.AsList(":"), - Target: config.Target, - Isolation: container.Isolation(config.Isolation), - } -} diff --git a/pkg/compose/build_test.go b/pkg/compose/build_test.go deleted file mode 100644 index fa0a9e2c4e5..00000000000 --- a/pkg/compose/build_test.go +++ /dev/null @@ -1,57 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "slices" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "gotest.tools/v3/assert" -) - -func Test_addBuildDependencies(t *testing.T) { - project := &types.Project{Services: types.Services{ - "test": types.ServiceConfig{ - Build: &types.BuildConfig{ - AdditionalContexts: map[string]string{ - "foo": "service:foo", - "bar": "service:bar", - }, - }, - }, - "foo": types.ServiceConfig{ - Build: &types.BuildConfig{ - AdditionalContexts: map[string]string{ - "zot": "service:zot", - }, - }, - }, - "bar": types.ServiceConfig{ - Build: &types.BuildConfig{}, - }, - "zot": types.ServiceConfig{ - Build: &types.BuildConfig{}, - }, - }} - - services := addBuildDependencies([]string{"test"}, project) - expected := []string{"test", "foo", "bar", "zot"} - slices.Sort(services) - slices.Sort(expected) - assert.DeepEqual(t, services, expected) -} diff --git a/pkg/compose/commit.go b/pkg/compose/commit.go deleted file mode 100644 index 70b22d5d783..00000000000 --- a/pkg/compose/commit.go +++ /dev/null @@ -1,79 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "strings" - - "github.com/docker/docker/api/types/container" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Commit(ctx context.Context, projectName string, options api.CommitOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.commit(ctx, projectName, options) - }, "commit", s.events) -} - -func (s *composeService) commit(ctx context.Context, projectName string, options api.CommitOptions) error { - projectName = strings.ToLower(projectName) - - ctr, err := s.getSpecifiedContainer(ctx, projectName, oneOffInclude, false, options.Service, options.Index) - if err != nil { - return err - } - - name := getCanonicalContainerName(ctr) - - s.events.On(api.Resource{ - ID: name, - Status: api.Working, - Text: api.StatusCommitting, - }) - - if s.dryRun { - s.events.On(api.Resource{ - ID: name, - Status: api.Done, - Text: api.StatusCommitted, - }) - - return nil - } - - response, err := s.apiClient().ContainerCommit(ctx, ctr.ID, container.CommitOptions{ - Reference: options.Reference, - Comment: options.Comment, - Author: options.Author, - Changes: options.Changes.GetSlice(), - Pause: options.Pause, - }) - if err != nil { - return err - } - - s.events.On(api.Resource{ - ID: name, - Text: fmt.Sprintf("Committed as %s", response.ID), - Status: api.Done, - }) - - return nil -} diff --git a/pkg/compose/compose.go b/pkg/compose/compose.go deleted file mode 100644 index 45bda0bf3ec..00000000000 --- a/pkg/compose/compose.go +++ /dev/null @@ -1,514 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "io" - "strconv" - "strings" - "sync" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/buildx/store/storeutil" - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/config/configfile" - "github.com/docker/cli/cli/flags" - "github.com/docker/cli/cli/streams" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/api/types/swarm" - "github.com/docker/docker/api/types/volume" - "github.com/docker/docker/client" - "github.com/jonboulle/clockwork" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/dryrun" -) - -type Option func(service *composeService) error - -// NewComposeService creates a Compose service using Docker CLI. -// This is the standard constructor that requires command.Cli for full functionality. -// -// Example usage: -// -// dockerCli, _ := command.NewDockerCli() -// service := NewComposeService(dockerCli) -// -// For advanced configuration with custom overrides, use ServiceOption functions: -// -// service := NewComposeService(dockerCli, -// WithPrompt(prompt.NewPrompt(cli.In(), cli.Out()).Confirm), -// WithOutputStream(customOut), -// WithErrorStream(customErr), -// WithInputStream(customIn)) -// -// Or set all streams at once: -// -// service := NewComposeService(dockerCli, -// WithStreams(customOut, customErr, customIn)) -func NewComposeService(dockerCli command.Cli, options ...Option) (api.Compose, error) { - s := &composeService{ - dockerCli: dockerCli, - clock: clockwork.NewRealClock(), - maxConcurrency: -1, - dryRun: false, - } - for _, option := range options { - if err := option(s); err != nil { - return nil, err - } - } - if s.prompt == nil { - s.prompt = func(message string, defaultValue bool) (bool, error) { - fmt.Println(message) - logrus.Warning("Compose is running without a 'prompt' component to interact with user") - return defaultValue, nil - } - } - if s.events == nil { - s.events = &ignore{} - } - - // If custom streams were provided, wrap the Docker CLI to use them - if s.outStream != nil || s.errStream != nil || s.inStream != nil { - s.dockerCli = s.wrapDockerCliWithStreams(dockerCli) - } - - return s, nil -} - -// WithStreams sets custom I/O streams for output and interaction -func WithStreams(out, err io.Writer, in io.Reader) Option { - return func(s *composeService) error { - s.outStream = out - s.errStream = err - s.inStream = in - return nil - } -} - -// WithOutputStream sets a custom output stream -func WithOutputStream(out io.Writer) Option { - return func(s *composeService) error { - s.outStream = out - return nil - } -} - -// WithErrorStream sets a custom error stream -func WithErrorStream(err io.Writer) Option { - return func(s *composeService) error { - s.errStream = err - return nil - } -} - -// WithInputStream sets a custom input stream -func WithInputStream(in io.Reader) Option { - return func(s *composeService) error { - s.inStream = in - return nil - } -} - -// WithContextInfo sets custom Docker context information -func WithContextInfo(info api.ContextInfo) Option { - return func(s *composeService) error { - s.contextInfo = info - return nil - } -} - -// WithProxyConfig sets custom HTTP proxy configuration for builds -func WithProxyConfig(config map[string]string) Option { - return func(s *composeService) error { - s.proxyConfig = config - return nil - } -} - -// WithPrompt configure a UI component for Compose service to interact with user and confirm actions -func WithPrompt(prompt Prompt) Option { - return func(s *composeService) error { - s.prompt = prompt - return nil - } -} - -// WithMaxConcurrency defines upper limit for concurrent operations against engine API -func WithMaxConcurrency(maxConcurrency int) Option { - return func(s *composeService) error { - s.maxConcurrency = maxConcurrency - return nil - } -} - -// WithDryRun configure Compose to run without actually applying changes -func WithDryRun(s *composeService) error { - s.dryRun = true - cli, err := command.NewDockerCli() - if err != nil { - return err - } - - options := flags.NewClientOptions() - options.Context = s.dockerCli.CurrentContext() - err = cli.Initialize(options, command.WithInitializeClient(func(cli *command.DockerCli) (client.APIClient, error) { - return dryrun.NewDryRunClient(s.apiClient(), s.dockerCli) - })) - if err != nil { - return err - } - s.dockerCli = cli - return nil -} - -type Prompt func(message string, defaultValue bool) (bool, error) - -// AlwaysOkPrompt returns a Prompt implementation that always returns true without user interaction. -func AlwaysOkPrompt() Prompt { - return func(message string, defaultValue bool) (bool, error) { - return true, nil - } -} - -// WithEventProcessor configure component to get notified on Compose operation and progress events. -// Typically used to configure a progress UI -func WithEventProcessor(bus api.EventProcessor) Option { - return func(s *composeService) error { - s.events = bus - return nil - } -} - -type composeService struct { - dockerCli command.Cli - // prompt is used to interact with user and confirm actions - prompt Prompt - // eventBus collects tasks execution events - events api.EventProcessor - - // Optional overrides for specific components (for SDK users) - outStream io.Writer - errStream io.Writer - inStream io.Reader - contextInfo api.ContextInfo - proxyConfig map[string]string - - clock clockwork.Clock - maxConcurrency int - dryRun bool -} - -// Close releases any connections/resources held by the underlying clients. -// -// In practice, this service has the same lifetime as the process, so everything -// will get cleaned up at about the same time regardless even if not invoked. -func (s *composeService) Close() error { - var errs []error - if s.dockerCli != nil { - errs = append(errs, s.apiClient().Close()) - } - return errors.Join(errs...) -} - -func (s *composeService) apiClient() client.APIClient { - return s.dockerCli.Client() -} - -func (s *composeService) configFile() *configfile.ConfigFile { - return s.dockerCli.ConfigFile() -} - -// getContextInfo returns the context info - either custom override or dockerCli adapter -func (s *composeService) getContextInfo() api.ContextInfo { - if s.contextInfo != nil { - return s.contextInfo - } - return &dockerCliContextInfo{cli: s.dockerCli} -} - -// getProxyConfig returns the proxy config - either custom override or environment-based -func (s *composeService) getProxyConfig() map[string]string { - if s.proxyConfig != nil { - return s.proxyConfig - } - return storeutil.GetProxyConfig(s.dockerCli) -} - -func (s *composeService) stdout() *streams.Out { - return s.dockerCli.Out() -} - -func (s *composeService) stdin() *streams.In { - return s.dockerCli.In() -} - -func (s *composeService) stderr() *streams.Out { - return s.dockerCli.Err() -} - -// readCloserAdapter adapts io.Reader to io.ReadCloser -type readCloserAdapter struct { - r io.Reader -} - -func (r *readCloserAdapter) Read(p []byte) (int, error) { - return r.r.Read(p) -} - -func (r *readCloserAdapter) Close() error { - return nil -} - -// wrapDockerCliWithStreams wraps the Docker CLI to intercept and override stream methods -func (s *composeService) wrapDockerCliWithStreams(baseCli command.Cli) command.Cli { - wrapper := &streamOverrideWrapper{ - Cli: baseCli, - } - - // Wrap custom streams in Docker CLI's stream types - if s.outStream != nil { - wrapper.outStream = streams.NewOut(s.outStream) - } - if s.errStream != nil { - wrapper.errStream = streams.NewOut(s.errStream) - } - if s.inStream != nil { - wrapper.inStream = streams.NewIn(&readCloserAdapter{r: s.inStream}) - } - - return wrapper -} - -// streamOverrideWrapper wraps command.Cli to override streams with custom implementations -type streamOverrideWrapper struct { - command.Cli - outStream *streams.Out - errStream *streams.Out - inStream *streams.In -} - -func (w *streamOverrideWrapper) Out() *streams.Out { - if w.outStream != nil { - return w.outStream - } - return w.Cli.Out() -} - -func (w *streamOverrideWrapper) Err() *streams.Out { - if w.errStream != nil { - return w.errStream - } - return w.Cli.Err() -} - -func (w *streamOverrideWrapper) In() *streams.In { - if w.inStream != nil { - return w.inStream - } - return w.Cli.In() -} - -func getCanonicalContainerName(c container.Summary) string { - if len(c.Names) == 0 { - // corner case, sometime happens on removal. return short ID as a safeguard value - return c.ID[:12] - } - // Names return container canonical name /foo + link aliases /linked_by/foo - for _, name := range c.Names { - if strings.LastIndex(name, "/") == 0 { - return name[1:] - } - } - - return strings.TrimPrefix(c.Names[0], "/") -} - -func getContainerNameWithoutProject(c container.Summary) string { - project := c.Labels[api.ProjectLabel] - defaultName := getDefaultContainerName(project, c.Labels[api.ServiceLabel], c.Labels[api.ContainerNumberLabel]) - name := getCanonicalContainerName(c) - if name != defaultName { - // service declares a custom container_name - return name - } - return name[len(project)+1:] -} - -// projectFromName builds a types.Project based on actual resources with compose labels set -func (s *composeService) projectFromName(containers Containers, projectName string, services ...string) (*types.Project, error) { - project := &types.Project{ - Name: projectName, - Services: types.Services{}, - } - if len(containers) == 0 { - return project, fmt.Errorf("no container found for project %q: %w", projectName, api.ErrNotFound) - } - set := types.Services{} - for _, c := range containers { - serviceLabel, ok := c.Labels[api.ServiceLabel] - if !ok { - serviceLabel = getCanonicalContainerName(c) - } - service, ok := set[serviceLabel] - if !ok { - service = types.ServiceConfig{ - Name: serviceLabel, - Image: c.Image, - Labels: c.Labels, - } - } - service.Scale = increment(service.Scale) - set[serviceLabel] = service - } - for name, service := range set { - dependencies := service.Labels[api.DependenciesLabel] - if dependencies != "" { - service.DependsOn = types.DependsOnConfig{} - for dc := range strings.SplitSeq(dependencies, ",") { - dcArr := strings.Split(dc, ":") - condition := ServiceConditionRunningOrHealthy - // Let's restart the dependency by default if we don't have the info stored in the label - restart := true - required := true - dependency := dcArr[0] - - // backward compatibility - if len(dcArr) > 1 { - condition = dcArr[1] - if len(dcArr) > 2 { - restart, _ = strconv.ParseBool(dcArr[2]) - } - } - service.DependsOn[dependency] = types.ServiceDependency{Condition: condition, Restart: restart, Required: required} - } - set[name] = service - } - } - project.Services = set - -SERVICES: - for _, qs := range services { - for _, es := range project.Services { - if es.Name == qs { - continue SERVICES - } - } - return project, fmt.Errorf("no such service: %q: %w", qs, api.ErrNotFound) - } - project, err := project.WithSelectedServices(services) - if err != nil { - return project, err - } - - return project, nil -} - -func increment(scale *int) *int { - i := 1 - if scale != nil { - i = *scale + 1 - } - return &i -} - -func (s *composeService) actualVolumes(ctx context.Context, projectName string) (types.Volumes, error) { - opts := volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(projectName)), - } - volumes, err := s.apiClient().VolumeList(ctx, opts) - if err != nil { - return nil, err - } - - actual := types.Volumes{} - for _, vol := range volumes.Volumes { - actual[vol.Labels[api.VolumeLabel]] = types.VolumeConfig{ - Name: vol.Name, - Driver: vol.Driver, - Labels: vol.Labels, - } - } - return actual, nil -} - -func (s *composeService) actualNetworks(ctx context.Context, projectName string) (types.Networks, error) { - networks, err := s.apiClient().NetworkList(ctx, network.ListOptions{ - Filters: filters.NewArgs(projectFilter(projectName)), - }) - if err != nil { - return nil, err - } - - actual := types.Networks{} - for _, net := range networks { - actual[net.Labels[api.NetworkLabel]] = types.NetworkConfig{ - Name: net.Name, - Driver: net.Driver, - Labels: net.Labels, - } - } - return actual, nil -} - -var swarmEnabled = struct { - once sync.Once - val bool - err error -}{} - -func (s *composeService) isSwarmEnabled(ctx context.Context) (bool, error) { - swarmEnabled.once.Do(func() { - info, err := s.apiClient().Info(ctx) - if err != nil { - swarmEnabled.err = err - } - switch info.Swarm.LocalNodeState { - case swarm.LocalNodeStateInactive, swarm.LocalNodeStateLocked: - swarmEnabled.val = false - default: - swarmEnabled.val = true - } - }) - return swarmEnabled.val, swarmEnabled.err -} - -type runtimeVersionCache struct { - once sync.Once - val string - err error -} - -var runtimeVersion runtimeVersionCache - -func (s *composeService) RuntimeVersion(ctx context.Context) (string, error) { - runtimeVersion.once.Do(func() { - version, err := s.apiClient().ServerVersion(ctx) - if err != nil { - runtimeVersion.err = err - } - runtimeVersion.val = version.APIVersion - }) - return runtimeVersion.val, runtimeVersion.err -} diff --git a/pkg/compose/container.go b/pkg/compose/container.go deleted file mode 100644 index 502547ddc9b..00000000000 --- a/pkg/compose/container.go +++ /dev/null @@ -1,58 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "io" - - moby "github.com/docker/docker/api/types" -) - -var _ io.ReadCloser = ContainerStdout{} - -// ContainerStdout implement ReadCloser for moby.HijackedResponse -type ContainerStdout struct { - moby.HijackedResponse -} - -// Read implement io.ReadCloser -func (l ContainerStdout) Read(p []byte) (n int, err error) { - return l.Reader.Read(p) -} - -// Close implement io.ReadCloser -func (l ContainerStdout) Close() error { - l.HijackedResponse.Close() - return nil -} - -var _ io.WriteCloser = ContainerStdin{} - -// ContainerStdin implement WriteCloser for moby.HijackedResponse -type ContainerStdin struct { - moby.HijackedResponse -} - -// Write implement io.WriteCloser -func (c ContainerStdin) Write(p []byte) (n int, err error) { - return c.Conn.Write(p) -} - -// Close implement io.WriteCloser -func (c ContainerStdin) Close() error { - return c.CloseWrite() -} diff --git a/pkg/compose/containers.go b/pkg/compose/containers.go deleted file mode 100644 index 50e8ad56016..00000000000 --- a/pkg/compose/containers.go +++ /dev/null @@ -1,182 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "slices" - "sort" - "strconv" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - - "github.com/docker/compose/v5/pkg/api" -) - -// Containers is a set of moby Container -type Containers []container.Summary - -type oneOff int - -const ( - oneOffInclude = oneOff(iota) - oneOffExclude - oneOffOnly -) - -func (s *composeService) getContainers(ctx context.Context, project string, oneOff oneOff, all bool, selectedServices ...string) (Containers, error) { - var containers Containers - f := getDefaultFilters(project, oneOff, selectedServices...) - containers, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - Filters: filters.NewArgs(f...), - All: all, - }) - if err != nil { - return nil, err - } - if len(selectedServices) > 1 { - containers = containers.filter(isService(selectedServices...)) - } - return containers, nil -} - -func getDefaultFilters(projectName string, oneOff oneOff, selectedServices ...string) []filters.KeyValuePair { - f := []filters.KeyValuePair{projectFilter(projectName)} - if len(selectedServices) == 1 { - f = append(f, serviceFilter(selectedServices[0])) - } - f = append(f, hasConfigHashLabel()) - switch oneOff { - case oneOffOnly: - f = append(f, oneOffFilter(true)) - case oneOffExclude: - f = append(f, oneOffFilter(false)) - case oneOffInclude: - } - return f -} - -func (s *composeService) getSpecifiedContainer(ctx context.Context, projectName string, oneOff oneOff, all bool, serviceName string, containerIndex int) (container.Summary, error) { - defaultFilters := getDefaultFilters(projectName, oneOff, serviceName) - if containerIndex > 0 { - defaultFilters = append(defaultFilters, containerNumberFilter(containerIndex)) - } - containers, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - Filters: filters.NewArgs( - defaultFilters..., - ), - All: all, - }) - if err != nil { - return container.Summary{}, err - } - if len(containers) < 1 { - if containerIndex > 0 { - return container.Summary{}, fmt.Errorf("service %q is not running container #%d", serviceName, containerIndex) - } - return container.Summary{}, fmt.Errorf("service %q is not running", serviceName) - } - - // Sort by container number first, then put one-off containers at the end - sort.Slice(containers, func(i, j int) bool { - numberLabelX, _ := strconv.Atoi(containers[i].Labels[api.ContainerNumberLabel]) - numberLabelY, _ := strconv.Atoi(containers[j].Labels[api.ContainerNumberLabel]) - IsOneOffLabelTrueX := containers[i].Labels[api.OneoffLabel] == "True" - IsOneOffLabelTrueY := containers[j].Labels[api.OneoffLabel] == "True" - - if IsOneOffLabelTrueX || IsOneOffLabelTrueY { - return !IsOneOffLabelTrueX && IsOneOffLabelTrueY - } - - return numberLabelX < numberLabelY - }) - return containers[0], nil -} - -// containerPredicate define a predicate we want container to satisfy for filtering operations -type containerPredicate func(c container.Summary) bool - -func matches(c container.Summary, predicates ...containerPredicate) bool { - for _, predicate := range predicates { - if !predicate(c) { - return false - } - } - return true -} - -func isService(services ...string) containerPredicate { - return func(c container.Summary) bool { - service := c.Labels[api.ServiceLabel] - return slices.Contains(services, service) - } -} - -// isOrphaned is a predicate to select containers without a matching service definition in compose project -func isOrphaned(project *types.Project) containerPredicate { - services := append(project.ServiceNames(), project.DisabledServiceNames()...) - return func(c container.Summary) bool { - // One-off container - v, ok := c.Labels[api.OneoffLabel] - if ok && v == "True" { - return c.State == container.StateExited || c.State == container.StateDead - } - // Service that is not defined in the compose model - service := c.Labels[api.ServiceLabel] - return !slices.Contains(services, service) - } -} - -func isNotOneOff(c container.Summary) bool { - v, ok := c.Labels[api.OneoffLabel] - return !ok || v == "False" -} - -// filter return Containers with elements to match predicate -func (containers Containers) filter(predicates ...containerPredicate) Containers { - var filtered Containers - for _, c := range containers { - if matches(c, predicates...) { - filtered = append(filtered, c) - } - } - return filtered -} - -func (containers Containers) names() []string { - var names []string - for _, c := range containers { - names = append(names, getCanonicalContainerName(c)) - } - return names -} - -func (containers Containers) forEach(fn func(container.Summary)) { - for _, c := range containers { - fn(c) - } -} - -func (containers Containers) sorted() Containers { - sort.Slice(containers, func(i, j int) bool { - return getCanonicalContainerName(containers[i]) < getCanonicalContainerName(containers[j]) - }) - return containers -} diff --git a/pkg/compose/convergence.go b/pkg/compose/convergence.go deleted file mode 100644 index baf34a227f7..00000000000 --- a/pkg/compose/convergence.go +++ /dev/null @@ -1,930 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "maps" - "slices" - "sort" - "strconv" - "strings" - "sync" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/platforms" - "github.com/docker/docker/api/types/container" - mmount "github.com/docker/docker/api/types/mount" - "github.com/docker/docker/api/types/versions" - specs "github.com/opencontainers/image-spec/specs-go/v1" - "github.com/sirupsen/logrus" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/trace" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -const ( - doubledContainerNameWarning = "WARNING: The %q service is using the custom container name %q. " + - "Docker requires each container to have a unique name. " + - "Remove the custom name to scale the service" -) - -// convergence manages service's container lifecycle. -// Based on initially observed state, it reconciles the existing container with desired state, which might include -// re-creating container, adding or removing replicas, or starting stopped containers. -// Cross services dependencies are managed by creating services in expected order and updating `service:xx` reference -// when a service has converged, so dependent ones can be managed with resolved containers references. -type convergence struct { - compose *composeService - services map[string]Containers - networks map[string]string - volumes map[string]string - stateMutex sync.Mutex -} - -func (c *convergence) getObservedState(serviceName string) Containers { - c.stateMutex.Lock() - defer c.stateMutex.Unlock() - return c.services[serviceName] -} - -func (c *convergence) setObservedState(serviceName string, containers Containers) { - c.stateMutex.Lock() - defer c.stateMutex.Unlock() - c.services[serviceName] = containers -} - -func newConvergence(services []string, state Containers, networks map[string]string, volumes map[string]string, s *composeService) *convergence { - observedState := map[string]Containers{} - for _, s := range services { - observedState[s] = Containers{} - } - for _, c := range state.filter(isNotOneOff) { - service := c.Labels[api.ServiceLabel] - observedState[service] = append(observedState[service], c) - } - return &convergence{ - compose: s, - services: observedState, - networks: networks, - volumes: volumes, - } -} - -func (c *convergence) apply(ctx context.Context, project *types.Project, options api.CreateOptions) error { - return InDependencyOrder(ctx, project, func(ctx context.Context, name string) error { - service, err := project.GetService(name) - if err != nil { - return err - } - - return tracing.SpanWrapFunc("service/apply", tracing.ServiceOptions(service), func(ctx context.Context) error { - strategy := options.RecreateDependencies - if slices.Contains(options.Services, name) { - strategy = options.Recreate - } - return c.ensureService(ctx, project, service, strategy, options.Inherit, options.Timeout) - })(ctx) - }) -} - -func (c *convergence) ensureService(ctx context.Context, project *types.Project, service types.ServiceConfig, recreate string, inherit bool, timeout *time.Duration) error { //nolint:gocyclo - if service.Provider != nil { - return c.compose.runPlugin(ctx, project, service, "up") - } - expected, err := getScale(service) - if err != nil { - return err - } - containers := c.getObservedState(service.Name) - actual := len(containers) - updated := make(Containers, expected) - - eg, ctx := errgroup.WithContext(ctx) - - err = c.resolveServiceReferences(&service) - if err != nil { - return err - } - - sort.Slice(containers, func(i, j int) bool { - // select obsolete containers first, so they get removed as we scale down - if obsolete, _ := c.mustRecreate(service, containers[i], recreate); obsolete { - // i is obsolete, so must be first in the list - return true - } - if obsolete, _ := c.mustRecreate(service, containers[j], recreate); obsolete { - // j is obsolete, so must be first in the list - return false - } - - // For up-to-date containers, sort by container number to preserve low-values in container numbers - ni, erri := strconv.Atoi(containers[i].Labels[api.ContainerNumberLabel]) - nj, errj := strconv.Atoi(containers[j].Labels[api.ContainerNumberLabel]) - if erri == nil && errj == nil { - return ni > nj - } - - // If we don't get a container number (?) just sort by creation date - return containers[i].Created < containers[j].Created - }) - - slices.Reverse(containers) - for i, ctr := range containers { - if i >= expected { - // Scale Down - // As we sorted containers, obsolete ones and/or highest number will be removed - ctr := ctr - traceOpts := append(tracing.ServiceOptions(service), tracing.ContainerOptions(ctr)...) - eg.Go(tracing.SpanWrapFuncForErrGroup(ctx, "service/scale/down", traceOpts, func(ctx context.Context) error { - return c.compose.stopAndRemoveContainer(ctx, ctr, &service, timeout, false) - })) - continue - } - - mustRecreate, err := c.mustRecreate(service, ctr, recreate) - if err != nil { - return err - } - if mustRecreate { - err := c.stopDependentContainers(ctx, project, service) - if err != nil { - return err - } - - i, ctr := i, ctr - eg.Go(tracing.SpanWrapFuncForErrGroup(ctx, "container/recreate", tracing.ContainerOptions(ctr), func(ctx context.Context) error { - recreated, err := c.compose.recreateContainer(ctx, project, service, ctr, inherit, timeout) - updated[i] = recreated - return err - })) - continue - } - - // Enforce non-diverged containers are running - name := getContainerProgressName(ctr) - switch ctr.State { - case container.StateRunning: - c.compose.events.On(runningEvent(name)) - case container.StateCreated: - case container.StateRestarting: - case container.StateExited: - default: - ctr := ctr - eg.Go(tracing.EventWrapFuncForErrGroup(ctx, "service/start", tracing.ContainerOptions(ctr), func(ctx context.Context) error { - return c.compose.startContainer(ctx, ctr) - })) - } - updated[i] = ctr - } - - next := nextContainerNumber(containers) - for i := 0; i < expected-actual; i++ { - // Scale UP - number := next + i - name := getContainerName(project.Name, service, number) - eventOpts := tracing.SpanOptions{trace.WithAttributes(attribute.String("container.name", name))} - eg.Go(tracing.EventWrapFuncForErrGroup(ctx, "service/scale/up", eventOpts, func(ctx context.Context) error { - opts := createOptions{ - AutoRemove: false, - AttachStdin: false, - UseNetworkAliases: true, - Labels: mergeLabels(service.Labels, service.CustomLabels), - } - ctr, err := c.compose.createContainer(ctx, project, service, name, number, opts) - updated[actual+i] = ctr - return err - })) - continue - } - - err = eg.Wait() - c.setObservedState(service.Name, updated) - return err -} - -func (c *convergence) stopDependentContainers(ctx context.Context, project *types.Project, service types.ServiceConfig) error { - // Stop dependent containers, so they will be restarted after service is re-created - dependents := project.GetDependentsForService(service, func(dependency types.ServiceDependency) bool { - return dependency.Restart - }) - if len(dependents) == 0 { - return nil - } - err := c.compose.stop(ctx, project.Name, api.StopOptions{ - Services: dependents, - Project: project, - }, nil) - if err != nil { - return err - } - - for _, name := range dependents { - dependentStates := c.getObservedState(name) - for i, dependent := range dependentStates { - dependent.State = container.StateExited - dependentStates[i] = dependent - } - c.setObservedState(name, dependentStates) - } - return nil -} - -func getScale(config types.ServiceConfig) (int, error) { - scale := config.GetScale() - if scale > 1 && config.ContainerName != "" { - return 0, fmt.Errorf(doubledContainerNameWarning, - config.Name, - config.ContainerName) - } - return scale, nil -} - -// resolveServiceReferences replaces reference to another service with reference to an actual container -func (c *convergence) resolveServiceReferences(service *types.ServiceConfig) error { - err := c.resolveVolumeFrom(service) - if err != nil { - return err - } - - err = c.resolveSharedNamespaces(service) - if err != nil { - return err - } - return nil -} - -func (c *convergence) resolveVolumeFrom(service *types.ServiceConfig) error { - for i, vol := range service.VolumesFrom { - spec := strings.Split(vol, ":") - if len(spec) == 0 { - continue - } - if spec[0] == "container" { - service.VolumesFrom[i] = spec[1] - continue - } - name := spec[0] - dependencies := c.getObservedState(name) - if len(dependencies) == 0 { - return fmt.Errorf("cannot share volume with service %s: container missing", name) - } - service.VolumesFrom[i] = dependencies.sorted()[0].ID - } - return nil -} - -func (c *convergence) resolveSharedNamespaces(service *types.ServiceConfig) error { - str := service.NetworkMode - if name := getDependentServiceFromMode(str); name != "" { - dependencies := c.getObservedState(name) - if len(dependencies) == 0 { - return fmt.Errorf("cannot share network namespace with service %s: container missing", name) - } - service.NetworkMode = types.ContainerPrefix + dependencies.sorted()[0].ID - } - - str = service.Ipc - if name := getDependentServiceFromMode(str); name != "" { - dependencies := c.getObservedState(name) - if len(dependencies) == 0 { - return fmt.Errorf("cannot share IPC namespace with service %s: container missing", name) - } - service.Ipc = types.ContainerPrefix + dependencies.sorted()[0].ID - } - - str = service.Pid - if name := getDependentServiceFromMode(str); name != "" { - dependencies := c.getObservedState(name) - if len(dependencies) == 0 { - return fmt.Errorf("cannot share PID namespace with service %s: container missing", name) - } - service.Pid = types.ContainerPrefix + dependencies.sorted()[0].ID - } - - return nil -} - -func (c *convergence) mustRecreate(expected types.ServiceConfig, actual container.Summary, policy string) (bool, error) { - if policy == api.RecreateNever { - return false, nil - } - if policy == api.RecreateForce { - return true, nil - } - configHash, err := ServiceHash(expected) - if err != nil { - return false, err - } - configChanged := actual.Labels[api.ConfigHashLabel] != configHash - imageUpdated := actual.Labels[api.ImageDigestLabel] != expected.CustomLabels[api.ImageDigestLabel] - if configChanged || imageUpdated { - return true, nil - } - - if c.networks != nil && actual.State == "running" { - if checkExpectedNetworks(expected, actual, c.networks) { - return true, nil - } - } - - if c.volumes != nil { - if checkExpectedVolumes(expected, actual, c.volumes) { - return true, nil - } - } - - return false, nil -} - -func checkExpectedNetworks(expected types.ServiceConfig, actual container.Summary, networks map[string]string) bool { - // check the networks container is connected to are the expected ones - for net := range expected.Networks { - id := networks[net] - if id == "swarm" { - // corner-case : swarm overlay network isn't visible until a container is attached - continue - } - found := false - for _, settings := range actual.NetworkSettings.Networks { - if settings.NetworkID == id { - found = true - break - } - } - if !found { - // config is up-to-date but container is not connected to network - return true - } - } - return false -} - -func checkExpectedVolumes(expected types.ServiceConfig, actual container.Summary, volumes map[string]string) bool { - // check container's volume mounts and search for the expected ones - for _, vol := range expected.Volumes { - if vol.Type != string(mmount.TypeVolume) { - continue - } - if vol.Source == "" { - continue - } - id := volumes[vol.Source] - found := false - for _, mount := range actual.Mounts { - if mount.Type != mmount.TypeVolume { - continue - } - if mount.Name == id { - found = true - break - } - } - if !found { - // config is up-to-date but container doesn't have volume mounted - return true - } - } - return false -} - -func getContainerName(projectName string, service types.ServiceConfig, number int) string { - name := getDefaultContainerName(projectName, service.Name, strconv.Itoa(number)) - if service.ContainerName != "" { - name = service.ContainerName - } - return name -} - -func getDefaultContainerName(projectName, serviceName, index string) string { - return strings.Join([]string{projectName, serviceName, index}, api.Separator) -} - -func getContainerProgressName(ctr container.Summary) string { - return "Container " + getCanonicalContainerName(ctr) -} - -func containerEvents(containers Containers, eventFunc func(string) api.Resource) []api.Resource { - events := []api.Resource{} - for _, ctr := range containers { - events = append(events, eventFunc(getContainerProgressName(ctr))) - } - return events -} - -func containerReasonEvents(containers Containers, eventFunc func(string, string) api.Resource, reason string) []api.Resource { - events := []api.Resource{} - for _, ctr := range containers { - events = append(events, eventFunc(getContainerProgressName(ctr), reason)) - } - return events -} - -// ServiceConditionRunningOrHealthy is a service condition on status running or healthy -const ServiceConditionRunningOrHealthy = "running_or_healthy" - -//nolint:gocyclo -func (s *composeService) waitDependencies(ctx context.Context, project *types.Project, dependant string, dependencies types.DependsOnConfig, containers Containers, timeout time.Duration) error { - if timeout > 0 { - withTimeout, cancelFunc := context.WithTimeout(ctx, timeout) - defer cancelFunc() - ctx = withTimeout - } - eg, ctx := errgroup.WithContext(ctx) - for dep, config := range dependencies { - if shouldWait, err := shouldWaitForDependency(dep, config, project); err != nil { - return err - } else if !shouldWait { - continue - } - - waitingFor := containers.filter(isService(dep), isNotOneOff) - s.events.On(containerEvents(waitingFor, waiting)...) - if len(waitingFor) == 0 { - if config.Required { - return fmt.Errorf("%s is missing dependency %s", dependant, dep) - } - logrus.Warnf("%s is missing dependency %s", dependant, dep) - continue - } - - eg.Go(func() error { - ticker := time.NewTicker(500 * time.Millisecond) - defer ticker.Stop() - for { - select { - case <-ticker.C: - case <-ctx.Done(): - return nil - } - switch config.Condition { - case ServiceConditionRunningOrHealthy: - isHealthy, err := s.isServiceHealthy(ctx, waitingFor, true) - if err != nil { - if !config.Required { - s.events.On(containerReasonEvents(waitingFor, skippedEvent, - fmt.Sprintf("optional dependency %q is not running or is unhealthy", dep))...) - logrus.Warnf("optional dependency %q is not running or is unhealthy: %s", dep, err.Error()) - return nil - } - return err - } - if isHealthy { - s.events.On(containerEvents(waitingFor, healthy)...) - return nil - } - case types.ServiceConditionHealthy: - isHealthy, err := s.isServiceHealthy(ctx, waitingFor, false) - if err != nil { - if !config.Required { - s.events.On(containerReasonEvents(waitingFor, skippedEvent, - fmt.Sprintf("optional dependency %q failed to start", dep))...) - logrus.Warnf("optional dependency %q failed to start: %s", dep, err.Error()) - return nil - } - s.events.On(containerEvents(waitingFor, func(s string) api.Resource { - return errorEventf(s, "dependency %s failed to start", dep) - })...) - return fmt.Errorf("dependency failed to start: %w", err) - } - if isHealthy { - s.events.On(containerEvents(waitingFor, healthy)...) - return nil - } - case types.ServiceConditionCompletedSuccessfully: - isExited, code, err := s.isServiceCompleted(ctx, waitingFor) - if err != nil { - return err - } - if isExited { - if code == 0 { - s.events.On(containerEvents(waitingFor, exited)...) - return nil - } - - messageSuffix := fmt.Sprintf("%q didn't complete successfully: exit %d", dep, code) - if !config.Required { - // optional -> mark as skipped & don't propagate error - s.events.On(containerReasonEvents(waitingFor, skippedEvent, - fmt.Sprintf("optional dependency %s", messageSuffix))...) - logrus.Warnf("optional dependency %s", messageSuffix) - return nil - } - - msg := fmt.Sprintf("service %s", messageSuffix) - s.events.On(containerEvents(waitingFor, func(s string) api.Resource { - return errorEventf(s, "service %s", messageSuffix) - })...) - return errors.New(msg) - } - default: - logrus.Warnf("unsupported depends_on condition: %s", config.Condition) - return nil - } - } - }) - } - err := eg.Wait() - if errors.Is(err, context.DeadlineExceeded) { - return fmt.Errorf("timeout waiting for dependencies") - } - return err -} - -func shouldWaitForDependency(serviceName string, dependencyConfig types.ServiceDependency, project *types.Project) (bool, error) { - if dependencyConfig.Condition == types.ServiceConditionStarted { - // already managed by InDependencyOrder - return false, nil - } - if service, err := project.GetService(serviceName); err != nil { - for _, ds := range project.DisabledServices { - if ds.Name == serviceName { - // don't wait for disabled service (--no-deps) - return false, nil - } - } - return false, err - } else if service.GetScale() == 0 { - // don't wait for the dependency which configured to have 0 containers running - return false, nil - } else if service.Provider != nil { - // don't wait for provider services - return false, nil - } - return true, nil -} - -func nextContainerNumber(containers []container.Summary) int { - maxNumber := 0 - for _, c := range containers { - s, ok := c.Labels[api.ContainerNumberLabel] - if !ok { - logrus.Warnf("container %s is missing %s label", c.ID, api.ContainerNumberLabel) - } - n, err := strconv.Atoi(s) - if err != nil { - logrus.Warnf("container %s has invalid %s label: %s", c.ID, api.ContainerNumberLabel, s) - continue - } - if n > maxNumber { - maxNumber = n - } - } - return maxNumber + 1 -} - -func (s *composeService) createContainer(ctx context.Context, project *types.Project, service types.ServiceConfig, - name string, number int, opts createOptions, -) (ctr container.Summary, err error) { - eventName := "Container " + name - s.events.On(creatingEvent(eventName)) - ctr, err = s.createMobyContainer(ctx, project, service, name, number, nil, opts) - if err != nil { - if ctx.Err() == nil { - s.events.On(api.Resource{ - ID: eventName, - Status: api.Error, - Text: err.Error(), - }) - } - return ctr, err - } - s.events.On(createdEvent(eventName)) - return ctr, nil -} - -func (s *composeService) recreateContainer(ctx context.Context, project *types.Project, service types.ServiceConfig, - replaced container.Summary, inherit bool, timeout *time.Duration, -) (created container.Summary, err error) { - eventName := getContainerProgressName(replaced) - s.events.On(newEvent(eventName, api.Working, "Recreate")) - defer func() { - if err != nil && ctx.Err() == nil { - s.events.On(api.Resource{ - ID: eventName, - Status: api.Error, - Text: err.Error(), - }) - } - }() - - number, err := strconv.Atoi(replaced.Labels[api.ContainerNumberLabel]) - if err != nil { - return created, err - } - - var inherited *container.Summary - if inherit { - inherited = &replaced - } - - replacedContainerName := service.ContainerName - if replacedContainerName == "" { - replacedContainerName = service.Name + api.Separator + strconv.Itoa(number) - } - name := getContainerName(project.Name, service, number) - tmpName := fmt.Sprintf("%s_%s", replaced.ID[:12], name) - opts := createOptions{ - AutoRemove: false, - AttachStdin: false, - UseNetworkAliases: true, - Labels: mergeLabels(service.Labels, service.CustomLabels).Add(api.ContainerReplaceLabel, replacedContainerName), - } - created, err = s.createMobyContainer(ctx, project, service, tmpName, number, inherited, opts) - if err != nil { - return created, err - } - - timeoutInSecond := utils.DurationSecondToInt(timeout) - err = s.apiClient().ContainerStop(ctx, replaced.ID, container.StopOptions{Timeout: timeoutInSecond}) - if err != nil { - return created, err - } - - err = s.apiClient().ContainerRemove(ctx, replaced.ID, container.RemoveOptions{}) - if err != nil { - return created, err - } - - err = s.apiClient().ContainerRename(ctx, tmpName, name) - if err != nil { - return created, err - } - - s.events.On(newEvent(eventName, api.Done, "Recreated")) - return created, err -} - -// force sequential calls to ContainerStart to prevent race condition in engine assigning ports from ranges -var startMx sync.Mutex - -func (s *composeService) startContainer(ctx context.Context, ctr container.Summary) error { - s.events.On(newEvent(getContainerProgressName(ctr), api.Working, "Restart")) - startMx.Lock() - defer startMx.Unlock() - err := s.apiClient().ContainerStart(ctx, ctr.ID, container.StartOptions{}) - if err != nil { - return err - } - s.events.On(newEvent(getContainerProgressName(ctr), api.Done, "Restarted")) - return nil -} - -func (s *composeService) createMobyContainer(ctx context.Context, project *types.Project, service types.ServiceConfig, - name string, number int, inherit *container.Summary, opts createOptions, -) (container.Summary, error) { - var created container.Summary - cfgs, err := s.getCreateConfigs(ctx, project, service, number, inherit, opts) - if err != nil { - return created, err - } - platform := service.Platform - if platform == "" { - platform = project.Environment["DOCKER_DEFAULT_PLATFORM"] - } - var plat *specs.Platform - if platform != "" { - var p specs.Platform - p, err = platforms.Parse(platform) - if err != nil { - return created, err - } - plat = &p - } - - response, err := s.apiClient().ContainerCreate(ctx, cfgs.Container, cfgs.Host, cfgs.Network, plat, name) - if err != nil { - return created, err - } - for _, warning := range response.Warnings { - s.events.On(api.Resource{ - ID: service.Name, - Status: api.Warning, - Text: warning, - }) - } - inspectedContainer, err := s.apiClient().ContainerInspect(ctx, response.ID) - if err != nil { - return created, err - } - created = container.Summary{ - ID: inspectedContainer.ID, - Labels: inspectedContainer.Config.Labels, - Names: []string{inspectedContainer.Name}, - NetworkSettings: &container.NetworkSettingsSummary{ - Networks: inspectedContainer.NetworkSettings.Networks, - }, - } - - apiVersion, err := s.RuntimeVersion(ctx) - if err != nil { - return created, err - } - // Starting API version 1.44, the ContainerCreate API call takes multiple networks - // so we include all the configurations there and can skip the one-by-one calls here - if versions.LessThan(apiVersion, APIVersion144) { - // the highest-priority network is the primary and is included in the ContainerCreate API - // call via container.NetworkMode & network.NetworkingConfig - // any remaining networks are connected one-by-one here after creation (but before start) - serviceNetworks := service.NetworksByPriority() - for _, networkKey := range serviceNetworks { - mobyNetworkName := project.Networks[networkKey].Name - if string(cfgs.Host.NetworkMode) == mobyNetworkName { - // primary network already configured as part of ContainerCreate - continue - } - epSettings := createEndpointSettings(project, service, number, networkKey, cfgs.Links, opts.UseNetworkAliases) - if err := s.apiClient().NetworkConnect(ctx, mobyNetworkName, created.ID, epSettings); err != nil { - return created, err - } - } - } - return created, nil -} - -// getLinks mimics V1 compose/service.py::Service::_get_links() -func (s *composeService) getLinks(ctx context.Context, projectName string, service types.ServiceConfig, number int) ([]string, error) { - var links []string - format := func(k, v string) string { - return fmt.Sprintf("%s:%s", k, v) - } - getServiceContainers := func(serviceName string) (Containers, error) { - return s.getContainers(ctx, projectName, oneOffExclude, true, serviceName) - } - - for _, rawLink := range service.Links { - // linkName if informed like in: "serviceName[:linkName]" - linkServiceName, linkName, ok := strings.Cut(rawLink, ":") - if !ok { - linkName = linkServiceName - } - cnts, err := getServiceContainers(linkServiceName) - if err != nil { - return nil, err - } - for _, c := range cnts { - containerName := getCanonicalContainerName(c) - links = append(links, - format(containerName, linkName), - format(containerName, linkServiceName+api.Separator+strconv.Itoa(number)), - format(containerName, strings.Join([]string{projectName, linkServiceName, strconv.Itoa(number)}, api.Separator)), - ) - } - } - - if service.Labels[api.OneoffLabel] == "True" { - cnts, err := getServiceContainers(service.Name) - if err != nil { - return nil, err - } - for _, c := range cnts { - containerName := getCanonicalContainerName(c) - links = append(links, - format(containerName, service.Name), - format(containerName, strings.TrimPrefix(containerName, projectName+api.Separator)), - format(containerName, containerName), - ) - } - } - - for _, rawExtLink := range service.ExternalLinks { - externalLink, linkName, ok := strings.Cut(rawExtLink, ":") - if !ok { - linkName = externalLink - } - links = append(links, format(externalLink, linkName)) - } - return links, nil -} - -func (s *composeService) isServiceHealthy(ctx context.Context, containers Containers, fallbackRunning bool) (bool, error) { - for _, c := range containers { - ctr, err := s.apiClient().ContainerInspect(ctx, c.ID) - if err != nil { - return false, err - } - name := ctr.Name[1:] - - if ctr.State.Status == container.StateExited { - return false, fmt.Errorf("container %s exited (%d)", name, ctr.State.ExitCode) - } - - noHealthcheck := ctr.Config.Healthcheck == nil || (len(ctr.Config.Healthcheck.Test) > 0 && ctr.Config.Healthcheck.Test[0] == "NONE") - if noHealthcheck && fallbackRunning { - // Container does not define a health check, but we can fall back to "running" state - return ctr.State != nil && ctr.State.Status == container.StateRunning, nil - } - - if ctr.State == nil || ctr.State.Health == nil { - return false, fmt.Errorf("container %s has no healthcheck configured", name) - } - switch ctr.State.Health.Status { - case container.Healthy: - // Continue by checking the next container. - case container.Unhealthy: - return false, fmt.Errorf("container %s is unhealthy", name) - case container.Starting: - return false, nil - default: - return false, fmt.Errorf("container %s had unexpected health status %q", name, ctr.State.Health.Status) - } - } - return true, nil -} - -func (s *composeService) isServiceCompleted(ctx context.Context, containers Containers) (bool, int, error) { - for _, c := range containers { - ctr, err := s.apiClient().ContainerInspect(ctx, c.ID) - if err != nil { - return false, 0, err - } - if ctr.State != nil && ctr.State.Status == container.StateExited { - return true, ctr.State.ExitCode, nil - } - } - return false, 0, nil -} - -func (s *composeService) startService(ctx context.Context, - project *types.Project, service types.ServiceConfig, - containers Containers, listener api.ContainerEventListener, - timeout time.Duration, -) error { - if service.Deploy != nil && service.Deploy.Replicas != nil && *service.Deploy.Replicas == 0 { - return nil - } - - err := s.waitDependencies(ctx, project, service.Name, service.DependsOn, containers, timeout) - if err != nil { - return err - } - - if len(containers) == 0 { - if service.GetScale() == 0 { - return nil - } - return fmt.Errorf("service %q has no container to start", service.Name) - } - - for _, ctr := range containers.filter(isService(service.Name)) { - if ctr.State == container.StateRunning { - continue - } - - err = s.injectSecrets(ctx, project, service, ctr.ID) - if err != nil { - return err - } - - err = s.injectConfigs(ctx, project, service, ctr.ID) - if err != nil { - return err - } - - eventName := getContainerProgressName(ctr) - s.events.On(startingEvent(eventName)) - err = s.apiClient().ContainerStart(ctx, ctr.ID, container.StartOptions{}) - if err != nil { - return err - } - - for _, hook := range service.PostStart { - err = s.runHook(ctx, ctr, service, hook, listener) - if err != nil { - return err - } - } - - s.events.On(startedEvent(eventName)) - } - return nil -} - -func mergeLabels(ls ...types.Labels) types.Labels { - merged := types.Labels{} - for _, l := range ls { - maps.Copy(merged, l) - } - return merged -} diff --git a/pkg/compose/convergence_test.go b/pkg/compose/convergence_test.go deleted file mode 100644 index eb4c1adfbde..00000000000 --- a/pkg/compose/convergence_test.go +++ /dev/null @@ -1,568 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "fmt" - "strings" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/config/configfile" - moby "github.com/docker/docker/api/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/api/types/network" - "github.com/docker/go-connections/nat" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/mocks" -) - -func TestContainerName(t *testing.T) { - s := types.ServiceConfig{ - Name: "testservicename", - ContainerName: "testcontainername", - Scale: intPtr(1), - Deploy: &types.DeployConfig{}, - } - ret, err := getScale(s) - assert.NilError(t, err) - assert.Equal(t, ret, *s.Scale) - - s.Scale = intPtr(0) - ret, err = getScale(s) - assert.NilError(t, err) - assert.Equal(t, ret, *s.Scale) - - s.Scale = intPtr(2) - _, err = getScale(s) - assert.Error(t, err, fmt.Sprintf(doubledContainerNameWarning, s.Name, s.ContainerName)) -} - -func intPtr(i int) *int { - return &i -} - -func TestServiceLinks(t *testing.T) { - const dbContainerName = "/" + testProject + "-db-1" - const webContainerName = "/" + testProject + "-web-1" - s := types.ServiceConfig{ - Name: "web", - Scale: intPtr(1), - } - - containerListOptions := container.ListOptions{ - Filters: filters.NewArgs( - projectFilter(testProject), - serviceFilter("db"), - oneOffFilter(false), - hasConfigHashLabel(), - ), - All: true, - } - - t.Run("service links default", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - s.Links = []string{"db"} - - c := testContainer("db", dbContainerName, false) - apiClient.EXPECT().ContainerList(gomock.Any(), containerListOptions).Return([]container.Summary{c}, nil) - - links, err := tested.(*composeService).getLinks(t.Context(), testProject, s, 1) - assert.NilError(t, err) - - assert.Equal(t, len(links), 3) - assert.Equal(t, links[0], "testProject-db-1:db") - assert.Equal(t, links[1], "testProject-db-1:db-1") - assert.Equal(t, links[2], "testProject-db-1:testProject-db-1") - }) - - t.Run("service links", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - s.Links = []string{"db:db"} - - c := testContainer("db", dbContainerName, false) - - apiClient.EXPECT().ContainerList(gomock.Any(), containerListOptions).Return([]container.Summary{c}, nil) - links, err := tested.(*composeService).getLinks(t.Context(), testProject, s, 1) - assert.NilError(t, err) - - assert.Equal(t, len(links), 3) - assert.Equal(t, links[0], "testProject-db-1:db") - assert.Equal(t, links[1], "testProject-db-1:db-1") - assert.Equal(t, links[2], "testProject-db-1:testProject-db-1") - }) - - t.Run("service links name", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - s.Links = []string{"db:dbname"} - - c := testContainer("db", dbContainerName, false) - apiClient.EXPECT().ContainerList(gomock.Any(), containerListOptions).Return([]container.Summary{c}, nil) - - links, err := tested.(*composeService).getLinks(t.Context(), testProject, s, 1) - assert.NilError(t, err) - - assert.Equal(t, len(links), 3) - assert.Equal(t, links[0], "testProject-db-1:dbname") - assert.Equal(t, links[1], "testProject-db-1:db-1") - assert.Equal(t, links[2], "testProject-db-1:testProject-db-1") - }) - - t.Run("service links external links", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - s.Links = []string{"db:dbname"} - s.ExternalLinks = []string{"db1:db2"} - - c := testContainer("db", dbContainerName, false) - apiClient.EXPECT().ContainerList(gomock.Any(), containerListOptions).Return([]container.Summary{c}, nil) - - links, err := tested.(*composeService).getLinks(t.Context(), testProject, s, 1) - assert.NilError(t, err) - - assert.Equal(t, len(links), 4) - assert.Equal(t, links[0], "testProject-db-1:dbname") - assert.Equal(t, links[1], "testProject-db-1:db-1") - assert.Equal(t, links[2], "testProject-db-1:testProject-db-1") - - // ExternalLink - assert.Equal(t, links[3], "db1:db2") - }) - - t.Run("service links itself oneoff", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - s.Links = []string{} - s.ExternalLinks = []string{} - s.Labels = s.Labels.Add(api.OneoffLabel, "True") - - c := testContainer("web", webContainerName, true) - containerListOptionsOneOff := container.ListOptions{ - Filters: filters.NewArgs( - projectFilter(testProject), - serviceFilter("web"), - oneOffFilter(false), - hasConfigHashLabel(), - ), - All: true, - } - apiClient.EXPECT().ContainerList(gomock.Any(), containerListOptionsOneOff).Return([]container.Summary{c}, nil) - - links, err := tested.(*composeService).getLinks(t.Context(), testProject, s, 1) - assert.NilError(t, err) - - assert.Equal(t, len(links), 3) - assert.Equal(t, links[0], "testProject-web-1:web") - assert.Equal(t, links[1], "testProject-web-1:web-1") - assert.Equal(t, links[2], "testProject-web-1:testProject-web-1") - }) -} - -func TestWaitDependencies(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - t.Run("should skip dependencies with scale 0", func(t *testing.T) { - dbService := types.ServiceConfig{Name: "db", Scale: intPtr(0)} - redisService := types.ServiceConfig{Name: "redis", Scale: intPtr(0)} - project := types.Project{Name: strings.ToLower(testProject), Services: types.Services{ - "db": dbService, - "redis": redisService, - }} - dependencies := types.DependsOnConfig{ - "db": {Condition: ServiceConditionRunningOrHealthy}, - "redis": {Condition: ServiceConditionRunningOrHealthy}, - } - assert.NilError(t, tested.(*composeService).waitDependencies(t.Context(), &project, "", dependencies, nil, 0)) - }) - t.Run("should skip dependencies with condition service_started", func(t *testing.T) { - dbService := types.ServiceConfig{Name: "db", Scale: intPtr(1)} - redisService := types.ServiceConfig{Name: "redis", Scale: intPtr(1)} - project := types.Project{Name: strings.ToLower(testProject), Services: types.Services{ - "db": dbService, - "redis": redisService, - }} - dependencies := types.DependsOnConfig{ - "db": {Condition: types.ServiceConditionStarted, Required: true}, - "redis": {Condition: types.ServiceConditionStarted, Required: true}, - } - assert.NilError(t, tested.(*composeService).waitDependencies(t.Context(), &project, "", dependencies, nil, 0)) - }) -} - -func TestIsServiceHealthy(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - ctx := t.Context() - - t.Run("disabled healthcheck with fallback to running", func(t *testing.T) { - containerID := "test-container-id" - containers := Containers{ - {ID: containerID}, - } - - // Container with disabled healthcheck (Test: ["NONE"]) - apiClient.EXPECT().ContainerInspect(ctx, containerID).Return(container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: containerID, - Name: "test-container", - State: &container.State{Status: "running"}, - }, - Config: &container.Config{ - Healthcheck: &container.HealthConfig{ - Test: []string{"NONE"}, - }, - }, - }, nil) - - isHealthy, err := tested.(*composeService).isServiceHealthy(ctx, containers, true) - assert.NilError(t, err) - assert.Equal(t, true, isHealthy, "Container with disabled healthcheck should be considered healthy when running with fallbackRunning=true") - }) - - t.Run("disabled healthcheck without fallback", func(t *testing.T) { - containerID := "test-container-id" - containers := Containers{ - {ID: containerID}, - } - - // Container with disabled healthcheck (Test: ["NONE"]) but fallbackRunning=false - apiClient.EXPECT().ContainerInspect(ctx, containerID).Return(container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: containerID, - Name: "test-container", - State: &container.State{Status: "running"}, - }, - Config: &container.Config{ - Healthcheck: &container.HealthConfig{ - Test: []string{"NONE"}, - }, - }, - }, nil) - - _, err := tested.(*composeService).isServiceHealthy(ctx, containers, false) - assert.ErrorContains(t, err, "has no healthcheck configured") - }) - - t.Run("no healthcheck with fallback to running", func(t *testing.T) { - containerID := "test-container-id" - containers := Containers{ - {ID: containerID}, - } - - // Container with no healthcheck at all - apiClient.EXPECT().ContainerInspect(ctx, containerID).Return(container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: containerID, - Name: "test-container", - State: &container.State{Status: "running"}, - }, - Config: &container.Config{ - Healthcheck: nil, - }, - }, nil) - - isHealthy, err := tested.(*composeService).isServiceHealthy(ctx, containers, true) - assert.NilError(t, err) - assert.Equal(t, true, isHealthy, "Container with no healthcheck should be considered healthy when running with fallbackRunning=true") - }) - - t.Run("exited container with disabled healthcheck", func(t *testing.T) { - containerID := "test-container-id" - containers := Containers{ - {ID: containerID}, - } - - // Container with disabled healthcheck but exited - apiClient.EXPECT().ContainerInspect(ctx, containerID).Return(container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: containerID, - Name: "test-container", - State: &container.State{ - Status: "exited", - ExitCode: 1, - }, - }, - Config: &container.Config{ - Healthcheck: &container.HealthConfig{ - Test: []string{"NONE"}, - }, - }, - }, nil) - - _, err := tested.(*composeService).isServiceHealthy(ctx, containers, true) - assert.ErrorContains(t, err, "exited") - }) - - t.Run("healthy container with healthcheck", func(t *testing.T) { - containerID := "test-container-id" - containers := Containers{ - {ID: containerID}, - } - - // Container with actual healthcheck that is healthy - apiClient.EXPECT().ContainerInspect(ctx, containerID).Return(container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: containerID, - Name: "test-container", - State: &container.State{ - Status: "running", - Health: &container.Health{ - Status: container.Healthy, - }, - }, - }, - Config: &container.Config{ - Healthcheck: &container.HealthConfig{ - Test: []string{"CMD", "curl", "-f", "http://localhost"}, - }, - }, - }, nil) - - isHealthy, err := tested.(*composeService).isServiceHealthy(ctx, containers, false) - assert.NilError(t, err) - assert.Equal(t, true, isHealthy, "Container with healthy status should be healthy") - }) -} - -func TestCreateMobyContainer(t *testing.T) { - t.Run("connects container networks one by one if API <1.44", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - cli.EXPECT().ConfigFile().Return(&configfile.ConfigFile{}).AnyTimes() - apiClient.EXPECT().DaemonHost().Return("").AnyTimes() - apiClient.EXPECT().ImageInspect(gomock.Any(), gomock.Any()).Return(image.InspectResponse{}, nil).AnyTimes() - // force `RuntimeVersion` to fetch again - runtimeVersion = runtimeVersionCache{} - apiClient.EXPECT().ServerVersion(gomock.Any()).Return(moby.Version{ - APIVersion: "1.43", - }, nil).AnyTimes() - - service := types.ServiceConfig{ - Name: "test", - Networks: map[string]*types.ServiceNetworkConfig{ - "a": { - Priority: 10, - }, - "b": { - Priority: 100, - }, - }, - } - project := types.Project{ - Name: "bork", - Services: types.Services{ - "test": service, - }, - Networks: types.Networks{ - "a": types.NetworkConfig{ - Name: "a-moby-name", - }, - "b": types.NetworkConfig{ - Name: "b-moby-name", - }, - }, - } - - var falseBool bool - apiClient.EXPECT().ContainerCreate(gomock.Any(), gomock.Any(), gomock.Eq( - &container.HostConfig{ - PortBindings: nat.PortMap{}, - ExtraHosts: []string{}, - Tmpfs: map[string]string{}, - Resources: container.Resources{ - OomKillDisable: &falseBool, - }, - NetworkMode: "b-moby-name", - }), gomock.Eq( - &network.NetworkingConfig{ - EndpointsConfig: map[string]*network.EndpointSettings{ - "b-moby-name": { - IPAMConfig: &network.EndpointIPAMConfig{}, - Aliases: []string{"bork-test-0"}, - }, - }, - }), gomock.Any(), gomock.Any()).Times(1).Return( - container.CreateResponse{ - ID: "an-id", - }, nil) - - apiClient.EXPECT().ContainerInspect(gomock.Any(), gomock.Eq("an-id")).Times(1).Return( - container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: "an-id", - Name: "a-name", - }, - Config: &container.Config{}, - NetworkSettings: &container.NetworkSettings{}, - }, nil) - - apiClient.EXPECT().NetworkConnect(gomock.Any(), "a-moby-name", "an-id", gomock.Eq( - &network.EndpointSettings{ - IPAMConfig: &network.EndpointIPAMConfig{}, - Aliases: []string{"bork-test-0"}, - })) - - _, err = tested.(*composeService).createMobyContainer(t.Context(), &project, service, "test", 0, nil, createOptions{ - Labels: make(types.Labels), - }) - assert.NilError(t, err) - }) - - t.Run("includes all container networks in ContainerCreate call if API >=1.44", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - cli.EXPECT().ConfigFile().Return(&configfile.ConfigFile{}).AnyTimes() - apiClient.EXPECT().DaemonHost().Return("").AnyTimes() - apiClient.EXPECT().ImageInspect(gomock.Any(), gomock.Any()).Return(image.InspectResponse{}, nil).AnyTimes() - // force `RuntimeVersion` to fetch fresh version - runtimeVersion = runtimeVersionCache{} - apiClient.EXPECT().ServerVersion(gomock.Any()).Return(moby.Version{ - APIVersion: APIVersion144, - }, nil).AnyTimes() - - service := types.ServiceConfig{ - Name: "test", - Networks: map[string]*types.ServiceNetworkConfig{ - "a": { - Priority: 10, - }, - "b": { - Priority: 100, - }, - }, - } - project := types.Project{ - Name: "bork", - Services: types.Services{ - "test": service, - }, - Networks: types.Networks{ - "a": types.NetworkConfig{ - Name: "a-moby-name", - }, - "b": types.NetworkConfig{ - Name: "b-moby-name", - }, - }, - } - - var falseBool bool - apiClient.EXPECT().ContainerCreate(gomock.Any(), gomock.Any(), gomock.Eq( - &container.HostConfig{ - PortBindings: nat.PortMap{}, - ExtraHosts: []string{}, - Tmpfs: map[string]string{}, - Resources: container.Resources{ - OomKillDisable: &falseBool, - }, - NetworkMode: "b-moby-name", - }), gomock.Eq( - &network.NetworkingConfig{ - EndpointsConfig: map[string]*network.EndpointSettings{ - "a-moby-name": { - IPAMConfig: &network.EndpointIPAMConfig{}, - Aliases: []string{"bork-test-0"}, - }, - "b-moby-name": { - IPAMConfig: &network.EndpointIPAMConfig{}, - Aliases: []string{"bork-test-0"}, - }, - }, - }), gomock.Any(), gomock.Any()).Times(1).Return( - container.CreateResponse{ - ID: "an-id", - }, nil) - - apiClient.EXPECT().ContainerInspect(gomock.Any(), gomock.Eq("an-id")).Times(1).Return( - container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: "an-id", - Name: "a-name", - }, - Config: &container.Config{}, - NetworkSettings: &container.NetworkSettings{}, - }, nil) - - _, err = tested.(*composeService).createMobyContainer(t.Context(), &project, service, "test", 0, nil, createOptions{ - Labels: make(types.Labels), - }) - assert.NilError(t, err) - }) -} diff --git a/pkg/compose/convert.go b/pkg/compose/convert.go deleted file mode 100644 index fd8fb7b3fb8..00000000000 --- a/pkg/compose/convert.go +++ /dev/null @@ -1,103 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "time" - - compose "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/versions" -) - -// ToMobyEnv convert into []string -func ToMobyEnv(environment compose.MappingWithEquals) []string { - var env []string - for k, v := range environment { - if v == nil { - env = append(env, k) - } else { - env = append(env, fmt.Sprintf("%s=%s", k, *v)) - } - } - return env -} - -// ToMobyHealthCheck convert into container.HealthConfig -func (s *composeService) ToMobyHealthCheck(ctx context.Context, check *compose.HealthCheckConfig) (*container.HealthConfig, error) { - if check == nil { - return nil, nil - } - var ( - interval time.Duration - timeout time.Duration - period time.Duration - retries int - ) - if check.Interval != nil { - interval = time.Duration(*check.Interval) - } - if check.Timeout != nil { - timeout = time.Duration(*check.Timeout) - } - if check.StartPeriod != nil { - period = time.Duration(*check.StartPeriod) - } - if check.Retries != nil { - retries = int(*check.Retries) - } - test := check.Test - if check.Disable { - test = []string{"NONE"} - } - var startInterval time.Duration - if check.StartInterval != nil { - version, err := s.RuntimeVersion(ctx) - if err != nil { - return nil, err - } - if versions.LessThan(version, APIVersion144) { - return nil, fmt.Errorf("can't set healthcheck.start_interval as feature require Docker Engine %s or later", DockerEngineV25) - } else { - startInterval = time.Duration(*check.StartInterval) - } - if check.StartPeriod == nil { - // see https://github.com/moby/moby/issues/48874 - return nil, errors.New("healthcheck.start_interval requires healthcheck.start_period to be set") - } - } - return &container.HealthConfig{ - Test: test, - Interval: interval, - Timeout: timeout, - StartPeriod: period, - StartInterval: startInterval, - Retries: retries, - }, nil -} - -// ToSeconds convert into seconds -func ToSeconds(d *compose.Duration) *int { - if d == nil { - return nil - } - s := int(time.Duration(*d).Seconds()) - return &s -} diff --git a/pkg/compose/cp.go b/pkg/compose/cp.go deleted file mode 100644 index f912c0827af..00000000000 --- a/pkg/compose/cp.go +++ /dev/null @@ -1,336 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "io" - "os" - "path/filepath" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/docker/docker/api/types/container" - "github.com/moby/go-archive" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -type copyDirection int - -const ( - fromService copyDirection = 1 << iota - toService - acrossServices = fromService | toService -) - -func (s *composeService) Copy(ctx context.Context, projectName string, options api.CopyOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.copy(ctx, projectName, options) - }, "copy", s.events) -} - -func (s *composeService) copy(ctx context.Context, projectName string, options api.CopyOptions) error { - projectName = strings.ToLower(projectName) - srcService, srcPath := splitCpArg(options.Source) - destService, dstPath := splitCpArg(options.Destination) - - var direction copyDirection - var serviceName string - var copyFunc func(ctx context.Context, containerID string, srcPath string, dstPath string, opts api.CopyOptions) error - if srcService != "" { - direction |= fromService - serviceName = srcService - copyFunc = s.copyFromContainer - } - if destService != "" { - direction |= toService - serviceName = destService - copyFunc = s.copyToContainer - } - if direction == acrossServices { - return errors.New("copying between services is not supported") - } - - if direction == 0 { - return errors.New("unknown copy direction") - } - - containers, err := s.listContainersTargetedForCopy(ctx, projectName, options, direction, serviceName) - if err != nil { - return err - } - - g := errgroup.Group{} - for _, cont := range containers { - ctr := cont - g.Go(func() error { - name := getCanonicalContainerName(ctr) - var msg string - if direction == fromService { - msg = fmt.Sprintf("%s:%s to %s", name, srcPath, dstPath) - } else { - msg = fmt.Sprintf("%s to %s:%s", srcPath, name, dstPath) - } - s.events.On(api.Resource{ - ID: name, - Text: api.StatusCopying, - Details: msg, - Status: api.Working, - }) - if err := copyFunc(ctx, ctr.ID, srcPath, dstPath, options); err != nil { - return err - } - s.events.On(api.Resource{ - ID: name, - Text: api.StatusCopied, - Details: msg, - Status: api.Done, - }) - return nil - }) - } - - return g.Wait() -} - -func (s *composeService) listContainersTargetedForCopy(ctx context.Context, projectName string, options api.CopyOptions, direction copyDirection, serviceName string) (Containers, error) { - var containers Containers - var err error - switch { - case options.Index > 0: - ctr, err := s.getSpecifiedContainer(ctx, projectName, oneOffExclude, true, serviceName, options.Index) - if err != nil { - return nil, err - } - return append(containers, ctr), nil - default: - withOneOff := oneOffExclude - if options.All { - withOneOff = oneOffInclude - } - containers, err = s.getContainers(ctx, projectName, withOneOff, true, serviceName) - if err != nil { - return nil, err - } - - if len(containers) < 1 { - return nil, fmt.Errorf("no container found for service %q", serviceName) - } - if direction == fromService { - return containers[:1], err - } - return containers, err - } -} - -func (s *composeService) copyToContainer(ctx context.Context, containerID string, srcPath string, dstPath string, opts api.CopyOptions) error { - var err error - if srcPath != "-" { - // Get an absolute source path. - srcPath, err = resolveLocalPath(srcPath) - if err != nil { - return err - } - } - - // Prepare destination copy info by stat-ing the container path. - dstInfo := archive.CopyInfo{Path: dstPath} - dstStat, err := s.apiClient().ContainerStatPath(ctx, containerID, dstPath) - - // If the destination is a symbolic link, we should evaluate it. - if err == nil && dstStat.Mode&os.ModeSymlink != 0 { - linkTarget := dstStat.LinkTarget - if !isAbs(linkTarget) { - // Join with the parent directory. - dstParent, _ := archive.SplitPathDirEntry(dstPath) - linkTarget = filepath.Join(dstParent, linkTarget) - } - - dstInfo.Path = linkTarget - dstStat, err = s.apiClient().ContainerStatPath(ctx, containerID, linkTarget) - } - - // Validate the destination path - if err := command.ValidateOutputPathFileMode(dstStat.Mode); err != nil { - return fmt.Errorf(`destination "%s:%s" must be a directory or a regular file: %w`, containerID, dstPath, err) - } - - // Ignore any error and assume that the parent directory of the destination - // path exists, in which case the copy may still succeed. If there is any - // type of conflict (e.g., non-directory overwriting an existing directory - // or vice versa) the extraction will fail. If the destination simply did - // not exist, but the parent directory does, the extraction will still - // succeed. - if err == nil { - dstInfo.Exists, dstInfo.IsDir = true, dstStat.Mode.IsDir() - } - - var ( - content io.Reader - resolvedDstPath string - ) - - if srcPath == "-" { - content = s.stdin() - resolvedDstPath = dstInfo.Path - if !dstInfo.IsDir { - return fmt.Errorf("destination \"%s:%s\" must be a directory", containerID, dstPath) - } - } else { - // Prepare source copy info. - srcInfo, err := archive.CopyInfoSourcePath(srcPath, opts.FollowLink) - if err != nil { - return err - } - - srcArchive, err := archive.TarResource(srcInfo) - if err != nil { - return err - } - defer srcArchive.Close() //nolint:errcheck - - // With the stat info about the local source as well as the - // destination, we have enough information to know whether we need to - // alter the archive that we upload so that when the server extracts - // it to the specified directory in the container we get the desired - // copy behavior. - - // See comments in the implementation of `archive.PrepareArchiveCopy` - // for exactly what goes into deciding how and whether the source - // archive needs to be altered for the correct copy behavior when it is - // extracted. This function also infers from the source and destination - // info which directory to extract to, which may be the parent of the - // destination that the user specified. - // Don't create the archive if running in Dry Run mode - if !s.dryRun { - dstDir, preparedArchive, err := archive.PrepareArchiveCopy(srcArchive, srcInfo, dstInfo) - if err != nil { - return err - } - defer preparedArchive.Close() //nolint:errcheck - - resolvedDstPath = dstDir - content = preparedArchive - } - } - - options := container.CopyToContainerOptions{ - AllowOverwriteDirWithFile: false, - CopyUIDGID: opts.CopyUIDGID, - } - return s.apiClient().CopyToContainer(ctx, containerID, resolvedDstPath, content, options) -} - -func (s *composeService) copyFromContainer(ctx context.Context, containerID, srcPath, dstPath string, opts api.CopyOptions) error { - var err error - if dstPath != "-" { - // Get an absolute destination path. - dstPath, err = resolveLocalPath(dstPath) - if err != nil { - return err - } - } - - if err := command.ValidateOutputPath(dstPath); err != nil { - return err - } - - // if client requests to follow symbol link, then must decide target file to be copied - var rebaseName string - if opts.FollowLink { - srcStat, err := s.apiClient().ContainerStatPath(ctx, containerID, srcPath) - - // If the destination is a symbolic link, we should follow it. - if err == nil && srcStat.Mode&os.ModeSymlink != 0 { - linkTarget := srcStat.LinkTarget - if !isAbs(linkTarget) { - // Join with the parent directory. - srcParent, _ := archive.SplitPathDirEntry(srcPath) - linkTarget = filepath.Join(srcParent, linkTarget) - } - - linkTarget, rebaseName = archive.GetRebaseName(srcPath, linkTarget) - srcPath = linkTarget - } - } - - content, stat, err := s.apiClient().CopyFromContainer(ctx, containerID, srcPath) - if err != nil { - return err - } - defer content.Close() //nolint:errcheck - - if dstPath == "-" { - _, err = io.Copy(s.stdout(), content) - return err - } - - srcInfo := archive.CopyInfo{ - Path: srcPath, - Exists: true, - IsDir: stat.Mode.IsDir(), - RebaseName: rebaseName, - } - - preArchive := content - if srcInfo.RebaseName != "" { - _, srcBase := archive.SplitPathDirEntry(srcInfo.Path) - preArchive = archive.RebaseArchiveEntries(content, srcBase, srcInfo.RebaseName) - } - - return archive.CopyTo(preArchive, srcInfo, dstPath) -} - -// IsAbs is a platform-agnostic wrapper for filepath.IsAbs. -// -// On Windows, golang filepath.IsAbs does not consider a path \windows\system32 -// as absolute as it doesn't start with a drive-letter/colon combination. However, -// in docker we need to verify things such as WORKDIR /windows/system32 in -// a Dockerfile (which gets translated to \windows\system32 when being processed -// by the daemon). This SHOULD be treated as absolute from a docker processing -// perspective. -func isAbs(path string) bool { - return filepath.IsAbs(path) || strings.HasPrefix(path, string(os.PathSeparator)) -} - -func splitCpArg(arg string) (ctr, path string) { - if isAbs(arg) { - // Explicit local absolute path, e.g., `C:\foo` or `/foo`. - return "", arg - } - - ctr, path, ok := strings.Cut(arg, ":") - - if !ok || strings.HasPrefix(ctr, ".") { - // Either there's no `:` in the arg - // OR it's an explicit local relative path like `./file:name.txt`. - return "", arg - } - - return ctr, path -} - -func resolveLocalPath(localPath string) (absPath string, err error) { - if absPath, err = filepath.Abs(localPath); err != nil { - return absPath, err - } - return archive.PreserveTrailingDotOrSeparator(absPath, localPath), nil -} diff --git a/pkg/compose/create.go b/pkg/compose/create.go deleted file mode 100644 index f01c3e9cec4..00000000000 --- a/pkg/compose/create.go +++ /dev/null @@ -1,1649 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "context" - "encoding/json" - "errors" - "fmt" - "os" - "path/filepath" - "slices" - "strconv" - "strings" - - "github.com/compose-spec/compose-go/v2/paths" - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/docker/docker/api/types/blkiodev" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/mount" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/api/types/versions" - volumetypes "github.com/docker/docker/api/types/volume" - "github.com/docker/go-connections/nat" - "github.com/sirupsen/logrus" - cdi "tags.cncf.io/container-device-interface/pkg/parser" - - "github.com/docker/compose/v5/pkg/api" -) - -type createOptions struct { - AutoRemove bool - AttachStdin bool - UseNetworkAliases bool - Labels types.Labels -} - -type createConfigs struct { - Container *container.Config - Host *container.HostConfig - Network *network.NetworkingConfig - Links []string -} - -func (s *composeService) Create(ctx context.Context, project *types.Project, createOpts api.CreateOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.create(ctx, project, createOpts) - }, "create", s.events) -} - -func (s *composeService) create(ctx context.Context, project *types.Project, options api.CreateOptions) error { - if len(options.Services) == 0 { - options.Services = project.ServiceNames() - } - - err := project.CheckContainerNameUnicity() - if err != nil { - return err - } - - err = s.ensureImagesExists(ctx, project, options.Build, options.QuietPull) - if err != nil { - return err - } - - err = s.ensureModels(ctx, project, options.QuietPull) - if err != nil { - return err - } - - prepareNetworks(project) - - networks, err := s.ensureNetworks(ctx, project) - if err != nil { - return err - } - - volumes, err := s.ensureProjectVolumes(ctx, project) - if err != nil { - return err - } - - var observedState Containers - observedState, err = s.getContainers(ctx, project.Name, oneOffInclude, true) - if err != nil { - return err - } - orphans := observedState.filter(isOrphaned(project)) - if len(orphans) > 0 && !options.IgnoreOrphans { - if options.RemoveOrphans { - err := s.removeContainers(ctx, orphans, nil, nil, false) - if err != nil { - return err - } - } else { - logrus.Warnf("Found orphan containers (%s) for this project. If "+ - "you removed or renamed this service in your compose "+ - "file, you can run this command with the "+ - "--remove-orphans flag to clean it up.", orphans.names()) - } - } - - // Temporary implementation of use_api_socket until we get actual support inside docker engine - project, err = s.useAPISocket(project) - if err != nil { - return err - } - - return newConvergence(options.Services, observedState, networks, volumes, s).apply(ctx, project, options) -} - -func prepareNetworks(project *types.Project) { - for k, nw := range project.Networks { - nw.CustomLabels = nw.CustomLabels. - Add(api.NetworkLabel, k). - Add(api.ProjectLabel, project.Name). - Add(api.VersionLabel, api.ComposeVersion) - project.Networks[k] = nw - } -} - -func (s *composeService) ensureNetworks(ctx context.Context, project *types.Project) (map[string]string, error) { - networks := map[string]string{} - for name, nw := range project.Networks { - id, err := s.ensureNetwork(ctx, project, name, &nw) - if err != nil { - return nil, err - } - networks[name] = id - project.Networks[name] = nw - } - return networks, nil -} - -func (s *composeService) ensureProjectVolumes(ctx context.Context, project *types.Project) (map[string]string, error) { - ids := map[string]string{} - for k, volume := range project.Volumes { - volume.CustomLabels = volume.CustomLabels.Add(api.VolumeLabel, k) - volume.CustomLabels = volume.CustomLabels.Add(api.ProjectLabel, project.Name) - volume.CustomLabels = volume.CustomLabels.Add(api.VersionLabel, api.ComposeVersion) - id, err := s.ensureVolume(ctx, k, volume, project) - if err != nil { - return nil, err - } - ids[k] = id - } - - return ids, nil -} - -//nolint:gocyclo -func (s *composeService) getCreateConfigs(ctx context.Context, - p *types.Project, - service types.ServiceConfig, - number int, - inherit *container.Summary, - opts createOptions, -) (createConfigs, error) { - labels, err := s.prepareLabels(opts.Labels, service, number) - if err != nil { - return createConfigs{}, err - } - - var runCmd, entrypoint []string - if service.Command != nil { - runCmd = service.Command - } - if service.Entrypoint != nil { - entrypoint = service.Entrypoint - } - - var ( - tty = service.Tty - stdinOpen = service.StdinOpen - ) - - proxyConfig := types.MappingWithEquals(s.configFile().ParseProxyConfig(s.apiClient().DaemonHost(), nil)) - env := proxyConfig.OverrideBy(service.Environment) - - var mainNwName string - var mainNw *types.ServiceNetworkConfig - if len(service.Networks) > 0 { - mainNwName = service.NetworksByPriority()[0] - mainNw = service.Networks[mainNwName] - } - - macAddress, err := s.prepareContainerMACAddress(ctx, service, mainNw, mainNwName) - if err != nil { - return createConfigs{}, err - } - - healthcheck, err := s.ToMobyHealthCheck(ctx, service.HealthCheck) - if err != nil { - return createConfigs{}, err - } - - exposed, err := buildContainerPorts(service) - if err != nil { - return createConfigs{}, err - } - - containerConfig := container.Config{ - Hostname: service.Hostname, - Domainname: service.DomainName, - User: service.User, - ExposedPorts: exposed, - Tty: tty, - OpenStdin: stdinOpen, - StdinOnce: opts.AttachStdin && stdinOpen, - AttachStdin: opts.AttachStdin, - AttachStderr: true, - AttachStdout: true, - Cmd: runCmd, - Image: api.GetImageNameOrDefault(service, p.Name), - WorkingDir: service.WorkingDir, - Entrypoint: entrypoint, - NetworkDisabled: service.NetworkMode == "disabled", - MacAddress: macAddress, // Field is deprecated since API v1.44, but kept for compatibility with older API versions. - Labels: labels, - StopSignal: service.StopSignal, - Env: ToMobyEnv(env), - Healthcheck: healthcheck, - StopTimeout: ToSeconds(service.StopGracePeriod), - } // VOLUMES/MOUNTS/FILESYSTEMS - tmpfs := map[string]string{} - for _, t := range service.Tmpfs { - k, v, _ := strings.Cut(t, ":") - tmpfs[k] = v - } - binds, mounts, err := s.buildContainerVolumes(ctx, *p, service, inherit) - if err != nil { - return createConfigs{}, err - } - - // NETWORKING - links, err := s.getLinks(ctx, p.Name, service, number) - if err != nil { - return createConfigs{}, err - } - apiVersion, err := s.RuntimeVersion(ctx) - if err != nil { - return createConfigs{}, err - } - networkMode, networkingConfig, err := defaultNetworkSettings(p, service, number, links, opts.UseNetworkAliases, apiVersion) - if err != nil { - return createConfigs{}, err - } - portBindings := buildContainerPortBindingOptions(service) - - // MISC - resources := getDeployResources(service) - var logConfig container.LogConfig - if service.Logging != nil { - logConfig = container.LogConfig{ - Type: service.Logging.Driver, - Config: service.Logging.Options, - } - } - securityOpts, unconfined, err := parseSecurityOpts(p, service.SecurityOpt) - if err != nil { - return createConfigs{}, err - } - - hostConfig := container.HostConfig{ - AutoRemove: opts.AutoRemove, - Annotations: service.Annotations, - Binds: binds, - Mounts: mounts, - CapAdd: service.CapAdd, - CapDrop: service.CapDrop, - NetworkMode: networkMode, - Init: service.Init, - IpcMode: container.IpcMode(service.Ipc), - CgroupnsMode: container.CgroupnsMode(service.Cgroup), - ReadonlyRootfs: service.ReadOnly, - RestartPolicy: getRestartPolicy(service), - ShmSize: int64(service.ShmSize), - Sysctls: service.Sysctls, - PortBindings: portBindings, - Resources: resources, - VolumeDriver: service.VolumeDriver, - VolumesFrom: service.VolumesFrom, - DNS: service.DNS, - DNSSearch: service.DNSSearch, - DNSOptions: service.DNSOpts, - ExtraHosts: service.ExtraHosts.AsList(":"), - SecurityOpt: securityOpts, - StorageOpt: service.StorageOpt, - UsernsMode: container.UsernsMode(service.UserNSMode), - UTSMode: container.UTSMode(service.Uts), - Privileged: service.Privileged, - PidMode: container.PidMode(service.Pid), - Tmpfs: tmpfs, - Isolation: container.Isolation(service.Isolation), - Runtime: service.Runtime, - LogConfig: logConfig, - GroupAdd: service.GroupAdd, - Links: links, - OomScoreAdj: int(service.OomScoreAdj), - } - - if unconfined { - hostConfig.MaskedPaths = []string{} - hostConfig.ReadonlyPaths = []string{} - } - - cfgs := createConfigs{ - Container: &containerConfig, - Host: &hostConfig, - Network: networkingConfig, - Links: links, - } - return cfgs, nil -} - -// prepareContainerMACAddress handles the service-level mac_address field and the newer mac_address field added to service -// network config. This newer field is only compatible with the Engine API v1.44 (and onwards), and this API version -// also deprecates the container-wide mac_address field. Thus, this method will validate service config and mutate the -// passed mainNw to provide backward-compatibility whenever possible. -// -// It returns the container-wide MAC address, but this value will be kept empty for newer API versions. -func (s *composeService) prepareContainerMACAddress(ctx context.Context, service types.ServiceConfig, mainNw *types.ServiceNetworkConfig, nwName string) (string, error) { - version, err := s.RuntimeVersion(ctx) - if err != nil { - return "", err - } - - // Engine API 1.44 added support for endpoint-specific MAC address and now returns a warning when a MAC address is - // set in container.Config. Thus, we have to jump through a number of hoops: - // - // 1. Top-level mac_address and main endpoint's MAC address should be the same ; - // 2. If supported by the API, top-level mac_address should be migrated to the main endpoint and container.Config - // should be kept empty ; - // 3. Otherwise, the endpoint mac_address should be set in container.Config and no other endpoint-specific - // mac_address can be specified. If that's the case, use top-level mac_address ; - // - // After that, if an endpoint mac_address is set, it's either user-defined or migrated by the code below, so - // there's no need to check for API version in defaultNetworkSettings. - macAddress := service.MacAddress - if macAddress != "" && mainNw != nil && mainNw.MacAddress != "" && mainNw.MacAddress != macAddress { - return "", fmt.Errorf("the service-level mac_address should have the same value as network %s", nwName) - } - if versions.GreaterThanOrEqualTo(version, APIVersion144) { - if mainNw != nil && mainNw.MacAddress == "" { - mainNw.MacAddress = macAddress - } - macAddress = "" - } else if len(service.Networks) > 0 { - var withMacAddress []string - for nwName, nw := range service.Networks { - if nw != nil && nw.MacAddress != "" { - withMacAddress = append(withMacAddress, nwName) - } - } - - if len(withMacAddress) > 1 { - return "", fmt.Errorf("a MAC address is specified for multiple networks (%s), but this feature requires Docker Engine %s or later", strings.Join(withMacAddress, ", "), DockerEngineV25) - } - - if mainNw != nil && mainNw.MacAddress != "" { - macAddress = mainNw.MacAddress - } - } - - return macAddress, nil -} - -func getAliases(project *types.Project, service types.ServiceConfig, serviceIndex int, cfg *types.ServiceNetworkConfig, useNetworkAliases bool) []string { - aliases := []string{getContainerName(project.Name, service, serviceIndex)} - if useNetworkAliases { - aliases = append(aliases, service.Name) - if cfg != nil { - aliases = append(aliases, cfg.Aliases...) - } - } - return aliases -} - -func createEndpointSettings(p *types.Project, service types.ServiceConfig, serviceIndex int, networkKey string, links []string, useNetworkAliases bool) *network.EndpointSettings { - const ifname = "com.docker.network.endpoint.ifname" - - config := service.Networks[networkKey] - var ipam *network.EndpointIPAMConfig - var ( - ipv4Address string - ipv6Address string - macAddress string - driverOpts types.Options - gwPriority int - ) - if config != nil { - ipv4Address = config.Ipv4Address - ipv6Address = config.Ipv6Address - ipam = &network.EndpointIPAMConfig{ - IPv4Address: ipv4Address, - IPv6Address: ipv6Address, - LinkLocalIPs: config.LinkLocalIPs, - } - macAddress = config.MacAddress - driverOpts = config.DriverOpts - if config.InterfaceName != "" { - if driverOpts == nil { - driverOpts = map[string]string{} - } - if name, ok := driverOpts[ifname]; ok && name != config.InterfaceName { - logrus.Warnf("ignoring services.%s.networks.%s.interface_name as %s driver_opts is already declared", service.Name, networkKey, ifname) - } - driverOpts[ifname] = config.InterfaceName - } - gwPriority = config.GatewayPriority - } - return &network.EndpointSettings{ - Aliases: getAliases(p, service, serviceIndex, config, useNetworkAliases), - Links: links, - IPAddress: ipv4Address, - IPv6Gateway: ipv6Address, - IPAMConfig: ipam, - MacAddress: macAddress, - DriverOpts: driverOpts, - GwPriority: gwPriority, - } -} - -// copy/pasted from https://github.com/docker/cli/blob/9de1b162f/cli/command/container/opts.go#L673-L697 + RelativePath -// TODO find so way to share this code with docker/cli -func parseSecurityOpts(p *types.Project, securityOpts []string) ([]string, bool, error) { - var ( - unconfined bool - parsed []string - ) - for _, opt := range securityOpts { - if opt == "systempaths=unconfined" { - unconfined = true - continue - } - con := strings.SplitN(opt, "=", 2) - if len(con) == 1 && con[0] != "no-new-privileges" { - if strings.Contains(opt, ":") { - con = strings.SplitN(opt, ":", 2) - } else { - return securityOpts, false, fmt.Errorf("invalid security-opt: %q", opt) - } - } - if con[0] == "seccomp" && con[1] != "unconfined" && con[1] != "builtin" { - f, err := os.ReadFile(p.RelativePath(con[1])) - if err != nil { - return securityOpts, false, fmt.Errorf("opening seccomp profile (%s) failed: %w", con[1], err) - } - b := bytes.NewBuffer(nil) - if err := json.Compact(b, f); err != nil { - return securityOpts, false, fmt.Errorf("compacting json for seccomp profile (%s) failed: %w", con[1], err) - } - parsed = append(parsed, fmt.Sprintf("seccomp=%s", b.Bytes())) - } else { - parsed = append(parsed, opt) - } - } - - return parsed, unconfined, nil -} - -func (s *composeService) prepareLabels(labels types.Labels, service types.ServiceConfig, number int) (map[string]string, error) { - hash, err := ServiceHash(service) - if err != nil { - return nil, err - } - labels[api.ConfigHashLabel] = hash - - if number > 0 { - // One-off containers are not indexed - labels[api.ContainerNumberLabel] = strconv.Itoa(number) - } - - var dependencies []string - for s, d := range service.DependsOn { - dependencies = append(dependencies, fmt.Sprintf("%s:%s:%t", s, d.Condition, d.Restart)) - } - labels[api.DependenciesLabel] = strings.Join(dependencies, ",") - return labels, nil -} - -// defaultNetworkSettings determines the container.NetworkMode and corresponding network.NetworkingConfig (nil if not applicable). -func defaultNetworkSettings(project *types.Project, - service types.ServiceConfig, serviceIndex int, - links []string, useNetworkAliases bool, - version string, -) (container.NetworkMode, *network.NetworkingConfig, error) { - if service.NetworkMode != "" { - return container.NetworkMode(service.NetworkMode), nil, nil - } - - if len(project.Networks) == 0 { - return "none", nil, nil - } - - var primaryNetworkKey string - if len(service.Networks) > 0 { - primaryNetworkKey = service.NetworksByPriority()[0] - } else { - primaryNetworkKey = "default" - } - primaryNetworkMobyNetworkName := project.Networks[primaryNetworkKey].Name - primaryNetworkEndpoint := createEndpointSettings(project, service, serviceIndex, primaryNetworkKey, links, useNetworkAliases) - endpointsConfig := map[string]*network.EndpointSettings{} - - // Starting from API version 1.44, the Engine will take several EndpointsConfigs - // so we can pass all the extra networks we want the container to be connected to - // in the network configuration instead of connecting the container to each extra - // network individually after creation. - if versions.GreaterThanOrEqualTo(version, APIVersion144) { - if len(service.Networks) > 1 { - serviceNetworks := service.NetworksByPriority() - for _, networkKey := range serviceNetworks[1:] { - mobyNetworkName := project.Networks[networkKey].Name - epSettings := createEndpointSettings(project, service, serviceIndex, networkKey, links, useNetworkAliases) - endpointsConfig[mobyNetworkName] = epSettings - } - } - if primaryNetworkEndpoint.MacAddress == "" { - primaryNetworkEndpoint.MacAddress = service.MacAddress - } - } - - if versions.LessThan(version, APIVersion149) { - for _, config := range service.Networks { - if config != nil && config.InterfaceName != "" { - return "", nil, fmt.Errorf("interface_name requires Docker Engine %s or later", DockerEngineV28_1) - } - } - } - - endpointsConfig[primaryNetworkMobyNetworkName] = primaryNetworkEndpoint - networkConfig := &network.NetworkingConfig{ - EndpointsConfig: endpointsConfig, - } - - // From the Engine API docs: - // > Supported standard values are: bridge, host, none, and container:<name|id>. - // > Any other value is taken as a custom network's name to which this container should connect to. - return container.NetworkMode(primaryNetworkMobyNetworkName), networkConfig, nil -} - -func getRestartPolicy(service types.ServiceConfig) container.RestartPolicy { - var restart container.RestartPolicy - if service.Restart != "" { - name, num, ok := strings.Cut(service.Restart, ":") - var attempts int - if ok { - attempts, _ = strconv.Atoi(num) - } - restart = container.RestartPolicy{ - Name: mapRestartPolicyCondition(name), - MaximumRetryCount: attempts, - } - } - if service.Deploy != nil && service.Deploy.RestartPolicy != nil { - policy := *service.Deploy.RestartPolicy - var attempts int - if policy.MaxAttempts != nil { - attempts = int(*policy.MaxAttempts) - } - restart = container.RestartPolicy{ - Name: mapRestartPolicyCondition(policy.Condition), - MaximumRetryCount: attempts, - } - } - return restart -} - -func mapRestartPolicyCondition(condition string) container.RestartPolicyMode { - // map definitions of deploy.restart_policy to engine definitions - switch condition { - case "none", "no": - return container.RestartPolicyDisabled - case "on-failure": - return container.RestartPolicyOnFailure - case "unless-stopped": - return container.RestartPolicyUnlessStopped - case "any", "always": - return container.RestartPolicyAlways - default: - return container.RestartPolicyMode(condition) - } -} - -func getDeployResources(s types.ServiceConfig) container.Resources { - var swappiness *int64 - if s.MemSwappiness != 0 { - val := int64(s.MemSwappiness) - swappiness = &val - } - resources := container.Resources{ - CgroupParent: s.CgroupParent, - Memory: int64(s.MemLimit), - MemorySwap: int64(s.MemSwapLimit), - MemorySwappiness: swappiness, - MemoryReservation: int64(s.MemReservation), - OomKillDisable: &s.OomKillDisable, - CPUCount: s.CPUCount, - CPUPeriod: s.CPUPeriod, - CPUQuota: s.CPUQuota, - CPURealtimePeriod: s.CPURTPeriod, - CPURealtimeRuntime: s.CPURTRuntime, - CPUShares: s.CPUShares, - NanoCPUs: int64(s.CPUS * 1e9), - CPUPercent: int64(s.CPUPercent * 100), - CpusetCpus: s.CPUSet, - DeviceCgroupRules: s.DeviceCgroupRules, - } - - if s.PidsLimit != 0 { - resources.PidsLimit = &s.PidsLimit - } - - setBlkio(s.BlkioConfig, &resources) - - if s.Deploy != nil { - setLimits(s.Deploy.Resources.Limits, &resources) - setReservations(s.Deploy.Resources.Reservations, &resources) - } - - var cdiDeviceNames []string - for _, device := range s.Devices { - - if device.Source == device.Target && cdi.IsQualifiedName(device.Source) { - cdiDeviceNames = append(cdiDeviceNames, device.Source) - continue - } - - resources.Devices = append(resources.Devices, container.DeviceMapping{ - PathOnHost: device.Source, - PathInContainer: device.Target, - CgroupPermissions: device.Permissions, - }) - } - - if len(cdiDeviceNames) > 0 { - resources.DeviceRequests = append(resources.DeviceRequests, container.DeviceRequest{ - Driver: "cdi", - DeviceIDs: cdiDeviceNames, - }) - } - - for _, gpus := range s.Gpus { - resources.DeviceRequests = append(resources.DeviceRequests, container.DeviceRequest{ - Driver: gpus.Driver, - Count: int(gpus.Count), - DeviceIDs: gpus.IDs, - Capabilities: [][]string{append(gpus.Capabilities, "gpu")}, - Options: gpus.Options, - }) - } - - ulimits := toUlimits(s.Ulimits) - resources.Ulimits = ulimits - return resources -} - -func toUlimits(m map[string]*types.UlimitsConfig) []*container.Ulimit { - var ulimits []*container.Ulimit - for name, u := range m { - soft := u.Single - if u.Soft != 0 { - soft = u.Soft - } - hard := u.Single - if u.Hard != 0 { - hard = u.Hard - } - ulimits = append(ulimits, &container.Ulimit{ - Name: name, - Hard: int64(hard), - Soft: int64(soft), - }) - } - return ulimits -} - -func setReservations(reservations *types.Resource, resources *container.Resources) { - if reservations == nil { - return - } - // Cpu reservation is a swarm option and PIDs is only a limit - // So we only need to map memory reservation and devices - if reservations.MemoryBytes != 0 { - resources.MemoryReservation = int64(reservations.MemoryBytes) - } - - for _, device := range reservations.Devices { - resources.DeviceRequests = append(resources.DeviceRequests, container.DeviceRequest{ - Capabilities: [][]string{device.Capabilities}, - Count: int(device.Count), - DeviceIDs: device.IDs, - Driver: device.Driver, - Options: device.Options, - }) - } -} - -func setLimits(limits *types.Resource, resources *container.Resources) { - if limits == nil { - return - } - if limits.MemoryBytes != 0 { - resources.Memory = int64(limits.MemoryBytes) - } - if limits.NanoCPUs != 0 { - resources.NanoCPUs = int64(limits.NanoCPUs * 1e9) - } - if limits.Pids > 0 { - resources.PidsLimit = &limits.Pids - } -} - -func setBlkio(blkio *types.BlkioConfig, resources *container.Resources) { - if blkio == nil { - return - } - resources.BlkioWeight = blkio.Weight - for _, b := range blkio.WeightDevice { - resources.BlkioWeightDevice = append(resources.BlkioWeightDevice, &blkiodev.WeightDevice{ - Path: b.Path, - Weight: b.Weight, - }) - } - for _, b := range blkio.DeviceReadBps { - resources.BlkioDeviceReadBps = append(resources.BlkioDeviceReadBps, &blkiodev.ThrottleDevice{ - Path: b.Path, - Rate: uint64(b.Rate), - }) - } - for _, b := range blkio.DeviceReadIOps { - resources.BlkioDeviceReadIOps = append(resources.BlkioDeviceReadIOps, &blkiodev.ThrottleDevice{ - Path: b.Path, - Rate: uint64(b.Rate), - }) - } - for _, b := range blkio.DeviceWriteBps { - resources.BlkioDeviceWriteBps = append(resources.BlkioDeviceWriteBps, &blkiodev.ThrottleDevice{ - Path: b.Path, - Rate: uint64(b.Rate), - }) - } - for _, b := range blkio.DeviceWriteIOps { - resources.BlkioDeviceWriteIOps = append(resources.BlkioDeviceWriteIOps, &blkiodev.ThrottleDevice{ - Path: b.Path, - Rate: uint64(b.Rate), - }) - } -} - -func buildContainerPorts(s types.ServiceConfig) (nat.PortSet, error) { - ports := nat.PortSet{} - for _, s := range s.Expose { - proto, port := nat.SplitProtoPort(s) - start, end, err := nat.ParsePortRange(port) - if err != nil { - return nil, err - } - for i := start; i <= end; i++ { - p := nat.Port(fmt.Sprintf("%d/%s", i, proto)) - ports[p] = struct{}{} - } - } - for _, p := range s.Ports { - p := nat.Port(fmt.Sprintf("%d/%s", p.Target, p.Protocol)) - ports[p] = struct{}{} - } - return ports, nil -} - -func buildContainerPortBindingOptions(s types.ServiceConfig) nat.PortMap { - bindings := nat.PortMap{} - for _, port := range s.Ports { - p := nat.Port(fmt.Sprintf("%d/%s", port.Target, port.Protocol)) - binding := nat.PortBinding{ - HostIP: port.HostIP, - HostPort: port.Published, - } - bindings[p] = append(bindings[p], binding) - } - return bindings -} - -func getDependentServiceFromMode(mode string) string { - if strings.HasPrefix( - mode, - types.NetworkModeServicePrefix, - ) { - return mode[len(types.NetworkModeServicePrefix):] - } - return "" -} - -func (s *composeService) buildContainerVolumes( - ctx context.Context, - p types.Project, - service types.ServiceConfig, - inherit *container.Summary, -) ([]string, []mount.Mount, error) { - var mounts []mount.Mount - var binds []string - - mountOptions, err := s.buildContainerMountOptions(ctx, p, service, inherit) - if err != nil { - return nil, nil, err - } - - for _, m := range mountOptions { - switch m.Type { - case mount.TypeBind: - // `Mount` is preferred but does not offer option to created host path if missing - // so `Bind` API is used here with raw volume string - // see https://github.com/moby/moby/issues/43483 - v := findVolumeByTarget(service.Volumes, m.Target) - if v != nil { - if v.Type != types.VolumeTypeBind { - v.Source = m.Source - } - if !bindRequiresMountAPI(v.Bind) { - source := m.Source - if vol := findVolumeByName(p.Volumes, m.Source); vol != nil { - source = m.Source - } - binds = append(binds, toBindString(source, v)) - continue - } - } - case mount.TypeVolume: - v := findVolumeByTarget(service.Volumes, m.Target) - vol := findVolumeByName(p.Volumes, m.Source) - if v != nil && vol != nil { - // Prefer the bind API if no advanced option is used, to preserve backward compatibility - if !volumeRequiresMountAPI(v.Volume) { - binds = append(binds, toBindString(vol.Name, v)) - continue - } - } - case mount.TypeImage: - version, err := s.RuntimeVersion(ctx) - if err != nil { - return nil, nil, err - } - if versions.LessThan(version, APIVersion148) { - return nil, nil, fmt.Errorf("volume with type=image require Docker Engine %s or later", DockerEngineV28) - } - } - mounts = append(mounts, m) - } - return binds, mounts, nil -} - -func toBindString(name string, v *types.ServiceVolumeConfig) string { - access := "rw" - if v.ReadOnly { - access = "ro" - } - options := []string{access} - if v.Bind != nil && v.Bind.SELinux != "" { - options = append(options, v.Bind.SELinux) - } - if v.Bind != nil && v.Bind.Propagation != "" { - options = append(options, v.Bind.Propagation) - } - if v.Volume != nil && v.Volume.NoCopy { - options = append(options, "nocopy") - } - return fmt.Sprintf("%s:%s:%s", name, v.Target, strings.Join(options, ",")) -} - -func findVolumeByName(volumes types.Volumes, name string) *types.VolumeConfig { - for _, vol := range volumes { - if vol.Name == name { - return &vol - } - } - return nil -} - -func findVolumeByTarget(volumes []types.ServiceVolumeConfig, target string) *types.ServiceVolumeConfig { - for _, v := range volumes { - if v.Target == target { - return &v - } - } - return nil -} - -// bindRequiresMountAPI check if Bind declaration can be implemented by the plain old Bind API or uses any of the advanced -// options which require use of Mount API -func bindRequiresMountAPI(bind *types.ServiceVolumeBind) bool { - switch { - case bind == nil: - return false - case !bool(bind.CreateHostPath): - return true - case bind.Propagation != "": - return true - case bind.Recursive != "": - return true - default: - return false - } -} - -// volumeRequiresMountAPI check if Volume declaration can be implemented by the plain old Bind API or uses any of the advanced -// options which require use of Mount API -func volumeRequiresMountAPI(vol *types.ServiceVolumeVolume) bool { - switch { - case vol == nil: - return false - case len(vol.Labels) > 0: - return true - case vol.Subpath != "": - return true - case vol.NoCopy: - return true - default: - return false - } -} - -func (s *composeService) buildContainerMountOptions(ctx context.Context, p types.Project, service types.ServiceConfig, inherit *container.Summary) ([]mount.Mount, error) { - mounts := map[string]mount.Mount{} - if inherit != nil { - for _, m := range inherit.Mounts { - if m.Type == "tmpfs" { - continue - } - src := m.Source - if m.Type == "volume" { - src = m.Name - } - - img, err := s.apiClient().ImageInspect(ctx, api.GetImageNameOrDefault(service, p.Name)) - if err != nil { - return nil, err - } - - if img.Config != nil { - if _, ok := img.Config.Volumes[m.Destination]; ok { - // inherit previous container's anonymous volume - mounts[m.Destination] = mount.Mount{ - Type: m.Type, - Source: src, - Target: m.Destination, - ReadOnly: !m.RW, - } - } - } - volumes := []types.ServiceVolumeConfig{} - for _, v := range service.Volumes { - if v.Target != m.Destination || v.Source != "" { - volumes = append(volumes, v) - continue - } - // inherit previous container's anonymous volume - mounts[m.Destination] = mount.Mount{ - Type: m.Type, - Source: src, - Target: m.Destination, - ReadOnly: !m.RW, - } - } - service.Volumes = volumes - } - } - - mounts, err := fillBindMounts(p, service, mounts) - if err != nil { - return nil, err - } - - values := make([]mount.Mount, 0, len(mounts)) - for _, v := range mounts { - values = append(values, v) - } - return values, nil -} - -func fillBindMounts(p types.Project, s types.ServiceConfig, m map[string]mount.Mount) (map[string]mount.Mount, error) { - for _, v := range s.Volumes { - bindMount, err := buildMount(p, v) - if err != nil { - return nil, err - } - m[bindMount.Target] = bindMount - } - - secrets, err := buildContainerSecretMounts(p, s) - if err != nil { - return nil, err - } - for _, s := range secrets { - if _, found := m[s.Target]; found { - continue - } - m[s.Target] = s - } - - configs, err := buildContainerConfigMounts(p, s) - if err != nil { - return nil, err - } - for _, c := range configs { - if _, found := m[c.Target]; found { - continue - } - m[c.Target] = c - } - return m, nil -} - -func buildContainerConfigMounts(p types.Project, s types.ServiceConfig) ([]mount.Mount, error) { - mounts := map[string]mount.Mount{} - - configsBaseDir := "/" - for _, config := range s.Configs { - target := config.Target - if config.Target == "" { - target = configsBaseDir + config.Source - } else if !isAbsTarget(config.Target) { - target = configsBaseDir + config.Target - } - - definedConfig := p.Configs[config.Source] - if definedConfig.External { - return nil, fmt.Errorf("unsupported external config %s", definedConfig.Name) - } - - if definedConfig.Driver != "" { - return nil, errors.New("Docker Compose does not support configs.*.driver") //nolint:staticcheck - } - if definedConfig.TemplateDriver != "" { - return nil, errors.New("Docker Compose does not support configs.*.template_driver") //nolint:staticcheck - } - - if definedConfig.Environment != "" || definedConfig.Content != "" { - continue - } - - if config.UID != "" || config.GID != "" || config.Mode != nil { - logrus.Warn("config `uid`, `gid` and `mode` are not supported, they will be ignored") - } - - bindMount, err := buildMount(p, types.ServiceVolumeConfig{ - Type: types.VolumeTypeBind, - Source: definedConfig.File, - Target: target, - ReadOnly: true, - }) - if err != nil { - return nil, err - } - mounts[target] = bindMount - } - values := make([]mount.Mount, 0, len(mounts)) - for _, v := range mounts { - values = append(values, v) - } - return values, nil -} - -func buildContainerSecretMounts(p types.Project, s types.ServiceConfig) ([]mount.Mount, error) { - mounts := map[string]mount.Mount{} - - secretsDir := "/run/secrets/" - for _, secret := range s.Secrets { - target := secret.Target - if secret.Target == "" { - target = secretsDir + secret.Source - } else if !isAbsTarget(secret.Target) { - target = secretsDir + secret.Target - } - - definedSecret := p.Secrets[secret.Source] - if definedSecret.External { - return nil, fmt.Errorf("unsupported external secret %s", definedSecret.Name) - } - - if definedSecret.Driver != "" { - return nil, errors.New("Docker Compose does not support secrets.*.driver") //nolint:staticcheck - } - if definedSecret.TemplateDriver != "" { - return nil, errors.New("Docker Compose does not support secrets.*.template_driver") //nolint:staticcheck - } - - if definedSecret.Environment != "" { - continue - } - - if secret.UID != "" || secret.GID != "" || secret.Mode != nil { - logrus.Warn("secrets `uid`, `gid` and `mode` are not supported, they will be ignored") - } - - if _, err := os.Stat(definedSecret.File); os.IsNotExist(err) { - logrus.Warnf("secret file %s does not exist", definedSecret.Name) - } - - mnt, err := buildMount(p, types.ServiceVolumeConfig{ - Type: types.VolumeTypeBind, - Source: definedSecret.File, - Target: target, - ReadOnly: true, - Bind: &types.ServiceVolumeBind{ - CreateHostPath: false, - }, - }) - if err != nil { - return nil, err - } - mounts[target] = mnt - } - values := make([]mount.Mount, 0, len(mounts)) - for _, v := range mounts { - values = append(values, v) - } - return values, nil -} - -func isAbsTarget(p string) bool { - return isUnixAbs(p) || isWindowsAbs(p) -} - -func isUnixAbs(p string) bool { - return strings.HasPrefix(p, "/") -} - -func isWindowsAbs(p string) bool { - return paths.IsWindowsAbs(p) -} - -func buildMount(project types.Project, volume types.ServiceVolumeConfig) (mount.Mount, error) { - source := volume.Source - switch volume.Type { - case types.VolumeTypeBind: - if !filepath.IsAbs(source) && !isUnixAbs(source) && !isWindowsAbs(source) { - // volume source has already been prefixed with workdir if required, by compose-go project loader - var err error - source, err = filepath.Abs(source) - if err != nil { - return mount.Mount{}, err - } - } - case types.VolumeTypeVolume: - if volume.Source != "" { - pVolume, ok := project.Volumes[volume.Source] - if ok { - source = pVolume.Name - } - } - } - - bind, vol, tmpfs, img := buildMountOptions(volume) - - if bind != nil { - volume.Type = types.VolumeTypeBind - } - - return mount.Mount{ - Type: mount.Type(volume.Type), - Source: source, - Target: volume.Target, - ReadOnly: volume.ReadOnly, - Consistency: mount.Consistency(volume.Consistency), - BindOptions: bind, - VolumeOptions: vol, - TmpfsOptions: tmpfs, - ImageOptions: img, - }, nil -} - -func buildMountOptions(volume types.ServiceVolumeConfig) (*mount.BindOptions, *mount.VolumeOptions, *mount.TmpfsOptions, *mount.ImageOptions) { - if volume.Type != types.VolumeTypeBind && volume.Bind != nil { - logrus.Warnf("mount of type `%s` should not define `bind` option", volume.Type) - } - if volume.Type != types.VolumeTypeVolume && volume.Volume != nil { - logrus.Warnf("mount of type `%s` should not define `volume` option", volume.Type) - } - if volume.Type != types.VolumeTypeTmpfs && volume.Tmpfs != nil { - logrus.Warnf("mount of type `%s` should not define `tmpfs` option", volume.Type) - } - if volume.Type != types.VolumeTypeImage && volume.Image != nil { - logrus.Warnf("mount of type `%s` should not define `image` option", volume.Type) - } - - switch volume.Type { - case "bind": - return buildBindOption(volume.Bind), nil, nil, nil - case "volume": - return nil, buildVolumeOptions(volume.Volume), nil, nil - case "tmpfs": - return nil, nil, buildTmpfsOptions(volume.Tmpfs), nil - case "image": - return nil, nil, nil, buildImageOptions(volume.Image) - } - return nil, nil, nil, nil -} - -func buildBindOption(bind *types.ServiceVolumeBind) *mount.BindOptions { - if bind == nil { - return nil - } - opts := &mount.BindOptions{ - Propagation: mount.Propagation(bind.Propagation), - CreateMountpoint: bool(bind.CreateHostPath), - } - switch bind.Recursive { - case "disabled": - opts.NonRecursive = true - case "writable": - opts.ReadOnlyNonRecursive = true - case "readonly": - opts.ReadOnlyForceRecursive = true - } - return opts -} - -func buildVolumeOptions(vol *types.ServiceVolumeVolume) *mount.VolumeOptions { - if vol == nil { - return nil - } - return &mount.VolumeOptions{ - NoCopy: vol.NoCopy, - Subpath: vol.Subpath, - Labels: vol.Labels, - // DriverConfig: , // FIXME missing from model ? - } -} - -func buildTmpfsOptions(tmpfs *types.ServiceVolumeTmpfs) *mount.TmpfsOptions { - if tmpfs == nil { - return nil - } - return &mount.TmpfsOptions{ - SizeBytes: int64(tmpfs.Size), - Mode: os.FileMode(tmpfs.Mode), - } -} - -func buildImageOptions(image *types.ServiceVolumeImage) *mount.ImageOptions { - if image == nil { - return nil - } - return &mount.ImageOptions{ - Subpath: image.SubPath, - } -} - -func (s *composeService) ensureNetwork(ctx context.Context, project *types.Project, name string, n *types.NetworkConfig) (string, error) { - if n.External { - return s.resolveExternalNetwork(ctx, n) - } - - id, err := s.resolveOrCreateNetwork(ctx, project, name, n) - if errdefs.IsConflict(err) { - // Maybe another execution of `docker compose up|run` created same network - // let's retry once - return s.resolveOrCreateNetwork(ctx, project, name, n) - } - return id, err -} - -func (s *composeService) resolveOrCreateNetwork(ctx context.Context, project *types.Project, name string, n *types.NetworkConfig) (string, error) { //nolint:gocyclo - // This is containers that could be left after a diverged network was removed - var dangledContainers Containers - - // First, try to find a unique network matching by name or ID - inspect, err := s.apiClient().NetworkInspect(ctx, n.Name, network.InspectOptions{}) - if err == nil { - // NetworkInspect will match on ID prefix, so double check we get the expected one - // as looking for network named `db` we could erroneously match network ID `db9086999caf` - if inspect.Name == n.Name || inspect.ID == n.Name { - p, ok := inspect.Labels[api.ProjectLabel] - if !ok { - logrus.Warnf("a network with name %s exists but was not created by compose.\n"+ - "Set `external: true` to use an existing network", n.Name) - } else if p != project.Name { - logrus.Warnf("a network with name %s exists but was not created for project %q.\n"+ - "Set `external: true` to use an existing network", n.Name, project.Name) - } - if inspect.Labels[api.NetworkLabel] != name { - return "", fmt.Errorf( - "network %s was found but has incorrect label %s set to %q (expected: %q)", - n.Name, - api.NetworkLabel, - inspect.Labels[api.NetworkLabel], - name, - ) - } - - hash := inspect.Labels[api.ConfigHashLabel] - expected, err := NetworkHash(n) - if err != nil { - return "", err - } - if hash == "" || hash == expected { - return inspect.ID, nil - } - - dangledContainers, err = s.removeDivergedNetwork(ctx, project, name, n) - if err != nil { - return "", err - } - } - } - // ignore other errors. Typically, an ambiguous request by name results in some generic `invalidParameter` error - - // Either not found, or name is ambiguous - use NetworkList to list by name - networks, err := s.apiClient().NetworkList(ctx, network.ListOptions{ - Filters: filters.NewArgs(filters.Arg("name", n.Name)), - }) - if err != nil { - return "", err - } - - // NetworkList Matches all or part of a network name, so we have to filter for a strict match - networks = slices.DeleteFunc(networks, func(net network.Summary) bool { - return net.Name != n.Name - }) - - for _, net := range networks { - if net.Labels[api.ProjectLabel] == project.Name && - net.Labels[api.NetworkLabel] == name { - return net.ID, nil - } - } - - // we could have set NetworkList with a projectFilter and networkFilter but not doing so allows to catch this - // scenario were a network with same name exists but doesn't have label, and use of `CheckDuplicate: true` - // prevents to create another one. - if len(networks) > 0 { - logrus.Warnf("a network with name %s exists but was not created by compose.\n"+ - "Set `external: true` to use an existing network", n.Name) - return networks[0].ID, nil - } - - var ipam *network.IPAM - if n.Ipam.Config != nil { - var config []network.IPAMConfig - for _, pool := range n.Ipam.Config { - config = append(config, network.IPAMConfig{ - Subnet: pool.Subnet, - IPRange: pool.IPRange, - Gateway: pool.Gateway, - AuxAddress: pool.AuxiliaryAddresses, - }) - } - ipam = &network.IPAM{ - Driver: n.Ipam.Driver, - Config: config, - } - } - hash, err := NetworkHash(n) - if err != nil { - return "", err - } - n.CustomLabels = n.CustomLabels.Add(api.ConfigHashLabel, hash) - createOpts := network.CreateOptions{ - Labels: mergeLabels(n.Labels, n.CustomLabels), - Driver: n.Driver, - Options: n.DriverOpts, - Internal: n.Internal, - Attachable: n.Attachable, - IPAM: ipam, - EnableIPv6: n.EnableIPv6, - EnableIPv4: n.EnableIPv4, - } - - if n.Ipam.Driver != "" || len(n.Ipam.Config) > 0 { - createOpts.IPAM = &network.IPAM{} - } - - if n.Ipam.Driver != "" { - createOpts.IPAM.Driver = n.Ipam.Driver - } - - for _, ipamConfig := range n.Ipam.Config { - config := network.IPAMConfig{ - Subnet: ipamConfig.Subnet, - IPRange: ipamConfig.IPRange, - Gateway: ipamConfig.Gateway, - AuxAddress: ipamConfig.AuxiliaryAddresses, - } - createOpts.IPAM.Config = append(createOpts.IPAM.Config, config) - } - - networkEventName := fmt.Sprintf("Network %s", n.Name) - s.events.On(creatingEvent(networkEventName)) - - resp, err := s.apiClient().NetworkCreate(ctx, n.Name, createOpts) - if err != nil { - s.events.On(errorEvent(networkEventName, err.Error())) - return "", fmt.Errorf("failed to create network %s: %w", n.Name, err) - } - s.events.On(createdEvent(networkEventName)) - - err = s.connectNetwork(ctx, n.Name, dangledContainers, nil) - if err != nil { - return "", err - } - - return resp.ID, nil -} - -func (s *composeService) removeDivergedNetwork(ctx context.Context, project *types.Project, name string, n *types.NetworkConfig) (Containers, error) { - // Remove services attached to this network to force recreation - var services []string - for _, service := range project.Services.Filter(func(config types.ServiceConfig) bool { - _, ok := config.Networks[name] - return ok - }) { - services = append(services, service.Name) - } - - // Stop containers so we can remove network - // They will be restarted (actually: recreated) with the updated network - err := s.stop(ctx, project.Name, api.StopOptions{ - Services: services, - Project: project, - }, nil) - if err != nil { - return nil, err - } - - containers, err := s.getContainers(ctx, project.Name, oneOffExclude, true, services...) - if err != nil { - return nil, err - } - - err = s.disconnectNetwork(ctx, n.Name, containers) - if err != nil { - return nil, err - } - - err = s.apiClient().NetworkRemove(ctx, n.Name) - eventName := fmt.Sprintf("Network %s", n.Name) - s.events.On(removedEvent(eventName)) - return containers, err -} - -func (s *composeService) disconnectNetwork( - ctx context.Context, - nwName string, - containers Containers, -) error { - for _, c := range containers { - err := s.apiClient().NetworkDisconnect(ctx, nwName, c.ID, true) - if err != nil { - return err - } - } - - return nil -} - -func (s *composeService) connectNetwork( - ctx context.Context, - nwName string, - containers Containers, - config *network.EndpointSettings, -) error { - for _, c := range containers { - err := s.apiClient().NetworkConnect(ctx, nwName, c.ID, config) - if err != nil { - return err - } - } - - return nil -} - -func (s *composeService) resolveExternalNetwork(ctx context.Context, n *types.NetworkConfig) (string, error) { - // NetworkInspect will match on ID prefix, so NetworkList with a name - // filter is used to look for an exact match to prevent e.g. a network - // named `db` from getting erroneously matched to a network with an ID - // like `db9086999caf` - networks, err := s.apiClient().NetworkList(ctx, network.ListOptions{ - Filters: filters.NewArgs(filters.Arg("name", n.Name)), - }) - if err != nil { - return "", err - } - - if len(networks) == 0 { - // in this instance, n.Name is really an ID - sn, err := s.apiClient().NetworkInspect(ctx, n.Name, network.InspectOptions{}) - if err == nil { - networks = append(networks, sn) - } else if !errdefs.IsNotFound(err) { - return "", err - } - - } - - // NetworkList API doesn't return the exact name match, so we can retrieve more than one network with a request - networks = slices.DeleteFunc(networks, func(net network.Inspect) bool { - // this function is called during the rebuild stage of `compose watch`. - // we still require just one network back, but we need to run the search on the ID - return net.Name != n.Name && net.ID != n.Name - }) - - switch len(networks) { - case 1: - return networks[0].ID, nil - case 0: - enabled, err := s.isSwarmEnabled(ctx) - if err != nil { - return "", err - } - if enabled { - // Swarm nodes do not register overlay networks that were - // created on a different node unless they're in use. - // So we can't preemptively check network exists, but - // networkAttach will later fail anyway if network actually doesn't exist - return "swarm", nil - } - return "", fmt.Errorf("network %s declared as external, but could not be found", n.Name) - default: - return "", fmt.Errorf("multiple networks with name %q were found. Use network ID as `name` to avoid ambiguity", n.Name) - } -} - -func (s *composeService) ensureVolume(ctx context.Context, name string, volume types.VolumeConfig, project *types.Project) (string, error) { - inspected, err := s.apiClient().VolumeInspect(ctx, volume.Name) - if err != nil { - if !errdefs.IsNotFound(err) { - return "", err - } - if volume.External { - return "", fmt.Errorf("external volume %q not found", volume.Name) - } - err = s.createVolume(ctx, volume) - return volume.Name, err - } - - if volume.External { - return volume.Name, nil - } - - // Volume exists with name, but let's double-check this is the expected one - p, ok := inspected.Labels[api.ProjectLabel] - if !ok { - logrus.Warnf("volume %q already exists but was not created by Docker Compose. Use `external: true` to use an existing volume", volume.Name) - } - if ok && p != project.Name { - logrus.Warnf("volume %q already exists but was created for project %q (expected %q). Use `external: true` to use an existing volume", volume.Name, p, project.Name) - } - - expected, err := VolumeHash(volume) - if err != nil { - return "", err - } - actual, ok := inspected.Labels[api.ConfigHashLabel] - if ok && actual != expected { - msg := fmt.Sprintf("Volume %q exists but doesn't match configuration in compose file. Recreate (data will be lost)?", volume.Name) - confirm, err := s.prompt(msg, false) - if err != nil { - return "", err - } - if confirm { - err = s.removeDivergedVolume(ctx, name, volume, project) - if err != nil { - return "", err - } - return volume.Name, s.createVolume(ctx, volume) - } - } - return inspected.Name, nil -} - -func (s *composeService) removeDivergedVolume(ctx context.Context, name string, volume types.VolumeConfig, project *types.Project) error { - // Remove services mounting divergent volume - var services []string - for _, service := range project.Services.Filter(func(config types.ServiceConfig) bool { - for _, cfg := range config.Volumes { - if cfg.Source == name { - return true - } - } - return false - }) { - services = append(services, service.Name) - } - - err := s.stop(ctx, project.Name, api.StopOptions{ - Services: services, - Project: project, - }, nil) - if err != nil { - return err - } - - containers, err := s.getContainers(ctx, project.Name, oneOffExclude, true, services...) - if err != nil { - return err - } - - // FIXME (ndeloof) we have to remove container so we can recreate volume - // but doing so we can't inherit anonymous volumes from previous instance - err = s.remove(ctx, containers, api.RemoveOptions{ - Services: services, - Project: project, - }) - if err != nil { - return err - } - - return s.apiClient().VolumeRemove(ctx, volume.Name, true) -} - -func (s *composeService) createVolume(ctx context.Context, volume types.VolumeConfig) error { - eventName := fmt.Sprintf("Volume %s", volume.Name) - s.events.On(creatingEvent(eventName)) - hash, err := VolumeHash(volume) - if err != nil { - return err - } - volume.CustomLabels.Add(api.ConfigHashLabel, hash) - _, err = s.apiClient().VolumeCreate(ctx, volumetypes.CreateOptions{ - Labels: mergeLabels(volume.Labels, volume.CustomLabels), - Name: volume.Name, - Driver: volume.Driver, - DriverOpts: volume.DriverOpts, - }) - if err != nil { - s.events.On(errorEvent(eventName, err.Error())) - return err - } - s.events.On(createdEvent(eventName)) - return nil -} diff --git a/pkg/compose/create_test.go b/pkg/compose/create_test.go deleted file mode 100644 index 41cd3bcb8ac..00000000000 --- a/pkg/compose/create_test.go +++ /dev/null @@ -1,456 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "os" - "path/filepath" - "sort" - "testing" - - composeloader "github.com/compose-spec/compose-go/v2/loader" - composetypes "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/image" - mountTypes "github.com/docker/docker/api/types/mount" - "github.com/docker/docker/api/types/network" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - "gotest.tools/v3/assert/cmp" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestBuildBindMount(t *testing.T) { - project := composetypes.Project{} - volume := composetypes.ServiceVolumeConfig{ - Type: composetypes.VolumeTypeBind, - Source: "", - Target: "/data", - } - mount, err := buildMount(project, volume) - assert.NilError(t, err) - assert.Assert(t, filepath.IsAbs(mount.Source)) - _, err = os.Stat(mount.Source) - assert.NilError(t, err) - assert.Equal(t, mount.Type, mountTypes.TypeBind) -} - -func TestBuildNamedPipeMount(t *testing.T) { - project := composetypes.Project{} - volume := composetypes.ServiceVolumeConfig{ - Type: composetypes.VolumeTypeNamedPipe, - Source: "\\\\.\\pipe\\docker_engine_windows", - Target: "\\\\.\\pipe\\docker_engine", - } - mount, err := buildMount(project, volume) - assert.NilError(t, err) - assert.Equal(t, mount.Type, mountTypes.TypeNamedPipe) -} - -func TestBuildVolumeMount(t *testing.T) { - project := composetypes.Project{ - Name: "myProject", - Volumes: composetypes.Volumes(map[string]composetypes.VolumeConfig{ - "myVolume": { - Name: "myProject_myVolume", - }, - }), - } - volume := composetypes.ServiceVolumeConfig{ - Type: composetypes.VolumeTypeVolume, - Source: "myVolume", - Target: "/data", - } - mount, err := buildMount(project, volume) - assert.NilError(t, err) - assert.Equal(t, mount.Source, "myProject_myVolume") - assert.Equal(t, mount.Type, mountTypes.TypeVolume) -} - -func TestServiceImageName(t *testing.T) { - assert.Equal(t, api.GetImageNameOrDefault(composetypes.ServiceConfig{Image: "myImage"}, "myProject"), "myImage") - assert.Equal(t, api.GetImageNameOrDefault(composetypes.ServiceConfig{Name: "aService"}, "myProject"), "myProject-aService") -} - -func TestPrepareNetworkLabels(t *testing.T) { - project := composetypes.Project{ - Name: "myProject", - Networks: composetypes.Networks(map[string]composetypes.NetworkConfig{"skynet": {}}), - } - prepareNetworks(&project) - assert.DeepEqual(t, project.Networks["skynet"].CustomLabels, composetypes.Labels(map[string]string{ - "com.docker.compose.network": "skynet", - "com.docker.compose.project": "myProject", - "com.docker.compose.version": api.ComposeVersion, - })) -} - -func TestBuildContainerMountOptions(t *testing.T) { - project := composetypes.Project{ - Name: "myProject", - Services: composetypes.Services{ - "myService": { - Name: "myService", - Volumes: []composetypes.ServiceVolumeConfig{ - { - Type: composetypes.VolumeTypeVolume, - Target: "/var/myvolume1", - }, - { - Type: composetypes.VolumeTypeVolume, - Target: "/var/myvolume2", - }, - { - Type: composetypes.VolumeTypeVolume, - Source: "myVolume3", - Target: "/var/myvolume3", - Volume: &composetypes.ServiceVolumeVolume{ - Subpath: "etc", - }, - }, - { - Type: composetypes.VolumeTypeNamedPipe, - Source: "\\\\.\\pipe\\docker_engine_windows", - Target: "\\\\.\\pipe\\docker_engine", - }, - }, - }, - }, - Volumes: composetypes.Volumes(map[string]composetypes.VolumeConfig{ - "myVolume1": { - Name: "myProject_myVolume1", - }, - "myVolume2": { - Name: "myProject_myVolume2", - }, - }), - } - - inherit := &container.Summary{ - Mounts: []container.MountPoint{ - { - Type: composetypes.VolumeTypeVolume, - Destination: "/var/myvolume1", - }, - { - Type: composetypes.VolumeTypeVolume, - Destination: "/var/myvolume2", - }, - }, - } - - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - mock, cli := prepareMocks(mockCtrl) - s := composeService{ - dockerCli: cli, - } - mock.EXPECT().ImageInspect(gomock.Any(), "myProject-myService").AnyTimes().Return(image.InspectResponse{}, nil) - - mounts, err := s.buildContainerMountOptions(t.Context(), project, project.Services["myService"], inherit) - sort.Slice(mounts, func(i, j int) bool { - return mounts[i].Target < mounts[j].Target - }) - assert.NilError(t, err) - assert.Assert(t, len(mounts) == 4) - assert.Equal(t, mounts[0].Target, "/var/myvolume1") - assert.Equal(t, mounts[1].Target, "/var/myvolume2") - assert.Equal(t, mounts[2].Target, "/var/myvolume3") - assert.Equal(t, mounts[2].VolumeOptions.Subpath, "etc") - assert.Equal(t, mounts[3].Target, "\\\\.\\pipe\\docker_engine") - - mounts, err = s.buildContainerMountOptions(t.Context(), project, project.Services["myService"], inherit) - sort.Slice(mounts, func(i, j int) bool { - return mounts[i].Target < mounts[j].Target - }) - assert.NilError(t, err) - assert.Assert(t, len(mounts) == 4) - assert.Equal(t, mounts[0].Target, "/var/myvolume1") - assert.Equal(t, mounts[1].Target, "/var/myvolume2") - assert.Equal(t, mounts[2].Target, "/var/myvolume3") - assert.Equal(t, mounts[2].VolumeOptions.Subpath, "etc") - assert.Equal(t, mounts[3].Target, "\\\\.\\pipe\\docker_engine") -} - -func TestDefaultNetworkSettings(t *testing.T) { - t.Run("returns the network with the highest priority when service has multiple networks", func(t *testing.T) { - service := composetypes.ServiceConfig{ - Name: "myService", - Networks: map[string]*composetypes.ServiceNetworkConfig{ - "myNetwork1": { - Priority: 10, - }, - "myNetwork2": { - Priority: 1000, - }, - }, - } - project := composetypes.Project{ - Name: "myProject", - Services: composetypes.Services{ - "myService": service, - }, - Networks: composetypes.Networks(map[string]composetypes.NetworkConfig{ - "myNetwork1": { - Name: "myProject_myNetwork1", - }, - "myNetwork2": { - Name: "myProject_myNetwork2", - }, - }), - } - - networkMode, networkConfig, err := defaultNetworkSettings(&project, service, 1, nil, true, "1.43") - assert.NilError(t, err) - assert.Equal(t, string(networkMode), "myProject_myNetwork2") - assert.Check(t, cmp.Len(networkConfig.EndpointsConfig, 1)) - assert.Check(t, cmp.Contains(networkConfig.EndpointsConfig, "myProject_myNetwork2")) - }) - - t.Run("returns default network when service has no networks", func(t *testing.T) { - service := composetypes.ServiceConfig{ - Name: "myService", - } - project := composetypes.Project{ - Name: "myProject", - Services: composetypes.Services{ - "myService": service, - }, - Networks: composetypes.Networks(map[string]composetypes.NetworkConfig{ - "myNetwork1": { - Name: "myProject_myNetwork1", - }, - "myNetwork2": { - Name: "myProject_myNetwork2", - }, - "default": { - Name: "myProject_default", - }, - }), - } - - networkMode, networkConfig, err := defaultNetworkSettings(&project, service, 1, nil, true, "1.43") - assert.NilError(t, err) - assert.Equal(t, string(networkMode), "myProject_default") - assert.Check(t, cmp.Len(networkConfig.EndpointsConfig, 1)) - assert.Check(t, cmp.Contains(networkConfig.EndpointsConfig, "myProject_default")) - }) - - t.Run("returns none if project has no networks", func(t *testing.T) { - service := composetypes.ServiceConfig{ - Name: "myService", - } - project := composetypes.Project{ - Name: "myProject", - Services: composetypes.Services{ - "myService": service, - }, - } - - networkMode, networkConfig, err := defaultNetworkSettings(&project, service, 1, nil, true, "1.43") - assert.NilError(t, err) - assert.Equal(t, string(networkMode), "none") - assert.Check(t, cmp.Nil(networkConfig)) - }) - - t.Run("returns defined network mode if explicitly set", func(t *testing.T) { - service := composetypes.ServiceConfig{ - Name: "myService", - NetworkMode: "host", - } - project := composetypes.Project{ - Name: "myProject", - Services: composetypes.Services{"myService": service}, - Networks: composetypes.Networks(map[string]composetypes.NetworkConfig{ - "default": { - Name: "myProject_default", - }, - }), - } - - networkMode, networkConfig, err := defaultNetworkSettings(&project, service, 1, nil, true, "1.43") - assert.NilError(t, err) - assert.Equal(t, string(networkMode), "host") - assert.Check(t, cmp.Nil(networkConfig)) - }) -} - -func TestCreateEndpointSettings(t *testing.T) { - eps := createEndpointSettings(&composetypes.Project{ - Name: "projName", - }, composetypes.ServiceConfig{ - Name: "serviceName", - ContainerName: "containerName", - Networks: map[string]*composetypes.ServiceNetworkConfig{ - "netName": { - Priority: 100, - Aliases: []string{"alias1", "alias2"}, - Ipv4Address: "10.16.17.18", - Ipv6Address: "fdb4:7a7f:373a:3f0c::42", - LinkLocalIPs: []string{"169.254.10.20"}, - MacAddress: "10:00:00:00:01", - DriverOpts: composetypes.Options{ - "driverOpt1": "optval1", - "driverOpt2": "optval2", - }, - }, - }, - }, 0, "netName", []string{"link1", "link2"}, true) - assert.Check(t, cmp.DeepEqual(eps, &network.EndpointSettings{ - IPAMConfig: &network.EndpointIPAMConfig{ - IPv4Address: "10.16.17.18", - IPv6Address: "fdb4:7a7f:373a:3f0c::42", - LinkLocalIPs: []string{"169.254.10.20"}, - }, - Links: []string{"link1", "link2"}, - Aliases: []string{"containerName", "serviceName", "alias1", "alias2"}, - MacAddress: "10:00:00:00:01", - DriverOpts: map[string]string{ - "driverOpt1": "optval1", - "driverOpt2": "optval2", - }, - - // FIXME(robmry) - IPAddress and IPv6Gateway are "operational data" fields... - // - The IPv6 address here is the container's address, not the gateway. - // - Both fields will be cleared by the daemon, but they could be removed from - // the request. - IPAddress: "10.16.17.18", - IPv6Gateway: "fdb4:7a7f:373a:3f0c::42", - })) -} - -func Test_buildContainerVolumes(t *testing.T) { - pwd, err := os.Getwd() - assert.NilError(t, err) - - tests := []struct { - name string - yaml string - binds []string - mounts []mountTypes.Mount - }{ - { - name: "bind mount local path", - yaml: ` -services: - test: - volumes: - - ./data:/data -`, - binds: []string{filepath.Join(pwd, "data") + ":/data:rw"}, - mounts: nil, - }, - { - name: "bind mount, not create host path", - yaml: ` -services: - test: - volumes: - - type: bind - source: ./data - target: /data - bind: - create_host_path: false -`, - binds: nil, - mounts: []mountTypes.Mount{ - { - Type: "bind", - Source: filepath.Join(pwd, "data"), - Target: "/data", - BindOptions: &mountTypes.BindOptions{CreateMountpoint: false}, - }, - }, - }, - { - name: "mount volume", - yaml: ` -services: - test: - volumes: - - data:/data -volumes: - data: - name: my_volume -`, - binds: []string{"my_volume:/data:rw"}, - mounts: nil, - }, - { - name: "mount volume, readonly", - yaml: ` -services: - test: - volumes: - - data:/data:ro -volumes: - data: - name: my_volume -`, - binds: []string{"my_volume:/data:ro"}, - mounts: nil, - }, - { - name: "mount volume subpath", - yaml: ` -services: - test: - volumes: - - type: volume - source: data - target: /data - volume: - subpath: test/ -volumes: - data: - name: my_volume -`, - binds: nil, - mounts: []mountTypes.Mount{ - { - Type: "volume", - Source: "my_volume", - Target: "/data", - VolumeOptions: &mountTypes.VolumeOptions{Subpath: "test/"}, - }, - }, - }, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - p, err := composeloader.LoadWithContext(t.Context(), composetypes.ConfigDetails{ - ConfigFiles: []composetypes.ConfigFile{ - { - Filename: "test", - Content: []byte(tt.yaml), - }, - }, - }, func(options *composeloader.Options) { - options.SkipValidation = true - options.SkipConsistencyCheck = true - }) - assert.NilError(t, err) - s := &composeService{} - binds, mounts, err := s.buildContainerVolumes(t.Context(), *p, p.Services["test"], nil) - assert.NilError(t, err) - assert.DeepEqual(t, tt.binds, binds) - assert.DeepEqual(t, tt.mounts, mounts) - }) - } -} diff --git a/pkg/compose/dependencies.go b/pkg/compose/dependencies.go deleted file mode 100644 index c448fd5a176..00000000000 --- a/pkg/compose/dependencies.go +++ /dev/null @@ -1,478 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "slices" - "strings" - "sync" - - "github.com/compose-spec/compose-go/v2/types" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -// ServiceStatus indicates the status of a service -type ServiceStatus int - -// Services status flags -const ( - ServiceStopped ServiceStatus = iota - ServiceStarted -) - -type graphTraversal struct { - mu sync.Mutex - seen map[string]struct{} - ignored map[string]struct{} - - extremityNodesFn func(*Graph) []*Vertex // leaves or roots - adjacentNodesFn func(*Vertex) []*Vertex // getParents or getChildren - filterAdjacentByStatusFn func(*Graph, string, ServiceStatus) []*Vertex // filterChildren or filterParents - targetServiceStatus ServiceStatus - adjacentServiceStatusToSkip ServiceStatus - - visitorFn func(context.Context, string) error - maxConcurrency int -} - -func upDirectionTraversal(visitorFn func(context.Context, string) error) *graphTraversal { - return &graphTraversal{ - extremityNodesFn: leaves, - adjacentNodesFn: getParents, - filterAdjacentByStatusFn: filterChildren, - adjacentServiceStatusToSkip: ServiceStopped, - targetServiceStatus: ServiceStarted, - visitorFn: visitorFn, - } -} - -func downDirectionTraversal(visitorFn func(context.Context, string) error) *graphTraversal { - return &graphTraversal{ - extremityNodesFn: roots, - adjacentNodesFn: getChildren, - filterAdjacentByStatusFn: filterParents, - adjacentServiceStatusToSkip: ServiceStarted, - targetServiceStatus: ServiceStopped, - visitorFn: visitorFn, - } -} - -// InDependencyOrder applies the function to the services of the project taking in account the dependency order -func InDependencyOrder(ctx context.Context, project *types.Project, fn func(context.Context, string) error, options ...func(*graphTraversal)) error { - graph, err := NewGraph(project, ServiceStopped) - if err != nil { - return err - } - t := upDirectionTraversal(fn) - for _, option := range options { - option(t) - } - return t.visit(ctx, graph) -} - -// InReverseDependencyOrder applies the function to the services of the project in reverse order of dependencies -func InReverseDependencyOrder(ctx context.Context, project *types.Project, fn func(context.Context, string) error, options ...func(*graphTraversal)) error { - graph, err := NewGraph(project, ServiceStarted) - if err != nil { - return err - } - t := downDirectionTraversal(fn) - for _, option := range options { - option(t) - } - return t.visit(ctx, graph) -} - -func WithRootNodesAndDown(nodes []string) func(*graphTraversal) { - return func(t *graphTraversal) { - if len(nodes) == 0 { - return - } - originalFn := t.extremityNodesFn - t.extremityNodesFn = func(graph *Graph) []*Vertex { - var want []string - for _, node := range nodes { - vertex := graph.Vertices[node] - want = append(want, vertex.Service) - for _, v := range getAncestors(vertex) { - want = append(want, v.Service) - } - } - - t.ignored = map[string]struct{}{} - for k := range graph.Vertices { - if !slices.Contains(want, k) { - t.ignored[k] = struct{}{} - } - } - - return originalFn(graph) - } - } -} - -func (t *graphTraversal) visit(ctx context.Context, g *Graph) error { - expect := len(g.Vertices) - if expect == 0 { - return nil - } - - eg, ctx := errgroup.WithContext(ctx) - if t.maxConcurrency > 0 { - eg.SetLimit(t.maxConcurrency + 1) - } - nodeCh := make(chan *Vertex, expect) - defer close(nodeCh) - // nodeCh need to allow n=expect writers while reader goroutine could have returner after ctx.Done - eg.Go(func() error { - for { - select { - case <-ctx.Done(): - return nil - case node := <-nodeCh: - expect-- - if expect == 0 { - return nil - } - t.run(ctx, g, eg, t.adjacentNodesFn(node), nodeCh) - } - } - }) - - nodes := t.extremityNodesFn(g) - t.run(ctx, g, eg, nodes, nodeCh) - - return eg.Wait() -} - -// Note: this could be `graph.walk` or whatever -func (t *graphTraversal) run(ctx context.Context, graph *Graph, eg *errgroup.Group, nodes []*Vertex, nodeCh chan *Vertex) { - for _, node := range nodes { - // Don't start this service yet if all of its children have - // not been started yet. - if len(t.filterAdjacentByStatusFn(graph, node.Key, t.adjacentServiceStatusToSkip)) != 0 { - continue - } - - if !t.consume(node.Key) { - // another worker already visited this node - continue - } - - eg.Go(func() error { - var err error - if _, ignore := t.ignored[node.Service]; !ignore { - err = t.visitorFn(ctx, node.Service) - } - if err == nil { - graph.UpdateStatus(node.Key, t.targetServiceStatus) - } - nodeCh <- node - return err - }) - } -} - -func (t *graphTraversal) consume(nodeKey string) bool { - t.mu.Lock() - defer t.mu.Unlock() - if t.seen == nil { - t.seen = make(map[string]struct{}) - } - if _, ok := t.seen[nodeKey]; ok { - return false - } - t.seen[nodeKey] = struct{}{} - return true -} - -// Graph represents project as service dependencies -type Graph struct { - Vertices map[string]*Vertex - lock sync.RWMutex -} - -// Vertex represents a service in the dependencies structure -type Vertex struct { - Key string - Service string - Status ServiceStatus - Children map[string]*Vertex - Parents map[string]*Vertex -} - -func getParents(v *Vertex) []*Vertex { - return v.GetParents() -} - -// GetParents returns a slice with the parent vertices of the Vertex -func (v *Vertex) GetParents() []*Vertex { - var res []*Vertex - for _, p := range v.Parents { - res = append(res, p) - } - return res -} - -func getChildren(v *Vertex) []*Vertex { - return v.GetChildren() -} - -// getAncestors return all descendents for a vertex, might contain duplicates -func getAncestors(v *Vertex) []*Vertex { - var descendents []*Vertex - for _, parent := range v.GetParents() { - descendents = append(descendents, parent) - descendents = append(descendents, getAncestors(parent)...) - } - return descendents -} - -// GetChildren returns a slice with the child vertices of the Vertex -func (v *Vertex) GetChildren() []*Vertex { - var res []*Vertex - for _, p := range v.Children { - res = append(res, p) - } - return res -} - -// NewGraph returns the dependency graph of the services -func NewGraph(project *types.Project, initialStatus ServiceStatus) (*Graph, error) { - graph := &Graph{ - lock: sync.RWMutex{}, - Vertices: map[string]*Vertex{}, - } - - for _, s := range project.Services { - graph.AddVertex(s.Name, s.Name, initialStatus) - } - - for index, s := range project.Services { - for _, name := range s.GetDependencies() { - err := graph.AddEdge(s.Name, name) - if err != nil { - if !s.DependsOn[name].Required { - delete(s.DependsOn, name) - project.Services[index] = s - continue - } - if api.IsNotFoundError(err) { - ds, err := project.GetDisabledService(name) - if err == nil { - return nil, fmt.Errorf("service %s is required by %s but is disabled. Can be enabled by profiles %s", name, s.Name, ds.Profiles) - } - } - return nil, err - } - } - } - - if b, err := graph.HasCycles(); b { - return nil, err - } - - return graph, nil -} - -// NewVertex is the constructor function for the Vertex -func NewVertex(key string, service string, initialStatus ServiceStatus) *Vertex { - return &Vertex{ - Key: key, - Service: service, - Status: initialStatus, - Parents: map[string]*Vertex{}, - Children: map[string]*Vertex{}, - } -} - -// AddVertex adds a vertex to the Graph -func (g *Graph) AddVertex(key string, service string, initialStatus ServiceStatus) { - g.lock.Lock() - defer g.lock.Unlock() - - v := NewVertex(key, service, initialStatus) - g.Vertices[key] = v -} - -// AddEdge adds a relationship of dependency between vertices `source` and `destination` -func (g *Graph) AddEdge(source string, destination string) error { - g.lock.Lock() - defer g.lock.Unlock() - - sourceVertex := g.Vertices[source] - destinationVertex := g.Vertices[destination] - - if sourceVertex == nil { - return fmt.Errorf("could not find %s: %w", source, api.ErrNotFound) - } - if destinationVertex == nil { - return fmt.Errorf("could not find %s: %w", destination, api.ErrNotFound) - } - - // If they are already connected - if _, ok := sourceVertex.Children[destination]; ok { - return nil - } - - sourceVertex.Children[destination] = destinationVertex - destinationVertex.Parents[source] = sourceVertex - - return nil -} - -func leaves(g *Graph) []*Vertex { - return g.Leaves() -} - -// Leaves returns the slice of leaves of the graph -func (g *Graph) Leaves() []*Vertex { - g.lock.Lock() - defer g.lock.Unlock() - - var res []*Vertex - for _, v := range g.Vertices { - if len(v.Children) == 0 { - res = append(res, v) - } - } - - return res -} - -func roots(g *Graph) []*Vertex { - return g.Roots() -} - -// Roots returns the slice of "Roots" of the graph -func (g *Graph) Roots() []*Vertex { - g.lock.Lock() - defer g.lock.Unlock() - - var res []*Vertex - for _, v := range g.Vertices { - if len(v.Parents) == 0 { - res = append(res, v) - } - } - return res -} - -// UpdateStatus updates the status of a certain vertex -func (g *Graph) UpdateStatus(key string, status ServiceStatus) { - g.lock.Lock() - defer g.lock.Unlock() - g.Vertices[key].Status = status -} - -func filterChildren(g *Graph, k string, s ServiceStatus) []*Vertex { - return g.FilterChildren(k, s) -} - -// FilterChildren returns children of a certain vertex that are in a certain status -func (g *Graph) FilterChildren(key string, status ServiceStatus) []*Vertex { - g.lock.Lock() - defer g.lock.Unlock() - - var res []*Vertex - vertex := g.Vertices[key] - - for _, child := range vertex.Children { - if child.Status == status { - res = append(res, child) - } - } - - return res -} - -func filterParents(g *Graph, k string, s ServiceStatus) []*Vertex { - return g.FilterParents(k, s) -} - -// FilterParents returns the parents of a certain vertex that are in a certain status -func (g *Graph) FilterParents(key string, status ServiceStatus) []*Vertex { - g.lock.Lock() - defer g.lock.Unlock() - - var res []*Vertex - vertex := g.Vertices[key] - - for _, parent := range vertex.Parents { - if parent.Status == status { - res = append(res, parent) - } - } - - return res -} - -// HasCycles detects cycles in the graph -func (g *Graph) HasCycles() (bool, error) { - discovered := []string{} - finished := []string{} - - for _, vertex := range g.Vertices { - path := []string{ - vertex.Key, - } - if !slices.Contains(discovered, vertex.Key) && !slices.Contains(finished, vertex.Key) { - var err error - discovered, finished, err = g.visit(vertex.Key, path, discovered, finished) - if err != nil { - return true, err - } - } - } - - return false, nil -} - -func (g *Graph) visit(key string, path []string, discovered []string, finished []string) ([]string, []string, error) { - discovered = append(discovered, key) - - for _, v := range g.Vertices[key].Children { - path := append(path, v.Key) - if slices.Contains(discovered, v.Key) { - return nil, nil, fmt.Errorf("cycle found: %s", strings.Join(path, " -> ")) - } - - if !slices.Contains(finished, v.Key) { - if _, _, err := g.visit(v.Key, path, discovered, finished); err != nil { - return nil, nil, err - } - } - } - - discovered = remove(discovered, key) - finished = append(finished, key) - return discovered, finished, nil -} - -func remove(slice []string, item string) []string { - var s []string - for _, i := range slice { - if i != item { - s = append(s, i) - } - } - return s -} diff --git a/pkg/compose/dependencies_test.go b/pkg/compose/dependencies_test.go deleted file mode 100644 index b22b68b9908..00000000000 --- a/pkg/compose/dependencies_test.go +++ /dev/null @@ -1,429 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "sort" - "sync" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - testify "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/utils" -) - -func createTestProject() *types.Project { - return &types.Project{ - Services: types.Services{ - "test1": { - Name: "test1", - DependsOn: map[string]types.ServiceDependency{ - "test2": {}, - }, - }, - "test2": { - Name: "test2", - DependsOn: map[string]types.ServiceDependency{ - "test3": {}, - }, - }, - "test3": { - Name: "test3", - }, - }, - } -} - -func TestTraversalWithMultipleParents(t *testing.T) { - dependent := types.ServiceConfig{ - Name: "dependent", - DependsOn: make(types.DependsOnConfig), - } - - project := types.Project{ - Services: types.Services{"dependent": dependent}, - } - - for i := 1; i <= 100; i++ { - name := fmt.Sprintf("svc_%d", i) - dependent.DependsOn[name] = types.ServiceDependency{} - - svc := types.ServiceConfig{Name: name} - project.Services[name] = svc - } - - svc := make(chan string, 10) - seen := make(map[string]int) - done := make(chan struct{}) - go func() { - for service := range svc { - seen[service]++ - } - done <- struct{}{} - }() - - err := InDependencyOrder(t.Context(), &project, func(ctx context.Context, service string) error { - svc <- service - return nil - }) - require.NoError(t, err, "Error during iteration") - close(svc) - <-done - - testify.Len(t, seen, 101) - for svc, count := range seen { - assert.Equal(t, 1, count, "Service: %s", svc) - } -} - -func TestInDependencyUpCommandOrder(t *testing.T) { - var order []string - err := InDependencyOrder(t.Context(), createTestProject(), func(ctx context.Context, service string) error { - order = append(order, service) - return nil - }) - require.NoError(t, err, "Error during iteration") - require.Equal(t, []string{"test3", "test2", "test1"}, order) -} - -func TestInDependencyReverseDownCommandOrder(t *testing.T) { - var order []string - err := InReverseDependencyOrder(t.Context(), createTestProject(), func(ctx context.Context, service string) error { - order = append(order, service) - return nil - }) - require.NoError(t, err, "Error during iteration") - require.Equal(t, []string{"test1", "test2", "test3"}, order) -} - -func TestBuildGraph(t *testing.T) { - testCases := []struct { - desc string - services types.Services - expectedVertices map[string]*Vertex - }{ - { - desc: "builds graph with single service", - services: types.Services{ - "test": { - Name: "test", - DependsOn: types.DependsOnConfig{}, - }, - }, - expectedVertices: map[string]*Vertex{ - "test": { - Key: "test", - Service: "test", - Status: ServiceStopped, - Children: map[string]*Vertex{}, - Parents: map[string]*Vertex{}, - }, - }, - }, - { - desc: "builds graph with two separate services", - services: types.Services{ - "test": { - Name: "test", - DependsOn: types.DependsOnConfig{}, - }, - "another": { - Name: "another", - DependsOn: types.DependsOnConfig{}, - }, - }, - expectedVertices: map[string]*Vertex{ - "test": { - Key: "test", - Service: "test", - Status: ServiceStopped, - Children: map[string]*Vertex{}, - Parents: map[string]*Vertex{}, - }, - "another": { - Key: "another", - Service: "another", - Status: ServiceStopped, - Children: map[string]*Vertex{}, - Parents: map[string]*Vertex{}, - }, - }, - }, - { - desc: "builds graph with a service and a dependency", - services: types.Services{ - "test": { - Name: "test", - DependsOn: types.DependsOnConfig{ - "another": types.ServiceDependency{}, - }, - }, - "another": { - Name: "another", - DependsOn: types.DependsOnConfig{}, - }, - }, - expectedVertices: map[string]*Vertex{ - "test": { - Key: "test", - Service: "test", - Status: ServiceStopped, - Children: map[string]*Vertex{ - "another": {}, - }, - Parents: map[string]*Vertex{}, - }, - "another": { - Key: "another", - Service: "another", - Status: ServiceStopped, - Children: map[string]*Vertex{}, - Parents: map[string]*Vertex{ - "test": {}, - }, - }, - }, - }, - { - desc: "builds graph with multiple dependency levels", - services: types.Services{ - "test": { - Name: "test", - DependsOn: types.DependsOnConfig{ - "another": types.ServiceDependency{}, - }, - }, - "another": { - Name: "another", - DependsOn: types.DependsOnConfig{ - "another_dep": types.ServiceDependency{}, - }, - }, - "another_dep": { - Name: "another_dep", - DependsOn: types.DependsOnConfig{}, - }, - }, - expectedVertices: map[string]*Vertex{ - "test": { - Key: "test", - Service: "test", - Status: ServiceStopped, - Children: map[string]*Vertex{ - "another": {}, - }, - Parents: map[string]*Vertex{}, - }, - "another": { - Key: "another", - Service: "another", - Status: ServiceStopped, - Children: map[string]*Vertex{ - "another_dep": {}, - }, - Parents: map[string]*Vertex{ - "test": {}, - }, - }, - "another_dep": { - Key: "another_dep", - Service: "another_dep", - Status: ServiceStopped, - Children: map[string]*Vertex{}, - Parents: map[string]*Vertex{ - "another": {}, - }, - }, - }, - }, - } - for _, tC := range testCases { - t.Run(tC.desc, func(t *testing.T) { - project := types.Project{ - Services: tC.services, - } - - graph, err := NewGraph(&project, ServiceStopped) - assert.NilError(t, err, fmt.Sprintf("failed to build graph for: %s", tC.desc)) - - for k, vertex := range graph.Vertices { - expected, ok := tC.expectedVertices[k] - assert.Equal(t, true, ok) - assert.Equal(t, true, isVertexEqual(*expected, *vertex)) - } - }) - } -} - -func TestBuildGraphDependsOn(t *testing.T) { - testCases := []struct { - desc string - services types.Services - expectedVertices map[string]*Vertex - }{ - { - desc: "service depends on init container which is already removed", - services: types.Services{ - "test": { - Name: "test", - DependsOn: types.DependsOnConfig{ - "test-removed-init-container": types.ServiceDependency{ - Condition: "service_completed_successfully", - Restart: false, - Extensions: types.Extensions(nil), - Required: false, - }, - }, - }, - }, - expectedVertices: map[string]*Vertex{ - "test": { - Key: "test", - Service: "test", - Status: ServiceStopped, - Children: map[string]*Vertex{}, - Parents: map[string]*Vertex{}, - }, - }, - }, - } - for _, tC := range testCases { - t.Run(tC.desc, func(t *testing.T) { - project := types.Project{ - Services: tC.services, - } - - graph, err := NewGraph(&project, ServiceStopped) - assert.NilError(t, err, fmt.Sprintf("failed to build graph for: %s", tC.desc)) - - for k, vertex := range graph.Vertices { - expected, ok := tC.expectedVertices[k] - assert.Equal(t, true, ok) - assert.Equal(t, true, isVertexEqual(*expected, *vertex)) - } - }) - } -} - -func isVertexEqual(a, b Vertex) bool { - childrenEquality := true - for c := range a.Children { - if _, ok := b.Children[c]; !ok { - childrenEquality = false - } - } - parentEquality := true - for p := range a.Parents { - if _, ok := b.Parents[p]; !ok { - parentEquality = false - } - } - return a.Key == b.Key && - a.Service == b.Service && - childrenEquality && - parentEquality -} - -func TestWith_RootNodesAndUp(t *testing.T) { - graph := &Graph{ - lock: sync.RWMutex{}, - Vertices: map[string]*Vertex{}, - } - - /** graph topology: - A B - / \ / \ - G C E - \ / - D - | - F - */ - - graph.AddVertex("A", "A", 0) - graph.AddVertex("B", "B", 0) - graph.AddVertex("C", "C", 0) - graph.AddVertex("D", "D", 0) - graph.AddVertex("E", "E", 0) - graph.AddVertex("F", "F", 0) - graph.AddVertex("G", "G", 0) - - _ = graph.AddEdge("C", "A") - _ = graph.AddEdge("C", "B") - _ = graph.AddEdge("E", "B") - _ = graph.AddEdge("D", "C") - _ = graph.AddEdge("D", "E") - _ = graph.AddEdge("F", "D") - _ = graph.AddEdge("G", "A") - - tests := []struct { - name string - nodes []string - want []string - }{ - { - name: "whole graph", - nodes: []string{"A", "B"}, - want: []string{"A", "B", "C", "D", "E", "F", "G"}, - }, - { - name: "only leaves", - nodes: []string{"F", "G"}, - want: []string{"F", "G"}, - }, - { - name: "simple dependent", - nodes: []string{"D"}, - want: []string{"D", "F"}, - }, - { - name: "diamond dependents", - nodes: []string{"B"}, - want: []string{"B", "C", "D", "E", "F"}, - }, - { - name: "partial graph", - nodes: []string{"A"}, - want: []string{"A", "C", "D", "F", "G"}, - }, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - mx := sync.Mutex{} - expected := utils.Set[string]{} - expected.AddAll("C", "G", "D", "F") - var visited []string - - gt := downDirectionTraversal(func(ctx context.Context, s string) error { - mx.Lock() - defer mx.Unlock() - visited = append(visited, s) - return nil - }) - WithRootNodesAndDown(tt.nodes)(gt) - err := gt.visit(t.Context(), graph) - assert.NilError(t, err) - sort.Strings(visited) - assert.DeepEqual(t, tt.want, visited) - }) - } -} diff --git a/pkg/compose/desktop.go b/pkg/compose/desktop.go deleted file mode 100644 index 9b985407b23..00000000000 --- a/pkg/compose/desktop.go +++ /dev/null @@ -1,41 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strings" -) - -// engineLabelDesktopAddress is used to detect that Compose is running with a -// Docker Desktop context. When this label is present, the value is an endpoint -// address for an in-memory socket (AF_UNIX or named pipe). -const engineLabelDesktopAddress = "com.docker.desktop.address" - -func (s *composeService) isDesktopIntegrationActive(ctx context.Context) (bool, error) { - info, err := s.apiClient().Info(ctx) - if err != nil { - return false, err - } - for _, l := range info.Labels { - k, _, ok := strings.Cut(l, "=") - if ok && k == engineLabelDesktopAddress { - return true, nil - } - } - return false, nil -} diff --git a/pkg/compose/docker_cli_providers.go b/pkg/compose/docker_cli_providers.go deleted file mode 100644 index 207fa3e37a7..00000000000 --- a/pkg/compose/docker_cli_providers.go +++ /dev/null @@ -1,38 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "github.com/docker/cli/cli/command" -) - -// dockerCliContextInfo implements api.ContextInfo using Docker CLI -type dockerCliContextInfo struct { - cli command.Cli -} - -func (c *dockerCliContextInfo) CurrentContext() string { - return c.cli.CurrentContext() -} - -func (c *dockerCliContextInfo) ServerOSType() string { - return c.cli.ServerInfo().OSType -} - -func (c *dockerCliContextInfo) BuildKitEnabled() (bool, error) { - return c.cli.BuildKitEnabled() -} diff --git a/pkg/compose/down.go b/pkg/compose/down.go deleted file mode 100644 index 35eec82ebe3..00000000000 --- a/pkg/compose/down.go +++ /dev/null @@ -1,402 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "strings" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - containerType "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - imageapi "github.com/docker/docker/api/types/image" - "github.com/docker/docker/api/types/network" - "github.com/sirupsen/logrus" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -type downOp func() error - -func (s *composeService) Down(ctx context.Context, projectName string, options api.DownOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.down(ctx, strings.ToLower(projectName), options) - }, "down", s.events) -} - -func (s *composeService) down(ctx context.Context, projectName string, options api.DownOptions) error { //nolint:gocyclo - resourceToRemove := false - - include := oneOffExclude - if options.RemoveOrphans { - include = oneOffInclude - } - containers, err := s.getContainers(ctx, projectName, include, true) - if err != nil { - return err - } - - project := options.Project - if project == nil { - project, err = s.getProjectWithResources(ctx, containers, projectName) - if err != nil { - return err - } - } - - // Check requested services exists in model - services, err := checkSelectedServices(options, project) - if err != nil { - return err - } - - if len(options.Services) > 0 && len(services) == 0 { - logrus.Infof("Any of the services %v not running in project %q", options.Services, projectName) - return nil - } - - options.Services = services - - if len(containers) > 0 { - resourceToRemove = true - } - - err = InReverseDependencyOrder(ctx, project, func(c context.Context, service string) error { - serv := project.Services[service] - if serv.Provider != nil { - return s.runPlugin(ctx, project, serv, "down") - } - serviceContainers := containers.filter(isService(service)) - err := s.removeContainers(ctx, serviceContainers, &serv, options.Timeout, options.Volumes) - return err - }, WithRootNodesAndDown(options.Services)) - if err != nil { - return err - } - - orphans := containers.filter(isOrphaned(project)) - if options.RemoveOrphans && len(orphans) > 0 { - err := s.removeContainers(ctx, orphans, nil, options.Timeout, false) - if err != nil { - return err - } - } - - ops := s.ensureNetworksDown(ctx, project) - - if options.Images != "" { - imgOps, err := s.ensureImagesDown(ctx, project, options) - if err != nil { - return err - } - ops = append(ops, imgOps...) - } - - if options.Volumes { - ops = append(ops, s.ensureVolumesDown(ctx, project)...) - } - - if !resourceToRemove && len(ops) == 0 { - logrus.Warnf("Warning: No resource found to remove for project %q.", projectName) - } - - eg, ctx := errgroup.WithContext(ctx) - for _, op := range ops { - eg.Go(op) - } - return eg.Wait() -} - -func checkSelectedServices(options api.DownOptions, project *types.Project) ([]string, error) { - var services []string - for _, service := range options.Services { - _, err := project.GetService(service) - if err != nil { - if options.Project != nil { - // ran with an explicit compose.yaml file, so we should not ignore - return nil, err - } - // ran without an explicit compose.yaml file, so can't distinguish typo vs container already removed - } else { - services = append(services, service) - } - } - return services, nil -} - -func (s *composeService) ensureVolumesDown(ctx context.Context, project *types.Project) []downOp { - var ops []downOp - for _, vol := range project.Volumes { - if vol.External { - continue - } - volumeName := vol.Name - ops = append(ops, func() error { - return s.removeVolume(ctx, volumeName) - }) - } - - return ops -} - -func (s *composeService) ensureImagesDown(ctx context.Context, project *types.Project, options api.DownOptions) ([]downOp, error) { - imagePruner := NewImagePruner(s.apiClient(), project) - pruneOpts := ImagePruneOptions{ - Mode: ImagePruneMode(options.Images), - RemoveOrphans: options.RemoveOrphans, - } - images, err := imagePruner.ImagesToPrune(ctx, pruneOpts) - if err != nil { - return nil, err - } - - var ops []downOp - for i := range images { - img := images[i] - ops = append(ops, func() error { - return s.removeImage(ctx, img) - }) - } - return ops, nil -} - -func (s *composeService) ensureNetworksDown(ctx context.Context, project *types.Project) []downOp { - var ops []downOp - for key, n := range project.Networks { - if n.External { - continue - } - // loop capture variable for op closure - networkKey := key - idOrName := n.Name - ops = append(ops, func() error { - return s.removeNetwork(ctx, networkKey, project.Name, idOrName) - }) - } - return ops -} - -func (s *composeService) removeNetwork(ctx context.Context, composeNetworkName string, projectName string, name string) error { - networks, err := s.apiClient().NetworkList(ctx, network.ListOptions{ - Filters: filters.NewArgs( - projectFilter(projectName), - networkFilter(composeNetworkName)), - }) - if err != nil { - return fmt.Errorf("failed to list networks: %w", err) - } - - if len(networks) == 0 { - return nil - } - - eventName := fmt.Sprintf("Network %s", name) - s.events.On(removingEvent(eventName)) - - var found int - for _, net := range networks { - if net.Name != name { - continue - } - nw, err := s.apiClient().NetworkInspect(ctx, net.ID, network.InspectOptions{}) - if errdefs.IsNotFound(err) { - s.events.On(newEvent(eventName, api.Warning, "No resource found to remove")) - return nil - } - if err != nil { - return err - } - if len(nw.Containers) > 0 { - s.events.On(newEvent(eventName, api.Warning, "Resource is still in use")) - found++ - continue - } - - if err := s.apiClient().NetworkRemove(ctx, net.ID); err != nil { - if errdefs.IsNotFound(err) { - continue - } - s.events.On(errorEvent(eventName, err.Error())) - return fmt.Errorf("failed to remove network %s: %w", name, err) - } - s.events.On(removedEvent(eventName)) - found++ - } - - if found == 0 { - // in practice, it's extremely unlikely for this to ever occur, as it'd - // mean the network was present when we queried at the start of this - // method but was then deleted by something else in the interim - s.events.On(newEvent(eventName, api.Warning, "No resource found to remove")) - return nil - } - return nil -} - -func (s *composeService) removeImage(ctx context.Context, image string) error { - id := fmt.Sprintf("Image %s", image) - s.events.On(newEvent(id, api.Working, "Removing")) - _, err := s.apiClient().ImageRemove(ctx, image, imageapi.RemoveOptions{}) - if err == nil { - s.events.On(newEvent(id, api.Done, "Removed")) - return nil - } - if errdefs.IsConflict(err) { - s.events.On(newEvent(id, api.Warning, "Resource is still in use")) - return nil - } - if errdefs.IsNotFound(err) { - s.events.On(newEvent(id, api.Done, "Warning: No resource found to remove")) - return nil - } - return err -} - -func (s *composeService) removeVolume(ctx context.Context, id string) error { - resource := fmt.Sprintf("Volume %s", id) - - _, err := s.apiClient().VolumeInspect(ctx, id) - if errdefs.IsNotFound(err) { - // Already gone - return nil - } - - s.events.On(newEvent(resource, api.Working, "Removing")) - err = s.apiClient().VolumeRemove(ctx, id, true) - if err == nil { - s.events.On(newEvent(resource, api.Done, "Removed")) - return nil - } - if errdefs.IsConflict(err) { - s.events.On(newEvent(resource, api.Warning, "Resource is still in use")) - return nil - } - if errdefs.IsNotFound(err) { - s.events.On(newEvent(resource, api.Done, "Warning: No resource found to remove")) - return nil - } - return err -} - -func (s *composeService) stopContainer(ctx context.Context, service *types.ServiceConfig, ctr containerType.Summary, timeout *time.Duration, listener api.ContainerEventListener) error { - eventName := getContainerProgressName(ctr) - s.events.On(stoppingEvent(eventName)) - - if service != nil { - for _, hook := range service.PreStop { - err := s.runHook(ctx, ctr, *service, hook, listener) - if err != nil { - // Ignore errors indicating that some containers were already stopped or removed. - if errdefs.IsNotFound(err) || errdefs.IsConflict(err) { - return nil - } - return err - } - } - } - - timeoutInSecond := utils.DurationSecondToInt(timeout) - err := s.apiClient().ContainerStop(ctx, ctr.ID, containerType.StopOptions{Timeout: timeoutInSecond}) - if err != nil { - s.events.On(errorEvent(eventName, "Error while Stopping")) - return err - } - s.events.On(stoppedEvent(eventName)) - return nil -} - -func (s *composeService) stopContainers(ctx context.Context, serv *types.ServiceConfig, containers []containerType.Summary, timeout *time.Duration, listener api.ContainerEventListener) error { - eg, ctx := errgroup.WithContext(ctx) - for _, ctr := range containers { - eg.Go(func() error { - return s.stopContainer(ctx, serv, ctr, timeout, listener) - }) - } - return eg.Wait() -} - -func (s *composeService) removeContainers(ctx context.Context, containers []containerType.Summary, service *types.ServiceConfig, timeout *time.Duration, volumes bool) error { - eg, ctx := errgroup.WithContext(ctx) - for _, ctr := range containers { - eg.Go(func() error { - return s.stopAndRemoveContainer(ctx, ctr, service, timeout, volumes) - }) - } - return eg.Wait() -} - -func (s *composeService) stopAndRemoveContainer(ctx context.Context, ctr containerType.Summary, service *types.ServiceConfig, timeout *time.Duration, volumes bool) error { - eventName := getContainerProgressName(ctr) - err := s.stopContainer(ctx, service, ctr, timeout, nil) - if errdefs.IsNotFound(err) { - s.events.On(removedEvent(eventName)) - return nil - } - if err != nil { - return err - } - s.events.On(removingEvent(eventName)) - err = s.apiClient().ContainerRemove(ctx, ctr.ID, containerType.RemoveOptions{ - Force: true, - RemoveVolumes: volumes, - }) - if err != nil && !errdefs.IsNotFound(err) && !errdefs.IsConflict(err) { - s.events.On(errorEvent(eventName, "Error while Removing")) - return err - } - s.events.On(removedEvent(eventName)) - return nil -} - -func (s *composeService) getProjectWithResources(ctx context.Context, containers Containers, projectName string) (*types.Project, error) { - containers = containers.filter(isNotOneOff) - p, err := s.projectFromName(containers, projectName) - if err != nil && !api.IsNotFoundError(err) { - return nil, err - } - project, err := p.WithServicesTransform(func(name string, service types.ServiceConfig) (types.ServiceConfig, error) { - for k := range service.DependsOn { - if dependency, ok := service.DependsOn[k]; ok { - dependency.Required = false - service.DependsOn[k] = dependency - } - } - return service, nil - }) - if err != nil { - return nil, err - } - - volumes, err := s.actualVolumes(ctx, projectName) - if err != nil { - return nil, err - } - project.Volumes = volumes - - networks, err := s.actualNetworks(ctx, projectName) - if err != nil { - return nil, err - } - project.Networks = networks - - return project, nil -} diff --git a/pkg/compose/down_test.go b/pkg/compose/down_test.go deleted file mode 100644 index 0b5852a2fa3..00000000000 --- a/pkg/compose/down_test.go +++ /dev/null @@ -1,419 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "fmt" - "os" - "strings" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/docker/cli/cli/streams" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/api/types/volume" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - compose "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/mocks" -) - -func TestDown(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)).Return( - []container.Summary{ - testContainer("service1", "123", false), - testContainer("service2", "456", false), - testContainer("service2", "789", false), - testContainer("service_orphan", "321", true), - }, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - - // network names are not guaranteed to be unique, ensure Compose handles - // cleanup properly if duplicates are inadvertently created - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{ - {ID: "abc123", Name: "myProject_default", Labels: map[string]string{compose.NetworkLabel: "default"}}, - {ID: "def456", Name: "myProject_default", Labels: map[string]string{compose.NetworkLabel: "default"}}, - }, nil) - - stopOptions := container.StopOptions{} - api.EXPECT().ContainerStop(gomock.Any(), "123", stopOptions).Return(nil) - api.EXPECT().ContainerStop(gomock.Any(), "456", stopOptions).Return(nil) - api.EXPECT().ContainerStop(gomock.Any(), "789", stopOptions).Return(nil) - - api.EXPECT().ContainerRemove(gomock.Any(), "123", container.RemoveOptions{Force: true}).Return(nil) - api.EXPECT().ContainerRemove(gomock.Any(), "456", container.RemoveOptions{Force: true}).Return(nil) - api.EXPECT().ContainerRemove(gomock.Any(), "789", container.RemoveOptions{Force: true}).Return(nil) - - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{ - Filters: filters.NewArgs( - projectFilter(strings.ToLower(testProject)), - networkFilter("default")), - }).Return([]network.Summary{ - {ID: "abc123", Name: "myProject_default"}, - {ID: "def456", Name: "myProject_default"}, - }, nil) - api.EXPECT().NetworkInspect(gomock.Any(), "abc123", gomock.Any()).Return(network.Inspect{ID: "abc123"}, nil) - api.EXPECT().NetworkInspect(gomock.Any(), "def456", gomock.Any()).Return(network.Inspect{ID: "def456"}, nil) - api.EXPECT().NetworkRemove(gomock.Any(), "abc123").Return(nil) - api.EXPECT().NetworkRemove(gomock.Any(), "def456").Return(nil) - - err = tested.Down(t.Context(), strings.ToLower(testProject), compose.DownOptions{}) - assert.NilError(t, err) -} - -func TestDownWithGivenServices(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)).Return( - []container.Summary{ - testContainer("service1", "123", false), - testContainer("service2", "456", false), - testContainer("service2", "789", false), - testContainer("service_orphan", "321", true), - }, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - - // network names are not guaranteed to be unique, ensure Compose handles - // cleanup properly if duplicates are inadvertently created - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{ - {ID: "abc123", Name: "myProject_default", Labels: map[string]string{compose.NetworkLabel: "default"}}, - {ID: "def456", Name: "myProject_default", Labels: map[string]string{compose.NetworkLabel: "default"}}, - }, nil) - - stopOptions := container.StopOptions{} - api.EXPECT().ContainerStop(gomock.Any(), "123", stopOptions).Return(nil) - - api.EXPECT().ContainerRemove(gomock.Any(), "123", container.RemoveOptions{Force: true}).Return(nil) - - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{ - Filters: filters.NewArgs( - projectFilter(strings.ToLower(testProject)), - networkFilter("default")), - }).Return([]network.Summary{ - {ID: "abc123", Name: "myProject_default"}, - }, nil) - api.EXPECT().NetworkInspect(gomock.Any(), "abc123", gomock.Any()).Return(network.Inspect{ID: "abc123"}, nil) - api.EXPECT().NetworkRemove(gomock.Any(), "abc123").Return(nil) - - err = tested.Down(t.Context(), strings.ToLower(testProject), compose.DownOptions{ - Services: []string{"service1", "not-running-service"}, - }) - assert.NilError(t, err) -} - -func TestDownWithSpecifiedServiceButTheServicesAreNotRunning(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)).Return( - []container.Summary{ - testContainer("service1", "123", false), - testContainer("service2", "456", false), - testContainer("service2", "789", false), - testContainer("service_orphan", "321", true), - }, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - - // network names are not guaranteed to be unique, ensure Compose handles - // cleanup properly if duplicates are inadvertently created - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{ - {ID: "abc123", Name: "myProject_default", Labels: map[string]string{compose.NetworkLabel: "default"}}, - {ID: "def456", Name: "myProject_default", Labels: map[string]string{compose.NetworkLabel: "default"}}, - }, nil) - - err = tested.Down(t.Context(), strings.ToLower(testProject), compose.DownOptions{ - Services: []string{"not-running-service1", "not-running-service2"}, - }) - assert.NilError(t, err) -} - -func TestDownRemoveOrphans(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(true)).Return( - []container.Summary{ - testContainer("service1", "123", false), - testContainer("service2", "789", false), - testContainer("service_orphan", "321", true), - }, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{ - { - Name: "myProject_default", - Labels: map[string]string{compose.NetworkLabel: "default"}, - }, - }, nil) - - stopOptions := container.StopOptions{} - api.EXPECT().ContainerStop(gomock.Any(), "123", stopOptions).Return(nil) - api.EXPECT().ContainerStop(gomock.Any(), "789", stopOptions).Return(nil) - api.EXPECT().ContainerStop(gomock.Any(), "321", stopOptions).Return(nil) - - api.EXPECT().ContainerRemove(gomock.Any(), "123", container.RemoveOptions{Force: true}).Return(nil) - api.EXPECT().ContainerRemove(gomock.Any(), "789", container.RemoveOptions{Force: true}).Return(nil) - api.EXPECT().ContainerRemove(gomock.Any(), "321", container.RemoveOptions{Force: true}).Return(nil) - - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{ - Filters: filters.NewArgs( - networkFilter("default"), - projectFilter(strings.ToLower(testProject)), - ), - }).Return([]network.Summary{{ID: "abc123", Name: "myProject_default"}}, nil) - api.EXPECT().NetworkInspect(gomock.Any(), "abc123", gomock.Any()).Return(network.Inspect{ID: "abc123"}, nil) - api.EXPECT().NetworkRemove(gomock.Any(), "abc123").Return(nil) - - err = tested.Down(t.Context(), strings.ToLower(testProject), compose.DownOptions{RemoveOrphans: true}) - assert.NilError(t, err) -} - -func TestDownRemoveVolumes(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)).Return( - []container.Summary{testContainer("service1", "123", false)}, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{ - Volumes: []*volume.Volume{{Name: "myProject_volume"}}, - }, nil) - api.EXPECT().VolumeInspect(gomock.Any(), "myProject_volume"). - Return(volume.Volume{}, nil) - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return(nil, nil) - - api.EXPECT().ContainerStop(gomock.Any(), "123", container.StopOptions{}).Return(nil) - api.EXPECT().ContainerRemove(gomock.Any(), "123", container.RemoveOptions{Force: true, RemoveVolumes: true}).Return(nil) - - api.EXPECT().VolumeRemove(gomock.Any(), "myProject_volume", true).Return(nil) - - err = tested.Down(t.Context(), strings.ToLower(testProject), compose.DownOptions{Volumes: true}) - assert.NilError(t, err) -} - -func TestDownRemoveImages(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - opts := compose.DownOptions{ - Project: &types.Project{ - Name: strings.ToLower(testProject), - Services: types.Services{ - "local-anonymous": {Name: "local-anonymous"}, - "local-named": {Name: "local-named", Image: "local-named-image"}, - "remote": {Name: "remote", Image: "remote-image"}, - "remote-tagged": {Name: "remote-tagged", Image: "registry.example.com/remote-image-tagged:v1.0"}, - "no-images-anonymous": {Name: "no-images-anonymous"}, - "no-images-named": {Name: "no-images-named", Image: "missing-named-image"}, - }, - }, - } - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)). - Return([]container.Summary{ - testContainer("service1", "123", false), - }, nil). - AnyTimes() - - api.EXPECT().ImageList(gomock.Any(), image.ListOptions{ - Filters: filters.NewArgs( - projectFilter(strings.ToLower(testProject)), - filters.Arg("dangling", "false"), - ), - }).Return([]image.Summary{ - { - Labels: types.Labels{compose.ServiceLabel: "local-anonymous"}, - RepoTags: []string{"testproject-local-anonymous:latest"}, - }, - { - Labels: types.Labels{compose.ServiceLabel: "local-named"}, - RepoTags: []string{"local-named-image:latest"}, - }, - }, nil).AnyTimes() - - imagesToBeInspected := map[string]bool{ - "testproject-local-anonymous": true, - "local-named-image": true, - "remote-image": true, - "testproject-no-images-anonymous": false, - "missing-named-image": false, - } - for img, exists := range imagesToBeInspected { - var resp image.InspectResponse - var err error - if exists { - resp.RepoTags = []string{img} - } else { - err = errdefs.ErrNotFound.WithMessage(fmt.Sprintf("test specified that image %q should not exist", img)) - } - - api.EXPECT().ImageInspect(gomock.Any(), img). - Return(resp, err). - AnyTimes() - } - - api.EXPECT().ImageInspect(gomock.Any(), "registry.example.com/remote-image-tagged:v1.0"). - Return(image.InspectResponse{RepoTags: []string{"registry.example.com/remote-image-tagged:v1.0"}}, nil). - AnyTimes() - - localImagesToBeRemoved := []string{ - "testproject-local-anonymous:latest", - "local-named-image:latest", - } - for _, img := range localImagesToBeRemoved { - // test calls down --rmi=local then down --rmi=all, so local images - // get "removed" 2x, while other images are only 1x - api.EXPECT().ImageRemove(gomock.Any(), img, image.RemoveOptions{}). - Return(nil, nil). - Times(2) - } - - t.Log("-> docker compose down --rmi=local") - opts.Images = "local" - err = tested.Down(t.Context(), strings.ToLower(testProject), opts) - assert.NilError(t, err) - - otherImagesToBeRemoved := []string{ - "remote-image:latest", - "registry.example.com/remote-image-tagged:v1.0", - } - for _, img := range otherImagesToBeRemoved { - api.EXPECT().ImageRemove(gomock.Any(), img, image.RemoveOptions{}). - Return(nil, nil). - Times(1) - } - - t.Log("-> docker compose down --rmi=all") - opts.Images = "all" - err = tested.Down(t.Context(), strings.ToLower(testProject), opts) - assert.NilError(t, err) -} - -func TestDownRemoveImages_NoLabel(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - ctr := testContainer("service1", "123", false) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)).Return( - []container.Summary{ctr}, nil) - - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{ - Volumes: []*volume.Volume{{Name: "myProject_volume"}}, - }, nil) - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return(nil, nil) - - // ImageList returns no images for the project since they were unlabeled - // (created by an older version of Compose) - api.EXPECT().ImageList(gomock.Any(), image.ListOptions{ - Filters: filters.NewArgs( - projectFilter(strings.ToLower(testProject)), - filters.Arg("dangling", "false"), - ), - }).Return(nil, nil) - - api.EXPECT().ImageInspect(gomock.Any(), "testproject-service1"). - Return(image.InspectResponse{}, nil) - - api.EXPECT().ContainerStop(gomock.Any(), "123", container.StopOptions{}).Return(nil) - api.EXPECT().ContainerRemove(gomock.Any(), "123", container.RemoveOptions{Force: true}).Return(nil) - - api.EXPECT().ImageRemove(gomock.Any(), "testproject-service1:latest", image.RemoveOptions{}).Return(nil, nil) - - err = tested.Down(t.Context(), strings.ToLower(testProject), compose.DownOptions{Images: "local"}) - assert.NilError(t, err) -} - -func prepareMocks(mockCtrl *gomock.Controller) (*mocks.MockAPIClient, *mocks.MockCli) { - api := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - cli.EXPECT().Client().Return(api).AnyTimes() - cli.EXPECT().Err().Return(streams.NewOut(os.Stderr)).AnyTimes() - cli.EXPECT().Out().Return(streams.NewOut(os.Stdout)).AnyTimes() - return api, cli -} diff --git a/pkg/compose/envresolver.go b/pkg/compose/envresolver.go deleted file mode 100644 index a86d9351919..00000000000 --- a/pkg/compose/envresolver.go +++ /dev/null @@ -1,66 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "runtime" - "strings" -) - -// isCaseInsensitiveEnvVars is true on platforms where environment variable names are treated case-insensitively. -var isCaseInsensitiveEnvVars = (runtime.GOOS == "windows") - -// envResolver returns resolver for environment variables suitable for the current platform. -// Expected to be used with `MappingWithEquals.Resolve`. -// Updates in `environment` may not be reflected. -func envResolver(environment map[string]string) func(string) (string, bool) { - return envResolverWithCase(environment, isCaseInsensitiveEnvVars) -} - -// envResolverWithCase returns resolver for environment variables with the specified case-sensitive condition. -// Expected to be used with `MappingWithEquals.Resolve`. -// Updates in `environment` may not be reflected. -func envResolverWithCase(environment map[string]string, caseInsensitive bool) func(string) (string, bool) { - if environment == nil { - return func(s string) (string, bool) { - return "", false - } - } - if !caseInsensitive { - return func(s string) (string, bool) { - v, ok := environment[s] - return v, ok - } - } - // variable names must be treated case-insensitively. - // Resolves in this way: - // * Return the value if its name matches with the passed name case-sensitively. - // * Otherwise, return the value if its lower-cased name matches lower-cased passed name. - // * The value is indefinite if multiple variable matches. - loweredEnvironment := make(map[string]string, len(environment)) - for k, v := range environment { - loweredEnvironment[strings.ToLower(k)] = v - } - return func(s string) (string, bool) { - v, ok := environment[s] - if ok { - return v, ok - } - v, ok = loweredEnvironment[strings.ToLower(s)] - return v, ok - } -} diff --git a/pkg/compose/envresolver_test.go b/pkg/compose/envresolver_test.go deleted file mode 100644 index fca5f719186..00000000000 --- a/pkg/compose/envresolver_test.go +++ /dev/null @@ -1,115 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "gotest.tools/v3/assert" -) - -func Test_EnvResolverWithCase(t *testing.T) { - tests := []struct { - name string - environment map[string]string - caseInsensitive bool - search string - expectedValue string - expectedOk bool - }{ - { - name: "case sensitive/case match", - environment: map[string]string{ - "Env1": "Value1", - "Env2": "Value2", - }, - caseInsensitive: false, - search: "Env1", - expectedValue: "Value1", - expectedOk: true, - }, - { - name: "case sensitive/case unmatch", - environment: map[string]string{ - "Env1": "Value1", - "Env2": "Value2", - }, - caseInsensitive: false, - search: "ENV1", - expectedValue: "", - expectedOk: false, - }, - { - name: "case sensitive/nil environment", - environment: nil, - caseInsensitive: false, - search: "Env1", - expectedValue: "", - expectedOk: false, - }, - { - name: "case insensitive/case match", - environment: map[string]string{ - "Env1": "Value1", - "Env2": "Value2", - }, - caseInsensitive: true, - search: "Env1", - expectedValue: "Value1", - expectedOk: true, - }, - { - name: "case insensitive/case unmatch", - environment: map[string]string{ - "Env1": "Value1", - "Env2": "Value2", - }, - caseInsensitive: true, - search: "ENV1", - expectedValue: "Value1", - expectedOk: true, - }, - { - name: "case insensitive/unmatch", - environment: map[string]string{ - "Env1": "Value1", - "Env2": "Value2", - }, - caseInsensitive: true, - search: "Env3", - expectedValue: "", - expectedOk: false, - }, - { - name: "case insensitive/nil environment", - environment: nil, - caseInsensitive: true, - search: "Env1", - expectedValue: "", - expectedOk: false, - }, - } - - for _, test := range tests { - t.Run(test.name, func(t *testing.T) { - f := envResolverWithCase(test.environment, test.caseInsensitive) - v, ok := f(test.search) - assert.Equal(t, v, test.expectedValue) - assert.Equal(t, ok, test.expectedOk) - }) - } -} diff --git a/pkg/compose/events.go b/pkg/compose/events.go deleted file mode 100644 index af75bd7c530..00000000000 --- a/pkg/compose/events.go +++ /dev/null @@ -1,83 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "slices" - "strings" - "time" - - "github.com/docker/docker/api/types/events" - "github.com/docker/docker/api/types/filters" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Events(ctx context.Context, projectName string, options api.EventsOptions) error { - projectName = strings.ToLower(projectName) - evts, errors := s.apiClient().Events(ctx, events.ListOptions{ - Filters: filters.NewArgs(projectFilter(projectName)), - Since: options.Since, - Until: options.Until, - }) - for { - select { - case event := <-evts: - // TODO: support other event types - if event.Type != "container" { - continue - } - - oneOff := event.Actor.Attributes[api.OneoffLabel] - if oneOff == "True" { - // ignore - continue - } - service := event.Actor.Attributes[api.ServiceLabel] - if len(options.Services) > 0 && !slices.Contains(options.Services, service) { - continue - } - - attributes := map[string]string{} - for k, v := range event.Actor.Attributes { - if strings.HasPrefix(k, "com.docker.compose.") { - continue - } - attributes[k] = v - } - - timestamp := time.Unix(event.Time, 0) - if event.TimeNano != 0 { - timestamp = time.Unix(0, event.TimeNano) - } - err := options.Consumer(api.Event{ - Timestamp: timestamp, - Service: service, - Container: event.Actor.ID, - Status: string(event.Action), - Attributes: attributes, - }) - if err != nil { - return err - } - - case err := <-errors: - return err - } - } -} diff --git a/pkg/compose/exec.go b/pkg/compose/exec.go deleted file mode 100644 index 311aebaa2c9..00000000000 --- a/pkg/compose/exec.go +++ /dev/null @@ -1,63 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "strings" - - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command/container" - containerType "github.com/docker/docker/api/types/container" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Exec(ctx context.Context, projectName string, options api.RunOptions) (int, error) { - projectName = strings.ToLower(projectName) - target, err := s.getExecTarget(ctx, projectName, options) - if err != nil { - return 0, err - } - - exec := container.NewExecOptions() - exec.Interactive = options.Interactive - exec.TTY = options.Tty - exec.Detach = options.Detach - exec.User = options.User - exec.Privileged = options.Privileged - exec.Workdir = options.WorkingDir - exec.Command = options.Command - for _, v := range options.Environment { - err := exec.Env.Set(v) - if err != nil { - return 0, err - } - } - - err = container.RunExec(ctx, s.dockerCli, target.ID, exec) - var sterr cli.StatusError - if errors.As(err, &sterr) { - return sterr.StatusCode, err - } - return 0, err -} - -func (s *composeService) getExecTarget(ctx context.Context, projectName string, opts api.RunOptions) (containerType.Summary, error) { - return s.getSpecifiedContainer(ctx, projectName, oneOffInclude, false, opts.Service, opts.Index) -} diff --git a/pkg/compose/export.go b/pkg/compose/export.go deleted file mode 100644 index 5de9d837be9..00000000000 --- a/pkg/compose/export.go +++ /dev/null @@ -1,94 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/moby/sys/atomicwriter" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Export(ctx context.Context, projectName string, options api.ExportOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.export(ctx, projectName, options) - }, "export", s.events) -} - -func (s *composeService) export(ctx context.Context, projectName string, options api.ExportOptions) error { - projectName = strings.ToLower(projectName) - - container, err := s.getSpecifiedContainer(ctx, projectName, oneOffInclude, false, options.Service, options.Index) - if err != nil { - return err - } - - if options.Output == "" { - if s.stdout().IsTerminal() { - return fmt.Errorf("output option is required when exporting to terminal") - } - } else if err := command.ValidateOutputPath(options.Output); err != nil { - return fmt.Errorf("failed to export container: %w", err) - } - - name := getCanonicalContainerName(container) - s.events.On(api.Resource{ - ID: name, - Text: api.StatusExporting, - Status: api.Working, - }) - - responseBody, err := s.apiClient().ContainerExport(ctx, container.ID) - if err != nil { - return err - } - - defer func() { - if err := responseBody.Close(); err != nil { - s.events.On(errorEventf(name, "Failed to close response body: %s", err.Error())) - } - }() - - if !s.dryRun { - if options.Output == "" { - _, err := io.Copy(s.stdout(), responseBody) - return err - } else { - writer, err := atomicwriter.New(options.Output, 0o600) - if err != nil { - return err - } - defer func() { _ = writer.Close() }() - - _, err = io.Copy(writer, responseBody) - return err - } - } - - s.events.On(api.Resource{ - ID: name, - Text: api.StatusExported, - Status: api.Done, - }) - - return nil -} diff --git a/pkg/compose/filters.go b/pkg/compose/filters.go deleted file mode 100644 index f3038ab1444..00000000000 --- a/pkg/compose/filters.go +++ /dev/null @@ -1,57 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "fmt" - - "github.com/docker/docker/api/types/filters" - - "github.com/docker/compose/v5/pkg/api" -) - -func projectFilter(projectName string) filters.KeyValuePair { - return filters.Arg("label", fmt.Sprintf("%s=%s", api.ProjectLabel, projectName)) -} - -func serviceFilter(serviceName string) filters.KeyValuePair { - return filters.Arg("label", fmt.Sprintf("%s=%s", api.ServiceLabel, serviceName)) -} - -func networkFilter(name string) filters.KeyValuePair { - return filters.Arg("label", fmt.Sprintf("%s=%s", api.NetworkLabel, name)) -} - -func oneOffFilter(b bool) filters.KeyValuePair { - v := "False" - if b { - v = "True" - } - return filters.Arg("label", fmt.Sprintf("%s=%s", api.OneoffLabel, v)) -} - -func containerNumberFilter(index int) filters.KeyValuePair { - return filters.Arg("label", fmt.Sprintf("%s=%d", api.ContainerNumberLabel, index)) -} - -func hasProjectLabelFilter() filters.KeyValuePair { - return filters.Arg("label", api.ProjectLabel) -} - -func hasConfigHashLabel() filters.KeyValuePair { - return filters.Arg("label", api.ConfigHashLabel) -} diff --git a/pkg/compose/generate.go b/pkg/compose/generate.go deleted file mode 100644 index a23bfe8f217..00000000000 --- a/pkg/compose/generate.go +++ /dev/null @@ -1,248 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "maps" - "slices" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/mount" - "github.com/docker/docker/api/types/network" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Generate(ctx context.Context, options api.GenerateOptions) (*types.Project, error) { - filtersListNames := filters.NewArgs() - filtersListIDs := filters.NewArgs() - for _, containerName := range options.Containers { - filtersListNames.Add("name", containerName) - filtersListIDs.Add("id", containerName) - } - containers, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - Filters: filtersListNames, - All: true, - }) - if err != nil { - return nil, err - } - - containersByIds, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - Filters: filtersListIDs, - All: true, - }) - if err != nil { - return nil, err - } - - for _, ctr := range containersByIds { - if !slices.ContainsFunc(containers, func(summary container.Summary) bool { - return summary.ID == ctr.ID - }) { - containers = append(containers, ctr) - } - } - - if len(containers) == 0 { - return nil, fmt.Errorf("no container(s) found with the following name(s): %s", strings.Join(options.Containers, ",")) - } - - return s.createProjectFromContainers(containers, options.ProjectName) -} - -func (s *composeService) createProjectFromContainers(containers []container.Summary, projectName string) (*types.Project, error) { - project := &types.Project{} - services := types.Services{} - networks := types.Networks{} - volumes := types.Volumes{} - secrets := types.Secrets{} - - if projectName != "" { - project.Name = projectName - } - - for _, c := range containers { - // if the container is from a previous Compose application, use the existing service name - serviceLabel, ok := c.Labels[api.ServiceLabel] - if !ok { - serviceLabel = getCanonicalContainerName(c) - } - service, ok := services[serviceLabel] - if !ok { - service = types.ServiceConfig{ - Name: serviceLabel, - Image: c.Image, - Labels: c.Labels, - } - } - service.Scale = increment(service.Scale) - - inspect, err := s.apiClient().ContainerInspect(context.Background(), c.ID) - if err != nil { - services[serviceLabel] = service - continue - } - s.extractComposeConfiguration(&service, inspect, volumes, secrets, networks) - service.Labels = cleanDockerPreviousLabels(service.Labels) - services[serviceLabel] = service - } - - project.Services = services - project.Networks = networks - project.Volumes = volumes - project.Secrets = secrets - return project, nil -} - -func (s *composeService) extractComposeConfiguration(service *types.ServiceConfig, inspect container.InspectResponse, volumes types.Volumes, secrets types.Secrets, networks types.Networks) { - service.Environment = types.NewMappingWithEquals(inspect.Config.Env) - if inspect.Config.Healthcheck != nil { - healthConfig := inspect.Config.Healthcheck - service.HealthCheck = s.toComposeHealthCheck(healthConfig) - } - if len(inspect.Mounts) > 0 { - detectedVolumes, volumeConfigs, detectedSecrets, secretsConfigs := s.toComposeVolumes(inspect.Mounts) - service.Volumes = append(service.Volumes, volumeConfigs...) - service.Secrets = append(service.Secrets, secretsConfigs...) - maps.Copy(volumes, detectedVolumes) - maps.Copy(secrets, detectedSecrets) - } - if len(inspect.NetworkSettings.Networks) > 0 { - detectedNetworks, networkConfigs := s.toComposeNetwork(inspect.NetworkSettings.Networks) - service.Networks = networkConfigs - maps.Copy(networks, detectedNetworks) - } - if len(inspect.HostConfig.PortBindings) > 0 { - for key, portBindings := range inspect.HostConfig.PortBindings { - for _, portBinding := range portBindings { - service.Ports = append(service.Ports, types.ServicePortConfig{ - Target: uint32(key.Int()), - Published: portBinding.HostPort, - Protocol: key.Proto(), - HostIP: portBinding.HostIP, - }) - } - } - } -} - -func (s *composeService) toComposeHealthCheck(healthConfig *container.HealthConfig) *types.HealthCheckConfig { - var healthCheck types.HealthCheckConfig - healthCheck.Test = healthConfig.Test - if healthConfig.Timeout != 0 { - timeout := types.Duration(healthConfig.Timeout) - healthCheck.Timeout = &timeout - } - if healthConfig.Interval != 0 { - interval := types.Duration(healthConfig.Interval) - healthCheck.Interval = &interval - } - if healthConfig.StartPeriod != 0 { - startPeriod := types.Duration(healthConfig.StartPeriod) - healthCheck.StartPeriod = &startPeriod - } - if healthConfig.StartInterval != 0 { - startInterval := types.Duration(healthConfig.StartInterval) - healthCheck.StartInterval = &startInterval - } - if healthConfig.Retries != 0 { - retries := uint64(healthConfig.Retries) - healthCheck.Retries = &retries - } - return &healthCheck -} - -func (s *composeService) toComposeVolumes(volumes []container.MountPoint) (map[string]types.VolumeConfig, - []types.ServiceVolumeConfig, map[string]types.SecretConfig, []types.ServiceSecretConfig, -) { - volumeConfigs := make(map[string]types.VolumeConfig) - secretConfigs := make(map[string]types.SecretConfig) - var serviceVolumeConfigs []types.ServiceVolumeConfig - var serviceSecretConfigs []types.ServiceSecretConfig - - for _, volume := range volumes { - serviceVC := types.ServiceVolumeConfig{ - Type: string(volume.Type), - Source: volume.Source, - Target: volume.Destination, - ReadOnly: !volume.RW, - } - switch volume.Type { - case mount.TypeVolume: - serviceVC.Source = volume.Name - vol := types.VolumeConfig{} - if volume.Driver != "local" { - vol.Driver = volume.Driver - vol.Name = volume.Name - } - volumeConfigs[volume.Name] = vol - serviceVolumeConfigs = append(serviceVolumeConfigs, serviceVC) - case mount.TypeBind: - if strings.HasPrefix(volume.Destination, "/run/secrets") { - destination := strings.Split(volume.Destination, "/") - secret := types.SecretConfig{ - Name: destination[len(destination)-1], - File: strings.TrimPrefix(volume.Source, "/host_mnt"), - } - secretConfigs[secret.Name] = secret - serviceSecretConfigs = append(serviceSecretConfigs, types.ServiceSecretConfig{ - Source: secret.Name, - Target: volume.Destination, - }) - } else { - serviceVolumeConfigs = append(serviceVolumeConfigs, serviceVC) - } - } - } - return volumeConfigs, serviceVolumeConfigs, secretConfigs, serviceSecretConfigs -} - -func (s *composeService) toComposeNetwork(networks map[string]*network.EndpointSettings) (map[string]types.NetworkConfig, map[string]*types.ServiceNetworkConfig) { - networkConfigs := make(map[string]types.NetworkConfig) - serviceNetworkConfigs := make(map[string]*types.ServiceNetworkConfig) - - for name, net := range networks { - inspect, err := s.apiClient().NetworkInspect(context.Background(), name, network.InspectOptions{}) - if err != nil { - networkConfigs[name] = types.NetworkConfig{} - } else { - networkConfigs[name] = types.NetworkConfig{ - Internal: inspect.Internal, - } - } - serviceNetworkConfigs[name] = &types.ServiceNetworkConfig{ - Aliases: net.Aliases, - } - } - return networkConfigs, serviceNetworkConfigs -} - -func cleanDockerPreviousLabels(labels types.Labels) types.Labels { - cleanedLabels := types.Labels{} - for key, value := range labels { - if !strings.HasPrefix(key, "com.docker.compose.") && !strings.HasPrefix(key, "desktop.docker.io") { - cleanedLabels[key] = value - } - } - return cleanedLabels -} diff --git a/pkg/compose/hash.go b/pkg/compose/hash.go deleted file mode 100644 index 24c6cce2e88..00000000000 --- a/pkg/compose/hash.go +++ /dev/null @@ -1,64 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "encoding/json" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/opencontainers/go-digest" -) - -// ServiceHash computes the configuration hash for a service. -func ServiceHash(o types.ServiceConfig) (string, error) { - // remove the Build config when generating the service hash - o.Build = nil - o.PullPolicy = "" - o.Scale = nil - if o.Deploy != nil { - o.Deploy.Replicas = nil - } - o.DependsOn = nil - o.Profiles = nil - - bytes, err := json.Marshal(o) - if err != nil { - return "", err - } - return digest.SHA256.FromBytes(bytes).Encoded(), nil -} - -// NetworkHash computes the configuration hash for a network. -func NetworkHash(o *types.NetworkConfig) (string, error) { - bytes, err := json.Marshal(o) - if err != nil { - return "", err - } - return digest.SHA256.FromBytes(bytes).Encoded(), nil -} - -// VolumeHash computes the configuration hash for a volume. -func VolumeHash(o types.VolumeConfig) (string, error) { - if o.Driver == "" { // (TODO: jhrotko) This probably should be fixed in compose-go - o.Driver = "local" - } - bytes, err := json.Marshal(o) - if err != nil { - return "", err - } - return digest.SHA256.FromBytes(bytes).Encoded(), nil -} diff --git a/pkg/compose/hash_test.go b/pkg/compose/hash_test.go deleted file mode 100644 index 73b7f387735..00000000000 --- a/pkg/compose/hash_test.go +++ /dev/null @@ -1,43 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "gotest.tools/v3/assert" -) - -func TestServiceHash(t *testing.T) { - hash1, err := ServiceHash(serviceConfig(1)) - assert.NilError(t, err) - hash2, err := ServiceHash(serviceConfig(2)) - assert.NilError(t, err) - assert.Equal(t, hash1, hash2) -} - -func serviceConfig(replicas int) types.ServiceConfig { - return types.ServiceConfig{ - Scale: &replicas, - Deploy: &types.DeployConfig{ - Replicas: &replicas, - }, - Name: "foo", - Image: "bar", - } -} diff --git a/pkg/compose/hook.go b/pkg/compose/hook.go deleted file mode 100644 index 45357e388e8..00000000000 --- a/pkg/compose/hook.go +++ /dev/null @@ -1,121 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/pkg/stdcopy" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -func (s composeService) runHook(ctx context.Context, ctr container.Summary, service types.ServiceConfig, hook types.ServiceHook, listener api.ContainerEventListener) error { - wOut := utils.GetWriter(func(line string) { - listener(api.ContainerEvent{ - Type: api.HookEventLog, - Source: getContainerNameWithoutProject(ctr) + " ->", - ID: ctr.ID, - Service: service.Name, - Line: line, - }) - }) - defer wOut.Close() //nolint:errcheck - - detached := listener == nil - exec, err := s.apiClient().ContainerExecCreate(ctx, ctr.ID, container.ExecOptions{ - User: hook.User, - Privileged: hook.Privileged, - Env: ToMobyEnv(hook.Environment), - WorkingDir: hook.WorkingDir, - Cmd: hook.Command, - AttachStdout: !detached, - AttachStderr: !detached, - }) - if err != nil { - return err - } - - if detached { - return s.runWaitExec(ctx, exec.ID, service, listener) - } - - height, width := s.stdout().GetTtySize() - consoleSize := &[2]uint{height, width} - attach, err := s.apiClient().ContainerExecAttach(ctx, exec.ID, container.ExecAttachOptions{ - Tty: service.Tty, - ConsoleSize: consoleSize, - }) - if err != nil { - return err - } - defer attach.Close() - - if service.Tty { - _, err = io.Copy(wOut, attach.Reader) - } else { - _, err = stdcopy.StdCopy(wOut, wOut, attach.Reader) - } - if err != nil { - return err - } - - inspected, err := s.apiClient().ContainerExecInspect(ctx, exec.ID) - if err != nil { - return err - } - if inspected.ExitCode != 0 { - return fmt.Errorf("%s hook exited with status %d", service.Name, inspected.ExitCode) - } - return nil -} - -func (s composeService) runWaitExec(ctx context.Context, execID string, service types.ServiceConfig, listener api.ContainerEventListener) error { - err := s.apiClient().ContainerExecStart(ctx, execID, container.ExecStartOptions{ - Detach: listener == nil, - Tty: service.Tty, - }) - if err != nil { - return nil - } - - // We miss a ContainerExecWait API - tick := time.NewTicker(100 * time.Millisecond) - for { - select { - case <-ctx.Done(): - return nil - case <-tick.C: - inspect, err := s.apiClient().ContainerExecInspect(ctx, execID) - if err != nil { - return nil - } - if !inspect.Running { - if inspect.ExitCode != 0 { - return fmt.Errorf("%s hook exited with status %d", service.Name, inspect.ExitCode) - } - return nil - } - } - } -} diff --git a/pkg/compose/image_pruner.go b/pkg/compose/image_pruner.go deleted file mode 100644 index 6e09d901442..00000000000 --- a/pkg/compose/image_pruner.go +++ /dev/null @@ -1,247 +0,0 @@ -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "sort" - "sync" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/distribution/reference" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/client" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -// ImagePruneMode controls how aggressively images associated with the project -// are removed from the engine. -type ImagePruneMode string - -const ( - // ImagePruneNone indicates that no project images should be removed. - ImagePruneNone ImagePruneMode = "" - // ImagePruneLocal indicates that only images built locally by Compose - // should be removed. - ImagePruneLocal ImagePruneMode = "local" - // ImagePruneAll indicates that all project-associated images, including - // remote images should be removed. - ImagePruneAll ImagePruneMode = "all" -) - -// ImagePruneOptions controls the behavior of image pruning. -type ImagePruneOptions struct { - Mode ImagePruneMode - - // RemoveOrphans will result in the removal of images that were built for - // the project regardless of whether they are for a known service if true. - RemoveOrphans bool -} - -// ImagePruner handles image removal during Compose `down` operations. -type ImagePruner struct { - client client.ImageAPIClient - project *types.Project -} - -// NewImagePruner creates an ImagePruner object for a project. -func NewImagePruner(imageClient client.ImageAPIClient, project *types.Project) *ImagePruner { - return &ImagePruner{ - client: imageClient, - project: project, - } -} - -// ImagesToPrune returns the set of images that should be removed. -func (p *ImagePruner) ImagesToPrune(ctx context.Context, opts ImagePruneOptions) ([]string, error) { - if opts.Mode == ImagePruneNone { - return nil, nil - } else if opts.Mode != ImagePruneLocal && opts.Mode != ImagePruneAll { - return nil, fmt.Errorf("unsupported image prune mode: %s", opts.Mode) - } - var images []string - - if opts.Mode == ImagePruneAll { - namedImages, err := p.namedImages(ctx) - if err != nil { - return nil, err - } - images = append(images, namedImages...) - } - - projectImages, err := p.labeledLocalImages(ctx) - if err != nil { - return nil, err - } - for _, img := range projectImages { - if len(img.RepoTags) == 0 { - // currently, we're only pruning the tagged references, but - // if we start removing the dangling images and grouping by - // service, we can remove this (and should rely on `Image::ID`) - continue - } - - var shouldPrune bool - if opts.RemoveOrphans { - // indiscriminately prune all project images even if they're not - // referenced by the current Compose state (e.g. the service was - // removed from YAML) - shouldPrune = true - } else { - // only prune the image if it belongs to a known service for the project. - if _, err := p.project.GetService(img.Labels[api.ServiceLabel]); err == nil { - shouldPrune = true - } - } - - if shouldPrune { - images = append(images, img.RepoTags[0]) - } - } - - fallbackImages, err := p.unlabeledLocalImages(ctx) - if err != nil { - return nil, err - } - images = append(images, fallbackImages...) - - images = normalizeAndDedupeImages(images) - return images, nil -} - -// namedImages are those that are explicitly named in the service config. -// -// These could be registry-only images (no local build), hybrid (support build -// as a fallback if cannot pull), or local-only (image does not exist in a -// registry). -func (p *ImagePruner) namedImages(ctx context.Context) ([]string, error) { - var images []string - for _, service := range p.project.Services { - if service.Image == "" { - continue - } - images = append(images, service.Image) - } - return p.filterImagesByExistence(ctx, images) -} - -// labeledLocalImages are images that were locally-built by a current version of -// Compose (it did not always label built images). -// -// The image name could either have been defined by the user or implicitly -// created from the project + service name. -func (p *ImagePruner) labeledLocalImages(ctx context.Context) ([]image.Summary, error) { - imageListOpts := image.ListOptions{ - Filters: filters.NewArgs( - projectFilter(p.project.Name), - // TODO(milas): we should really clean up the dangling images as - // well (historically we have NOT); need to refactor this to handle - // it gracefully without producing confusing CLI output, i.e. we - // do not want to print out a bunch of untagged/dangling image IDs, - // they should be grouped into a logical operation for the relevant - // service - filters.Arg("dangling", "false"), - ), - } - projectImages, err := p.client.ImageList(ctx, imageListOpts) - if err != nil { - return nil, err - } - return projectImages, nil -} - -// unlabeledLocalImages are images that match the implicit naming convention -// for locally-built images but did not get labeled, presumably because they -// were produced by an older version of Compose. -// -// This is transitional to ensure `down` continues to work as expected on -// projects built/launched by previous versions of Compose. It can safely -// be removed after some time. -func (p *ImagePruner) unlabeledLocalImages(ctx context.Context) ([]string, error) { - var images []string - for _, service := range p.project.Services { - if service.Image != "" { - continue - } - img := api.GetImageNameOrDefault(service, p.project.Name) - images = append(images, img) - } - return p.filterImagesByExistence(ctx, images) -} - -// filterImagesByExistence returns the subset of images that exist in the -// engine store. -// -// NOTE: Any transient errors communicating with the API will result in an -// image being returned as "existing", as this method is exclusively used to -// find images to remove, so the worst case of being conservative here is an -// attempt to remove an image that doesn't exist, which will cause a warning -// but is otherwise harmless. -func (p *ImagePruner) filterImagesByExistence(ctx context.Context, imageNames []string) ([]string, error) { - var mu sync.Mutex - var ret []string - - eg, ctx := errgroup.WithContext(ctx) - for _, img := range imageNames { - eg.Go(func() error { - _, err := p.client.ImageInspect(ctx, img) - if errdefs.IsNotFound(err) { - // err on the side of caution: only skip if we successfully - // queried the API and got back a definitive "not exists" - return nil - } - mu.Lock() - defer mu.Unlock() - ret = append(ret, img) - return nil - }) - } - - if err := eg.Wait(); err != nil { - return nil, err - } - - return ret, nil -} - -// normalizeAndDedupeImages returns the unique set of images after normalization. -func normalizeAndDedupeImages(images []string) []string { - seen := make(map[string]struct{}, len(images)) - for _, img := range images { - // since some references come from user input (service.image) and some - // come from the engine API, we standardize them, opting for the - // familiar name format since they'll also be displayed in the CLI - ref, err := reference.ParseNormalizedNamed(img) - if err == nil { - ref = reference.TagNameOnly(ref) - img = reference.FamiliarString(ref) - } - seen[img] = struct{}{} - } - ret := make([]string, 0, len(seen)) - for v := range seen { - ret = append(ret, v) - } - // ensure a deterministic return result - the actual ordering is not useful - sort.Strings(ret) - return ret -} diff --git a/pkg/compose/images.go b/pkg/compose/images.go deleted file mode 100644 index e322920e59c..00000000000 --- a/pkg/compose/images.go +++ /dev/null @@ -1,163 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "slices" - "strings" - "sync" - "time" - - "github.com/containerd/errdefs" - "github.com/containerd/platforms" - "github.com/distribution/reference" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/versions" - "github.com/docker/docker/client" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Images(ctx context.Context, projectName string, options api.ImagesOptions) (map[string]api.ImageSummary, error) { - projectName = strings.ToLower(projectName) - allContainers, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - All: true, - Filters: filters.NewArgs(projectFilter(projectName)), - }) - if err != nil { - return nil, err - } - var containers []container.Summary - if len(options.Services) > 0 { - // filter service containers - for _, c := range allContainers { - if slices.Contains(options.Services, c.Labels[api.ServiceLabel]) { - containers = append(containers, c) - } - } - } else { - containers = allContainers - } - - version, err := s.RuntimeVersion(ctx) - if err != nil { - return nil, err - } - withPlatform := versions.GreaterThanOrEqualTo(version, APIVersion149) - - summary := map[string]api.ImageSummary{} - var mux sync.Mutex - eg, ctx := errgroup.WithContext(ctx) - for _, c := range containers { - eg.Go(func() error { - image, err := s.apiClient().ImageInspect(ctx, c.Image) - if err != nil { - return err - } - id := image.ID // platform-specific image ID can't be combined with image tag, see https://github.com/moby/moby/issues/49995 - - if withPlatform && c.ImageManifestDescriptor != nil && c.ImageManifestDescriptor.Platform != nil { - image, err = s.apiClient().ImageInspect(ctx, c.Image, client.ImageInspectWithPlatform(c.ImageManifestDescriptor.Platform)) - if err != nil { - return err - } - } - - var repository, tag string - ref, err := reference.ParseDockerRef(c.Image) - if err == nil { - // ParseDockerRef will reject a local image ID - repository = reference.FamiliarName(ref) - if tagged, ok := ref.(reference.Tagged); ok { - tag = tagged.Tag() - } - } - - var created *time.Time - if image.Created != "" { - t, err := time.Parse(time.RFC3339Nano, image.Created) - if err != nil { - return err - } - created = &t - } - - mux.Lock() - defer mux.Unlock() - summary[getCanonicalContainerName(c)] = api.ImageSummary{ - ID: id, - Repository: repository, - Tag: tag, - Platform: platforms.Platform{ - Architecture: image.Architecture, - OS: image.Os, - OSVersion: image.OsVersion, - Variant: image.Variant, - }, - Size: image.Size, - Created: created, - LastTagTime: image.Metadata.LastTagTime, - } - return nil - }) - } - - err = eg.Wait() - return summary, err -} - -func (s *composeService) getImageSummaries(ctx context.Context, repoTags []string) (map[string]api.ImageSummary, error) { - summary := map[string]api.ImageSummary{} - l := sync.Mutex{} - eg, ctx := errgroup.WithContext(ctx) - for _, repoTag := range repoTags { - eg.Go(func() error { - inspect, err := s.apiClient().ImageInspect(ctx, repoTag) - if err != nil { - if errdefs.IsNotFound(err) { - return nil - } - return fmt.Errorf("unable to get image '%s': %w", repoTag, err) - } - tag := "" - repository := "" - ref, err := reference.ParseDockerRef(repoTag) - if err == nil { - // ParseDockerRef will reject a local image ID - repository = reference.FamiliarName(ref) - if tagged, ok := ref.(reference.Tagged); ok { - tag = tagged.Tag() - } - } - l.Lock() - summary[repoTag] = api.ImageSummary{ - ID: inspect.ID, - Repository: repository, - Tag: tag, - Size: inspect.Size, - LastTagTime: inspect.Metadata.LastTagTime, - } - l.Unlock() - return nil - }) - } - return summary, eg.Wait() -} diff --git a/pkg/compose/images_test.go b/pkg/compose/images_test.go deleted file mode 100644 index b6eb9cce138..00000000000 --- a/pkg/compose/images_test.go +++ /dev/null @@ -1,108 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "strings" - "testing" - "time" - - "github.com/docker/docker/api/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - compose "github.com/docker/compose/v5/pkg/api" -) - -func TestImages(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - args := filters.NewArgs(projectFilter(strings.ToLower(testProject))) - listOpts := container.ListOptions{All: true, Filters: args} - api.EXPECT().ServerVersion(gomock.Any()).Return(types.Version{APIVersion: "1.96"}, nil).AnyTimes() - timeStr1 := "2025-06-06T06:06:06.000000000Z" - created1, _ := time.Parse(time.RFC3339Nano, timeStr1) - timeStr2 := "2025-03-03T03:03:03.000000000Z" - created2, _ := time.Parse(time.RFC3339Nano, timeStr2) - image1 := imageInspect("image1", "foo:1", 12345, timeStr1) - image2 := imageInspect("image2", "bar:2", 67890, timeStr2) - api.EXPECT().ImageInspect(anyCancellableContext(), "foo:1").Return(image1, nil).MaxTimes(2) - api.EXPECT().ImageInspect(anyCancellableContext(), "bar:2").Return(image2, nil) - c1 := containerDetail("service1", "123", "running", "foo:1") - c2 := containerDetail("service1", "456", "running", "bar:2") - c2.Ports = []container.Port{{PublicPort: 80, PrivatePort: 90, IP: "localhost"}} - c3 := containerDetail("service2", "789", "exited", "foo:1") - api.EXPECT().ContainerList(t.Context(), listOpts).Return([]container.Summary{c1, c2, c3}, nil) - - images, err := tested.Images(t.Context(), strings.ToLower(testProject), compose.ImagesOptions{}) - - expected := map[string]compose.ImageSummary{ - "123": { - ID: "image1", - Repository: "foo", - Tag: "1", - Size: 12345, - Created: &created1, - }, - "456": { - ID: "image2", - Repository: "bar", - Tag: "2", - Size: 67890, - Created: &created2, - }, - "789": { - ID: "image1", - Repository: "foo", - Tag: "1", - Size: 12345, - Created: &created1, - }, - } - assert.NilError(t, err) - assert.DeepEqual(t, images, expected) -} - -func imageInspect(id string, imageReference string, size int64, created string) image.InspectResponse { - return image.InspectResponse{ - ID: id, - RepoTags: []string{ - "someRepo:someTag", - imageReference, - }, - Size: size, - Created: created, - } -} - -func containerDetail(service string, id string, status string, imageName string) container.Summary { - return container.Summary{ - ID: id, - Names: []string{"/" + id}, - Image: imageName, - Labels: containerLabels(service, false), - State: status, - } -} diff --git a/pkg/compose/kill.go b/pkg/compose/kill.go deleted file mode 100644 index 63d369902af..00000000000 --- a/pkg/compose/kill.go +++ /dev/null @@ -1,74 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strings" - - "github.com/docker/docker/api/types/container" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Kill(ctx context.Context, projectName string, options api.KillOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.kill(ctx, strings.ToLower(projectName), options) - }, "kill", s.events) -} - -func (s *composeService) kill(ctx context.Context, projectName string, options api.KillOptions) error { - services := options.Services - - var containers Containers - containers, err := s.getContainers(ctx, projectName, oneOffInclude, options.All, services...) - if err != nil { - return err - } - - project := options.Project - if project == nil { - project, err = s.getProjectWithResources(ctx, containers, projectName) - if err != nil { - return err - } - } - - if !options.RemoveOrphans { - containers = containers.filter(isService(project.ServiceNames()...)) - } - if len(containers) == 0 { - return api.ErrNoResources - } - - eg, ctx := errgroup.WithContext(ctx) - containers.forEach(func(ctr container.Summary) { - eg.Go(func() error { - eventName := getContainerProgressName(ctr) - s.events.On(killingEvent(eventName)) - err := s.apiClient().ContainerKill(ctx, ctr.ID, options.Signal) - if err != nil { - s.events.On(errorEvent(eventName, "Error while Killing")) - return err - } - s.events.On(killedEvent(eventName)) - return nil - }) - }) - return eg.Wait() -} diff --git a/pkg/compose/kill_test.go b/pkg/compose/kill_test.go deleted file mode 100644 index f27606b7f2e..00000000000 --- a/pkg/compose/kill_test.go +++ /dev/null @@ -1,148 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "path/filepath" - "strings" - "testing" - - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/api/types/volume" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - compose "github.com/docker/compose/v5/pkg/api" -) - -const testProject = "testProject" - -func TestKillAll(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - name := strings.ToLower(testProject) - - api.EXPECT().ContainerList(t.Context(), container.ListOptions{ - Filters: filters.NewArgs(projectFilter(name), hasConfigHashLabel()), - }).Return( - []container.Summary{testContainer("service1", "123", false), testContainer("service1", "456", false), testContainer("service2", "789", false)}, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{ - {ID: "abc123", Name: "testProject_default"}, - }, nil) - api.EXPECT().ContainerKill(anyCancellableContext(), "123", "").Return(nil) - api.EXPECT().ContainerKill(anyCancellableContext(), "456", "").Return(nil) - api.EXPECT().ContainerKill(anyCancellableContext(), "789", "").Return(nil) - - err = tested.Kill(t.Context(), name, compose.KillOptions{}) - assert.NilError(t, err) -} - -func TestKillSignal(t *testing.T) { - const serviceName = "service1" - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - name := strings.ToLower(testProject) - listOptions := container.ListOptions{ - Filters: filters.NewArgs(projectFilter(name), serviceFilter(serviceName), hasConfigHashLabel()), - } - - api.EXPECT().ContainerList(t.Context(), listOptions).Return([]container.Summary{testContainer(serviceName, "123", false)}, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{ - {ID: "abc123", Name: "testProject_default"}, - }, nil) - api.EXPECT().ContainerKill(anyCancellableContext(), "123", "SIGTERM").Return(nil) - - err = tested.Kill(t.Context(), name, compose.KillOptions{Services: []string{serviceName}, Signal: "SIGTERM"}) - assert.NilError(t, err) -} - -func testContainer(service string, id string, oneOff bool) container.Summary { - // canonical docker names in the API start with a leading slash, some - // parts of Compose code will attempt to strip this off, so make sure - // it's consistently present - name := "/" + strings.TrimPrefix(id, "/") - return container.Summary{ - ID: id, - Names: []string{name}, - Labels: containerLabels(service, oneOff), - State: container.StateExited, - } -} - -func containerLabels(service string, oneOff bool) map[string]string { - workingdir := "/src/pkg/compose/testdata" - composefile := filepath.Join(workingdir, "compose.yaml") - labels := map[string]string{ - compose.ServiceLabel: service, - compose.ConfigFilesLabel: composefile, - compose.WorkingDirLabel: workingdir, - compose.ProjectLabel: strings.ToLower(testProject), - } - if oneOff { - labels[compose.OneoffLabel] = "True" - } - return labels -} - -func anyCancellableContext() gomock.Matcher { - //nolint:forbidigo // This creates a context type for gomock matching, not for actual test usage - ctxWithCancel, cancel := context.WithCancel(context.Background()) - cancel() - return gomock.AssignableToTypeOf(ctxWithCancel) -} - -func projectFilterListOpt(withOneOff bool) container.ListOptions { - filter := filters.NewArgs( - projectFilter(strings.ToLower(testProject)), - hasConfigHashLabel(), - ) - if !withOneOff { - filter.Add("label", fmt.Sprintf("%s=False", compose.OneoffLabel)) - } - return container.ListOptions{ - Filters: filter, - All: true, - } -} diff --git a/pkg/compose/loader.go b/pkg/compose/loader.go deleted file mode 100644 index 9a0699da7c6..00000000000 --- a/pkg/compose/loader.go +++ /dev/null @@ -1,157 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "os" - "strings" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/loader" - "github.com/compose-spec/compose-go/v2/types" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/remote" - "github.com/docker/compose/v5/pkg/utils" -) - -// LoadProject implements api.Compose.LoadProject -// It loads and validates a Compose project from configuration files. -func (s *composeService) LoadProject(ctx context.Context, options api.ProjectLoadOptions) (*types.Project, error) { - // Setup remote loaders (Git, OCI) - remoteLoaders := s.createRemoteLoaders(options) - - projectOptions, err := s.buildProjectOptions(options, remoteLoaders) - if err != nil { - return nil, err - } - - // Register all user-provided listeners (e.g., for metrics collection) - for _, listener := range options.LoadListeners { - if listener != nil { - projectOptions.WithListeners(listener) - } - } - - if options.Compatibility || utils.StringToBool(projectOptions.Environment[api.ComposeCompatibility]) { - api.Separator = "_" - } - - project, err := projectOptions.LoadProject(ctx) - if err != nil { - return nil, err - } - - // Post-processing: service selection, environment resolution, etc. - project, err = s.postProcessProject(project, options) - if err != nil { - return nil, err - } - - return project, nil -} - -// createRemoteLoaders creates Git and OCI remote loaders if not in offline mode -func (s *composeService) createRemoteLoaders(options api.ProjectLoadOptions) []loader.ResourceLoader { - if options.Offline { - return nil - } - git := remote.NewGitRemoteLoader(s.dockerCli, options.Offline) - oci := remote.NewOCIRemoteLoader(s.dockerCli, options.Offline, options.OCI) - return []loader.ResourceLoader{git, oci} -} - -// buildProjectOptions constructs compose-go ProjectOptions from API options -func (s *composeService) buildProjectOptions(options api.ProjectLoadOptions, remoteLoaders []loader.ResourceLoader) (*cli.ProjectOptions, error) { - opts := []cli.ProjectOptionsFn{ - cli.WithWorkingDirectory(options.WorkingDir), - cli.WithOsEnv, - } - - // Add PWD if not present - if _, present := os.LookupEnv("PWD"); !present { - if pwd, err := os.Getwd(); err == nil { - opts = append(opts, cli.WithEnv([]string{"PWD=" + pwd})) - } - } - - // Add remote loaders - for _, r := range remoteLoaders { - opts = append(opts, cli.WithResourceLoader(r)) - } - - opts = append(opts, - // Load PWD/.env if present and no explicit --env-file has been set - cli.WithEnvFiles(options.EnvFiles...), - // read dot env file to populate project environment - cli.WithDotEnv, - // get compose file path set by COMPOSE_FILE - cli.WithConfigFileEnv, - // if none was selected, get default compose.yaml file from current dir or parent folder - cli.WithDefaultConfigPath, - // .. and then, a project directory != PWD maybe has been set so let's load .env file - cli.WithEnvFiles(options.EnvFiles...), //nolint:gocritic // intentionally applying cli.WithEnvFiles twice. - cli.WithDotEnv, //nolint:gocritic // intentionally applying cli.WithDotEnv twice. - // eventually COMPOSE_PROFILES should have been set - cli.WithDefaultProfiles(options.Profiles...), - cli.WithName(options.ProjectName), - ) - - return cli.NewProjectOptions(options.ConfigPaths, append(options.ProjectOptionsFns, opts...)...) -} - -// postProcessProject applies post-loading transformations to the project -func (s *composeService) postProcessProject(project *types.Project, options api.ProjectLoadOptions) (*types.Project, error) { - if project.Name == "" { - return nil, errors.New("project name can't be empty. Use ProjectName option to set a valid name") - } - - project, err := project.WithServicesEnabled(options.Services...) - if err != nil { - return nil, err - } - - // Add custom labels - for name, s := range project.Services { - s.CustomLabels = map[string]string{ - api.ProjectLabel: project.Name, - api.ServiceLabel: name, - api.VersionLabel: api.ComposeVersion, - api.WorkingDirLabel: project.WorkingDir, - api.ConfigFilesLabel: strings.Join(project.ComposeFiles, ","), - api.OneoffLabel: "False", - } - if len(options.EnvFiles) != 0 { - s.CustomLabels[api.EnvironmentFileLabel] = strings.Join(options.EnvFiles, ",") - } - project.Services[name] = s - } - - project, err = project.WithSelectedServices(options.Services) - if err != nil { - return nil, err - } - - // Remove unnecessary resources if not All - if !options.All { - project = project.WithoutUnnecessaryResources() - } - - return project, nil -} diff --git a/pkg/compose/loader_test.go b/pkg/compose/loader_test.go deleted file mode 100644 index 8006d4169d0..00000000000 --- a/pkg/compose/loader_test.go +++ /dev/null @@ -1,322 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "os" - "path/filepath" - "testing" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestLoadProject_Basic(t *testing.T) { - // Create a temporary compose file - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -name: test-project -services: - web: - image: nginx:latest - ports: - - "8080:80" - db: - image: postgres:latest - environment: - POSTGRES_PASSWORD: secret -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Load the project - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - }) - - // Assertions - require.NoError(t, err) - assert.NotNil(t, project) - assert.Equal(t, "test-project", project.Name) - assert.Len(t, project.Services, 2) - assert.Contains(t, project.Services, "web") - assert.Contains(t, project.Services, "db") - - // Check labels were applied - webService := project.Services["web"] - assert.Equal(t, "test-project", webService.CustomLabels[api.ProjectLabel]) - assert.Equal(t, "web", webService.CustomLabels[api.ServiceLabel]) -} - -func TestLoadProject_WithEnvironmentResolution(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -services: - app: - image: myapp:latest - environment: - - TEST_VAR=${TEST_VAR} - - LITERAL_VAR=literal_value -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - // Set environment variable - t.Setenv("TEST_VAR", "resolved_value") - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Test with environment resolution (default) - t.Run("WithResolution", func(t *testing.T) { - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - }) - require.NoError(t, err) - - appService := project.Services["app"] - // Environment should be resolved - assert.NotNil(t, appService.Environment["TEST_VAR"]) - assert.Equal(t, "resolved_value", *appService.Environment["TEST_VAR"]) - assert.NotNil(t, appService.Environment["LITERAL_VAR"]) - assert.Equal(t, "literal_value", *appService.Environment["LITERAL_VAR"]) - }) - - // Test without environment resolution - t.Run("WithoutResolution", func(t *testing.T) { - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - ProjectOptionsFns: []cli.ProjectOptionsFn{cli.WithoutEnvironmentResolution}, - }) - require.NoError(t, err) - - appService := project.Services["app"] - // Environment should NOT be resolved, keeping raw values - // Note: This depends on compose-go behavior, which may still have some resolution - assert.NotNil(t, appService.Environment) - }) -} - -func TestLoadProject_ServiceSelection(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -services: - web: - image: nginx:latest - db: - image: postgres:latest - cache: - image: redis:latest -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Load only specific services - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - Services: []string{"web", "db"}, - }) - - require.NoError(t, err) - assert.Len(t, project.Services, 2) - assert.Contains(t, project.Services, "web") - assert.Contains(t, project.Services, "db") - assert.NotContains(t, project.Services, "cache") -} - -func TestLoadProject_WithProfiles(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -services: - web: - image: nginx:latest - debug: - image: busybox:latest - profiles: ["debug"] -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Without debug profile - t.Run("WithoutProfile", func(t *testing.T) { - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - }) - require.NoError(t, err) - assert.Len(t, project.Services, 1) - assert.Contains(t, project.Services, "web") - }) - - // With debug profile - t.Run("WithProfile", func(t *testing.T) { - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - Profiles: []string{"debug"}, - }) - require.NoError(t, err) - assert.Len(t, project.Services, 2) - assert.Contains(t, project.Services, "web") - assert.Contains(t, project.Services, "debug") - }) -} - -func TestLoadProject_WithLoadListeners(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -services: - web: - image: nginx:latest -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Track events received - var events []string - listener := func(event string, metadata map[string]any) { - events = append(events, event) - } - - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - LoadListeners: []api.LoadListener{listener}, - }) - - require.NoError(t, err) - assert.NotNil(t, project) - - // Listeners should have been called (exact events depend on compose-go implementation) - // The slice itself is always initialized (non-nil), even if empty - _ = events // events may or may not have entries depending on compose-go behavior -} - -func TestLoadProject_ProjectNameInference(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -services: - web: - image: nginx:latest -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Without explicit project name - t.Run("InferredName", func(t *testing.T) { - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - }) - require.NoError(t, err) - // Project name should be inferred from directory - assert.NotEmpty(t, project.Name) - }) - - // With explicit project name - t.Run("ExplicitName", func(t *testing.T) { - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - ProjectName: "my-custom-project", - }) - require.NoError(t, err) - assert.Equal(t, "my-custom-project", project.Name) - }) -} - -func TestLoadProject_Compatibility(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -services: - web: - image: nginx:latest -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // With compatibility mode - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - Compatibility: true, - }) - - require.NoError(t, err) - assert.NotNil(t, project) - // In compatibility mode, separator should be "_" - assert.Equal(t, "_", api.Separator) - - // Reset separator - api.Separator = "-" -} - -func TestLoadProject_InvalidComposeFile(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -this is not valid yaml: [[[ -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Should return an error for invalid YAML - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - }) - - require.Error(t, err) - assert.Nil(t, project) -} - -func TestLoadProject_MissingComposeFile(t *testing.T) { - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Should return an error for missing file - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{"/nonexistent/compose.yaml"}, - }) - - require.Error(t, err) - assert.Nil(t, project) -} diff --git a/pkg/compose/logs.go b/pkg/compose/logs.go deleted file mode 100644 index c633f5c311f..00000000000 --- a/pkg/compose/logs.go +++ /dev/null @@ -1,148 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "io" - - "github.com/containerd/errdefs" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/pkg/stdcopy" - "github.com/sirupsen/logrus" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -func (s *composeService) Logs( - ctx context.Context, - projectName string, - consumer api.LogConsumer, - options api.LogOptions, -) error { - var containers Containers - var err error - - if options.Index > 0 { - ctr, err := s.getSpecifiedContainer(ctx, projectName, oneOffExclude, true, options.Services[0], options.Index) - if err != nil { - return err - } - containers = append(containers, ctr) - } else { - containers, err = s.getContainers(ctx, projectName, oneOffExclude, true, options.Services...) - if err != nil { - return err - } - } - - if options.Project != nil && len(options.Services) == 0 { - // we run with an explicit compose.yaml, so only consider services defined in this file - options.Services = options.Project.ServiceNames() - containers = containers.filter(isService(options.Services...)) - } - - eg, ctx := errgroup.WithContext(ctx) - for _, ctr := range containers { - eg.Go(func() error { - err := s.logContainer(ctx, consumer, ctr, options) - if errdefs.IsNotImplemented(err) { - logrus.Warnf("Can't retrieve logs for %q: %s", getCanonicalContainerName(ctr), err.Error()) - return nil - } - return err - }) - } - - if options.Follow { - printer := newLogPrinter(consumer) - - monitor := newMonitor(s.apiClient(), projectName) - if len(options.Services) > 0 { - monitor.withServices(options.Services) - } else if options.Project != nil { - monitor.withServices(options.Project.ServiceNames()) - } - monitor.withListener(printer.HandleEvent) - monitor.withListener(func(event api.ContainerEvent) { - if event.Type == api.ContainerEventStarted { - eg.Go(func() error { - ctr, err := s.apiClient().ContainerInspect(ctx, event.ID) - if err != nil { - return err - } - - err = s.doLogContainer(ctx, consumer, event.Source, ctr, api.LogOptions{ - Follow: options.Follow, - Since: ctr.State.StartedAt, - Until: options.Until, - Tail: options.Tail, - Timestamps: options.Timestamps, - }) - if errdefs.IsNotImplemented(err) { - // ignore - return nil - } - return err - }) - } - }) - eg.Go(func() error { - // pass ctx so monitor will immediately stop on SIGINT - return monitor.Start(ctx) - }) - } - - return eg.Wait() -} - -func (s *composeService) logContainer(ctx context.Context, consumer api.LogConsumer, c container.Summary, options api.LogOptions) error { - ctr, err := s.apiClient().ContainerInspect(ctx, c.ID) - if err != nil { - return err - } - name := getContainerNameWithoutProject(c) - return s.doLogContainer(ctx, consumer, name, ctr, options) -} - -func (s *composeService) doLogContainer(ctx context.Context, consumer api.LogConsumer, name string, ctr container.InspectResponse, options api.LogOptions) error { - r, err := s.apiClient().ContainerLogs(ctx, ctr.ID, container.LogsOptions{ - ShowStdout: true, - ShowStderr: true, - Follow: options.Follow, - Since: options.Since, - Until: options.Until, - Tail: options.Tail, - Timestamps: options.Timestamps, - }) - if err != nil { - return err - } - defer r.Close() //nolint:errcheck - - w := utils.GetWriter(func(line string) { - consumer.Log(name, line) - }) - if ctr.Config.Tty { - _, err = io.Copy(w, r) - } else { - _, err = stdcopy.StdCopy(w, w, r) - } - return err -} diff --git a/pkg/compose/logs_test.go b/pkg/compose/logs_test.go deleted file mode 100644 index b5823c908da..00000000000 --- a/pkg/compose/logs_test.go +++ /dev/null @@ -1,191 +0,0 @@ -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "io" - "strings" - "sync" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - containerType "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/pkg/stdcopy" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - "go.uber.org/mock/gomock" - - compose "github.com/docker/compose/v5/pkg/api" -) - -func TestComposeService_Logs_Demux(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - require.NoError(t, err) - - name := strings.ToLower(testProject) - - api.EXPECT().ContainerList(t.Context(), containerType.ListOptions{ - All: true, - Filters: filters.NewArgs(oneOffFilter(false), projectFilter(name), hasConfigHashLabel()), - }).Return( - []containerType.Summary{ - testContainer("service", "c", false), - }, - nil, - ) - - api.EXPECT(). - ContainerInspect(anyCancellableContext(), "c"). - Return(containerType.InspectResponse{ - ContainerJSONBase: &containerType.ContainerJSONBase{ID: "c"}, - Config: &containerType.Config{Tty: false}, - }, nil) - c1Reader, c1Writer := io.Pipe() - t.Cleanup(func() { - _ = c1Reader.Close() - _ = c1Writer.Close() - }) - c1Stdout := stdcopy.NewStdWriter(c1Writer, stdcopy.Stdout) - c1Stderr := stdcopy.NewStdWriter(c1Writer, stdcopy.Stderr) - go func() { - _, err := c1Stdout.Write([]byte("hello stdout\n")) - assert.NoError(t, err, "Writing to fake stdout") - _, err = c1Stderr.Write([]byte("hello stderr\n")) - assert.NoError(t, err, "Writing to fake stderr") - _ = c1Writer.Close() - }() - api.EXPECT().ContainerLogs(anyCancellableContext(), "c", gomock.Any()). - Return(c1Reader, nil) - - opts := compose.LogOptions{ - Project: &types.Project{ - Services: types.Services{ - "service": {Name: "service"}, - }, - }, - } - - consumer := &testLogConsumer{} - err = tested.Logs(t.Context(), name, consumer, opts) - require.NoError(t, err) - - require.Equal( - t, - []string{"hello stdout", "hello stderr"}, - consumer.LogsForContainer("c"), - ) -} - -// TestComposeService_Logs_ServiceFiltering ensures that we do not include -// logs from out-of-scope services based on the Compose file vs actual state. -// -// NOTE(milas): This test exists because each method is currently duplicating -// a lot of the project/service filtering logic. We should consider moving it -// to an earlier point in the loading process, at which point this test could -// safely be removed. -func TestComposeService_Logs_ServiceFiltering(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - require.NoError(t, err) - - name := strings.ToLower(testProject) - - api.EXPECT().ContainerList(t.Context(), containerType.ListOptions{ - All: true, - Filters: filters.NewArgs(oneOffFilter(false), projectFilter(name), hasConfigHashLabel()), - }).Return( - []containerType.Summary{ - testContainer("serviceA", "c1", false), - testContainer("serviceA", "c2", false), - // serviceB will be filtered out by the project definition to - // ensure we ignore "orphan" containers - testContainer("serviceB", "c3", false), - testContainer("serviceC", "c4", false), - }, - nil, - ) - - for _, id := range []string{"c1", "c2", "c4"} { - api.EXPECT(). - ContainerInspect(anyCancellableContext(), id). - Return( - containerType.InspectResponse{ - ContainerJSONBase: &containerType.ContainerJSONBase{ID: id}, - Config: &containerType.Config{Tty: true}, - }, - nil, - ) - api.EXPECT().ContainerLogs(anyCancellableContext(), id, gomock.Any()). - Return(io.NopCloser(strings.NewReader("hello "+id+"\n")), nil). - Times(1) - } - - // this simulates passing `--filename` with a Compose file that does NOT - // reference `serviceB` even though it has running services for this proj - proj := &types.Project{ - Services: types.Services{ - "serviceA": {Name: "serviceA"}, - "serviceC": {Name: "serviceC"}, - }, - } - consumer := &testLogConsumer{} - opts := compose.LogOptions{ - Project: proj, - } - err = tested.Logs(t.Context(), name, consumer, opts) - require.NoError(t, err) - - require.Equal(t, []string{"hello c1"}, consumer.LogsForContainer("c1")) - require.Equal(t, []string{"hello c2"}, consumer.LogsForContainer("c2")) - require.Empty(t, consumer.LogsForContainer("c3")) - require.Equal(t, []string{"hello c4"}, consumer.LogsForContainer("c4")) -} - -type testLogConsumer struct { - mu sync.Mutex - // logs is keyed by container ID; values are log lines - logs map[string][]string -} - -func (l *testLogConsumer) Log(containerName, message string) { - l.mu.Lock() - defer l.mu.Unlock() - if l.logs == nil { - l.logs = make(map[string][]string) - } - l.logs[containerName] = append(l.logs[containerName], message) -} - -func (l *testLogConsumer) Err(containerName, message string) { - l.Log(containerName, message) -} - -func (l *testLogConsumer) Status(containerName, msg string) {} - -func (l *testLogConsumer) LogsForContainer(containerName string) []string { - l.mu.Lock() - defer l.mu.Unlock() - return l.logs[containerName] -} diff --git a/pkg/compose/ls.go b/pkg/compose/ls.go deleted file mode 100644 index b999e828bdc..00000000000 --- a/pkg/compose/ls.go +++ /dev/null @@ -1,136 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "slices" - "sort" - "strings" - - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) List(ctx context.Context, opts api.ListOptions) ([]api.Stack, error) { - list, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - Filters: filters.NewArgs(hasProjectLabelFilter(), hasConfigHashLabel()), - All: opts.All, - }) - if err != nil { - return nil, err - } - - return containersToStacks(list) -} - -func containersToStacks(containers []container.Summary) ([]api.Stack, error) { - containersByLabel, keys, err := groupContainerByLabel(containers, api.ProjectLabel) - if err != nil { - return nil, err - } - var projects []api.Stack - for _, project := range keys { - configFiles, err := combinedConfigFiles(containersByLabel[project]) - if err != nil { - logrus.Warn(err.Error()) - configFiles = "N/A" - } - - projects = append(projects, api.Stack{ - ID: project, - Name: project, - Status: combinedStatus(containerToState(containersByLabel[project])), - ConfigFiles: configFiles, - }) - } - return projects, nil -} - -func combinedConfigFiles(containers []container.Summary) (string, error) { - configFiles := []string{} - - for _, c := range containers { - files, ok := c.Labels[api.ConfigFilesLabel] - if !ok { - return "", fmt.Errorf("no label %q set on container %q of compose project", api.ConfigFilesLabel, c.ID) - } - - for f := range strings.SplitSeq(files, ",") { - if !slices.Contains(configFiles, f) { - configFiles = append(configFiles, f) - } - } - } - - return strings.Join(configFiles, ","), nil -} - -func containerToState(containers []container.Summary) []string { - statuses := []string{} - for _, c := range containers { - statuses = append(statuses, c.State) - } - return statuses -} - -func combinedStatus(statuses []string) string { - nbByStatus := map[string]int{} - keys := []string{} - for _, status := range statuses { - nb, ok := nbByStatus[status] - if !ok { - nb = 0 - keys = append(keys, status) - } - nbByStatus[status] = nb + 1 - } - sort.Strings(keys) - result := "" - for _, status := range keys { - nb := nbByStatus[status] - if result != "" { - result += ", " - } - result += fmt.Sprintf("%s(%d)", status, nb) - } - return result -} - -func groupContainerByLabel(containers []container.Summary, labelName string) (map[string][]container.Summary, []string, error) { - containersByLabel := map[string][]container.Summary{} - keys := []string{} - for _, c := range containers { - label, ok := c.Labels[labelName] - if !ok { - return nil, nil, fmt.Errorf("no label %q set on container %q of compose project", labelName, c.ID) - } - labelContainers, ok := containersByLabel[label] - if !ok { - labelContainers = []container.Summary{} - keys = append(keys, label) - } - labelContainers = append(labelContainers, c) - containersByLabel[label] = labelContainers - } - sort.Strings(keys) - return containersByLabel, keys, nil -} diff --git a/pkg/compose/ls_test.go b/pkg/compose/ls_test.go deleted file mode 100644 index b444abd3476..00000000000 --- a/pkg/compose/ls_test.go +++ /dev/null @@ -1,122 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "fmt" - "testing" - - "github.com/docker/docker/api/types/container" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestContainersToStacks(t *testing.T) { - containers := []container.Summary{ - { - ID: "service1", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project1", api.ConfigFilesLabel: "/home/docker-compose.yaml"}, - }, - { - ID: "service2", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project1", api.ConfigFilesLabel: "/home/docker-compose.yaml"}, - }, - { - ID: "service3", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project2", api.ConfigFilesLabel: "/home/project2-docker-compose.yaml"}, - }, - } - stacks, err := containersToStacks(containers) - assert.NilError(t, err) - assert.DeepEqual(t, stacks, []api.Stack{ - { - ID: "project1", - Name: "project1", - Status: "running(2)", - ConfigFiles: "/home/docker-compose.yaml", - }, - { - ID: "project2", - Name: "project2", - Status: "running(1)", - ConfigFiles: "/home/project2-docker-compose.yaml", - }, - }) -} - -func TestStacksMixedStatus(t *testing.T) { - assert.Equal(t, combinedStatus([]string{"running"}), "running(1)") - assert.Equal(t, combinedStatus([]string{"running", "running", "running"}), "running(3)") - assert.Equal(t, combinedStatus([]string{"running", "exited", "running"}), "exited(1), running(2)") -} - -func TestCombinedConfigFiles(t *testing.T) { - containersByLabel := map[string][]container.Summary{ - "project1": { - { - ID: "service1", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project1", api.ConfigFilesLabel: "/home/docker-compose.yaml"}, - }, - { - ID: "service2", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project1", api.ConfigFilesLabel: "/home/docker-compose.yaml"}, - }, - }, - "project2": { - { - ID: "service3", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project2", api.ConfigFilesLabel: "/home/project2-docker-compose.yaml"}, - }, - }, - "project3": { - { - ID: "service4", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project3"}, - }, - }, - } - - testData := map[string]struct { - ConfigFiles string - Error error - }{ - "project1": {ConfigFiles: "/home/docker-compose.yaml", Error: nil}, - "project2": {ConfigFiles: "/home/project2-docker-compose.yaml", Error: nil}, - "project3": {ConfigFiles: "", Error: fmt.Errorf("no label %q set on container %q of compose project", api.ConfigFilesLabel, "service4")}, - } - - for project, containers := range containersByLabel { - configFiles, err := combinedConfigFiles(containers) - - expected := testData[project] - - if expected.Error != nil { - assert.Error(t, err, expected.Error.Error()) - } else { - assert.Equal(t, err, expected.Error) - } - assert.Equal(t, configFiles, expected.ConfigFiles) - } -} diff --git a/pkg/compose/model.go b/pkg/compose/model.go deleted file mode 100644 index e4614d199c3..00000000000 --- a/pkg/compose/model.go +++ /dev/null @@ -1,294 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bufio" - "context" - "encoding/json" - "fmt" - "os/exec" - "slices" - "strconv" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/docker/cli/cli-plugins/manager" - "github.com/docker/docker/api/types/versions" - "github.com/spf13/cobra" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) ensureModels(ctx context.Context, project *types.Project, quietPull bool) error { - if len(project.Models) == 0 { - return nil - } - - mdlAPI, err := s.newModelAPI(project) - if err != nil { - return err - } - defer mdlAPI.Close() - availableModels, err := mdlAPI.ListModels(ctx) - - eg, ctx := errgroup.WithContext(ctx) - eg.Go(func() error { - return mdlAPI.SetModelVariables(ctx, project) - }) - - for name, config := range project.Models { - if config.Name == "" { - config.Name = name - } - eg.Go(func() error { - if !slices.Contains(availableModels, config.Model) { - err = mdlAPI.PullModel(ctx, config, quietPull, s.events) - if err != nil { - return err - } - } - return mdlAPI.ConfigureModel(ctx, config, s.events) - }) - } - return eg.Wait() -} - -type modelAPI struct { - path string - env []string - prepare func(ctx context.Context, cmd *exec.Cmd) error - cleanup func() - version string -} - -func (s *composeService) newModelAPI(project *types.Project) (*modelAPI, error) { - dockerModel, err := manager.GetPlugin("model", s.dockerCli, &cobra.Command{}) - if err != nil { - if errdefs.IsNotFound(err) { - return nil, fmt.Errorf("'models' support requires Docker Model plugin") - } - return nil, err - } - if dockerModel.Err != nil { - return nil, fmt.Errorf("failed to load Docker Model plugin: %w", dockerModel.Err) - } - endpoint, cleanup, err := s.propagateDockerEndpoint() - if err != nil { - return nil, err - } - return &modelAPI{ - path: dockerModel.Path, - version: dockerModel.Version, - prepare: func(ctx context.Context, cmd *exec.Cmd) error { - return s.prepareShellOut(ctx, project.Environment, cmd) - }, - cleanup: cleanup, - env: append(project.Environment.Values(), endpoint...), - }, nil -} - -func (m *modelAPI) Close() { - m.cleanup() -} - -func (m *modelAPI) PullModel(ctx context.Context, model types.ModelConfig, quietPull bool, events api.EventProcessor) error { - events.On(api.Resource{ - ID: model.Name, - Status: api.Working, - Text: api.StatusPulling, - }) - - cmd := exec.CommandContext(ctx, m.path, "pull", model.Model) - err := m.prepare(ctx, cmd) - if err != nil { - return err - } - stream, err := cmd.StdoutPipe() - if err != nil { - return err - } - - err = cmd.Start() - if err != nil { - return err - } - - scanner := bufio.NewScanner(stream) - for scanner.Scan() { - msg := scanner.Text() - if msg == "" { - continue - } - - if !quietPull { - events.On(api.Resource{ - ID: model.Name, - Status: api.Working, - Text: api.StatusPulling, - }) - } - } - - err = cmd.Wait() - if err != nil { - events.On(errorEvent(model.Name, err.Error())) - } - events.On(api.Resource{ - ID: model.Name, - Status: api.Working, - Text: api.StatusPulled, - }) - return err -} - -func (m *modelAPI) ConfigureModel(ctx context.Context, config types.ModelConfig, events api.EventProcessor) error { - events.On(api.Resource{ - ID: config.Name, - Status: api.Working, - Text: api.StatusConfiguring, - }) - // configure [--context-size=<n>] MODEL [-- <runtime-flags...>] - args := []string{"configure"} - if config.ContextSize > 0 { - args = append(args, "--context-size", strconv.Itoa(config.ContextSize)) - } - args = append(args, config.Model) - // Only append RuntimeFlags if docker model CLI version is >= v1.0.6 - if len(config.RuntimeFlags) != 0 && m.supportsRuntimeFlags() { - args = append(args, "--") - args = append(args, config.RuntimeFlags...) - } - cmd := exec.CommandContext(ctx, m.path, args...) - err := m.prepare(ctx, cmd) - if err != nil { - return err - } - err = cmd.Run() - if err != nil { - events.On(errorEvent(config.Name, err.Error())) - return err - } - events.On(api.Resource{ - ID: config.Name, - Status: api.Done, - Text: api.StatusConfigured, - }) - return nil -} - -func (m *modelAPI) SetModelVariables(ctx context.Context, project *types.Project) error { - cmd := exec.CommandContext(ctx, m.path, "status", "--json") - err := m.prepare(ctx, cmd) - if err != nil { - return err - } - - statusOut, err := cmd.CombinedOutput() - if err != nil { - return fmt.Errorf("error checking docker-model status: %w", err) - } - type Status struct { - Endpoint string `json:"endpoint"` - } - - var status Status - err = json.Unmarshal(statusOut, &status) - if err != nil { - return err - } - - for _, service := range project.Services { - for ref, modelConfig := range service.Models { - model := project.Models[ref] - varPrefix := strings.ReplaceAll(strings.ToUpper(ref), "-", "_") - var variable string - if modelConfig != nil && modelConfig.ModelVariable != "" { - variable = modelConfig.ModelVariable - } else { - variable = varPrefix + "_MODEL" - } - service.Environment[variable] = &model.Model - - if modelConfig != nil && modelConfig.EndpointVariable != "" { - variable = modelConfig.EndpointVariable - } else { - variable = varPrefix + "_URL" - } - service.Environment[variable] = &status.Endpoint - } - } - return nil -} - -type Model struct { - Id string `json:"id"` - Tags []string `json:"tags"` - Created int `json:"created"` - Config struct { - Format string `json:"format"` - Quantization string `json:"quantization"` - Parameters string `json:"parameters"` - Architecture string `json:"architecture"` - Size string `json:"size"` - } `json:"config"` -} - -func (m *modelAPI) ListModels(ctx context.Context) ([]string, error) { - cmd := exec.CommandContext(ctx, m.path, "ls", "--json") - err := m.prepare(ctx, cmd) - if err != nil { - return nil, err - } - - output, err := cmd.CombinedOutput() - if err != nil { - return nil, fmt.Errorf("error checking available models: %w", err) - } - - type AvailableModel struct { - Id string `json:"id"` - Tags []string `json:"tags"` - Created int `json:"created"` - } - - models := []AvailableModel{} - err = json.Unmarshal(output, &models) - if err != nil { - return nil, fmt.Errorf("error unmarshalling available models: %w", err) - } - var availableModels []string - for _, model := range models { - availableModels = append(availableModels, model.Tags...) - } - return availableModels, nil -} - -// supportsRuntimeFlags checks if the docker model version supports runtime flags -// Runtime flags are supported in version >= v1.0.6 -func (m *modelAPI) supportsRuntimeFlags() bool { - // If version is not cached, don't append runtime flags to be safe - if m.version == "" { - return false - } - - // Strip 'v' prefix if present (e.g., "v1.0.6" -> "1.0.6") - versionStr := strings.TrimPrefix(m.version, "v") - return !versions.LessThan(versionStr, "1.0.6") -} diff --git a/pkg/compose/monitor.go b/pkg/compose/monitor.go deleted file mode 100644 index 70dd30ada00..00000000000 --- a/pkg/compose/monitor.go +++ /dev/null @@ -1,215 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strconv" - - "github.com/containerd/errdefs" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/events" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/client" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -type monitor struct { - apiClient client.APIClient - project string - // services tells us which service to consider and those we can ignore, maybe ran by a concurrent compose command - services map[string]bool - listeners []api.ContainerEventListener -} - -func newMonitor(apiClient client.APIClient, project string) *monitor { - return &monitor{ - apiClient: apiClient, - project: project, - services: map[string]bool{}, - } -} - -func (c *monitor) withServices(services []string) { - for _, name := range services { - c.services[name] = true - } -} - -// Start runs monitor to detect application events and return after termination -// -//nolint:gocyclo -func (c *monitor) Start(ctx context.Context) error { - // collect initial application container - initialState, err := c.apiClient.ContainerList(ctx, container.ListOptions{ - All: true, - Filters: filters.NewArgs( - projectFilter(c.project), - oneOffFilter(false), - hasConfigHashLabel(), - ), - }) - if err != nil { - return err - } - - // containers is the set if container IDs the application is based on - containers := utils.Set[string]{} - for _, ctr := range initialState { - if len(c.services) == 0 || c.services[ctr.Labels[api.ServiceLabel]] { - containers.Add(ctr.ID) - } - } - restarting := utils.Set[string]{} - - evtCh, errCh := c.apiClient.Events(ctx, events.ListOptions{ - Filters: filters.NewArgs( - filters.Arg("type", "container"), - projectFilter(c.project)), - }) - for { - if len(containers) == 0 { - return nil - } - select { - case <-ctx.Done(): - return nil - case err := <-errCh: - return err - case event := <-evtCh: - if len(c.services) > 0 && !c.services[event.Actor.Attributes[api.ServiceLabel]] { - continue - } - ctr, err := c.getContainerSummary(event) - if err != nil { - return err - } - - switch event.Action { - case events.ActionCreate: - if len(c.services) == 0 || c.services[ctr.Labels[api.ServiceLabel]] { - containers.Add(ctr.ID) - } - evtType := api.ContainerEventCreated - if _, ok := ctr.Labels[api.ContainerReplaceLabel]; ok { - evtType = api.ContainerEventRecreated - } - for _, listener := range c.listeners { - listener(newContainerEvent(event.TimeNano, ctr, evtType)) - } - logrus.Debugf("container %s created", ctr.Name) - case events.ActionStart: - restarted := restarting.Has(ctr.ID) - if restarted { - logrus.Debugf("container %s restarted", ctr.Name) - for _, listener := range c.listeners { - listener(newContainerEvent(event.TimeNano, ctr, api.ContainerEventStarted, func(e *api.ContainerEvent) { - e.Restarting = restarted - })) - } - } else { - logrus.Debugf("container %s started", ctr.Name) - for _, listener := range c.listeners { - listener(newContainerEvent(event.TimeNano, ctr, api.ContainerEventStarted)) - } - } - if len(c.services) == 0 || c.services[ctr.Labels[api.ServiceLabel]] { - containers.Add(ctr.ID) - } - case events.ActionRestart: - for _, listener := range c.listeners { - listener(newContainerEvent(event.TimeNano, ctr, api.ContainerEventRestarted)) - } - logrus.Debugf("container %s restarted", ctr.Name) - case events.ActionDie: - logrus.Debugf("container %s exited with code %d", ctr.Name, ctr.ExitCode) - inspect, err := c.apiClient.ContainerInspect(ctx, event.Actor.ID) - if errdefs.IsNotFound(err) { - // Source is already removed - } else if err != nil { - return err - } - - if inspect.State != nil && (inspect.State.Restarting || inspect.State.Running) { - // State.Restarting is set by engine when container is configured to restart on exit - // on ContainerRestart it doesn't (see https://github.com/moby/moby/issues/45538) - // container state still is reported as "running" - logrus.Debugf("container %s is restarting", ctr.Name) - restarting.Add(ctr.ID) - for _, listener := range c.listeners { - listener(newContainerEvent(event.TimeNano, ctr, api.ContainerEventExited, func(e *api.ContainerEvent) { - e.Restarting = true - })) - } - } else { - for _, listener := range c.listeners { - listener(newContainerEvent(event.TimeNano, ctr, api.ContainerEventExited)) - } - containers.Remove(ctr.ID) - } - } - } - } -} - -func newContainerEvent(timeNano int64, ctr *api.ContainerSummary, eventType int, opts ...func(e *api.ContainerEvent)) api.ContainerEvent { - name := ctr.Name - defaultName := getDefaultContainerName(ctr.Project, ctr.Labels[api.ServiceLabel], ctr.Labels[api.ContainerNumberLabel]) - if name == defaultName { - // remove project- prefix - name = name[len(ctr.Project)+1:] - } - - event := api.ContainerEvent{ - Type: eventType, - Container: ctr, - Time: timeNano, - Source: name, - ID: ctr.ID, - Service: ctr.Service, - ExitCode: ctr.ExitCode, - } - for _, opt := range opts { - opt(&event) - } - return event -} - -func (c *monitor) getContainerSummary(event events.Message) (*api.ContainerSummary, error) { - ctr := &api.ContainerSummary{ - ID: event.Actor.ID, - Name: event.Actor.Attributes["name"], - Project: c.project, - Service: event.Actor.Attributes[api.ServiceLabel], - Labels: event.Actor.Attributes, // More than just labels, but that'c the closest the API gives us - } - if ec, ok := event.Actor.Attributes["exitCode"]; ok { - exitCode, err := strconv.Atoi(ec) - if err != nil { - return nil, err - } - ctr.ExitCode = exitCode - } - return ctr, nil -} - -func (c *monitor) withListener(listener api.ContainerEventListener) { - c.listeners = append(c.listeners, listener) -} diff --git a/pkg/compose/pause.go b/pkg/compose/pause.go deleted file mode 100644 index 35d476eeac2..00000000000 --- a/pkg/compose/pause.go +++ /dev/null @@ -1,87 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strings" - - "github.com/docker/docker/api/types/container" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Pause(ctx context.Context, projectName string, options api.PauseOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.pause(ctx, strings.ToLower(projectName), options) - }, "pause", s.events) -} - -func (s *composeService) pause(ctx context.Context, projectName string, options api.PauseOptions) error { - containers, err := s.getContainers(ctx, projectName, oneOffExclude, false, options.Services...) - if err != nil { - return err - } - - if options.Project != nil { - containers = containers.filter(isService(options.Project.ServiceNames()...)) - } - - eg, ctx := errgroup.WithContext(ctx) - containers.forEach(func(container container.Summary) { - eg.Go(func() error { - err := s.apiClient().ContainerPause(ctx, container.ID) - if err == nil { - eventName := getContainerProgressName(container) - s.events.On(newEvent(eventName, api.Done, "Paused")) - } - return err - }) - }) - return eg.Wait() -} - -func (s *composeService) UnPause(ctx context.Context, projectName string, options api.PauseOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.unPause(ctx, strings.ToLower(projectName), options) - }, "unpause", s.events) -} - -func (s *composeService) unPause(ctx context.Context, projectName string, options api.PauseOptions) error { - containers, err := s.getContainers(ctx, projectName, oneOffExclude, false, options.Services...) - if err != nil { - return err - } - - if options.Project != nil { - containers = containers.filter(isService(options.Project.ServiceNames()...)) - } - - eg, ctx := errgroup.WithContext(ctx) - containers.forEach(func(ctr container.Summary) { - eg.Go(func() error { - err = s.apiClient().ContainerUnpause(ctx, ctr.ID) - if err == nil { - eventName := getContainerProgressName(ctr) - s.events.On(newEvent(eventName, api.Done, "Unpaused")) - } - return err - }) - }) - return eg.Wait() -} diff --git a/pkg/compose/plugins.go b/pkg/compose/plugins.go deleted file mode 100644 index 159394f6bf1..00000000000 --- a/pkg/compose/plugins.go +++ /dev/null @@ -1,283 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "context" - "encoding/json" - "errors" - "fmt" - "io" - "os" - "os/exec" - "path/filepath" - "strings" - "sync" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/docker/cli/cli-plugins/manager" - "github.com/docker/cli/cli/config" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" -) - -type JsonMessage struct { - Type string `json:"type"` - Message string `json:"message"` -} - -const ( - ErrorType = "error" - InfoType = "info" - SetEnvType = "setenv" - DebugType = "debug" - providerMetadataDirectory = "compose/providers" -) - -var mux sync.Mutex - -func (s *composeService) runPlugin(ctx context.Context, project *types.Project, service types.ServiceConfig, command string) error { - provider := *service.Provider - - plugin, err := s.getPluginBinaryPath(provider.Type) - if err != nil { - return err - } - - cmd, err := s.setupPluginCommand(ctx, project, service, plugin, command) - if err != nil { - return err - } - - variables, err := s.executePlugin(cmd, command, service) - if err != nil { - return err - } - - mux.Lock() - defer mux.Unlock() - for name, s := range project.Services { - if _, ok := s.DependsOn[service.Name]; ok { - prefix := strings.ToUpper(service.Name) + "_" - for key, val := range variables { - s.Environment[prefix+key] = &val - } - project.Services[name] = s - } - } - return nil -} - -func (s *composeService) executePlugin(cmd *exec.Cmd, command string, service types.ServiceConfig) (types.Mapping, error) { - var action string - switch command { - case "up": - s.events.On(creatingEvent(service.Name)) - action = "create" - case "down": - s.events.On(removingEvent(service.Name)) - action = "remove" - default: - return nil, fmt.Errorf("unsupported plugin command: %s", command) - } - - stdout, err := cmd.StdoutPipe() - if err != nil { - return nil, err - } - - err = cmd.Start() - if err != nil { - return nil, err - } - - decoder := json.NewDecoder(stdout) - defer func() { _ = stdout.Close() }() - - variables := types.Mapping{} - - for { - var msg JsonMessage - err = decoder.Decode(&msg) - if errors.Is(err, io.EOF) { - break - } - if err != nil { - return nil, err - } - switch msg.Type { - case ErrorType: - s.events.On(newEvent(service.Name, api.Error, msg.Message)) - return nil, errors.New(msg.Message) - case InfoType: - s.events.On(newEvent(service.Name, api.Working, msg.Message)) - case SetEnvType: - key, val, found := strings.Cut(msg.Message, "=") - if !found { - return nil, fmt.Errorf("invalid response from plugin: %s", msg.Message) - } - variables[key] = val - case DebugType: - logrus.Debugf("%s: %s", service.Name, msg.Message) - default: - return nil, fmt.Errorf("invalid response from plugin: %s", msg.Type) - } - } - - err = cmd.Wait() - if err != nil { - s.events.On(errorEvent(service.Name, err.Error())) - return nil, fmt.Errorf("failed to %s service provider: %s", action, err.Error()) - } - switch command { - case "up": - s.events.On(createdEvent(service.Name)) - case "down": - s.events.On(removedEvent(service.Name)) - } - return variables, nil -} - -func (s *composeService) getPluginBinaryPath(provider string) (path string, err error) { - if provider == "compose" { - return "", errors.New("'compose' is not a valid provider type") - } - plugin, err := manager.GetPlugin(provider, s.dockerCli, &cobra.Command{}) - if err == nil { - path = plugin.Path - } - if errdefs.IsNotFound(err) { - path, err = exec.LookPath(executable(provider)) - } - return path, err -} - -func (s *composeService) setupPluginCommand(ctx context.Context, project *types.Project, service types.ServiceConfig, path, command string) (*exec.Cmd, error) { - cmdOptionsMetadata := s.getPluginMetadata(path, service.Provider.Type, project) - var currentCommandMetadata CommandMetadata - switch command { - case "up": - currentCommandMetadata = cmdOptionsMetadata.Up - case "down": - currentCommandMetadata = cmdOptionsMetadata.Down - } - - provider := *service.Provider - commandMetadataIsEmpty := cmdOptionsMetadata.IsEmpty() - if err := currentCommandMetadata.CheckRequiredParameters(provider); !commandMetadataIsEmpty && err != nil { - return nil, err - } - - args := []string{"compose", fmt.Sprintf("--project-name=%s", project.Name), command} - for k, v := range provider.Options { - for _, value := range v { - if _, ok := currentCommandMetadata.GetParameter(k); commandMetadataIsEmpty || ok { - args = append(args, fmt.Sprintf("--%s=%s", k, value)) - } - } - } - args = append(args, service.Name) - - cmd := exec.CommandContext(ctx, path, args...) - - err := s.prepareShellOut(ctx, project.Environment, cmd) - if err != nil { - return nil, err - } - return cmd, nil -} - -func (s *composeService) getPluginMetadata(path, command string, project *types.Project) ProviderMetadata { - cmd := exec.Command(path, "compose", "metadata") - err := s.prepareShellOut(context.Background(), project.Environment, cmd) - if err != nil { - logrus.Debugf("failed to prepare plugin metadata command: %v", err) - return ProviderMetadata{} - } - stdout := &bytes.Buffer{} - cmd.Stdout = stdout - - if err := cmd.Run(); err != nil { - logrus.Debugf("failed to start plugin metadata command: %v", err) - return ProviderMetadata{} - } - - var metadata ProviderMetadata - if err := json.Unmarshal(stdout.Bytes(), &metadata); err != nil { - output, _ := io.ReadAll(stdout) - logrus.Debugf("failed to decode plugin metadata: %v - %s", err, output) - return ProviderMetadata{} - } - // Save metadata into docker home directory to be used by Docker LSP tool - // Just log the error as it's not a critical error for the main flow - metadataDir := filepath.Join(config.Dir(), providerMetadataDirectory) - if err := os.MkdirAll(metadataDir, 0o700); err == nil { - metadataFilePath := filepath.Join(metadataDir, command+".json") - if err := os.WriteFile(metadataFilePath, stdout.Bytes(), 0o600); err != nil { - logrus.Debugf("failed to save plugin metadata: %v", err) - } - } else { - logrus.Debugf("failed to create plugin metadata directory: %v", err) - } - return metadata -} - -type ProviderMetadata struct { - Description string `json:"description"` - Up CommandMetadata `json:"up"` - Down CommandMetadata `json:"down"` -} - -func (p ProviderMetadata) IsEmpty() bool { - return p.Description == "" && p.Up.Parameters == nil && p.Down.Parameters == nil -} - -type CommandMetadata struct { - Parameters []ParameterMetadata `json:"parameters"` -} - -type ParameterMetadata struct { - Name string `json:"name"` - Description string `json:"description"` - Required bool `json:"required"` - Type string `json:"type"` - Default string `json:"default,omitempty"` -} - -func (c CommandMetadata) GetParameter(paramName string) (ParameterMetadata, bool) { - for _, p := range c.Parameters { - if p.Name == paramName { - return p, true - } - } - return ParameterMetadata{}, false -} - -func (c CommandMetadata) CheckRequiredParameters(provider types.ServiceProviderConfig) error { - for _, p := range c.Parameters { - if p.Required { - if _, ok := provider.Options[p.Name]; !ok { - return fmt.Errorf("required parameter %q is missing from provider %q definition", p.Name, provider.Type) - } - } - } - return nil -} diff --git a/pkg/compose/plugins_windows.go b/pkg/compose/plugins_windows.go deleted file mode 100644 index 327b3e4c102..00000000000 --- a/pkg/compose/plugins_windows.go +++ /dev/null @@ -1,23 +0,0 @@ -//go:build windows - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -func executable(s string) string { - return s + ".exe" -} diff --git a/pkg/compose/port.go b/pkg/compose/port.go deleted file mode 100644 index ac64b4dfef1..00000000000 --- a/pkg/compose/port.go +++ /dev/null @@ -1,55 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "strings" - - "github.com/docker/docker/api/types/container" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Port(ctx context.Context, projectName string, service string, port uint16, options api.PortOptions) (string, int, error) { - projectName = strings.ToLower(projectName) - ctr, err := s.getSpecifiedContainer(ctx, projectName, oneOffInclude, false, service, options.Index) - if err != nil { - return "", 0, err - } - for _, p := range ctr.Ports { - if p.PrivatePort == port && p.Type == options.Protocol { - return p.IP, int(p.PublicPort), nil - } - } - return "", 0, portNotFoundError(options.Protocol, port, ctr) -} - -func portNotFoundError(protocol string, port uint16, ctr container.Summary) error { - formatPort := func(protocol string, port uint16) string { - return fmt.Sprintf("%d/%s", port, protocol) - } - - var containerPorts []string - for _, p := range ctr.Ports { - containerPorts = append(containerPorts, formatPort(p.Type, p.PublicPort)) - } - - name := strings.TrimPrefix(ctr.Names[0], "/") - return fmt.Errorf("no port %s for container %s: %s", formatPort(protocol, port), name, strings.Join(containerPorts, ", ")) -} diff --git a/pkg/compose/printer.go b/pkg/compose/printer.go deleted file mode 100644 index 6fe6b9fda6e..00000000000 --- a/pkg/compose/printer.go +++ /dev/null @@ -1,57 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "fmt" - - "github.com/docker/compose/v5/pkg/api" -) - -// logPrinter watch application containers and collect their logs -type logPrinter interface { - HandleEvent(event api.ContainerEvent) -} - -type printer struct { - consumer api.LogConsumer -} - -// newLogPrinter builds a LogPrinter passing containers logs to LogConsumer -func newLogPrinter(consumer api.LogConsumer) logPrinter { - printer := printer{ - consumer: consumer, - } - return &printer -} - -func (p *printer) HandleEvent(event api.ContainerEvent) { - switch event.Type { - case api.ContainerEventExited: - if event.Restarting { - p.consumer.Status(event.Source, fmt.Sprintf("exited with code %d (restarting)", event.ExitCode)) - } else { - p.consumer.Status(event.Source, fmt.Sprintf("exited with code %d", event.ExitCode)) - } - case api.ContainerEventRecreated: - p.consumer.Status(event.Container.Labels[api.ContainerReplaceLabel], "has been recreated") - case api.ContainerEventLog, api.HookEventLog: - p.consumer.Log(event.Source, event.Line) - case api.ContainerEventErr: - p.consumer.Err(event.Source, event.Line) - } -} diff --git a/pkg/compose/progress.go b/pkg/compose/progress.go deleted file mode 100644 index 26f9b5d8590..00000000000 --- a/pkg/compose/progress.go +++ /dev/null @@ -1,176 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - - "github.com/docker/compose/v5/pkg/api" -) - -type progressFunc func(context.Context) error - -func Run(ctx context.Context, pf progressFunc, operation string, bus api.EventProcessor) error { - bus.Start(ctx, operation) - err := pf(ctx) - bus.Done(operation, err != nil) - return err -} - -// errorEvent creates a new Error Resource with message -func errorEvent(id string, msg string) api.Resource { - return api.Resource{ - ID: id, - Status: api.Error, - Text: api.StatusError, - Details: msg, - } -} - -// errorEventf creates a new Error Resource with format message -func errorEventf(id string, msg string, args ...any) api.Resource { - return errorEvent(id, fmt.Sprintf(msg, args...)) -} - -// creatingEvent creates a new Create in progress Resource -func creatingEvent(id string) api.Resource { - return newEvent(id, api.Working, api.StatusCreating) -} - -// startingEvent creates a new Starting in progress Resource -func startingEvent(id string) api.Resource { - return newEvent(id, api.Working, api.StatusStarting) -} - -// startedEvent creates a new Started in progress Resource -func startedEvent(id string) api.Resource { - return newEvent(id, api.Done, api.StatusStarted) -} - -// waiting creates a new waiting event -func waiting(id string) api.Resource { - return newEvent(id, api.Working, api.StatusWaiting) -} - -// healthy creates a new healthy event -func healthy(id string) api.Resource { - return newEvent(id, api.Done, api.StatusHealthy) -} - -// exited creates a new exited event -func exited(id string) api.Resource { - return newEvent(id, api.Done, api.StatusExited) -} - -// restartingEvent creates a new Restarting in progress Resource -func restartingEvent(id string) api.Resource { - return newEvent(id, api.Working, api.StatusRestarting) -} - -// runningEvent creates a new Running in progress Resource -func runningEvent(id string) api.Resource { - return newEvent(id, api.Done, api.StatusRunning) -} - -// createdEvent creates a new Created (done) Resource -func createdEvent(id string) api.Resource { - return newEvent(id, api.Done, api.StatusCreated) -} - -// stoppingEvent creates a new Stopping in progress Resource -func stoppingEvent(id string) api.Resource { - return newEvent(id, api.Working, api.StatusStopping) -} - -// stoppedEvent creates a new Stopping in progress Resource -func stoppedEvent(id string) api.Resource { - return newEvent(id, api.Done, api.StatusStopped) -} - -// killingEvent creates a new Killing in progress Resource -func killingEvent(id string) api.Resource { - return newEvent(id, api.Working, api.StatusKilling) -} - -// killedEvent creates a new Killed in progress Resource -func killedEvent(id string) api.Resource { - return newEvent(id, api.Done, api.StatusKilled) -} - -// removingEvent creates a new Removing in progress Resource -func removingEvent(id string) api.Resource { - return newEvent(id, api.Working, api.StatusRemoving) -} - -// removedEvent creates a new removed (done) Resource -func removedEvent(id string) api.Resource { - return newEvent(id, api.Done, api.StatusRemoved) -} - -// buildingEvent creates a new Building in progress Resource -func buildingEvent(id string) api.Resource { - return newEvent("Image "+id, api.Working, api.StatusBuilding) -} - -// builtEvent creates a new built (done) Resource -func builtEvent(id string) api.Resource { - return newEvent("Image "+id, api.Done, api.StatusBuilt) -} - -// pullingEvent creates a new pulling (in progress) Resource -func pullingEvent(id string) api.Resource { - return newEvent("Image "+id, api.Working, api.StatusPulling) -} - -// pulledEvent creates a new pulled (done) Resource -func pulledEvent(id string) api.Resource { - return newEvent("Image "+id, api.Done, api.StatusPulled) -} - -// skippedEvent creates a new Skipped Resource -func skippedEvent(id string, reason string) api.Resource { - return api.Resource{ - ID: id, - Status: api.Warning, - Text: "Skipped: " + reason, - } -} - -// newEvent new event -func newEvent(id string, status api.EventStatus, text string, reason ...string) api.Resource { - r := api.Resource{ - ID: id, - Status: status, - Text: text, - } - if len(reason) > 0 { - r.Details = reason[0] - } - return r -} - -type ignore struct{} - -func (q *ignore) Start(_ context.Context, _ string) { -} - -func (q *ignore) Done(_ string, _ bool) { -} - -func (q *ignore) On(_ ...api.Resource) { -} diff --git a/pkg/compose/ps.go b/pkg/compose/ps.go deleted file mode 100644 index fb32358aebe..00000000000 --- a/pkg/compose/ps.go +++ /dev/null @@ -1,128 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "sort" - "strings" - - "github.com/docker/docker/api/types/container" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Ps(ctx context.Context, projectName string, options api.PsOptions) ([]api.ContainerSummary, error) { - projectName = strings.ToLower(projectName) - oneOff := oneOffExclude - if options.All { - oneOff = oneOffInclude - } - containers, err := s.getContainers(ctx, projectName, oneOff, options.All, options.Services...) - if err != nil { - return nil, err - } - - if len(options.Services) != 0 { - containers = containers.filter(isService(options.Services...)) - } - summary := make([]api.ContainerSummary, len(containers)) - eg, ctx := errgroup.WithContext(ctx) - for i, ctr := range containers { - eg.Go(func() error { - publishers := make([]api.PortPublisher, len(ctr.Ports)) - sort.Slice(ctr.Ports, func(i, j int) bool { - return ctr.Ports[i].PrivatePort < ctr.Ports[j].PrivatePort - }) - for i, p := range ctr.Ports { - publishers[i] = api.PortPublisher{ - URL: p.IP, - TargetPort: int(p.PrivatePort), - PublishedPort: int(p.PublicPort), - Protocol: p.Type, - } - } - - inspect, err := s.apiClient().ContainerInspect(ctx, ctr.ID) - if err != nil { - return err - } - - var ( - health container.HealthStatus - exitCode int - ) - if inspect.State != nil { - switch inspect.State.Status { - case container.StateRunning: - if inspect.State.Health != nil { - health = inspect.State.Health.Status - } - case container.StateExited, container.StateDead: - exitCode = inspect.State.ExitCode - } - } - - var ( - local int - mounts []string - ) - for _, m := range ctr.Mounts { - name := m.Name - if name == "" { - name = m.Source - } - if m.Driver == "local" { - local++ - } - mounts = append(mounts, name) - } - - var networks []string - if ctr.NetworkSettings != nil { - for k := range ctr.NetworkSettings.Networks { - networks = append(networks, k) - } - } - - summary[i] = api.ContainerSummary{ - ID: ctr.ID, - Name: getCanonicalContainerName(ctr), - Names: ctr.Names, - Image: ctr.Image, - Project: ctr.Labels[api.ProjectLabel], - Service: ctr.Labels[api.ServiceLabel], - Command: ctr.Command, - State: ctr.State, - Status: ctr.Status, - Created: ctr.Created, - Labels: ctr.Labels, - SizeRw: ctr.SizeRw, - SizeRootFs: ctr.SizeRootFs, - Mounts: mounts, - LocalVolumes: local, - Networks: networks, - Health: health, - ExitCode: exitCode, - Publishers: publishers, - } - return nil - }) - } - return summary, eg.Wait() -} diff --git a/pkg/compose/ps_test.go b/pkg/compose/ps_test.go deleted file mode 100644 index b006d2f57fa..00000000000 --- a/pkg/compose/ps_test.go +++ /dev/null @@ -1,114 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "strings" - "testing" - - containerType "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - compose "github.com/docker/compose/v5/pkg/api" -) - -func TestPs(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - args := filters.NewArgs(projectFilter(strings.ToLower(testProject)), hasConfigHashLabel()) - args.Add("label", "com.docker.compose.oneoff=False") - listOpts := containerType.ListOptions{Filters: args, All: false} - c1, inspect1 := containerDetails("service1", "123", containerType.StateRunning, containerType.Healthy, 0) - c2, inspect2 := containerDetails("service1", "456", containerType.StateRunning, "", 0) - c2.Ports = []containerType.Port{{PublicPort: 80, PrivatePort: 90, IP: "localhost"}} - c3, inspect3 := containerDetails("service2", "789", containerType.StateExited, "", 130) - api.EXPECT().ContainerList(t.Context(), listOpts).Return([]containerType.Summary{c1, c2, c3}, nil) - api.EXPECT().ContainerInspect(anyCancellableContext(), "123").Return(inspect1, nil) - api.EXPECT().ContainerInspect(anyCancellableContext(), "456").Return(inspect2, nil) - api.EXPECT().ContainerInspect(anyCancellableContext(), "789").Return(inspect3, nil) - - containers, err := tested.Ps(t.Context(), strings.ToLower(testProject), compose.PsOptions{}) - - expected := []compose.ContainerSummary{ - { - ID: "123", Name: "123", Names: []string{"/123"}, Image: "foo", Project: strings.ToLower(testProject), Service: "service1", - State: containerType.StateRunning, - Health: containerType.Healthy, - Publishers: []compose.PortPublisher{}, - Labels: map[string]string{ - compose.ProjectLabel: strings.ToLower(testProject), - compose.ConfigFilesLabel: "/src/pkg/compose/testdata/compose.yaml", - compose.WorkingDirLabel: "/src/pkg/compose/testdata", - compose.ServiceLabel: "service1", - }, - }, - { - ID: "456", Name: "456", Names: []string{"/456"}, Image: "foo", Project: strings.ToLower(testProject), Service: "service1", - State: containerType.StateRunning, - Health: "", - Publishers: []compose.PortPublisher{{URL: "localhost", TargetPort: 90, PublishedPort: 80}}, - Labels: map[string]string{ - compose.ProjectLabel: strings.ToLower(testProject), - compose.ConfigFilesLabel: "/src/pkg/compose/testdata/compose.yaml", - compose.WorkingDirLabel: "/src/pkg/compose/testdata", - compose.ServiceLabel: "service1", - }, - }, - { - ID: "789", Name: "789", Names: []string{"/789"}, Image: "foo", Project: strings.ToLower(testProject), Service: "service2", - State: containerType.StateExited, - Health: "", - ExitCode: 130, - Publishers: []compose.PortPublisher{}, - Labels: map[string]string{ - compose.ProjectLabel: strings.ToLower(testProject), - compose.ConfigFilesLabel: "/src/pkg/compose/testdata/compose.yaml", - compose.WorkingDirLabel: "/src/pkg/compose/testdata", - compose.ServiceLabel: "service2", - }, - }, - } - assert.NilError(t, err) - assert.DeepEqual(t, containers, expected) -} - -func containerDetails(service string, id string, status containerType.ContainerState, health containerType.HealthStatus, exitCode int) (containerType.Summary, containerType.InspectResponse) { - ctr := containerType.Summary{ - ID: id, - Names: []string{"/" + id}, - Image: "foo", - Labels: containerLabels(service, false), - State: status, - } - inspect := containerType.InspectResponse{ - ContainerJSONBase: &containerType.ContainerJSONBase{ - State: &containerType.State{ - Status: status, - Health: &containerType.Health{Status: health}, - ExitCode: exitCode, - }, - }, - } - return ctr, inspect -} diff --git a/pkg/compose/publish.go b/pkg/compose/publish.go deleted file mode 100644 index fb466607559..00000000000 --- a/pkg/compose/publish.go +++ /dev/null @@ -1,505 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "context" - "crypto/sha256" - "encoding/json" - "errors" - "fmt" - "io" - "os" - "strings" - - "github.com/DefangLabs/secret-detector/pkg/scanner" - "github.com/DefangLabs/secret-detector/pkg/secrets" - "github.com/compose-spec/compose-go/v2/loader" - "github.com/compose-spec/compose-go/v2/types" - "github.com/distribution/reference" - "github.com/opencontainers/go-digest" - "github.com/opencontainers/image-spec/specs-go" - v1 "github.com/opencontainers/image-spec/specs-go/v1" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/internal/oci" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose/transform" -) - -func (s *composeService) Publish(ctx context.Context, project *types.Project, repository string, options api.PublishOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.publish(ctx, project, repository, options) - }, "publish", s.events) -} - -//nolint:gocyclo -func (s *composeService) publish(ctx context.Context, project *types.Project, repository string, options api.PublishOptions) error { - project, err := project.WithProfiles([]string{"*"}) - if err != nil { - return err - } - accept, err := s.preChecks(project, options) - if err != nil { - return err - } - if !accept { - return nil - } - err = s.Push(ctx, project, api.PushOptions{IgnoreFailures: true, ImageMandatory: true}) - if err != nil { - return err - } - - layers, err := s.createLayers(ctx, project, options) - if err != nil { - return err - } - - s.events.On(api.Resource{ - ID: repository, - Text: "publishing", - Status: api.Working, - }) - if logrus.IsLevelEnabled(logrus.DebugLevel) { - logrus.Debug("publishing layers") - for _, layer := range layers { - indent, _ := json.MarshalIndent(layer, "", " ") - fmt.Println(string(indent)) - } - } - if !s.dryRun { - named, err := reference.ParseDockerRef(repository) - if err != nil { - return err - } - - var insecureRegistries []string - if options.InsecureRegistry { - insecureRegistries = append(insecureRegistries, reference.Domain(named)) - } - - resolver := oci.NewResolver(s.configFile(), insecureRegistries...) - - descriptor, err := oci.PushManifest(ctx, resolver, named, layers, options.OCIVersion) - if err != nil { - s.events.On(api.Resource{ - ID: repository, - Text: "publishing", - Status: api.Error, - }) - return err - } - - if options.Application { - manifests := []v1.Descriptor{} - for _, service := range project.Services { - ref, err := reference.ParseDockerRef(service.Image) - if err != nil { - return err - } - - manifest, err := oci.Copy(ctx, resolver, ref, named) - if err != nil { - return err - } - manifests = append(manifests, manifest) - } - - descriptor.Data = nil - index, err := json.Marshal(v1.Index{ - Versioned: specs.Versioned{SchemaVersion: 2}, - MediaType: v1.MediaTypeImageIndex, - Manifests: manifests, - Subject: &descriptor, - Annotations: map[string]string{ - "com.docker.compose.version": api.ComposeVersion, - }, - }) - if err != nil { - return err - } - imagesDescriptor := v1.Descriptor{ - MediaType: v1.MediaTypeImageIndex, - ArtifactType: oci.ComposeProjectArtifactType, - Digest: digest.FromString(string(index)), - Size: int64(len(index)), - Annotations: map[string]string{ - "com.docker.compose.version": api.ComposeVersion, - }, - Data: index, - } - err = oci.Push(ctx, resolver, reference.TrimNamed(named), imagesDescriptor) - if err != nil { - return err - } - } - } - s.events.On(api.Resource{ - ID: repository, - Text: "published", - Status: api.Done, - }) - return nil -} - -func (s *composeService) createLayers(ctx context.Context, project *types.Project, options api.PublishOptions) ([]v1.Descriptor, error) { - var layers []v1.Descriptor - extFiles := map[string]string{} - envFiles := map[string]string{} - for _, file := range project.ComposeFiles { - data, err := processFile(ctx, file, project, extFiles, envFiles) - if err != nil { - return nil, err - } - - layerDescriptor := oci.DescriptorForComposeFile(file, data) - layers = append(layers, layerDescriptor) - } - - extLayers, err := processExtends(ctx, project, extFiles) - if err != nil { - return nil, err - } - layers = append(layers, extLayers...) - - if options.WithEnvironment { - layers = append(layers, envFileLayers(envFiles)...) - } - - if options.ResolveImageDigests { - yaml, err := s.generateImageDigestsOverride(ctx, project) - if err != nil { - return nil, err - } - - layerDescriptor := oci.DescriptorForComposeFile("image-digests.yaml", yaml) - layers = append(layers, layerDescriptor) - } - return layers, nil -} - -func processExtends(ctx context.Context, project *types.Project, extFiles map[string]string) ([]v1.Descriptor, error) { - var layers []v1.Descriptor - moreExtFiles := map[string]string{} - for xf, hash := range extFiles { - data, err := processFile(ctx, xf, project, moreExtFiles, nil) - if err != nil { - return nil, err - } - - layerDescriptor := oci.DescriptorForComposeFile(hash, data) - layerDescriptor.Annotations["com.docker.compose.extends"] = "true" - layers = append(layers, layerDescriptor) - } - for f, hash := range moreExtFiles { - if _, ok := extFiles[f]; ok { - delete(moreExtFiles, f) - } - extFiles[f] = hash - } - if len(moreExtFiles) > 0 { - extLayers, err := processExtends(ctx, project, moreExtFiles) - if err != nil { - return nil, err - } - layers = append(layers, extLayers...) - } - return layers, nil -} - -func processFile(ctx context.Context, file string, project *types.Project, extFiles map[string]string, envFiles map[string]string) ([]byte, error) { - f, err := os.ReadFile(file) - if err != nil { - return nil, err - } - - base, err := loader.LoadWithContext(ctx, types.ConfigDetails{ - WorkingDir: project.WorkingDir, - Environment: project.Environment, - ConfigFiles: []types.ConfigFile{ - { - Filename: file, - Content: f, - }, - }, - }, func(options *loader.Options) { - options.SkipValidation = true - options.SkipExtends = true - options.SkipConsistencyCheck = true - options.ResolvePaths = true - options.SkipInclude = true - options.Profiles = project.Profiles - }) - if err != nil { - return nil, err - } - for name, service := range base.Services { - for i, envFile := range service.EnvFiles { - hash := fmt.Sprintf("%x.env", sha256.Sum256([]byte(envFile.Path))) - envFiles[envFile.Path] = hash - f, err = transform.ReplaceEnvFile(f, name, i, hash) - if err != nil { - return nil, err - } - } - - if service.Extends == nil { - continue - } - xf := service.Extends.File - if xf == "" { - continue - } - if _, err = os.Stat(service.Extends.File); os.IsNotExist(err) { - // No local file, while we loaded the project successfully: This is actually a remote resource - continue - } - - hash := fmt.Sprintf("%x.yaml", sha256.Sum256([]byte(xf))) - extFiles[xf] = hash - - f, err = transform.ReplaceExtendsFile(f, name, hash) - if err != nil { - return nil, err - } - } - return f, nil -} - -func (s *composeService) generateImageDigestsOverride(ctx context.Context, project *types.Project) ([]byte, error) { - project, err := project.WithImagesResolved(ImageDigestResolver(ctx, s.configFile(), s.apiClient())) - if err != nil { - return nil, err - } - override := types.Project{ - Services: types.Services{}, - } - for name, service := range project.Services { - override.Services[name] = types.ServiceConfig{ - Image: service.Image, - } - } - return override.MarshalYAML() -} - -func (s *composeService) preChecks(project *types.Project, options api.PublishOptions) (bool, error) { - if ok, err := s.checkOnlyBuildSection(project); !ok || err != nil { - return false, err - } - bindMounts := s.checkForBindMount(project) - if len(bindMounts) > 0 { - b := strings.Builder{} - b.WriteString("you are about to publish bind mounts declaration within your OCI artifact.\n" + - "only the bind mount declarations will be added to the OCI artifact (not content)\n" + - "please double check that you are not mounting potential user's sensitive directories or data\n") - for key, val := range bindMounts { - b.WriteString(key) - for _, v := range val { - b.WriteString(v.String()) - b.WriteRune('\n') - } - } - b.WriteString("Are you ok to publish these bind mount declarations?") - confirm, err := s.prompt(b.String(), false) - if err != nil || !confirm { - return false, err - } - } - detectedSecrets, err := s.checkForSensitiveData(project) - if err != nil { - return false, err - } - if len(detectedSecrets) > 0 { - b := strings.Builder{} - b.WriteString("you are about to publish sensitive data within your OCI artifact.\n" + - "please double check that you are not leaking sensitive data\n") - for _, val := range detectedSecrets { - b.WriteString(val.Type) - b.WriteRune('\n') - b.WriteString(fmt.Sprintf("%q: %s\n", val.Key, val.Value)) - } - b.WriteString("Are you ok to publish these sensitive data?") - confirm, err := s.prompt(b.String(), false) - if err != nil || !confirm { - return false, err - } - } - err = s.checkEnvironmentVariables(project, options) - if err != nil { - return false, err - } - return true, nil -} - -func (s *composeService) checkEnvironmentVariables(project *types.Project, options api.PublishOptions) error { - errorList := map[string][]string{} - - for _, service := range project.Services { - if len(service.EnvFiles) > 0 { - errorList[service.Name] = append(errorList[service.Name], fmt.Sprintf("service %q has env_file declared.", service.Name)) - } - } - - if !options.WithEnvironment && len(errorList) > 0 { - errorMsgSuffix := "To avoid leaking sensitive data, you must either explicitly allow the sending of environment variables by using the --with-env flag,\n" + - "or remove sensitive data from your Compose configuration" - var errorMsg strings.Builder - for _, errors := range errorList { - for _, err := range errors { - errorMsg.WriteString(fmt.Sprintf("%s\n", err)) - } - } - return fmt.Errorf("%s%s", errorMsg.String(), errorMsgSuffix) - - } - return nil -} - -func envFileLayers(files map[string]string) []v1.Descriptor { - var layers []v1.Descriptor - for file, hash := range files { - f, err := os.ReadFile(file) - if err != nil { - // if we can't read the file, skip to the next one - continue - } - layerDescriptor := oci.DescriptorForEnvFile(hash, f) - layers = append(layers, layerDescriptor) - } - return layers -} - -func (s *composeService) checkOnlyBuildSection(project *types.Project) (bool, error) { - errorList := []string{} - for _, service := range project.Services { - if service.Image == "" && service.Build != nil { - errorList = append(errorList, service.Name) - } - } - if len(errorList) > 0 { - var errMsg strings.Builder - errMsg.WriteString("your Compose stack cannot be published as it only contains a build section for service(s):\n") - for _, serviceInError := range errorList { - errMsg.WriteString(fmt.Sprintf("- %q\n", serviceInError)) - } - return false, errors.New(errMsg.String()) - } - return true, nil -} - -func (s *composeService) checkForBindMount(project *types.Project) map[string][]types.ServiceVolumeConfig { - allFindings := map[string][]types.ServiceVolumeConfig{} - for serviceName, config := range project.Services { - bindMounts := []types.ServiceVolumeConfig{} - for _, volume := range config.Volumes { - if volume.Type == types.VolumeTypeBind { - bindMounts = append(bindMounts, volume) - } - } - if len(bindMounts) > 0 { - allFindings[serviceName] = bindMounts - } - } - return allFindings -} - -func (s *composeService) checkForSensitiveData(project *types.Project) ([]secrets.DetectedSecret, error) { - var allFindings []secrets.DetectedSecret - scan := scanner.NewDefaultScanner() - // Check all compose files - for _, file := range project.ComposeFiles { - in, err := composeFileAsByteReader(file, project) - if err != nil { - return nil, err - } - - findings, err := scan.ScanReader(in) - if err != nil { - return nil, fmt.Errorf("failed to scan compose file %s: %w", file, err) - } - allFindings = append(allFindings, findings...) - } - for _, service := range project.Services { - // Check env files - for _, envFile := range service.EnvFiles { - findings, err := scan.ScanFile(envFile.Path) - if err != nil { - return nil, fmt.Errorf("failed to scan env file %s: %w", envFile.Path, err) - } - allFindings = append(allFindings, findings...) - } - } - - // Check configs defined by files - for _, config := range project.Configs { - if config.File != "" { - findings, err := scan.ScanFile(config.File) - if err != nil { - return nil, fmt.Errorf("failed to scan config file %s: %w", config.File, err) - } - allFindings = append(allFindings, findings...) - } - } - - // Check secrets defined by files - for _, secret := range project.Secrets { - if secret.File != "" { - findings, err := scan.ScanFile(secret.File) - if err != nil { - return nil, fmt.Errorf("failed to scan secret file %s: %w", secret.File, err) - } - allFindings = append(allFindings, findings...) - } - } - - return allFindings, nil -} - -func composeFileAsByteReader(filePath string, project *types.Project) (io.Reader, error) { - composeFile, err := os.ReadFile(filePath) - if err != nil { - return nil, fmt.Errorf("failed to open compose file %s: %w", filePath, err) - } - base, err := loader.LoadWithContext(context.TODO(), types.ConfigDetails{ - WorkingDir: project.WorkingDir, - Environment: project.Environment, - ConfigFiles: []types.ConfigFile{ - { - Filename: filePath, - Content: composeFile, - }, - }, - }, func(options *loader.Options) { - options.SkipValidation = true - options.SkipExtends = true - options.SkipConsistencyCheck = true - options.ResolvePaths = true - options.SkipInterpolation = true - options.SkipResolveEnvironment = true - }) - if err != nil { - return nil, err - } - - in, err := base.MarshalYAML() - if err != nil { - return nil, err - } - return bytes.NewBuffer(in), nil -} diff --git a/pkg/compose/publish_test.go b/pkg/compose/publish_test.go deleted file mode 100644 index 8f91f663e69..00000000000 --- a/pkg/compose/publish_test.go +++ /dev/null @@ -1,102 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "slices" - "testing" - - "github.com/compose-spec/compose-go/v2/loader" - "github.com/compose-spec/compose-go/v2/types" - "github.com/google/go-cmp/cmp" - v1 "github.com/opencontainers/image-spec/specs-go/v1" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/internal" - "github.com/docker/compose/v5/pkg/api" -) - -func Test_createLayers(t *testing.T) { - project, err := loader.LoadWithContext(t.Context(), types.ConfigDetails{ - WorkingDir: "testdata/publish/", - Environment: types.Mapping{}, - ConfigFiles: []types.ConfigFile{ - { - Filename: "testdata/publish/compose.yaml", - }, - }, - }) - assert.NilError(t, err) - project.ComposeFiles = []string{"testdata/publish/compose.yaml"} - - service := &composeService{} - layers, err := service.createLayers(t.Context(), project, api.PublishOptions{ - WithEnvironment: true, - }) - assert.NilError(t, err) - - published := string(layers[0].Data) - assert.Equal(t, published, `name: test -services: - test: - extends: - file: f8f9ede3d201ec37d5a5e3a77bbadab79af26035e53135e19571f50d541d390c.yaml - service: foo - - string: - image: test - env_file: 5efca9cdbac9f5394c6c2e2094b1b42661f988f57fcab165a0bf72b205451af3.env - - list: - image: test - env_file: - - 5efca9cdbac9f5394c6c2e2094b1b42661f988f57fcab165a0bf72b205451af3.env - - mapping: - image: test - env_file: - - path: 5efca9cdbac9f5394c6c2e2094b1b42661f988f57fcab165a0bf72b205451af3.env -`) - - expectedLayers := []v1.Descriptor{ - { - MediaType: "application/vnd.docker.compose.file+yaml", - Annotations: map[string]string{ - "com.docker.compose.file": "compose.yaml", - "com.docker.compose.version": internal.Version, - }, - }, - { - MediaType: "application/vnd.docker.compose.file+yaml", - Annotations: map[string]string{ - "com.docker.compose.extends": "true", - "com.docker.compose.file": "f8f9ede3d201ec37d5a5e3a77bbadab79af26035e53135e19571f50d541d390c", - "com.docker.compose.version": internal.Version, - }, - }, - { - MediaType: "application/vnd.docker.compose.envfile", - Annotations: map[string]string{ - "com.docker.compose.envfile": "5efca9cdbac9f5394c6c2e2094b1b42661f988f57fcab165a0bf72b205451af3", - "com.docker.compose.version": internal.Version, - }, - }, - } - assert.DeepEqual(t, expectedLayers, layers, cmp.FilterPath(func(path cmp.Path) bool { - return !slices.Contains([]string{".Data", ".Digest", ".Size"}, path.String()) - }, cmp.Ignore())) -} diff --git a/pkg/compose/pull.go b/pkg/compose/pull.go deleted file mode 100644 index bcc466646a6..00000000000 --- a/pkg/compose/pull.go +++ /dev/null @@ -1,454 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "encoding/base64" - "encoding/json" - "errors" - "fmt" - "io" - "strings" - "sync" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/distribution/reference" - "github.com/docker/cli/cli/config/configfile" - clitypes "github.com/docker/cli/cli/config/types" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/client" - "github.com/docker/docker/pkg/jsonmessage" - "github.com/opencontainers/go-digest" - "github.com/sirupsen/logrus" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/internal/registry" - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Pull(ctx context.Context, project *types.Project, options api.PullOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.pull(ctx, project, options) - }, "pull", s.events) -} - -func (s *composeService) pull(ctx context.Context, project *types.Project, opts api.PullOptions) error { //nolint:gocyclo - images, err := s.getLocalImagesDigests(ctx, project) - if err != nil { - return err - } - - eg, ctx := errgroup.WithContext(ctx) - eg.SetLimit(s.maxConcurrency) - - var ( - mustBuild []string - pullErrors = make([]error, len(project.Services)) - imagesBeingPulled = map[string]string{} - ) - - i := 0 - for name, service := range project.Services { - if service.Image == "" { - s.events.On(api.Resource{ - ID: name, - Status: api.Done, - Text: "Skipped", - Details: "No image to be pulled", - }) - continue - } - - switch service.PullPolicy { - case types.PullPolicyNever, types.PullPolicyBuild: - s.events.On(api.Resource{ - ID: "Image " + service.Image, - Status: api.Done, - Text: "Skipped", - }) - continue - case types.PullPolicyMissing, types.PullPolicyIfNotPresent: - if imageAlreadyPresent(service.Image, images) { - s.events.On(api.Resource{ - ID: "Image " + service.Image, - Status: api.Done, - Text: "Skipped", - Details: "Image is already present locally", - }) - continue - } - } - - if service.Build != nil && opts.IgnoreBuildable { - s.events.On(api.Resource{ - ID: "Image " + service.Image, - Status: api.Done, - Text: "Skipped", - Details: "Image can be built", - }) - continue - } - - if _, ok := imagesBeingPulled[service.Image]; ok { - continue - } - - imagesBeingPulled[service.Image] = service.Name - - idx := i - eg.Go(func() error { - _, err := s.pullServiceImage(ctx, service, opts.Quiet, project.Environment["DOCKER_DEFAULT_PLATFORM"]) - if err != nil { - pullErrors[idx] = err - if service.Build != nil { - mustBuild = append(mustBuild, service.Name) - } - if !opts.IgnoreFailures && service.Build == nil { - if s.dryRun { - s.events.On(errorEventf("Image "+service.Image, - "error pulling image: %s", service.Image)) - } - // fail fast if image can't be pulled nor built - return err - } - } - return nil - }) - i++ - } - - err = eg.Wait() - - if len(mustBuild) > 0 { - logrus.Warnf("WARNING: Some service image(s) must be built from source by running:\n docker compose build %s", strings.Join(mustBuild, " ")) - } - - if err != nil { - return err - } - if opts.IgnoreFailures { - return nil - } - return errors.Join(pullErrors...) -} - -func imageAlreadyPresent(serviceImage string, localImages map[string]api.ImageSummary) bool { - normalizedImage, err := reference.ParseDockerRef(serviceImage) - if err != nil { - return false - } - switch refType := normalizedImage.(type) { - case reference.NamedTagged: - _, ok := localImages[serviceImage] - return ok && refType.Tag() != "latest" - default: - _, ok := localImages[serviceImage] - return ok - } -} - -func getUnwrappedErrorMessage(err error) string { - derr := errors.Unwrap(err) - if derr != nil { - return getUnwrappedErrorMessage(derr) - } - return err.Error() -} - -func (s *composeService) pullServiceImage(ctx context.Context, service types.ServiceConfig, quietPull bool, defaultPlatform string) (string, error) { - resource := "Image " + service.Image - s.events.On(pullingEvent(service.Image)) - ref, err := reference.ParseNormalizedNamed(service.Image) - if err != nil { - return "", err - } - - encodedAuth, err := encodedAuth(ref, s.configFile()) - if err != nil { - return "", err - } - - platform := service.Platform - if platform == "" { - platform = defaultPlatform - } - - stream, err := s.apiClient().ImagePull(ctx, service.Image, image.PullOptions{ - RegistryAuth: encodedAuth, - Platform: platform, - }) - - if ctx.Err() != nil { - s.events.On(api.Resource{ - ID: resource, - Status: api.Warning, - Text: "Interrupted", - }) - return "", nil - } - - // check if has error and the service has a build section - // then the status should be warning instead of error - if err != nil && service.Build != nil { - s.events.On(api.Resource{ - ID: resource, - Status: api.Warning, - Text: getUnwrappedErrorMessage(err), - }) - return "", err - } - - if err != nil { - s.events.On(errorEvent(resource, getUnwrappedErrorMessage(err))) - return "", err - } - - dec := json.NewDecoder(stream) - for { - var jm jsonmessage.JSONMessage - if err := dec.Decode(&jm); err != nil { - if errors.Is(err, io.EOF) { - break - } - return "", err - } - if jm.Error != nil { - return "", errors.New(jm.Error.Message) - } - if !quietPull { - toPullProgressEvent(resource, jm, s.events) - } - } - s.events.On(pulledEvent(service.Image)) - - inspected, err := s.apiClient().ImageInspect(ctx, service.Image) - if err != nil { - return "", err - } - return inspected.ID, nil -} - -// ImageDigestResolver creates a func able to resolve image digest from a docker ref, -func ImageDigestResolver(ctx context.Context, file *configfile.ConfigFile, apiClient client.APIClient) func(named reference.Named) (digest.Digest, error) { - return func(named reference.Named) (digest.Digest, error) { - auth, err := encodedAuth(named, file) - if err != nil { - return "", err - } - inspect, err := apiClient.DistributionInspect(ctx, named.String(), auth) - if err != nil { - return "", - fmt.Errorf("failed to resolve digest for %s: %w", named.String(), err) - } - return inspect.Descriptor.Digest, nil - } -} - -type authProvider interface { - GetAuthConfig(registryHostname string) (clitypes.AuthConfig, error) -} - -func encodedAuth(ref reference.Named, configFile authProvider) (string, error) { - authConfig, err := configFile.GetAuthConfig(registry.GetAuthConfigKey(reference.Domain(ref))) - if err != nil { - return "", err - } - - buf, err := json.Marshal(authConfig) - if err != nil { - return "", err - } - return base64.URLEncoding.EncodeToString(buf), nil -} - -func (s *composeService) pullRequiredImages(ctx context.Context, project *types.Project, images map[string]api.ImageSummary, quietPull bool) error { - needPull := map[string]types.ServiceConfig{} - for name, service := range project.Services { - pull, err := mustPull(service, images) - if err != nil { - return err - } - if pull { - needPull[name] = service - } - for i, vol := range service.Volumes { - if vol.Type == types.VolumeTypeImage { - if _, ok := images[vol.Source]; !ok { - // Hack: create a fake ServiceConfig so we pull missing volume image - n := fmt.Sprintf("%s:volume %d", name, i) - needPull[n] = types.ServiceConfig{ - Name: n, - Image: vol.Source, - } - } - } - } - - } - if len(needPull) == 0 { - return nil - } - - eg, ctx := errgroup.WithContext(ctx) - eg.SetLimit(s.maxConcurrency) - pulledImages := map[string]api.ImageSummary{} - var mutex sync.Mutex - for name, service := range needPull { - eg.Go(func() error { - id, err := s.pullServiceImage(ctx, service, quietPull, project.Environment["DOCKER_DEFAULT_PLATFORM"]) - mutex.Lock() - defer mutex.Unlock() - pulledImages[name] = api.ImageSummary{ - ID: id, - Repository: service.Image, - LastTagTime: time.Now(), - } - if err != nil && isServiceImageToBuild(service, project.Services) { - // image can be built, so we can ignore pull failure - return nil - } - return err - }) - } - err := eg.Wait() - for i, service := range needPull { - if pulledImages[i].ID != "" { - images[service.Image] = pulledImages[i] - } - } - return err -} - -func mustPull(service types.ServiceConfig, images map[string]api.ImageSummary) (bool, error) { - if service.Provider != nil { - return false, nil - } - if service.Image == "" { - return false, nil - } - policy, duration, err := service.GetPullPolicy() - if err != nil { - return false, err - } - switch policy { - case types.PullPolicyAlways: - // force pull - return true, nil - case types.PullPolicyNever, types.PullPolicyBuild: - return false, nil - case types.PullPolicyRefresh: - img, ok := images[service.Image] - if !ok { - return true, nil - } - return time.Now().After(img.LastTagTime.Add(duration)), nil - default: // Pull if missing - _, ok := images[service.Image] - return !ok, nil - } -} - -func isServiceImageToBuild(service types.ServiceConfig, services types.Services) bool { - if service.Build != nil { - return true - } - - if service.Image == "" { - // N.B. this should be impossible as service must have either `build` or `image` (or both) - return false - } - - // look through the other services to see if another has a build definition for the same - // image name - for _, svc := range services { - if svc.Image == service.Image && svc.Build != nil { - return true - } - } - return false -} - -const ( - PreparingPhase = "Preparing" - WaitingPhase = "waiting" - PullingFsPhase = "Pulling fs layer" - DownloadingPhase = "Downloading" - DownloadCompletePhase = "Download complete" - ExtractingPhase = "Extracting" - VerifyingChecksumPhase = "Verifying Checksum" - AlreadyExistsPhase = "Already exists" - PullCompletePhase = "Pull complete" -) - -func toPullProgressEvent(parent string, jm jsonmessage.JSONMessage, events api.EventProcessor) { - if jm.ID == "" || jm.Progress == nil { - return - } - - var ( - progress string - total int64 - percent int - current int64 - status = api.Working - ) - - progress = jm.Progress.String() - - switch jm.Status { - case PreparingPhase, WaitingPhase, PullingFsPhase: - percent = 0 - case DownloadingPhase, ExtractingPhase, VerifyingChecksumPhase: - if jm.Progress != nil { - current = jm.Progress.Current - total = jm.Progress.Total - if jm.Progress.Total > 0 { - percent = int(jm.Progress.Current * 100 / jm.Progress.Total) - if percent > 100 { - percent = 100 - } - } - } - case DownloadCompletePhase, AlreadyExistsPhase, PullCompletePhase: - status = api.Done - percent = 100 - } - - if strings.Contains(jm.Status, "Image is up to date") || - strings.Contains(jm.Status, "Downloaded newer image") { - status = api.Done - percent = 100 - } - - if jm.Error != nil { - status = api.Error - progress = jm.Error.Message - } - - events.On(api.Resource{ - ID: jm.ID, - ParentID: parent, - Current: current, - Total: total, - Percent: percent, - Status: status, - Text: jm.Status, - Details: progress, - }) -} diff --git a/pkg/compose/push.go b/pkg/compose/push.go deleted file mode 100644 index a1e9b2686e4..00000000000 --- a/pkg/compose/push.go +++ /dev/null @@ -1,184 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "encoding/base64" - "encoding/json" - "errors" - "fmt" - "io" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/distribution/reference" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/pkg/jsonmessage" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/internal/registry" - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Push(ctx context.Context, project *types.Project, options api.PushOptions) error { - if options.Quiet { - return s.push(ctx, project, options) - } - return Run(ctx, func(ctx context.Context) error { - return s.push(ctx, project, options) - }, "push", s.events) -} - -func (s *composeService) push(ctx context.Context, project *types.Project, options api.PushOptions) error { - eg, ctx := errgroup.WithContext(ctx) - eg.SetLimit(s.maxConcurrency) - - for _, service := range project.Services { - if service.Build == nil || service.Image == "" { - if options.ImageMandatory && service.Image == "" && service.Provider == nil { - return fmt.Errorf("%q attribute is mandatory to push an image for service %q", "service.image", service.Name) - } - s.events.On(api.Resource{ - ID: service.Name, - Status: api.Done, - Text: "Skipped", - }) - continue - } - tags := []string{service.Image} - if service.Build != nil { - tags = append(tags, service.Build.Tags...) - } - - for _, tag := range tags { - eg.Go(func() error { - s.events.On(newEvent(tag, api.Working, "Pushing")) - err := s.pushServiceImage(ctx, tag, options.Quiet) - if err != nil { - if !options.IgnoreFailures { - s.events.On(newEvent(tag, api.Error, err.Error())) - return err - } - s.events.On(newEvent(tag, api.Warning, err.Error())) - } else { - s.events.On(newEvent(tag, api.Done, "Pushed")) - } - return nil - }) - } - } - return eg.Wait() -} - -func (s *composeService) pushServiceImage(ctx context.Context, tag string, quietPush bool) error { - ref, err := reference.ParseNormalizedNamed(tag) - if err != nil { - return err - } - - authConfig, err := s.configFile().GetAuthConfig(registry.GetAuthConfigKey(reference.Domain(ref))) - if err != nil { - return err - } - - buf, err := json.Marshal(authConfig) - if err != nil { - return err - } - - stream, err := s.apiClient().ImagePush(ctx, tag, image.PushOptions{ - RegistryAuth: base64.URLEncoding.EncodeToString(buf), - }) - if err != nil { - return err - } - dec := json.NewDecoder(stream) - for { - var jm jsonmessage.JSONMessage - if err := dec.Decode(&jm); err != nil { - if errors.Is(err, io.EOF) { - break - } - return err - } - if jm.Error != nil { - return errors.New(jm.Error.Message) - } - - if !quietPush { - toPushProgressEvent(tag, jm, s.events) - } - } - - return nil -} - -func toPushProgressEvent(prefix string, jm jsonmessage.JSONMessage, events api.EventProcessor) { - if jm.ID == "" { - // skipped - return - } - var ( - text string - status = api.Working - total int64 - current int64 - percent int - ) - if isDone(jm) { - status = api.Done - percent = 100 - } - if jm.Error != nil { - status = api.Error - text = jm.Error.Message - } - if jm.Progress != nil { - text = jm.Progress.String() - if jm.Progress.Total != 0 { - current = jm.Progress.Current - total = jm.Progress.Total - if jm.Progress.Total > 0 { - percent = int(jm.Progress.Current * 100 / jm.Progress.Total) - if percent > 100 { - percent = 100 - } - } - } - } - - events.On(api.Resource{ - ParentID: prefix, - ID: jm.ID, - Text: text, - Status: status, - Current: current, - Total: total, - Percent: percent, - }) -} - -func isDone(msg jsonmessage.JSONMessage) bool { - // TODO there should be a better way to detect push is done than such a status message check - switch strings.ToLower(msg.Status) { - case "pushed", "layer already exists": - return true - default: - return false - } -} diff --git a/pkg/compose/remove.go b/pkg/compose/remove.go deleted file mode 100644 index 3f63c9a6cce..00000000000 --- a/pkg/compose/remove.go +++ /dev/null @@ -1,115 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "strings" - - "github.com/docker/docker/api/types/container" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Remove(ctx context.Context, projectName string, options api.RemoveOptions) error { - projectName = strings.ToLower(projectName) - - if options.Stop { - err := s.Stop(ctx, projectName, api.StopOptions{ - Services: options.Services, - Project: options.Project, - }) - if err != nil { - return err - } - } - - containers, err := s.getContainers(ctx, projectName, oneOffExclude, true, options.Services...) - if err != nil { - if api.IsNotFoundError(err) { - _, _ = fmt.Fprintln(s.stderr(), "No stopped containers") - return nil - } - return err - } - - if options.Project != nil { - containers = containers.filter(isService(options.Project.ServiceNames()...)) - } - - var stoppedContainers Containers - for _, ctr := range containers { - // We have to inspect containers, as State reported by getContainers suffers a race condition - inspected, err := s.apiClient().ContainerInspect(ctx, ctr.ID) - if api.IsNotFoundError(err) { - // Already removed. Maybe configured with auto-remove - continue - } - if err != nil { - return err - } - if !inspected.State.Running || (options.Stop && s.dryRun) { - stoppedContainers = append(stoppedContainers, ctr) - } - } - - var names []string - stoppedContainers.forEach(func(c container.Summary) { - names = append(names, getCanonicalContainerName(c)) - }) - - if len(names) == 0 { - return api.ErrNoResources - } - - msg := fmt.Sprintf("Going to remove %s", strings.Join(names, ", ")) - if options.Force { - _, _ = fmt.Fprintln(s.stdout(), msg) - } else { - confirm, err := s.prompt(msg, false) - if err != nil { - return err - } - if !confirm { - return nil - } - } - return Run(ctx, func(ctx context.Context) error { - return s.remove(ctx, stoppedContainers, options) - }, "remove", s.events) -} - -func (s *composeService) remove(ctx context.Context, containers Containers, options api.RemoveOptions) error { - eg, ctx := errgroup.WithContext(ctx) - for _, ctr := range containers { - eg.Go(func() error { - eventName := getContainerProgressName(ctr) - s.events.On(removingEvent(eventName)) - err := s.apiClient().ContainerRemove(ctx, ctr.ID, container.RemoveOptions{ - RemoveVolumes: options.Volumes, - Force: options.Force, - }) - if err == nil { - s.events.On(removedEvent(eventName)) - } - return err - }) - } - return eg.Wait() -} diff --git a/pkg/compose/restart.go b/pkg/compose/restart.go deleted file mode 100644 index ca73b962896..00000000000 --- a/pkg/compose/restart.go +++ /dev/null @@ -1,114 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -func (s *composeService) Restart(ctx context.Context, projectName string, options api.RestartOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.restart(ctx, strings.ToLower(projectName), options) - }, "restart", s.events) -} - -func (s *composeService) restart(ctx context.Context, projectName string, options api.RestartOptions) error { //nolint:gocyclo - containers, err := s.getContainers(ctx, projectName, oneOffExclude, true) - if err != nil { - return err - } - - project := options.Project - if project == nil { - project, err = s.getProjectWithResources(ctx, containers, projectName) - if err != nil { - return err - } - } - - if options.NoDeps { - project, err = project.WithSelectedServices(options.Services, types.IgnoreDependencies) - if err != nil { - return err - } - } - - // ignore depends_on relations which are not impacted by restarting service or not required - project, err = project.WithServicesTransform(func(_ string, s types.ServiceConfig) (types.ServiceConfig, error) { - for name, r := range s.DependsOn { - if !r.Restart { - delete(s.DependsOn, name) - } - } - return s, nil - }) - if err != nil { - return err - } - - if len(options.Services) != 0 { - project, err = project.WithSelectedServices(options.Services, types.IncludeDependents) - if err != nil { - return err - } - } - - return InDependencyOrder(ctx, project, func(c context.Context, service string) error { - config := project.Services[service] - err = s.waitDependencies(ctx, project, service, config.DependsOn, containers, 0) - if err != nil { - return err - } - - eg, ctx := errgroup.WithContext(ctx) - for _, ctr := range containers.filter(isService(service)) { - eg.Go(func() error { - def := project.Services[service] - for _, hook := range def.PreStop { - err = s.runHook(ctx, ctr, def, hook, nil) - if err != nil { - return err - } - } - eventName := getContainerProgressName(ctr) - s.events.On(restartingEvent(eventName)) - timeout := utils.DurationSecondToInt(options.Timeout) - err = s.apiClient().ContainerRestart(ctx, ctr.ID, container.StopOptions{Timeout: timeout}) - if err != nil { - return err - } - s.events.On(startedEvent(eventName)) - for _, hook := range def.PostStart { - err = s.runHook(ctx, ctr, def, hook, nil) - if err != nil { - return err - } - } - return nil - }) - } - return eg.Wait() - }) -} diff --git a/pkg/compose/run.go b/pkg/compose/run.go deleted file mode 100644 index e5a68fd0447..00000000000 --- a/pkg/compose/run.go +++ /dev/null @@ -1,227 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "os" - "os/signal" - "slices" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli" - cmd "github.com/docker/cli/cli/command/container" - "github.com/docker/docker/pkg/stringid" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) RunOneOffContainer(ctx context.Context, project *types.Project, opts api.RunOptions) (int, error) { - containerID, err := s.prepareRun(ctx, project, opts) - if err != nil { - return 0, err - } - - // remove cancellable context signal handler so we can forward signals to container without compose from exiting - signal.Reset() - - sigc := make(chan os.Signal, 128) - signal.Notify(sigc) - go cmd.ForwardAllSignals(ctx, s.apiClient(), containerID, sigc) - defer signal.Stop(sigc) - - err = cmd.RunStart(ctx, s.dockerCli, &cmd.StartOptions{ - OpenStdin: !opts.Detach && opts.Interactive, - Attach: !opts.Detach, - Containers: []string{containerID}, - DetachKeys: s.configFile().DetachKeys, - }) - var stErr cli.StatusError - if errors.As(err, &stErr) { - return stErr.StatusCode, nil - } - return 0, err -} - -func (s *composeService) prepareRun(ctx context.Context, project *types.Project, opts api.RunOptions) (string, error) { - // Temporary implementation of use_api_socket until we get actual support inside docker engine - project, err := s.useAPISocket(project) - if err != nil { - return "", err - } - - err = Run(ctx, func(ctx context.Context) error { - return s.startDependencies(ctx, project, opts) - }, "run", s.events) - if err != nil { - return "", err - } - - service, err := project.GetService(opts.Service) - if err != nil { - return "", err - } - - applyRunOptions(project, &service, opts) - - if err := s.stdin().CheckTty(opts.Interactive, service.Tty); err != nil { - return "", err - } - - slug := stringid.GenerateRandomID() - if service.ContainerName == "" { - service.ContainerName = fmt.Sprintf("%[1]s%[4]s%[2]s%[4]srun%[4]s%[3]s", project.Name, service.Name, stringid.TruncateID(slug), api.Separator) - } - one := 1 - service.Scale = &one - service.Restart = "" - if service.Deploy != nil { - service.Deploy.RestartPolicy = nil - } - service.CustomLabels = service.CustomLabels. - Add(api.SlugLabel, slug). - Add(api.OneoffLabel, "True") - - // Only ensure image exists for the target service, dependencies were already handled by startDependencies - buildOpts := prepareBuildOptions(opts) - if err := s.ensureImagesExists(ctx, project, buildOpts, opts.QuietPull); err != nil { // all dependencies already checked, but might miss service img - return "", err - } - - observedState, err := s.getContainers(ctx, project.Name, oneOffInclude, true) - if err != nil { - return "", err - } - - if !opts.NoDeps { - if err := s.waitDependencies(ctx, project, service.Name, service.DependsOn, observedState, 0); err != nil { - return "", err - } - } - createOpts := createOptions{ - AutoRemove: opts.AutoRemove, - AttachStdin: opts.Interactive, - UseNetworkAliases: opts.UseNetworkAliases, - Labels: mergeLabels(service.Labels, service.CustomLabels), - } - - err = newConvergence(project.ServiceNames(), observedState, nil, nil, s).resolveServiceReferences(&service) - if err != nil { - return "", err - } - - err = s.ensureModels(ctx, project, opts.QuietPull) - if err != nil { - return "", err - } - - created, err := s.createContainer(ctx, project, service, service.ContainerName, -1, createOpts) - if err != nil { - return "", err - } - - ctr, err := s.apiClient().ContainerInspect(ctx, created.ID) - if err != nil { - return "", err - } - - err = s.injectSecrets(ctx, project, service, ctr.ID) - if err != nil { - return created.ID, err - } - - err = s.injectConfigs(ctx, project, service, ctr.ID) - return created.ID, err -} - -func prepareBuildOptions(opts api.RunOptions) *api.BuildOptions { - if opts.Build == nil { - return nil - } - // Create a copy of build options and restrict to only the target service - buildOptsCopy := *opts.Build - buildOptsCopy.Services = []string{opts.Service} - return &buildOptsCopy -} - -func applyRunOptions(project *types.Project, service *types.ServiceConfig, opts api.RunOptions) { - service.Tty = opts.Tty - service.StdinOpen = opts.Interactive - service.ContainerName = opts.Name - - if len(opts.Command) > 0 { - service.Command = opts.Command - } - if opts.User != "" { - service.User = opts.User - } - - if len(opts.CapAdd) > 0 { - service.CapAdd = append(service.CapAdd, opts.CapAdd...) - service.CapDrop = slices.DeleteFunc(service.CapDrop, func(e string) bool { return slices.Contains(opts.CapAdd, e) }) - } - if len(opts.CapDrop) > 0 { - service.CapDrop = append(service.CapDrop, opts.CapDrop...) - service.CapAdd = slices.DeleteFunc(service.CapAdd, func(e string) bool { return slices.Contains(opts.CapDrop, e) }) - } - if opts.WorkingDir != "" { - service.WorkingDir = opts.WorkingDir - } - if opts.Entrypoint != nil { - service.Entrypoint = opts.Entrypoint - if len(opts.Command) == 0 { - service.Command = []string{} - } - } - if len(opts.Environment) > 0 { - cmdEnv := types.NewMappingWithEquals(opts.Environment) - serviceOverrideEnv := cmdEnv.Resolve(func(s string) (string, bool) { - v, ok := envResolver(project.Environment)(s) - return v, ok - }).RemoveEmpty() - if service.Environment == nil { - service.Environment = types.MappingWithEquals{} - } - service.Environment.OverrideBy(serviceOverrideEnv) - } - for k, v := range opts.Labels { - service.Labels = service.Labels.Add(k, v) - } -} - -func (s *composeService) startDependencies(ctx context.Context, project *types.Project, options api.RunOptions) error { - project = project.WithServicesDisabled(options.Service) - - err := s.Create(ctx, project, api.CreateOptions{ - Build: options.Build, - IgnoreOrphans: options.IgnoreOrphans, - RemoveOrphans: options.RemoveOrphans, - QuietPull: options.QuietPull, - }) - if err != nil { - return err - } - - if len(project.Services) > 0 { - return s.Start(ctx, project.Name, api.StartOptions{ - Project: project, - }) - } - return nil -} diff --git a/pkg/compose/scale.go b/pkg/compose/scale.go deleted file mode 100644 index 4ef8134a264..00000000000 --- a/pkg/compose/scale.go +++ /dev/null @@ -1,35 +0,0 @@ -/* -Copyright 2020 Docker Compose CLI authors - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package compose - -import ( - "context" - - "github.com/compose-spec/compose-go/v2/types" - - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Scale(ctx context.Context, project *types.Project, options api.ScaleOptions) error { - return Run(ctx, tracing.SpanWrapFunc("project/scale", tracing.ProjectOptions(ctx, project), func(ctx context.Context) error { - err := s.create(ctx, project, api.CreateOptions{Services: options.Services}) - if err != nil { - return err - } - return s.start(ctx, project.Name, api.StartOptions{Project: project, Services: options.Services}, nil) - }), "scale", s.events) -} diff --git a/pkg/compose/secrets.go b/pkg/compose/secrets.go deleted file mode 100644 index 72bc4b5c8e8..00000000000 --- a/pkg/compose/secrets.go +++ /dev/null @@ -1,179 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "archive/tar" - "bytes" - "context" - "fmt" - "strconv" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" -) - -type mountType string - -const ( - secretMount mountType = "secret" - configMount mountType = "config" -) - -func (s *composeService) injectSecrets(ctx context.Context, project *types.Project, service types.ServiceConfig, id string) error { - return s.injectFileReferences(ctx, project, service, id, secretMount) -} - -func (s *composeService) injectConfigs(ctx context.Context, project *types.Project, service types.ServiceConfig, id string) error { - return s.injectFileReferences(ctx, project, service, id, configMount) -} - -func (s *composeService) injectFileReferences(ctx context.Context, project *types.Project, service types.ServiceConfig, id string, mountType mountType) error { - mounts, sources := s.getFilesAndMap(project, service, mountType) - - for _, mount := range mounts { - content, err := s.resolveFileContent(project, sources[mount.Source], mountType) - if err != nil { - return err - } - if content == "" { - continue - } - - if service.ReadOnly { - return fmt.Errorf("cannot create %s %q in read-only service %s: `file` is the sole supported option", mountType, sources[mount.Source].Name, service.Name) - } - - s.setDefaultTarget(&mount, mountType) - - if err := s.copyFileToContainer(ctx, id, content, mount); err != nil { - return err - } - } - return nil -} - -func (s *composeService) getFilesAndMap(project *types.Project, service types.ServiceConfig, mountType mountType) ([]types.FileReferenceConfig, map[string]types.FileObjectConfig) { - var files []types.FileReferenceConfig - var fileMap map[string]types.FileObjectConfig - - switch mountType { - case secretMount: - files = make([]types.FileReferenceConfig, len(service.Secrets)) - for i, config := range service.Secrets { - files[i] = types.FileReferenceConfig(config) - } - fileMap = make(map[string]types.FileObjectConfig) - for k, v := range project.Secrets { - fileMap[k] = types.FileObjectConfig(v) - } - case configMount: - files = make([]types.FileReferenceConfig, len(service.Configs)) - for i, config := range service.Configs { - files[i] = types.FileReferenceConfig(config) - } - fileMap = make(map[string]types.FileObjectConfig) - for k, v := range project.Configs { - fileMap[k] = types.FileObjectConfig(v) - } - } - return files, fileMap -} - -func (s *composeService) resolveFileContent(project *types.Project, source types.FileObjectConfig, mountType mountType) (string, error) { - if source.Content != "" { - // inlined, or already resolved by include - return source.Content, nil - } - if source.Environment != "" { - env, ok := project.Environment[source.Environment] - if !ok { - return "", fmt.Errorf("environment variable %q required by %s %q is not set", source.Environment, mountType, source.Name) - } - return env, nil - } - return "", nil -} - -func (s *composeService) setDefaultTarget(file *types.FileReferenceConfig, mountType mountType) { - if file.Target == "" { - if mountType == secretMount { - file.Target = "/run/secrets/" + file.Source - } else { - file.Target = "/" + file.Source - } - } else if mountType == secretMount && !isAbsTarget(file.Target) { - file.Target = "/run/secrets/" + file.Target - } -} - -func (s *composeService) copyFileToContainer(ctx context.Context, id, content string, file types.FileReferenceConfig) error { - b, err := createTar(content, file) - if err != nil { - return err - } - - return s.apiClient().CopyToContainer(ctx, id, "/", &b, container.CopyToContainerOptions{ - CopyUIDGID: file.UID != "" || file.GID != "", - }) -} - -func createTar(env string, config types.FileReferenceConfig) (bytes.Buffer, error) { - value := []byte(env) - b := bytes.Buffer{} - tarWriter := tar.NewWriter(&b) - mode := types.FileMode(0o444) - if config.Mode != nil { - mode = *config.Mode - } - - var uid, gid int - if config.UID != "" { - v, err := strconv.Atoi(config.UID) - if err != nil { - return b, err - } - uid = v - } - if config.GID != "" { - v, err := strconv.Atoi(config.GID) - if err != nil { - return b, err - } - gid = v - } - - header := &tar.Header{ - Name: config.Target, - Size: int64(len(value)), - Mode: int64(mode), - ModTime: time.Now(), - Uid: uid, - Gid: gid, - } - err := tarWriter.WriteHeader(header) - if err != nil { - return bytes.Buffer{}, err - } - _, err = tarWriter.Write(value) - if err != nil { - return bytes.Buffer{}, err - } - err = tarWriter.Close() - return b, err -} diff --git a/pkg/compose/shellout.go b/pkg/compose/shellout.go deleted file mode 100644 index ceea0788214..00000000000 --- a/pkg/compose/shellout.go +++ /dev/null @@ -1,90 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "os" - "os/exec" - "path/filepath" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli-plugins/metadata" - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/flags" - "github.com/docker/docker/client" - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/propagation" - - "github.com/docker/compose/v5/internal" -) - -// prepareShellOut prepare a shell-out command to be ran by Compose -func (s *composeService) prepareShellOut(gctx context.Context, env types.Mapping, cmd *exec.Cmd) error { - env = env.Clone() - // remove DOCKER_CLI_PLUGIN... variable so a docker-cli plugin will detect it run standalone - delete(env, metadata.ReexecEnvvar) - - // propagate opentelemetry context to child process, see https://github.com/open-telemetry/oteps/blob/main/text/0258-env-context-baggage-carriers.md - carrier := propagation.MapCarrier{} - otel.GetTextMapPropagator().Inject(gctx, &carrier) - env.Merge(types.Mapping(carrier)) - - cmd.Env = env.Values() - return nil -} - -// propagateDockerEndpoint produces DOCKER_* env vars for a child CLI plugin to target the same docker endpoint -// `cleanup` func MUST be called after child process completion to enforce removal of cert files -func (s *composeService) propagateDockerEndpoint() ([]string, func(), error) { - cleanup := func() {} - env := types.Mapping{} - - env[command.EnvOverrideContext] = s.dockerCli.CurrentContext() - env["USER_AGENT"] = "compose/" + internal.Version - - endpoint := s.dockerCli.DockerEndpoint() - env[client.EnvOverrideHost] = endpoint.Host - if endpoint.TLSData != nil { - certs, err := os.MkdirTemp("", "compose") - if err != nil { - return nil, cleanup, err - } - cleanup = func() { - _ = os.RemoveAll(certs) - } - env[client.EnvOverrideCertPath] = certs - env["DOCKER_TLS"] = "1" - if !endpoint.SkipTLSVerify { - env[client.EnvTLSVerify] = "1" - } - - err = os.WriteFile(filepath.Join(certs, flags.DefaultKeyFile), endpoint.TLSData.Key, 0o600) - if err != nil { - return nil, cleanup, err - } - err = os.WriteFile(filepath.Join(certs, flags.DefaultCertFile), endpoint.TLSData.Cert, 0o600) - if err != nil { - return nil, cleanup, err - } - err = os.WriteFile(filepath.Join(certs, flags.DefaultCaFile), endpoint.TLSData.CA, 0o600) - if err != nil { - return nil, cleanup, err - } - } - return env.Values(), cleanup, nil -} diff --git a/pkg/compose/start.go b/pkg/compose/start.go deleted file mode 100644 index eeb232fa66d..00000000000 --- a/pkg/compose/start.go +++ /dev/null @@ -1,115 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - containerType "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Start(ctx context.Context, projectName string, options api.StartOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.start(ctx, strings.ToLower(projectName), options, nil) - }, "start", s.events) -} - -func (s *composeService) start(ctx context.Context, projectName string, options api.StartOptions, listener api.ContainerEventListener) error { - project := options.Project - if project == nil { - var containers Containers - containers, err := s.getContainers(ctx, projectName, oneOffExclude, true) - if err != nil { - return err - } - - project, err = s.projectFromName(containers, projectName, options.AttachTo...) - if err != nil { - return err - } - } - - var containers Containers - containers, err := s.apiClient().ContainerList(ctx, containerType.ListOptions{ - Filters: filters.NewArgs( - projectFilter(project.Name), - oneOffFilter(false), - ), - All: true, - }) - if err != nil { - return err - } - - err = InDependencyOrder(ctx, project, func(c context.Context, name string) error { - service, err := project.GetService(name) - if err != nil { - return err - } - - return s.startService(ctx, project, service, containers, listener, options.WaitTimeout) - }) - if err != nil { - return err - } - - if options.Wait { - depends := types.DependsOnConfig{} - for _, s := range project.Services { - depends[s.Name] = types.ServiceDependency{ - Condition: getDependencyCondition(s, project), - Required: true, - } - } - if options.WaitTimeout > 0 { - withTimeout, cancel := context.WithTimeout(ctx, options.WaitTimeout) - ctx = withTimeout - defer cancel() - } - - err = s.waitDependencies(ctx, project, project.Name, depends, containers, 0) - if err != nil { - if errors.Is(ctx.Err(), context.DeadlineExceeded) { - return fmt.Errorf("application not healthy after %s", options.WaitTimeout) - } - return err - } - } - - return nil -} - -// getDependencyCondition checks if service is depended on by other services -// with service_completed_successfully condition, and applies that condition -// instead, or --wait will never finish waiting for one-shot containers -func getDependencyCondition(service types.ServiceConfig, project *types.Project) string { - for _, services := range project.Services { - for dependencyService, dependencyConfig := range services.DependsOn { - if dependencyService == service.Name && dependencyConfig.Condition == types.ServiceConditionCompletedSuccessfully { - return types.ServiceConditionCompletedSuccessfully - } - } - } - return ServiceConditionRunningOrHealthy -} diff --git a/pkg/compose/stop.go b/pkg/compose/stop.go deleted file mode 100644 index 79272513c0b..00000000000 --- a/pkg/compose/stop.go +++ /dev/null @@ -1,58 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "slices" - "strings" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Stop(ctx context.Context, projectName string, options api.StopOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.stop(ctx, strings.ToLower(projectName), options, nil) - }, "stop", s.events) -} - -func (s *composeService) stop(ctx context.Context, projectName string, options api.StopOptions, event api.ContainerEventListener) error { - containers, err := s.getContainers(ctx, projectName, oneOffExclude, true) - if err != nil { - return err - } - - project := options.Project - if project == nil { - project, err = s.getProjectWithResources(ctx, containers, projectName) - if err != nil { - return err - } - } - - if len(options.Services) == 0 { - options.Services = project.ServiceNames() - } - - return InReverseDependencyOrder(ctx, project, func(c context.Context, service string) error { - if !slices.Contains(options.Services, service) { - return nil - } - serv := project.Services[service] - return s.stopContainers(ctx, &serv, containers.filter(isService(service)).filter(isNotOneOff), options.Timeout, event) - }) -} diff --git a/pkg/compose/stop_test.go b/pkg/compose/stop_test.go deleted file mode 100644 index adecd1d615b..00000000000 --- a/pkg/compose/stop_test.go +++ /dev/null @@ -1,68 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "strings" - "testing" - "time" - - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/api/types/volume" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - compose "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -func TestStopTimeout(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)).Return( - []container.Summary{ - testContainer("service1", "123", false), - testContainer("service1", "456", false), - testContainer("service2", "789", false), - }, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{}, nil) - - timeout := 2 * time.Second - stopConfig := container.StopOptions{Timeout: utils.DurationSecondToInt(&timeout)} - api.EXPECT().ContainerStop(gomock.Any(), "123", stopConfig).Return(nil) - api.EXPECT().ContainerStop(gomock.Any(), "456", stopConfig).Return(nil) - api.EXPECT().ContainerStop(gomock.Any(), "789", stopConfig).Return(nil) - - err = tested.Stop(t.Context(), strings.ToLower(testProject), compose.StopOptions{ - Timeout: &timeout, - }) - assert.NilError(t, err) -} diff --git a/pkg/compose/suffix_unix.go b/pkg/compose/suffix_unix.go deleted file mode 100644 index 59595384ca4..00000000000 --- a/pkg/compose/suffix_unix.go +++ /dev/null @@ -1,23 +0,0 @@ -//go:build !windows - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -func executable(s string) string { - return s -} diff --git a/pkg/compose/testdata/compose.yaml b/pkg/compose/testdata/compose.yaml deleted file mode 100644 index 4e3e6cb98c1..00000000000 --- a/pkg/compose/testdata/compose.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - service1: - image: nginx - service2: - image: mysql diff --git a/pkg/compose/testdata/publish/common.yaml b/pkg/compose/testdata/publish/common.yaml deleted file mode 100644 index ce048e36623..00000000000 --- a/pkg/compose/testdata/publish/common.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - foo: - image: bar diff --git a/pkg/compose/testdata/publish/compose.yaml b/pkg/compose/testdata/publish/compose.yaml deleted file mode 100644 index 9c9f3659b5e..00000000000 --- a/pkg/compose/testdata/publish/compose.yaml +++ /dev/null @@ -1,20 +0,0 @@ -name: test -services: - test: - extends: - file: common.yaml - service: foo - - string: - image: test - env_file: test.env - - list: - image: test - env_file: - - test.env - - mapping: - image: test - env_file: - - path: test.env diff --git a/pkg/compose/testdata/publish/test.env b/pkg/compose/testdata/publish/test.env deleted file mode 100644 index 6e1f61b59ea..00000000000 --- a/pkg/compose/testdata/publish/test.env +++ /dev/null @@ -1 +0,0 @@ -HELLO=WORLD \ No newline at end of file diff --git a/pkg/compose/top.go b/pkg/compose/top.go deleted file mode 100644 index d2efdc1bc8c..00000000000 --- a/pkg/compose/top.go +++ /dev/null @@ -1,65 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strings" - - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Top(ctx context.Context, projectName string, services []string) ([]api.ContainerProcSummary, error) { - projectName = strings.ToLower(projectName) - var containers Containers - containers, err := s.getContainers(ctx, projectName, oneOffInclude, false) - if err != nil { - return nil, err - } - if len(services) > 0 { - containers = containers.filter(isService(services...)) - } - summary := make([]api.ContainerProcSummary, len(containers)) - eg, ctx := errgroup.WithContext(ctx) - for i, ctr := range containers { - eg.Go(func() error { - topContent, err := s.apiClient().ContainerTop(ctx, ctr.ID, []string{}) - if err != nil { - return err - } - name := getCanonicalContainerName(ctr) - s := api.ContainerProcSummary{ - ID: ctr.ID, - Name: name, - Processes: topContent.Processes, - Titles: topContent.Titles, - Service: name, - } - if service, exists := ctr.Labels[api.ServiceLabel]; exists { - s.Service = service - } - if replica, exists := ctr.Labels[api.ContainerNumberLabel]; exists { - s.Replica = replica - } - summary[i] = s - return nil - }) - } - return summary, eg.Wait() -} diff --git a/pkg/compose/transform/replace.go b/pkg/compose/transform/replace.go deleted file mode 100644 index 36266a066f4..00000000000 --- a/pkg/compose/transform/replace.go +++ /dev/null @@ -1,149 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package transform - -import ( - "fmt" - - "go.yaml.in/yaml/v4" -) - -// ReplaceExtendsFile changes value for service.extends.file in input yaml stream, preserving formatting -func ReplaceExtendsFile(in []byte, service string, value string) ([]byte, error) { - var doc yaml.Node - err := yaml.Unmarshal(in, &doc) - if err != nil { - return nil, err - } - if doc.Kind != yaml.DocumentNode { - return nil, fmt.Errorf("expected document kind %v, got %v", yaml.DocumentNode, doc.Kind) - } - root := doc.Content[0] - if root.Kind != yaml.MappingNode { - return nil, fmt.Errorf("expected document root to be a mapping, got %v", root.Kind) - } - - services, err := getMapping(root, "services") - if err != nil { - return nil, err - } - - target, err := getMapping(services, service) - if err != nil { - return nil, err - } - - extends, err := getMapping(target, "extends") - if err != nil { - return nil, err - } - - file, err := getMapping(extends, "file") - if err != nil { - return nil, err - } - - // we've found target `file` yaml node. Let's replace value in stream at node position - return replace(in, file.Line, file.Column, value), nil -} - -// ReplaceEnvFile changes value for service.extends.env_file in input yaml stream, preserving formatting -func ReplaceEnvFile(in []byte, service string, i int, value string) ([]byte, error) { - var doc yaml.Node - err := yaml.Unmarshal(in, &doc) - if err != nil { - return nil, err - } - if doc.Kind != yaml.DocumentNode { - return nil, fmt.Errorf("expected document kind %v, got %v", yaml.DocumentNode, doc.Kind) - } - root := doc.Content[0] - if root.Kind != yaml.MappingNode { - return nil, fmt.Errorf("expected document root to be a mapping, got %v", root.Kind) - } - - services, err := getMapping(root, "services") - if err != nil { - return nil, err - } - - target, err := getMapping(services, service) - if err != nil { - return nil, err - } - - envFile, err := getMapping(target, "env_file") - if err != nil { - return nil, err - } - - // env_file can be either a string, sequence of strings, or sequence of mappings with path attribute - if envFile.Kind == yaml.SequenceNode { - envFile = envFile.Content[i] - if envFile.Kind == yaml.MappingNode { - envFile, err = getMapping(envFile, "path") - if err != nil { - return nil, err - } - } - return replace(in, envFile.Line, envFile.Column, value), nil - } else { - return replace(in, envFile.Line, envFile.Column, value), nil - } -} - -func getMapping(root *yaml.Node, key string) (*yaml.Node, error) { - var node *yaml.Node - l := len(root.Content) - for i := 0; i < l; i += 2 { - k := root.Content[i] - if k.Kind != yaml.ScalarNode || k.Tag != "!!str" { - return nil, fmt.Errorf("expected mapping key to be a string, got %v %v", root.Kind, k.Tag) - } - if k.Value == key { - node = root.Content[i+1] - return node, nil - } - } - return nil, fmt.Errorf("key %v not found", key) -} - -// replace changes yaml node value in stream at position, preserving content -func replace(in []byte, line int, column int, value string) []byte { - var out []byte - l := 1 - pos := 0 - for _, b := range in { - if b == '\n' { - l++ - if l == line { - break - } - } - pos++ - } - pos += column - out = append(out, in[0:pos]...) - out = append(out, []byte(value)...) - for ; pos < len(in); pos++ { - if in[pos] == '\n' { - break - } - } - out = append(out, in[pos:]...) - return out -} diff --git a/pkg/compose/transform/replace_test.go b/pkg/compose/transform/replace_test.go deleted file mode 100644 index 6f7477e0350..00000000000 --- a/pkg/compose/transform/replace_test.go +++ /dev/null @@ -1,85 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package transform - -import ( - "reflect" - "testing" - - "gotest.tools/v3/assert" -) - -func TestReplace(t *testing.T) { - tests := []struct { - name string - in string - want string - }{ - { - name: "simple", - in: `services: - test: - extends: - file: foo.yaml - service: foo -`, - want: `services: - test: - extends: - file: REPLACED - service: foo -`, - }, - { - name: "last line", - in: `services: - test: - extends: - service: foo - file: foo.yaml -`, - want: `services: - test: - extends: - service: foo - file: REPLACED -`, - }, - { - name: "last line no CR", - in: `services: - test: - extends: - service: foo - file: foo.yaml`, - want: `services: - test: - extends: - service: foo - file: REPLACED`, - }, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - got, err := ReplaceExtendsFile([]byte(tt.in), "test", "REPLACED") - assert.NilError(t, err) - if !reflect.DeepEqual(got, []byte(tt.want)) { - t.Errorf("ReplaceExtendsFile() got = %v, want %v", got, tt.want) - } - }) - } -} diff --git a/pkg/compose/up.go b/pkg/compose/up.go deleted file mode 100644 index d5eb4d87691..00000000000 --- a/pkg/compose/up.go +++ /dev/null @@ -1,303 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "os" - "os/signal" - "slices" - "sync" - "sync/atomic" - "syscall" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/docker/cli/cli" - "github.com/eiannone/keyboard" - "github.com/sirupsen/logrus" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Up(ctx context.Context, project *types.Project, options api.UpOptions) error { //nolint:gocyclo - err := Run(ctx, tracing.SpanWrapFunc("project/up", tracing.ProjectOptions(ctx, project), func(ctx context.Context) error { - err := s.create(ctx, project, options.Create) - if err != nil { - return err - } - if options.Start.Attach == nil { - return s.start(ctx, project.Name, options.Start, nil) - } - return nil - }), "up", s.events) - if err != nil { - return err - } - - if options.Start.Attach == nil { - return err - } - if s.dryRun { - _, _ = fmt.Fprintln(s.stdout(), "end of 'compose up' output, interactive run is not supported in dry-run mode") - return err - } - - // if we get a second signal during shutdown, we kill the services - // immediately, so the channel needs to have sufficient capacity or - // we might miss a signal while setting up the second channel read - // (this is also why signal.Notify is used vs signal.NotifyContext) - signalChan := make(chan os.Signal, 2) - signal.Notify(signalChan, syscall.SIGINT, syscall.SIGTERM) - defer signal.Stop(signalChan) - var isTerminated atomic.Bool - - var ( - logConsumer = options.Start.Attach - navigationMenu *formatter.LogKeyboard - kEvents <-chan keyboard.KeyEvent - ) - if options.Start.NavigationMenu { - kEvents, err = keyboard.GetKeys(100) - if err != nil { - logrus.Warnf("could not start menu, an error occurred while starting: %v", err) - options.Start.NavigationMenu = false - } else { - defer keyboard.Close() //nolint:errcheck - isDockerDesktopActive, err := s.isDesktopIntegrationActive(ctx) - if err != nil { - return err - } - tracing.KeyboardMetrics(ctx, options.Start.NavigationMenu, isDockerDesktopActive) - navigationMenu = formatter.NewKeyboardManager(isDockerDesktopActive, signalChan) - logConsumer = navigationMenu.Decorate(logConsumer) - } - } - - watcher, err := NewWatcher(project, options, s.watch, logConsumer) - if err != nil && options.Start.Watch { - return err - } - - if navigationMenu != nil && watcher != nil { - navigationMenu.EnableWatch(options.Start.Watch, watcher) - } - - printer := newLogPrinter(logConsumer) - - // global context to handle canceling goroutines - globalCtx, cancel := context.WithCancel(ctx) - defer cancel() - - if navigationMenu != nil { - navigationMenu.EnableDetach(cancel) - } - - var ( - eg errgroup.Group - mu sync.Mutex - errs []error - ) - - appendErr := func(err error) { - if err != nil { - mu.Lock() - errs = append(errs, err) - mu.Unlock() - } - } - - eg.Go(func() error { - first := true - gracefulTeardown := func() { - first = false - s.events.On(newEvent(api.ResourceCompose, api.Working, api.StatusStopping, "Gracefully Stopping... press Ctrl+C again to force")) - eg.Go(func() error { - err = s.stop(context.WithoutCancel(globalCtx), project.Name, api.StopOptions{ - Services: options.Create.Services, - Project: project, - }, printer.HandleEvent) - appendErr(err) - return nil - }) - isTerminated.Store(true) - } - - for { - select { - case <-globalCtx.Done(): - if watcher != nil { - return watcher.Stop() - } - return nil - case <-ctx.Done(): - if first { - gracefulTeardown() - } - case <-signalChan: - if first { - _ = keyboard.Close() - gracefulTeardown() - break - } - eg.Go(func() error { - err := s.kill(context.WithoutCancel(globalCtx), project.Name, api.KillOptions{ - Services: options.Create.Services, - Project: project, - All: true, - }) - // Ignore errors indicating that some of the containers were already stopped or removed. - if errdefs.IsNotFound(err) || errdefs.IsConflict(err) || errors.Is(err, api.ErrNoResources) { - return nil - } - - appendErr(err) - return nil - }) - return nil - case event := <-kEvents: - navigationMenu.HandleKeyEvents(globalCtx, event, project, options) - } - } - }) - - if options.Start.Watch && watcher != nil { - if err := watcher.Start(globalCtx); err != nil { - // cancel the global context to terminate background goroutines - cancel() - _ = eg.Wait() - return err - } - } - - monitor := newMonitor(s.apiClient(), project.Name) - if len(options.Start.Services) > 0 { - monitor.withServices(options.Start.Services) - } else { - // Start.AttachTo have been already curated with only the services to monitor - monitor.withServices(options.Start.AttachTo) - } - monitor.withListener(printer.HandleEvent) - - var exitCode int - if options.Start.OnExit != api.CascadeIgnore { - once := true - // detect first container to exit to trigger application shutdown - monitor.withListener(func(event api.ContainerEvent) { - if once && event.Type == api.ContainerEventExited { - if options.Start.OnExit == api.CascadeFail && event.ExitCode == 0 { - return - } - once = false - exitCode = event.ExitCode - s.events.On(newEvent(api.ResourceCompose, api.Working, api.StatusStopping, "Aborting on container exit...")) - eg.Go(func() error { - err = s.stop(context.WithoutCancel(globalCtx), project.Name, api.StopOptions{ - Services: options.Create.Services, - Project: project, - }, printer.HandleEvent) - appendErr(err) - return nil - }) - } - }) - } - - if options.Start.ExitCodeFrom != "" { - once := true - // capture exit code from first container to exit with selected service - monitor.withListener(func(event api.ContainerEvent) { - if once && event.Type == api.ContainerEventExited && event.Service == options.Start.ExitCodeFrom { - exitCode = event.ExitCode - once = false - } - }) - } - - containers, err := s.attach(globalCtx, project, printer.HandleEvent, options.Start.AttachTo) - if err != nil { - cancel() - _ = eg.Wait() - return err - } - attached := make([]string, len(containers)) - for i, ctr := range containers { - attached[i] = ctr.ID - } - - monitor.withListener(func(event api.ContainerEvent) { - if event.Type != api.ContainerEventStarted { - return - } - if slices.Contains(attached, event.ID) && !event.Restarting { - return - } - eg.Go(func() error { - ctr, err := s.apiClient().ContainerInspect(globalCtx, event.ID) - if err != nil { - appendErr(err) - return nil - } - - err = s.doLogContainer(globalCtx, options.Start.Attach, event.Source, ctr, api.LogOptions{ - Follow: true, - Since: ctr.State.StartedAt, - }) - if errdefs.IsNotImplemented(err) { - // container may be configured with logging_driver: none - // as container already started, we might miss the very first logs. But still better than none - err := s.doAttachContainer(globalCtx, event.Service, event.ID, event.Source, printer.HandleEvent) - appendErr(err) - return nil - } - appendErr(err) - return nil - }) - }) - - eg.Go(func() error { - err := monitor.Start(globalCtx) - // cancel the global context to terminate signal-handler goroutines - cancel() - appendErr(err) - return nil - }) - - // We use the parent context without cancellation as we manage sigterm to stop the stack - err = s.start(context.WithoutCancel(ctx), project.Name, options.Start, printer.HandleEvent) - if err != nil && !isTerminated.Load() { // Ignore error if the process is terminated - cancel() - _ = eg.Wait() - return err - } - - _ = eg.Wait() - err = errors.Join(errs...) - if exitCode != 0 { - errMsg := "" - if err != nil { - errMsg = err.Error() - } - return cli.StatusError{StatusCode: exitCode, Status: errMsg} - } - return err -} diff --git a/pkg/compose/viz.go b/pkg/compose/viz.go deleted file mode 100644 index cf8c4401254..00000000000 --- a/pkg/compose/viz.go +++ /dev/null @@ -1,141 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strconv" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - - "github.com/docker/compose/v5/pkg/api" -) - -// maps a service with the services it depends on -type vizGraph map[*types.ServiceConfig][]*types.ServiceConfig - -func (s *composeService) Viz(_ context.Context, project *types.Project, opts api.VizOptions) (string, error) { - graph := make(vizGraph) - for _, service := range project.Services { - graph[&service] = make([]*types.ServiceConfig, 0, len(service.DependsOn)) - for dependencyName := range service.DependsOn { - // no error should be returned since dependencyName should exist - dependency, _ := project.GetService(dependencyName) - graph[&service] = append(graph[&service], &dependency) - } - } - - // build graphviz graph - var graphBuilder strings.Builder - - // graph name - graphBuilder.WriteString("digraph ") - writeQuoted(&graphBuilder, project.Name) - graphBuilder.WriteString(" {\n") - - // graph layout - // dot is the perfect layout for this use case since graph is directed and hierarchical - graphBuilder.WriteString(opts.Indentation + "layout=dot;\n") - - addNodes(&graphBuilder, graph, project.Name, &opts) - graphBuilder.WriteByte('\n') - - addEdges(&graphBuilder, graph, &opts) - graphBuilder.WriteString("}\n") - - return graphBuilder.String(), nil -} - -// addNodes adds the corresponding graphviz representation of all the nodes in the given graph to the graphBuilder -// returns the same graphBuilder -func addNodes(graphBuilder *strings.Builder, graph vizGraph, projectName string, opts *api.VizOptions) *strings.Builder { - for serviceNode := range graph { - // write: - // "service name" [style="filled" label<service name - graphBuilder.WriteString(opts.Indentation) - writeQuoted(graphBuilder, serviceNode.Name) - graphBuilder.WriteString(" [style=\"filled\" label=<") - graphBuilder.WriteString(serviceNode.Name) - graphBuilder.WriteString("") - - if opts.IncludeNetworks && len(serviceNode.Networks) > 0 { - graphBuilder.WriteString("") - graphBuilder.WriteString(" Networks:") - for _, networkName := range serviceNode.NetworksByPriority() { - graphBuilder.WriteString(" ") - graphBuilder.WriteString(networkName) - } - graphBuilder.WriteString("") - } - - if opts.IncludePorts && len(serviceNode.Ports) > 0 { - graphBuilder.WriteString("") - graphBuilder.WriteString(" Ports:") - for _, portConfig := range serviceNode.Ports { - graphBuilder.WriteString(" ") - if portConfig.HostIP != "" { - graphBuilder.WriteString(portConfig.HostIP) - graphBuilder.WriteByte(':') - } - graphBuilder.WriteString(portConfig.Published) - graphBuilder.WriteByte(':') - graphBuilder.WriteString(strconv.Itoa(int(portConfig.Target))) - graphBuilder.WriteString(" (") - graphBuilder.WriteString(portConfig.Protocol) - graphBuilder.WriteString(", ") - graphBuilder.WriteString(portConfig.Mode) - graphBuilder.WriteString(")") - } - graphBuilder.WriteString("") - } - - if opts.IncludeImageName { - graphBuilder.WriteString("") - graphBuilder.WriteString(" Image: ") - graphBuilder.WriteString(api.GetImageNameOrDefault(*serviceNode, projectName)) - graphBuilder.WriteString("") - } - - graphBuilder.WriteString(">];\n") - } - - return graphBuilder -} - -// addEdges adds the corresponding graphviz representation of all edges in the given graph to the graphBuilder -// returns the same graphBuilder -func addEdges(graphBuilder *strings.Builder, graph vizGraph, opts *api.VizOptions) *strings.Builder { - for parent, children := range graph { - for _, child := range children { - graphBuilder.WriteString(opts.Indentation) - writeQuoted(graphBuilder, parent.Name) - graphBuilder.WriteString(" -> ") - writeQuoted(graphBuilder, child.Name) - graphBuilder.WriteString(";\n") - } - } - - return graphBuilder -} - -// writeQuoted writes "str" to builder -func writeQuoted(builder *strings.Builder, str string) { - builder.WriteByte('"') - builder.WriteString(str) - builder.WriteByte('"') -} diff --git a/pkg/compose/viz_test.go b/pkg/compose/viz_test.go deleted file mode 100644 index dad8bac36a3..00000000000 --- a/pkg/compose/viz_test.go +++ /dev/null @@ -1,216 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "strconv" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - "go.uber.org/mock/gomock" - - compose "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/mocks" -) - -func TestViz(t *testing.T) { - project := types.Project{ - Name: "viz-test", - WorkingDir: "/home", - Services: types.Services{ - "service1": { - Name: "service1", - Image: "image-for-service1", - Ports: []types.ServicePortConfig{ - { - Published: "80", - Target: 80, - Protocol: "tcp", - }, - { - Published: "53", - Target: 533, - Protocol: "udp", - }, - }, - Networks: map[string]*types.ServiceNetworkConfig{ - "internal": nil, - }, - }, - "service2": { - Name: "service2", - Image: "image-for-service2", - Ports: []types.ServicePortConfig{}, - }, - "service3": { - Name: "service3", - Image: "some-image", - DependsOn: map[string]types.ServiceDependency{ - "service2": {}, - "service1": {}, - }, - }, - "service4": { - Name: "service4", - Image: "another-image", - DependsOn: map[string]types.ServiceDependency{ - "service3": {}, - }, - Ports: []types.ServicePortConfig{ - { - Published: "8080", - Target: 80, - }, - }, - Networks: map[string]*types.ServiceNetworkConfig{ - "external": nil, - }, - }, - "With host IP": { - Name: "With host IP", - Image: "user/image-name", - DependsOn: map[string]types.ServiceDependency{ - "service1": {}, - }, - Ports: []types.ServicePortConfig{ - { - Published: "8888", - Target: 8080, - HostIP: "127.0.0.1", - }, - }, - }, - }, - Networks: types.Networks{ - "internal": types.NetworkConfig{}, - "external": types.NetworkConfig{}, - "not-used": types.NetworkConfig{}, - }, - Volumes: nil, - Secrets: nil, - Configs: nil, - Extensions: nil, - ComposeFiles: nil, - Environment: nil, - DisabledServices: nil, - Profiles: nil, - } - - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - require.NoError(t, err) - - t.Run("viz (no ports, networks or image)", func(t *testing.T) { - graphStr, err := tested.Viz(t.Context(), &project, compose.VizOptions{ - Indentation: " ", - IncludePorts: false, - IncludeImageName: false, - IncludeNetworks: false, - }) - require.NoError(t, err, "viz command failed") - - // check indentation - assert.Contains(t, graphStr, "\n ", graphStr) - assert.NotContains(t, graphStr, "\n ", graphStr) - - // check digraph name - assert.Contains(t, graphStr, "digraph \""+project.Name+"\"", graphStr) - - // check nodes - for _, service := range project.Services { - assert.Contains(t, graphStr, "\""+service.Name+"\" [style=\"filled\"", graphStr) - } - - // check node attributes - assert.NotContains(t, graphStr, "Networks", graphStr) - assert.NotContains(t, graphStr, "Image", graphStr) - assert.NotContains(t, graphStr, "Ports", graphStr) - - // check edges that SHOULD exist in the generated graph - allowedEdges := make(map[string][]string) - for name, service := range project.Services { - allowed := make([]string, 0, len(service.DependsOn)) - for depName := range service.DependsOn { - allowed = append(allowed, depName) - } - allowedEdges[name] = allowed - } - for serviceName, dependencies := range allowedEdges { - for _, dependencyName := range dependencies { - assert.Contains(t, graphStr, "\""+serviceName+"\" -> \""+dependencyName+"\"", graphStr) - } - } - - // check edges that SHOULD NOT exist in the generated graph - forbiddenEdges := make(map[string][]string) - for name, service := range project.Services { - forbiddenEdges[name] = make([]string, 0, len(project.ServiceNames())-len(service.DependsOn)) - for _, serviceName := range project.ServiceNames() { - _, edgeExists := service.DependsOn[serviceName] - if !edgeExists { - forbiddenEdges[name] = append(forbiddenEdges[name], serviceName) - } - } - } - for serviceName, forbiddenDeps := range forbiddenEdges { - for _, forbiddenDep := range forbiddenDeps { - assert.NotContains(t, graphStr, "\""+serviceName+"\" -> \""+forbiddenDep+"\"") - } - } - }) - - t.Run("viz (with ports, networks and image)", func(t *testing.T) { - graphStr, err := tested.Viz(t.Context(), &project, compose.VizOptions{ - Indentation: "\t", - IncludePorts: true, - IncludeImageName: true, - IncludeNetworks: true, - }) - require.NoError(t, err, "viz command failed") - - // check indentation - assert.Contains(t, graphStr, "\n\t", graphStr) - assert.NotContains(t, graphStr, "\n\t\t", graphStr) - - // check digraph name - assert.Contains(t, graphStr, "digraph \""+project.Name+"\"", graphStr) - - // check nodes - for _, service := range project.Services { - assert.Contains(t, graphStr, "\""+service.Name+"\" [style=\"filled\"", graphStr) - } - - // check node attributes - assert.Contains(t, graphStr, "Networks", graphStr) - assert.Contains(t, graphStr, ">internal<", graphStr) - assert.Contains(t, graphStr, ">external<", graphStr) - assert.Contains(t, graphStr, "Image", graphStr) - for _, service := range project.Services { - assert.Contains(t, graphStr, ">"+service.Image+"<", graphStr) - } - assert.Contains(t, graphStr, "Ports", graphStr) - for _, service := range project.Services { - for _, portConfig := range service.Ports { - assert.NotContains(t, graphStr, ">"+portConfig.Published+":"+strconv.Itoa(int(portConfig.Target))+"<", graphStr) - } - } - }) -} diff --git a/pkg/compose/volumes.go b/pkg/compose/volumes.go deleted file mode 100644 index 84e42b37c30..00000000000 --- a/pkg/compose/volumes.go +++ /dev/null @@ -1,83 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "slices" - - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/volume" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Volumes(ctx context.Context, project string, options api.VolumesOptions) ([]api.VolumesSummary, error) { - allContainers, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - Filters: filters.NewArgs(projectFilter(project)), - }) - if err != nil { - return nil, err - } - - var containers []container.Summary - - if len(options.Services) > 0 { - // filter service containers - for _, c := range allContainers { - if slices.Contains(options.Services, c.Labels[api.ServiceLabel]) { - containers = append(containers, c) - } - } - } else { - containers = allContainers - } - - volumesResponse, err := s.apiClient().VolumeList(ctx, volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(project)), - }) - if err != nil { - return nil, err - } - - projectVolumes := volumesResponse.Volumes - - if len(options.Services) == 0 { - return projectVolumes, nil - } - - var volumes []api.VolumesSummary - - // create a name lookup of volumes used by containers - serviceVolumes := make(map[string]bool) - - for _, container := range containers { - for _, mount := range container.Mounts { - serviceVolumes[mount.Name] = true - } - } - - // append if volumes in this project are in serviceVolumes - for _, v := range projectVolumes { - if serviceVolumes[v.Name] { - volumes = append(volumes, v) - } - } - - return volumes, nil -} diff --git a/pkg/compose/volumes_test.go b/pkg/compose/volumes_test.go deleted file mode 100644 index 85a838eaeed..00000000000 --- a/pkg/compose/volumes_test.go +++ /dev/null @@ -1,85 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/volume" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestVolumes(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - mockApi, mockCli := prepareMocks(mockCtrl) - tested := composeService{ - dockerCli: mockCli, - } - - // Create test volumes - vol1 := &volume.Volume{Name: testProject + "_vol1"} - vol2 := &volume.Volume{Name: testProject + "_vol2"} - vol3 := &volume.Volume{Name: testProject + "_vol3"} - - // Create test containers with volume mounts - c1 := container.Summary{ - Labels: map[string]string{api.ServiceLabel: "service1"}, - Mounts: []container.MountPoint{ - {Name: testProject + "_vol1"}, - {Name: testProject + "_vol2"}, - }, - } - c2 := container.Summary{ - Labels: map[string]string{api.ServiceLabel: "service2"}, - Mounts: []container.MountPoint{ - {Name: testProject + "_vol3"}, - }, - } - - args := filters.NewArgs(projectFilter(testProject)) - listOpts := container.ListOptions{Filters: args} - volumeListArgs := filters.NewArgs(projectFilter(testProject)) - volumeListOpts := volume.ListOptions{Filters: volumeListArgs} - volumeReturn := volume.ListResponse{ - Volumes: []*volume.Volume{vol1, vol2, vol3}, - } - containerReturn := []container.Summary{c1, c2} - - mockApi.EXPECT().ContainerList(t.Context(), listOpts).Times(2).Return(containerReturn, nil) - mockApi.EXPECT().VolumeList(t.Context(), volumeListOpts).Times(2).Return(volumeReturn, nil) - - // Test without service filter - should return all project volumes - volumeOptions := api.VolumesOptions{} - volumes, err := tested.Volumes(t.Context(), testProject, volumeOptions) - expected := []api.VolumesSummary{vol1, vol2, vol3} - assert.NilError(t, err) - assert.DeepEqual(t, volumes, expected) - - // Test with service filter - should only return volumes used by service1 - volumeOptions = api.VolumesOptions{Services: []string{"service1"}} - volumes, err = tested.Volumes(t.Context(), testProject, volumeOptions) - expected = []api.VolumesSummary{vol1, vol2} - assert.NilError(t, err) - assert.DeepEqual(t, volumes, expected) -} diff --git a/pkg/compose/wait.go b/pkg/compose/wait.go deleted file mode 100644 index 30413cb533e..00000000000 --- a/pkg/compose/wait.go +++ /dev/null @@ -1,67 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Wait(ctx context.Context, projectName string, options api.WaitOptions) (int64, error) { - containers, err := s.getContainers(ctx, projectName, oneOffInclude, false, options.Services...) - if err != nil { - return 0, err - } - if len(containers) == 0 { - return 0, fmt.Errorf("no containers for project %q", projectName) - } - - eg, waitCtx := errgroup.WithContext(ctx) - var statusCode int64 - for _, ctr := range containers { - eg.Go(func() error { - var err error - resultC, errC := s.apiClient().ContainerWait(waitCtx, ctr.ID, "") - - select { - case result := <-resultC: - _, _ = fmt.Fprintf(s.stdout(), "container %q exited with status code %d\n", ctr.ID, result.StatusCode) - statusCode = result.StatusCode - case err = <-errC: - } - - return err - }) - } - - err = eg.Wait() - if err != nil { - return 42, err // Ignore abort flag in case of error in wait - } - - if options.DownProjectOnContainerExit { - return statusCode, s.Down(ctx, projectName, api.DownOptions{ - RemoveOrphans: true, - }) - } - - return statusCode, err -} diff --git a/pkg/compose/watch.go b/pkg/compose/watch.go deleted file mode 100644 index 77575bcdca2..00000000000 --- a/pkg/compose/watch.go +++ /dev/null @@ -1,858 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "io" - "io/fs" - "os" - "path" - "path/filepath" - "slices" - "strconv" - "strings" - gsync "sync" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/compose-spec/compose-go/v2/utils" - ccli "github.com/docker/cli/cli/command/container" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/go-viper/mapstructure/v2" - "github.com/moby/buildkit/util/progress/progressui" - "github.com/sirupsen/logrus" - "golang.org/x/sync/errgroup" - - pathutil "github.com/docker/compose/v5/internal/paths" - "github.com/docker/compose/v5/internal/sync" - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" - cutils "github.com/docker/compose/v5/pkg/utils" - "github.com/docker/compose/v5/pkg/watch" -) - -type WatchFunc func(ctx context.Context, project *types.Project, options api.WatchOptions) (func() error, error) - -type Watcher struct { - project *types.Project - options api.WatchOptions - watchFn WatchFunc - stopFn func() - errCh chan error -} - -func NewWatcher(project *types.Project, options api.UpOptions, w WatchFunc, consumer api.LogConsumer) (*Watcher, error) { - for i := range project.Services { - service := project.Services[i] - - if service.Develop != nil && service.Develop.Watch != nil { - build := options.Create.Build - return &Watcher{ - project: project, - options: api.WatchOptions{ - LogTo: consumer, - Build: build, - }, - watchFn: w, - errCh: make(chan error), - }, nil - } - } - // none of the services is eligible to watch - return nil, fmt.Errorf("none of the selected services is configured for watch, see https://docs.docker.com/compose/how-tos/file-watch/") -} - -// ensure state changes are atomic -var mx gsync.Mutex - -func (w *Watcher) Start(ctx context.Context) error { - mx.Lock() - defer mx.Unlock() - ctx, cancelFunc := context.WithCancel(ctx) - w.stopFn = cancelFunc - wait, err := w.watchFn(ctx, w.project, w.options) - if err != nil { - go func() { - w.errCh <- err - }() - return err - } - go func() { - w.errCh <- wait() - }() - return nil -} - -func (w *Watcher) Stop() error { - mx.Lock() - defer mx.Unlock() - if w.stopFn == nil { - return nil - } - w.stopFn() - w.stopFn = nil - err := <-w.errCh - return err -} - -// getSyncImplementation returns an appropriate sync implementation for the -// project. -// -// Currently, an implementation that batches files and transfers them using -// the Moby `Untar` API. -func (s *composeService) getSyncImplementation(project *types.Project) (sync.Syncer, error) { - var useTar bool - if useTarEnv, ok := os.LookupEnv("COMPOSE_EXPERIMENTAL_WATCH_TAR"); ok { - useTar, _ = strconv.ParseBool(useTarEnv) - } else { - useTar = true - } - if !useTar { - return nil, errors.New("no available sync implementation") - } - - return sync.NewTar(project.Name, tarDockerClient{s: s}), nil -} - -func (s *composeService) Watch(ctx context.Context, project *types.Project, options api.WatchOptions) error { - wait, err := s.watch(ctx, project, options) - if err != nil { - return err - } - return wait() -} - -type watchRule struct { - types.Trigger - include watch.PathMatcher - ignore watch.PathMatcher - service string -} - -func (r watchRule) Matches(event watch.FileEvent) *sync.PathMapping { - hostPath := string(event) - if !pathutil.IsChild(r.Path, hostPath) { - return nil - } - included, err := r.include.Matches(hostPath) - if err != nil { - logrus.Warnf("error include matching %q: %v", hostPath, err) - return nil - } - if !included { - logrus.Debugf("%s is not matching include pattern", hostPath) - return nil - } - isIgnored, err := r.ignore.Matches(hostPath) - if err != nil { - logrus.Warnf("error ignore matching %q: %v", hostPath, err) - return nil - } - - if isIgnored { - logrus.Debugf("%s is matching ignore pattern", hostPath) - return nil - } - - var containerPath string - if r.Target != "" { - rel, err := filepath.Rel(r.Path, hostPath) - if err != nil { - logrus.Warnf("error making %s relative to %s: %v", hostPath, r.Path, err) - return nil - } - // always use Unix-style paths for inside the container - containerPath = path.Join(r.Target, filepath.ToSlash(rel)) - } - return &sync.PathMapping{ - HostPath: hostPath, - ContainerPath: containerPath, - } -} - -func (s *composeService) watch(ctx context.Context, project *types.Project, options api.WatchOptions) (func() error, error) { //nolint: gocyclo - var err error - if project, err = project.WithSelectedServices(options.Services); err != nil { - return nil, err - } - syncer, err := s.getSyncImplementation(project) - if err != nil { - return nil, err - } - eg, ctx := errgroup.WithContext(ctx) - - var ( - rules []watchRule - paths []string - ) - for serviceName, service := range project.Services { - config, err := loadDevelopmentConfig(service, project) - if err != nil { - return nil, err - } - - if service.Develop != nil { - config = service.Develop - } - - if config == nil { - continue - } - - for _, trigger := range config.Watch { - if trigger.Action == types.WatchActionRebuild { - if service.Build == nil { - return nil, fmt.Errorf("can't watch service %q with action %s without a build context", service.Name, types.WatchActionRebuild) - } - if options.Build == nil { - return nil, fmt.Errorf("--no-build is incompatible with watch action %s in service %s", types.WatchActionRebuild, service.Name) - } - // set the service to always be built - watch triggers `Up()` when it receives a rebuild event - service.PullPolicy = types.PullPolicyBuild - project.Services[serviceName] = service - } - } - - for _, trigger := range config.Watch { - if isSync(trigger) && checkIfPathAlreadyBindMounted(trigger.Path, service.Volumes) { - logrus.Warnf("path '%s' also declared by a bind mount volume, this path won't be monitored!\n", trigger.Path) - continue - } else { - shouldInitialSync := trigger.InitialSync - - // Check legacy extension attribute for backward compatibility - if !shouldInitialSync { - var legacyInitialSync bool - success, err := trigger.Extensions.Get("x-initialSync", &legacyInitialSync) - if err == nil && success && legacyInitialSync { - shouldInitialSync = true - logrus.Warnf("x-initialSync is DEPRECATED, please use the official `initial_sync` attribute\n") - } - } - - if shouldInitialSync && isSync(trigger) { - // Need to check initial files are in container that are meant to be synced from watch action - err := s.initialSync(ctx, project, service, trigger, syncer) - if err != nil { - return nil, err - } - } - } - paths = append(paths, trigger.Path) - } - - serviceWatchRules, err := getWatchRules(config, service) - if err != nil { - return nil, err - } - rules = append(rules, serviceWatchRules...) - } - - if len(paths) == 0 { - return nil, fmt.Errorf("none of the selected services is configured for watch, consider setting a 'develop' section") - } - - watcher, err := watch.NewWatcher(paths) - if err != nil { - return nil, err - } - - err = watcher.Start() - if err != nil { - return nil, err - } - - eg.Go(func() error { - return s.watchEvents(ctx, project, options, watcher, syncer, rules) - }) - options.LogTo.Log(api.WatchLogger, "Watch enabled") - - return func() error { - err := eg.Wait() - if werr := watcher.Close(); werr != nil { - logrus.Debugf("Error closing Watcher: %v", werr) - } - return err - }, nil -} - -func getWatchRules(config *types.DevelopConfig, service types.ServiceConfig) ([]watchRule, error) { - var rules []watchRule - - dockerIgnores, err := watch.LoadDockerIgnore(service.Build) - if err != nil { - return nil, err - } - - // add a hardcoded set of ignores on top of what came from .dockerignore - // some of this should likely be configurable (e.g. there could be cases - // where you want `.git` to be synced) but this is suitable for now - dotGitIgnore, err := watch.NewDockerPatternMatcher("/", []string{".git/"}) - if err != nil { - return nil, err - } - - for _, trigger := range config.Watch { - ignore, err := watch.NewDockerPatternMatcher(trigger.Path, trigger.Ignore) - if err != nil { - return nil, err - } - - var include watch.PathMatcher - if len(trigger.Include) == 0 { - include = watch.AnyMatcher{} - } else { - include, err = watch.NewDockerPatternMatcher(trigger.Path, trigger.Include) - if err != nil { - return nil, err - } - } - - rules = append(rules, watchRule{ - Trigger: trigger, - include: include, - ignore: watch.NewCompositeMatcher( - dockerIgnores, - watch.EphemeralPathMatcher(), - dotGitIgnore, - ignore, - ), - service: service.Name, - }) - } - return rules, nil -} - -func isSync(trigger types.Trigger) bool { - return trigger.Action == types.WatchActionSync || trigger.Action == types.WatchActionSyncRestart -} - -func (s *composeService) watchEvents(ctx context.Context, project *types.Project, options api.WatchOptions, watcher watch.Notify, syncer sync.Syncer, rules []watchRule) error { - ctx, cancel := context.WithCancel(ctx) - defer cancel() - - // debounce and group filesystem events so that we capture IDE saving many files as one "batch" event - batchEvents := watch.BatchDebounceEvents(ctx, s.clock, watcher.Events()) - - for { - select { - case <-ctx.Done(): - options.LogTo.Log(api.WatchLogger, "Watch disabled") - // Ensure watcher is closed to release resources - _ = watcher.Close() - return nil - case err, open := <-watcher.Errors(): - if err != nil { - options.LogTo.Err(api.WatchLogger, "Watch disabled with errors: "+err.Error()) - } - if open { - continue - } - _ = watcher.Close() - return err - case batch, ok := <-batchEvents: - if !ok { - options.LogTo.Log(api.WatchLogger, "Watch disabled") - _ = watcher.Close() - return nil - } - if len(batch) > 1000 { - logrus.Warnf("Very large batch of file changes detected: %d files. This may impact performance.", len(batch)) - options.LogTo.Log(api.WatchLogger, "Large batch of file changes detected. If you just switched branches, this is expected.") - } - start := time.Now() - logrus.Debugf("batch start: count[%d]", len(batch)) - err := s.handleWatchBatch(ctx, project, options, batch, rules, syncer) - if err != nil { - logrus.Warnf("Error handling changed files: %v", err) - // If context was canceled, exit immediately - if ctx.Err() != nil { - _ = watcher.Close() - return ctx.Err() - } - } - logrus.Debugf("batch complete: duration[%s] count[%d]", time.Since(start), len(batch)) - } - } -} - -func loadDevelopmentConfig(service types.ServiceConfig, project *types.Project) (*types.DevelopConfig, error) { - var config types.DevelopConfig - y, ok := service.Extensions["x-develop"] - if !ok { - return nil, nil - } - logrus.Warnf("x-develop is DEPRECATED, please use the official `develop` attribute") - err := mapstructure.Decode(y, &config) - if err != nil { - return nil, err - } - baseDir, err := filepath.EvalSymlinks(project.WorkingDir) - if err != nil { - return nil, fmt.Errorf("resolving symlink for %q: %w", project.WorkingDir, err) - } - - for i, trigger := range config.Watch { - if !filepath.IsAbs(trigger.Path) { - trigger.Path = filepath.Join(baseDir, trigger.Path) - } - if p, err := filepath.EvalSymlinks(trigger.Path); err == nil { - // this might fail because the path doesn't exist, etc. - trigger.Path = p - } - trigger.Path = filepath.Clean(trigger.Path) - if trigger.Path == "" { - return nil, errors.New("watch rules MUST define a path") - } - - if trigger.Action == types.WatchActionRebuild && service.Build == nil { - return nil, fmt.Errorf("service %s doesn't have a build section, can't apply %s on watch", types.WatchActionRebuild, service.Name) - } - if trigger.Action == types.WatchActionSyncExec && len(trigger.Exec.Command) == 0 { - return nil, fmt.Errorf("can't watch with action %q on service %s without a command", types.WatchActionSyncExec, service.Name) - } - - config.Watch[i] = trigger - } - return &config, nil -} - -func checkIfPathAlreadyBindMounted(watchPath string, volumes []types.ServiceVolumeConfig) bool { - for _, volume := range volumes { - if volume.Bind != nil { - relPath, err := filepath.Rel(volume.Source, watchPath) - if err == nil && !strings.HasPrefix(relPath, "..") { - return true - } - } - } - return false -} - -type tarDockerClient struct { - s *composeService -} - -func (t tarDockerClient) ContainersForService(ctx context.Context, projectName string, serviceName string) ([]container.Summary, error) { - containers, err := t.s.getContainers(ctx, projectName, oneOffExclude, true, serviceName) - if err != nil { - return nil, err - } - return containers, nil -} - -func (t tarDockerClient) Exec(ctx context.Context, containerID string, cmd []string, in io.Reader) error { - execCfg := container.ExecOptions{ - Cmd: cmd, - AttachStdout: false, - AttachStderr: true, - AttachStdin: in != nil, - Tty: false, - } - execCreateResp, err := t.s.apiClient().ContainerExecCreate(ctx, containerID, execCfg) - if err != nil { - return err - } - - startCheck := container.ExecStartOptions{Tty: false, Detach: false} - conn, err := t.s.apiClient().ContainerExecAttach(ctx, execCreateResp.ID, startCheck) - if err != nil { - return err - } - defer conn.Close() - - var eg errgroup.Group - if in != nil { - eg.Go(func() error { - defer func() { - _ = conn.CloseWrite() - }() - _, err := io.Copy(conn.Conn, in) - return err - }) - } - eg.Go(func() error { - _, err := io.Copy(t.s.stdout(), conn.Reader) - return err - }) - - err = t.s.apiClient().ContainerExecStart(ctx, execCreateResp.ID, startCheck) - if err != nil { - return err - } - - // although the errgroup is not tied directly to the context, the operations - // in it are reading/writing to the connection, which is tied to the context, - // so they won't block indefinitely - if err := eg.Wait(); err != nil { - return err - } - - execResult, err := t.s.apiClient().ContainerExecInspect(ctx, execCreateResp.ID) - if err != nil { - return err - } - if execResult.Running { - return errors.New("process still running") - } - if execResult.ExitCode != 0 { - return fmt.Errorf("exit code %d", execResult.ExitCode) - } - return nil -} - -func (t tarDockerClient) Untar(ctx context.Context, id string, archive io.ReadCloser) error { - return t.s.apiClient().CopyToContainer(ctx, id, "/", archive, container.CopyToContainerOptions{ - CopyUIDGID: true, - }) -} - -//nolint:gocyclo -func (s *composeService) handleWatchBatch(ctx context.Context, project *types.Project, options api.WatchOptions, batch []watch.FileEvent, rules []watchRule, syncer sync.Syncer) error { - var ( - restart = map[string]bool{} - syncfiles = map[string][]*sync.PathMapping{} - exec = map[string][]int{} - rebuild = map[string]bool{} - ) - for _, event := range batch { - for i, rule := range rules { - mapping := rule.Matches(event) - if mapping == nil { - continue - } - - switch rule.Action { - case types.WatchActionRebuild: - rebuild[rule.service] = true - case types.WatchActionSync: - syncfiles[rule.service] = append(syncfiles[rule.service], mapping) - case types.WatchActionRestart: - restart[rule.service] = true - case types.WatchActionSyncRestart: - syncfiles[rule.service] = append(syncfiles[rule.service], mapping) - restart[rule.service] = true - case types.WatchActionSyncExec: - syncfiles[rule.service] = append(syncfiles[rule.service], mapping) - // We want to run exec hooks only once after syncfiles if multiple file events match - // as we can't compare ServiceHook to sort and compact a slice, collect rule indexes - exec[rule.service] = append(exec[rule.service], i) - } - } - } - - logrus.Debugf("watch actions: rebuild %d sync %d restart %d", len(rebuild), len(syncfiles), len(restart)) - - if len(rebuild) > 0 { - err := s.rebuild(ctx, project, utils.MapKeys(rebuild), options) - if err != nil { - return err - } - } - - for serviceName, pathMappings := range syncfiles { - writeWatchSyncMessage(options.LogTo, serviceName, pathMappings) - err := syncer.Sync(ctx, serviceName, pathMappings) - if err != nil { - return err - } - } - if len(restart) > 0 { - services := utils.MapKeys(restart) - err := s.restart(ctx, project.Name, api.RestartOptions{ - Services: services, - Project: project, - NoDeps: false, - }) - if err != nil { - return err - } - options.LogTo.Log( - api.WatchLogger, - fmt.Sprintf("service(s) %q restarted", services)) - } - - eg, ctx := errgroup.WithContext(ctx) - for service, rulesToExec := range exec { - slices.Sort(rulesToExec) - for _, i := range slices.Compact(rulesToExec) { - err := s.exec(ctx, project, service, rules[i].Exec, eg) - if err != nil { - return err - } - } - } - return eg.Wait() -} - -func (s *composeService) exec(ctx context.Context, project *types.Project, serviceName string, x types.ServiceHook, eg *errgroup.Group) error { - containers, err := s.getContainers(ctx, project.Name, oneOffExclude, false, serviceName) - if err != nil { - return err - } - for _, c := range containers { - eg.Go(func() error { - exec := ccli.NewExecOptions() - exec.User = x.User - exec.Privileged = x.Privileged - exec.Command = x.Command - exec.Workdir = x.WorkingDir - exec.DetachKeys = s.configFile().DetachKeys - for _, v := range x.Environment.ToMapping().Values() { - err := exec.Env.Set(v) - if err != nil { - return err - } - } - return ccli.RunExec(ctx, s.dockerCli, c.ID, exec) - }) - } - return nil -} - -func (s *composeService) rebuild(ctx context.Context, project *types.Project, services []string, options api.WatchOptions) error { - options.LogTo.Log(api.WatchLogger, fmt.Sprintf("Rebuilding service(s) %q after changes were detected...", services)) - // restrict the build to ONLY this service, not any of its dependencies - options.Build.Services = services - options.Build.Progress = string(progressui.PlainMode) - options.Build.Out = cutils.GetWriter(func(line string) { - options.LogTo.Log(api.WatchLogger, line) - }) - - var ( - imageNameToIdMap map[string]string - err error - ) - err = tracing.SpanWrapFunc("project/build", tracing.ProjectOptions(ctx, project), - func(ctx context.Context) error { - imageNameToIdMap, err = s.build(ctx, project, *options.Build, nil) - return err - })(ctx) - if err != nil { - options.LogTo.Log(api.WatchLogger, fmt.Sprintf("Build failed. Error: %v", err)) - return err - } - - if options.Prune { - s.pruneDanglingImagesOnRebuild(ctx, project.Name, imageNameToIdMap) - } - - options.LogTo.Log(api.WatchLogger, fmt.Sprintf("service(s) %q successfully built", services)) - - err = s.create(ctx, project, api.CreateOptions{ - Services: services, - Inherit: true, - Recreate: api.RecreateForce, - }) - if err != nil { - options.LogTo.Log(api.WatchLogger, fmt.Sprintf("Failed to recreate services after update. Error: %v", err)) - return err - } - - p, err := project.WithSelectedServices(services, types.IncludeDependents) - if err != nil { - return err - } - err = s.start(ctx, project.Name, api.StartOptions{ - Project: p, - Services: services, - AttachTo: services, - }, nil) - if err != nil { - options.LogTo.Log(api.WatchLogger, fmt.Sprintf("Application failed to start after update. Error: %v", err)) - } - return nil -} - -// writeWatchSyncMessage prints out a message about the sync for the changed paths. -func writeWatchSyncMessage(log api.LogConsumer, serviceName string, pathMappings []*sync.PathMapping) { - if logrus.IsLevelEnabled(logrus.DebugLevel) { - hostPathsToSync := make([]string, len(pathMappings)) - for i := range pathMappings { - hostPathsToSync[i] = pathMappings[i].HostPath - } - log.Log( - api.WatchLogger, - fmt.Sprintf( - "Syncing service %q after changes were detected: %s", - serviceName, - strings.Join(hostPathsToSync, ", "), - ), - ) - } else { - log.Log( - api.WatchLogger, - fmt.Sprintf("Syncing service %q after %d changes were detected", serviceName, len(pathMappings)), - ) - } -} - -func (s *composeService) pruneDanglingImagesOnRebuild(ctx context.Context, projectName string, imageNameToIdMap map[string]string) { - images, err := s.apiClient().ImageList(ctx, image.ListOptions{ - Filters: filters.NewArgs( - filters.Arg("dangling", "true"), - filters.Arg("label", api.ProjectLabel+"="+projectName), - ), - }) - if err != nil { - logrus.Debugf("Failed to list images: %v", err) - return - } - - for _, img := range images { - if _, ok := imageNameToIdMap[img.ID]; !ok { - _, err := s.apiClient().ImageRemove(ctx, img.ID, image.RemoveOptions{}) - if err != nil { - logrus.Debugf("Failed to remove image %s: %v", img.ID, err) - } - } - } -} - -// Walks develop.watch.path and checks which files should be copied inside the container -// ignores develop.watch.ignore, Dockerfile, compose files, bind mounted paths and .git -func (s *composeService) initialSync(ctx context.Context, project *types.Project, service types.ServiceConfig, trigger types.Trigger, syncer sync.Syncer) error { - dockerIgnores, err := watch.LoadDockerIgnore(service.Build) - if err != nil { - return err - } - - dotGitIgnore, err := watch.NewDockerPatternMatcher("/", []string{".git/"}) - if err != nil { - return err - } - - triggerIgnore, err := watch.NewDockerPatternMatcher(trigger.Path, trigger.Ignore) - if err != nil { - return err - } - // FIXME .dockerignore - ignoreInitialSync := watch.NewCompositeMatcher( - dockerIgnores, - watch.EphemeralPathMatcher(), - dotGitIgnore, - triggerIgnore) - - pathsToCopy, err := s.initialSyncFiles(ctx, project, service, trigger, ignoreInitialSync) - if err != nil { - return err - } - - return syncer.Sync(ctx, service.Name, pathsToCopy) -} - -// Syncs files from develop.watch.path if thy have been modified after the image has been created -// -//nolint:gocyclo -func (s *composeService) initialSyncFiles(ctx context.Context, project *types.Project, service types.ServiceConfig, trigger types.Trigger, ignore watch.PathMatcher) ([]*sync.PathMapping, error) { - fi, err := os.Stat(trigger.Path) - if err != nil { - return nil, err - } - timeImageCreated, err := s.imageCreatedTime(ctx, project, service.Name) - if err != nil { - return nil, err - } - var pathsToCopy []*sync.PathMapping - switch mode := fi.Mode(); { - case mode.IsDir(): - // process directory - err = filepath.WalkDir(trigger.Path, func(path string, d fs.DirEntry, err error) error { - if err != nil { - // handle possible path err, just in case... - return err - } - if trigger.Path == path { - // walk starts at the root directory - return nil - } - if shouldIgnore(filepath.Base(path), ignore) || checkIfPathAlreadyBindMounted(path, service.Volumes) { - // By definition sync ignores bind mounted paths - if d.IsDir() { - // skip folder - return fs.SkipDir - } - return nil // skip file - } - info, err := d.Info() - if err != nil { - return err - } - if !d.IsDir() { - if info.ModTime().Before(timeImageCreated) { - // skip file if it was modified before image creation - return nil - } - rel, err := filepath.Rel(trigger.Path, path) - if err != nil { - return err - } - // only copy files (and not full directories) - pathsToCopy = append(pathsToCopy, &sync.PathMapping{ - HostPath: path, - ContainerPath: filepath.Join(trigger.Target, rel), - }) - } - return nil - }) - case mode.IsRegular(): - // process file - if fi.ModTime().After(timeImageCreated) && !shouldIgnore(filepath.Base(trigger.Path), ignore) && !checkIfPathAlreadyBindMounted(trigger.Path, service.Volumes) { - pathsToCopy = append(pathsToCopy, &sync.PathMapping{ - HostPath: trigger.Path, - ContainerPath: trigger.Target, - }) - } - } - return pathsToCopy, err -} - -func shouldIgnore(name string, ignore watch.PathMatcher) bool { - shouldIgnore, _ := ignore.Matches(name) - // ignore files that match any ignore pattern - return shouldIgnore -} - -// gets the image creation time for a service -func (s *composeService) imageCreatedTime(ctx context.Context, project *types.Project, serviceName string) (time.Time, error) { - containers, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - All: true, - Filters: filters.NewArgs( - filters.Arg("label", fmt.Sprintf("%s=%s", api.ProjectLabel, project.Name)), - filters.Arg("label", fmt.Sprintf("%s=%s", api.ServiceLabel, serviceName))), - }) - if err != nil { - return time.Now(), err - } - if len(containers) == 0 { - return time.Now(), fmt.Errorf("could not get created time for service's image") - } - - img, err := s.apiClient().ImageInspect(ctx, containers[0].ImageID) - if err != nil { - return time.Now(), err - } - // Need to get the oldest one? - timeCreated, err := time.Parse(time.RFC3339Nano, img.Created) - if err != nil { - return time.Now(), err - } - return timeCreated, nil -} diff --git a/pkg/compose/watch_test.go b/pkg/compose/watch_test.go deleted file mode 100644 index 956ca4e27e7..00000000000 --- a/pkg/compose/watch_test.go +++ /dev/null @@ -1,188 +0,0 @@ -/* - - Copyright 2020 Docker Compose CLI authors - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - "testing" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/streams" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/jonboulle/clockwork" - "github.com/stretchr/testify/require" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/internal/sync" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/mocks" - "github.com/docker/compose/v5/pkg/watch" -) - -type testWatcher struct { - events chan watch.FileEvent - errors chan error -} - -func (t testWatcher) Start() error { - return nil -} - -func (t testWatcher) Close() error { - return nil -} - -func (t testWatcher) Events() chan watch.FileEvent { - return t.events -} - -func (t testWatcher) Errors() chan error { - return t.errors -} - -type stdLogger struct{} - -func (s stdLogger) Log(containerName, message string) { - fmt.Printf("%s: %s\n", containerName, message) -} - -func (s stdLogger) Err(containerName, message string) { - fmt.Fprintf(os.Stderr, "%s: %s\n", containerName, message) -} - -func (s stdLogger) Status(containerName, msg string) { - fmt.Printf("%s: %s\n", containerName, msg) -} - -func TestWatch_Sync(t *testing.T) { - mockCtrl := gomock.NewController(t) - cli := mocks.NewMockCli(mockCtrl) - cli.EXPECT().Err().Return(streams.NewOut(os.Stderr)).AnyTimes() - apiClient := mocks.NewMockAPIClient(mockCtrl) - apiClient.EXPECT().ContainerList(gomock.Any(), gomock.Any()).Return([]container.Summary{ - testContainer("test", "123", false), - }, nil).AnyTimes() - // we expect the image to be pruned - apiClient.EXPECT().ImageList(gomock.Any(), image.ListOptions{ - Filters: filters.NewArgs( - filters.Arg("dangling", "true"), - filters.Arg("label", api.ProjectLabel+"=myProjectName"), - ), - }).Return([]image.Summary{ - {ID: "123"}, - {ID: "456"}, - }, nil).Times(1) - apiClient.EXPECT().ImageRemove(gomock.Any(), "123", image.RemoveOptions{}).Times(1) - apiClient.EXPECT().ImageRemove(gomock.Any(), "456", image.RemoveOptions{}).Times(1) - // - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - ctx, cancelFunc := context.WithCancel(t.Context()) - t.Cleanup(cancelFunc) - - proj := types.Project{ - Name: "myProjectName", - Services: types.Services{ - "test": { - Name: "test", - }, - }, - } - - watcher := testWatcher{ - events: make(chan watch.FileEvent), - errors: make(chan error), - } - - syncer := newFakeSyncer() - clock := clockwork.NewFakeClock() - go func() { - service := composeService{ - dockerCli: cli, - clock: clock, - } - rules, err := getWatchRules(&types.DevelopConfig{ - Watch: []types.Trigger{ - { - Path: "/sync", - Action: "sync", - Target: "/work", - Ignore: []string{"ignore"}, - }, - { - Path: "/rebuild", - Action: "rebuild", - }, - }, - }, types.ServiceConfig{Name: "test"}) - assert.NilError(t, err) - - err = service.watchEvents(ctx, &proj, api.WatchOptions{ - Build: &api.BuildOptions{}, - LogTo: stdLogger{}, - Prune: true, - }, watcher, syncer, rules) - assert.NilError(t, err) - }() - - watcher.Events() <- watch.NewFileEvent("/sync/changed") - watcher.Events() <- watch.NewFileEvent("/sync/changed/sub") - err := clock.BlockUntilContext(ctx, 3) - assert.NilError(t, err) - clock.Advance(watch.QuietPeriod) - select { - case actual := <-syncer.synced: - require.ElementsMatch(t, []*sync.PathMapping{ - {HostPath: "/sync/changed", ContainerPath: "/work/changed"}, - {HostPath: "/sync/changed/sub", ContainerPath: "/work/changed/sub"}, - }, actual) - case <-time.After(100 * time.Millisecond): - t.Error("timeout") - } - - watcher.Events() <- watch.NewFileEvent("/rebuild") - watcher.Events() <- watch.NewFileEvent("/sync/changed") - err = clock.BlockUntilContext(ctx, 4) - assert.NilError(t, err) - clock.Advance(watch.QuietPeriod) - select { - case batch := <-syncer.synced: - t.Fatalf("received unexpected events: %v", batch) - case <-time.After(100 * time.Millisecond): - // expected - } - // TODO: there's not a great way to assert that the rebuild attempt happened -} - -type fakeSyncer struct { - synced chan []*sync.PathMapping -} - -func newFakeSyncer() *fakeSyncer { - return &fakeSyncer{ - synced: make(chan []*sync.PathMapping), - } -} - -func (f *fakeSyncer) Sync(ctx context.Context, service string, paths []*sync.PathMapping) error { - f.synced <- paths - return nil -} diff --git a/pkg/dryrun/dryrunclient.go b/pkg/dryrun/dryrunclient.go deleted file mode 100644 index 6755ae96b6a..00000000000 --- a/pkg/dryrun/dryrunclient.go +++ /dev/null @@ -1,692 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package dryrun - -import ( - "bytes" - "context" - "crypto/rand" - "encoding/json" - "errors" - "fmt" - "io" - "net" - "net/http" - "runtime" - "strings" - "sync" - - "github.com/docker/buildx/builder" - "github.com/docker/buildx/util/imagetools" - "github.com/docker/cli/cli/command" - moby "github.com/docker/docker/api/types" - "github.com/docker/docker/api/types/build" - "github.com/docker/docker/api/types/checkpoint" - containerType "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/events" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/api/types/registry" - "github.com/docker/docker/api/types/swarm" - "github.com/docker/docker/api/types/system" - "github.com/docker/docker/api/types/volume" - "github.com/docker/docker/client" - "github.com/docker/docker/pkg/jsonmessage" - specs "github.com/opencontainers/image-spec/specs-go/v1" -) - -var _ client.APIClient = &DryRunClient{} - -// DryRunClient implements APIClient by delegating to implementation functions. This allows lazy init and per-method overrides -type DryRunClient struct { - apiClient client.APIClient - containers []containerType.Summary - execs sync.Map - resolver *imagetools.Resolver -} - -type execDetails struct { - container string - command []string -} - -// NewDryRunClient produces a DryRunClient -func NewDryRunClient(apiClient client.APIClient, cli command.Cli) (*DryRunClient, error) { - b, err := builder.New(cli, builder.WithSkippedValidation()) - if err != nil { - return nil, err - } - configFile, err := b.ImageOpt() - if err != nil { - return nil, err - } - return &DryRunClient{ - apiClient: apiClient, - containers: []containerType.Summary{}, - execs: sync.Map{}, - resolver: imagetools.New(configFile), - }, nil -} - -func getCallingFunction() string { - pc, _, _, _ := runtime.Caller(2) - fullName := runtime.FuncForPC(pc).Name() - return fullName[strings.LastIndex(fullName, ".")+1:] -} - -// All methods and functions which need to be overridden for dry run. - -func (d *DryRunClient) ContainerAttach(ctx context.Context, container string, options containerType.AttachOptions) (moby.HijackedResponse, error) { - return moby.HijackedResponse{}, errors.New("interactive run is not supported in dry-run mode") -} - -func (d *DryRunClient) ContainerCreate(ctx context.Context, config *containerType.Config, hostConfig *containerType.HostConfig, - networkingConfig *network.NetworkingConfig, platform *specs.Platform, containerName string, -) (containerType.CreateResponse, error) { - d.containers = append(d.containers, containerType.Summary{ - ID: containerName, - Names: []string{containerName}, - Labels: config.Labels, - HostConfig: struct { - NetworkMode string `json:",omitempty"` - Annotations map[string]string `json:",omitempty"` - }{}, - }) - return containerType.CreateResponse{ID: containerName}, nil -} - -func (d *DryRunClient) ContainerInspect(ctx context.Context, container string) (containerType.InspectResponse, error) { - containerJSON, err := d.apiClient.ContainerInspect(ctx, container) - if err != nil { - id := "dryRunId" - for _, c := range d.containers { - if c.ID == container { - id = container - } - } - return containerType.InspectResponse{ - ContainerJSONBase: &containerType.ContainerJSONBase{ - ID: id, - Name: container, - State: &containerType.State{ - Status: containerType.StateRunning, // needed for --wait option - Health: &containerType.Health{ - Status: containerType.Healthy, // needed for healthcheck control - }, - }, - }, - Mounts: nil, - Config: &containerType.Config{}, - NetworkSettings: &containerType.NetworkSettings{}, - }, nil - } - return containerJSON, err -} - -func (d *DryRunClient) ContainerKill(ctx context.Context, container, signal string) error { - return nil -} - -func (d *DryRunClient) ContainerList(ctx context.Context, options containerType.ListOptions) ([]containerType.Summary, error) { - caller := getCallingFunction() - switch caller { - case "start": - return d.containers, nil - case "getContainers": - if len(d.containers) == 0 { - var err error - d.containers, err = d.apiClient.ContainerList(ctx, options) - return d.containers, err - } - } - return d.apiClient.ContainerList(ctx, options) -} - -func (d *DryRunClient) ContainerPause(ctx context.Context, container string) error { - return nil -} - -func (d *DryRunClient) ContainerRemove(ctx context.Context, container string, options containerType.RemoveOptions) error { - return nil -} - -func (d *DryRunClient) ContainerRename(ctx context.Context, container, newContainerName string) error { - return nil -} - -func (d *DryRunClient) ContainerRestart(ctx context.Context, container string, options containerType.StopOptions) error { - return nil -} - -func (d *DryRunClient) ContainerStart(ctx context.Context, container string, options containerType.StartOptions) error { - return nil -} - -func (d *DryRunClient) ContainerStop(ctx context.Context, container string, options containerType.StopOptions) error { - return nil -} - -func (d *DryRunClient) ContainerUnpause(ctx context.Context, container string) error { - return nil -} - -func (d *DryRunClient) CopyFromContainer(ctx context.Context, container, srcPath string) (io.ReadCloser, containerType.PathStat, error) { - rc := io.NopCloser(strings.NewReader("")) - if _, err := d.ContainerStatPath(ctx, container, srcPath); err != nil { - return rc, containerType.PathStat{}, fmt.Errorf("could not find the file %s in container %s", srcPath, container) - } - return rc, containerType.PathStat{}, nil -} - -func (d *DryRunClient) CopyToContainer(ctx context.Context, container, path string, content io.Reader, options containerType.CopyToContainerOptions) error { - return nil -} - -func (d *DryRunClient) ImageBuild(ctx context.Context, reader io.Reader, options build.ImageBuildOptions) (build.ImageBuildResponse, error) { - rc := io.NopCloser(bytes.NewReader(nil)) - - return build.ImageBuildResponse{ - Body: rc, - }, nil -} - -func (d *DryRunClient) ImageInspect(ctx context.Context, imageName string, options ...client.ImageInspectOption) (image.InspectResponse, error) { - caller := getCallingFunction() - switch caller { - case "pullServiceImage", "buildContainerVolumes": - return image.InspectResponse{ID: "dryRunId"}, nil - default: - return d.apiClient.ImageInspect(ctx, imageName, options...) - } -} - -// Deprecated: Use [DryRunClient.ImageInspect] instead; raw response can be obtained by [client.ImageInspectWithRawResponse] option. -func (d *DryRunClient) ImageInspectWithRaw(ctx context.Context, imageName string) (image.InspectResponse, []byte, error) { - var buf bytes.Buffer - resp, err := d.ImageInspect(ctx, imageName, client.ImageInspectWithRawResponse(&buf)) - if err != nil { - return image.InspectResponse{}, nil, err - } - return resp, buf.Bytes(), err -} - -func (d *DryRunClient) ImagePull(ctx context.Context, ref string, options image.PullOptions) (io.ReadCloser, error) { - if _, _, err := d.resolver.Resolve(ctx, ref); err != nil { - return nil, err - } - rc := io.NopCloser(strings.NewReader("")) - return rc, nil -} - -func (d *DryRunClient) ImagePush(ctx context.Context, ref string, options image.PushOptions) (io.ReadCloser, error) { - if _, _, err := d.resolver.Resolve(ctx, ref); err != nil { - return nil, err - } - jsonMessage, err := json.Marshal(&jsonmessage.JSONMessage{ - Status: "Pushed", - Progress: &jsonmessage.JSONProgress{ - Current: 100, - Total: 100, - Start: 0, - HideCounts: false, - Units: "Mb", - }, - ID: ref, - }) - if err != nil { - return nil, err - } - rc := io.NopCloser(bytes.NewReader(jsonMessage)) - return rc, nil -} - -func (d *DryRunClient) ImageRemove(ctx context.Context, imageName string, options image.RemoveOptions) ([]image.DeleteResponse, error) { - return nil, nil -} - -func (d *DryRunClient) NetworkConnect(ctx context.Context, networkName, container string, config *network.EndpointSettings) error { - return nil -} - -func (d *DryRunClient) NetworkCreate(ctx context.Context, name string, options network.CreateOptions) (network.CreateResponse, error) { - return network.CreateResponse{ - ID: name, - Warning: "", - }, nil -} - -func (d *DryRunClient) NetworkDisconnect(ctx context.Context, networkName, container string, force bool) error { - return nil -} - -func (d *DryRunClient) NetworkRemove(ctx context.Context, networkName string) error { - return nil -} - -func (d *DryRunClient) VolumeCreate(ctx context.Context, options volume.CreateOptions) (volume.Volume, error) { - return volume.Volume{ - ClusterVolume: nil, - Driver: options.Driver, - Labels: options.Labels, - Name: options.Name, - Options: options.DriverOpts, - }, nil -} - -func (d *DryRunClient) VolumeRemove(ctx context.Context, volumeID string, force bool) error { - return nil -} - -func (d *DryRunClient) ContainerExecCreate(ctx context.Context, container string, config containerType.ExecOptions) (containerType.ExecCreateResponse, error) { - b := make([]byte, 32) - _, _ = rand.Read(b) - id := fmt.Sprintf("%x", b) - d.execs.Store(id, execDetails{ - container: container, - command: config.Cmd, - }) - return containerType.ExecCreateResponse{ - ID: id, - }, nil -} - -func (d *DryRunClient) ContainerExecStart(ctx context.Context, execID string, config containerType.ExecStartOptions) error { - _, ok := d.execs.LoadAndDelete(execID) - if !ok { - return fmt.Errorf("invalid exec ID %q", execID) - } - return nil -} - -// Functions delegated to original APIClient (not used by Compose or not modifying the Compose stack) - -func (d *DryRunClient) ConfigList(ctx context.Context, options swarm.ConfigListOptions) ([]swarm.Config, error) { - return d.apiClient.ConfigList(ctx, options) -} - -func (d *DryRunClient) ConfigCreate(ctx context.Context, config swarm.ConfigSpec) (swarm.ConfigCreateResponse, error) { - return d.apiClient.ConfigCreate(ctx, config) -} - -func (d *DryRunClient) ConfigRemove(ctx context.Context, id string) error { - return d.apiClient.ConfigRemove(ctx, id) -} - -func (d *DryRunClient) ConfigInspectWithRaw(ctx context.Context, name string) (swarm.Config, []byte, error) { - return d.apiClient.ConfigInspectWithRaw(ctx, name) -} - -func (d *DryRunClient) ConfigUpdate(ctx context.Context, id string, version swarm.Version, config swarm.ConfigSpec) error { - return d.apiClient.ConfigUpdate(ctx, id, version, config) -} - -func (d *DryRunClient) ContainerCommit(ctx context.Context, container string, options containerType.CommitOptions) (containerType.CommitResponse, error) { - return d.apiClient.ContainerCommit(ctx, container, options) -} - -func (d *DryRunClient) ContainerDiff(ctx context.Context, container string) ([]containerType.FilesystemChange, error) { - return d.apiClient.ContainerDiff(ctx, container) -} - -func (d *DryRunClient) ContainerExecAttach(ctx context.Context, execID string, config containerType.ExecStartOptions) (moby.HijackedResponse, error) { - return moby.HijackedResponse{}, errors.New("interactive exec is not supported in dry-run mode") -} - -func (d *DryRunClient) ContainerExecInspect(ctx context.Context, execID string) (containerType.ExecInspect, error) { - return d.apiClient.ContainerExecInspect(ctx, execID) -} - -func (d *DryRunClient) ContainerExecResize(ctx context.Context, execID string, options containerType.ResizeOptions) error { - return d.apiClient.ContainerExecResize(ctx, execID, options) -} - -func (d *DryRunClient) ContainerExport(ctx context.Context, container string) (io.ReadCloser, error) { - return d.apiClient.ContainerExport(ctx, container) -} - -func (d *DryRunClient) ContainerInspectWithRaw(ctx context.Context, container string, getSize bool) (containerType.InspectResponse, []byte, error) { - return d.apiClient.ContainerInspectWithRaw(ctx, container, getSize) -} - -func (d *DryRunClient) ContainerLogs(ctx context.Context, container string, options containerType.LogsOptions) (io.ReadCloser, error) { - return d.apiClient.ContainerLogs(ctx, container, options) -} - -func (d *DryRunClient) ContainerResize(ctx context.Context, container string, options containerType.ResizeOptions) error { - return d.apiClient.ContainerResize(ctx, container, options) -} - -func (d *DryRunClient) ContainerStatPath(ctx context.Context, container, path string) (containerType.PathStat, error) { - return d.apiClient.ContainerStatPath(ctx, container, path) -} - -func (d *DryRunClient) ContainerStats(ctx context.Context, container string, stream bool) (containerType.StatsResponseReader, error) { - return d.apiClient.ContainerStats(ctx, container, stream) -} - -func (d *DryRunClient) ContainerStatsOneShot(ctx context.Context, container string) (containerType.StatsResponseReader, error) { - return d.apiClient.ContainerStatsOneShot(ctx, container) -} - -func (d *DryRunClient) ContainerTop(ctx context.Context, container string, arguments []string) (containerType.TopResponse, error) { - return d.apiClient.ContainerTop(ctx, container, arguments) -} - -func (d *DryRunClient) ContainerUpdate(ctx context.Context, container string, updateConfig containerType.UpdateConfig) (containerType.UpdateResponse, error) { - return d.apiClient.ContainerUpdate(ctx, container, updateConfig) -} - -func (d *DryRunClient) ContainerWait(ctx context.Context, container string, condition containerType.WaitCondition) (<-chan containerType.WaitResponse, <-chan error) { - return d.apiClient.ContainerWait(ctx, container, condition) -} - -func (d *DryRunClient) ContainersPrune(ctx context.Context, pruneFilters filters.Args) (containerType.PruneReport, error) { - return d.apiClient.ContainersPrune(ctx, pruneFilters) -} - -func (d *DryRunClient) DistributionInspect(ctx context.Context, imageName, encodedRegistryAuth string) (registry.DistributionInspect, error) { - return d.apiClient.DistributionInspect(ctx, imageName, encodedRegistryAuth) -} - -func (d *DryRunClient) BuildCachePrune(ctx context.Context, opts build.CachePruneOptions) (*build.CachePruneReport, error) { - return d.apiClient.BuildCachePrune(ctx, opts) -} - -func (d *DryRunClient) BuildCancel(ctx context.Context, id string) error { - return d.apiClient.BuildCancel(ctx, id) -} - -func (d *DryRunClient) ImageCreate(ctx context.Context, parentReference string, options image.CreateOptions) (io.ReadCloser, error) { - return d.apiClient.ImageCreate(ctx, parentReference, options) -} - -func (d *DryRunClient) ImageHistory(ctx context.Context, imageName string, options ...client.ImageHistoryOption) ([]image.HistoryResponseItem, error) { - return d.apiClient.ImageHistory(ctx, imageName, options...) -} - -func (d *DryRunClient) ImageImport(ctx context.Context, source image.ImportSource, ref string, options image.ImportOptions) (io.ReadCloser, error) { - return d.apiClient.ImageImport(ctx, source, ref, options) -} - -func (d *DryRunClient) ImageList(ctx context.Context, options image.ListOptions) ([]image.Summary, error) { - return d.apiClient.ImageList(ctx, options) -} - -func (d *DryRunClient) ImageLoad(ctx context.Context, input io.Reader, options ...client.ImageLoadOption) (image.LoadResponse, error) { - return d.apiClient.ImageLoad(ctx, input, options...) -} - -func (d *DryRunClient) ImageSearch(ctx context.Context, term string, options registry.SearchOptions) ([]registry.SearchResult, error) { - return d.apiClient.ImageSearch(ctx, term, options) -} - -func (d *DryRunClient) ImageSave(ctx context.Context, images []string, options ...client.ImageSaveOption) (io.ReadCloser, error) { - return d.apiClient.ImageSave(ctx, images, options...) -} - -func (d *DryRunClient) ImageTag(ctx context.Context, imageName, ref string) error { - return d.apiClient.ImageTag(ctx, imageName, ref) -} - -func (d *DryRunClient) ImagesPrune(ctx context.Context, pruneFilter filters.Args) (image.PruneReport, error) { - return d.apiClient.ImagesPrune(ctx, pruneFilter) -} - -func (d *DryRunClient) NodeInspectWithRaw(ctx context.Context, nodeID string) (swarm.Node, []byte, error) { - return d.apiClient.NodeInspectWithRaw(ctx, nodeID) -} - -func (d *DryRunClient) NodeList(ctx context.Context, options swarm.NodeListOptions) ([]swarm.Node, error) { - return d.apiClient.NodeList(ctx, options) -} - -func (d *DryRunClient) NodeRemove(ctx context.Context, nodeID string, options swarm.NodeRemoveOptions) error { - return d.apiClient.NodeRemove(ctx, nodeID, options) -} - -func (d *DryRunClient) NodeUpdate(ctx context.Context, nodeID string, version swarm.Version, node swarm.NodeSpec) error { - return d.apiClient.NodeUpdate(ctx, nodeID, version, node) -} - -func (d *DryRunClient) NetworkInspect(ctx context.Context, networkName string, options network.InspectOptions) (network.Inspect, error) { - return d.apiClient.NetworkInspect(ctx, networkName, options) -} - -func (d *DryRunClient) NetworkInspectWithRaw(ctx context.Context, networkName string, options network.InspectOptions) (network.Inspect, []byte, error) { - return d.apiClient.NetworkInspectWithRaw(ctx, networkName, options) -} - -func (d *DryRunClient) NetworkList(ctx context.Context, options network.ListOptions) ([]network.Inspect, error) { - return d.apiClient.NetworkList(ctx, options) -} - -func (d *DryRunClient) NetworksPrune(ctx context.Context, pruneFilter filters.Args) (network.PruneReport, error) { - return d.apiClient.NetworksPrune(ctx, pruneFilter) -} - -func (d *DryRunClient) PluginList(ctx context.Context, filter filters.Args) (moby.PluginsListResponse, error) { - return d.apiClient.PluginList(ctx, filter) -} - -func (d *DryRunClient) PluginRemove(ctx context.Context, name string, options moby.PluginRemoveOptions) error { - return d.apiClient.PluginRemove(ctx, name, options) -} - -func (d *DryRunClient) PluginEnable(ctx context.Context, name string, options moby.PluginEnableOptions) error { - return d.apiClient.PluginEnable(ctx, name, options) -} - -func (d *DryRunClient) PluginDisable(ctx context.Context, name string, options moby.PluginDisableOptions) error { - return d.apiClient.PluginDisable(ctx, name, options) -} - -func (d *DryRunClient) PluginInstall(ctx context.Context, name string, options moby.PluginInstallOptions) (io.ReadCloser, error) { - return d.apiClient.PluginInstall(ctx, name, options) -} - -func (d *DryRunClient) PluginUpgrade(ctx context.Context, name string, options moby.PluginInstallOptions) (io.ReadCloser, error) { - return d.apiClient.PluginUpgrade(ctx, name, options) -} - -func (d *DryRunClient) PluginPush(ctx context.Context, name string, registryAuth string) (io.ReadCloser, error) { - return d.apiClient.PluginPush(ctx, name, registryAuth) -} - -func (d *DryRunClient) PluginSet(ctx context.Context, name string, args []string) error { - return d.apiClient.PluginSet(ctx, name, args) -} - -func (d *DryRunClient) PluginInspectWithRaw(ctx context.Context, name string) (*moby.Plugin, []byte, error) { - return d.apiClient.PluginInspectWithRaw(ctx, name) -} - -func (d *DryRunClient) PluginCreate(ctx context.Context, createContext io.Reader, options moby.PluginCreateOptions) error { - return d.apiClient.PluginCreate(ctx, createContext, options) -} - -func (d *DryRunClient) ServiceCreate(ctx context.Context, service swarm.ServiceSpec, options swarm.ServiceCreateOptions) (swarm.ServiceCreateResponse, error) { - return d.apiClient.ServiceCreate(ctx, service, options) -} - -func (d *DryRunClient) ServiceInspectWithRaw(ctx context.Context, serviceID string, options swarm.ServiceInspectOptions) (swarm.Service, []byte, error) { - return d.apiClient.ServiceInspectWithRaw(ctx, serviceID, options) -} - -func (d *DryRunClient) ServiceList(ctx context.Context, options swarm.ServiceListOptions) ([]swarm.Service, error) { - return d.apiClient.ServiceList(ctx, options) -} - -func (d *DryRunClient) ServiceRemove(ctx context.Context, serviceID string) error { - return d.apiClient.ServiceRemove(ctx, serviceID) -} - -func (d *DryRunClient) ServiceUpdate(ctx context.Context, serviceID string, version swarm.Version, service swarm.ServiceSpec, options swarm.ServiceUpdateOptions) (swarm.ServiceUpdateResponse, error) { - return d.apiClient.ServiceUpdate(ctx, serviceID, version, service, options) -} - -func (d *DryRunClient) ServiceLogs(ctx context.Context, serviceID string, options containerType.LogsOptions) (io.ReadCloser, error) { - return d.apiClient.ServiceLogs(ctx, serviceID, options) -} - -func (d *DryRunClient) TaskLogs(ctx context.Context, taskID string, options containerType.LogsOptions) (io.ReadCloser, error) { - return d.apiClient.TaskLogs(ctx, taskID, options) -} - -func (d *DryRunClient) TaskInspectWithRaw(ctx context.Context, taskID string) (swarm.Task, []byte, error) { - return d.apiClient.TaskInspectWithRaw(ctx, taskID) -} - -func (d *DryRunClient) TaskList(ctx context.Context, options swarm.TaskListOptions) ([]swarm.Task, error) { - return d.apiClient.TaskList(ctx, options) -} - -func (d *DryRunClient) SwarmInit(ctx context.Context, req swarm.InitRequest) (string, error) { - return d.apiClient.SwarmInit(ctx, req) -} - -func (d *DryRunClient) SwarmJoin(ctx context.Context, req swarm.JoinRequest) error { - return d.apiClient.SwarmJoin(ctx, req) -} - -func (d *DryRunClient) SwarmGetUnlockKey(ctx context.Context) (swarm.UnlockKeyResponse, error) { - return d.apiClient.SwarmGetUnlockKey(ctx) -} - -func (d *DryRunClient) SwarmUnlock(ctx context.Context, req swarm.UnlockRequest) error { - return d.apiClient.SwarmUnlock(ctx, req) -} - -func (d *DryRunClient) SwarmLeave(ctx context.Context, force bool) error { - return d.apiClient.SwarmLeave(ctx, force) -} - -func (d *DryRunClient) SwarmInspect(ctx context.Context) (swarm.Swarm, error) { - return d.apiClient.SwarmInspect(ctx) -} - -func (d *DryRunClient) SwarmUpdate(ctx context.Context, version swarm.Version, swarmSpec swarm.Spec, flags swarm.UpdateFlags) error { - return d.apiClient.SwarmUpdate(ctx, version, swarmSpec, flags) -} - -func (d *DryRunClient) SecretList(ctx context.Context, options swarm.SecretListOptions) ([]swarm.Secret, error) { - return d.apiClient.SecretList(ctx, options) -} - -func (d *DryRunClient) SecretCreate(ctx context.Context, secret swarm.SecretSpec) (swarm.SecretCreateResponse, error) { - return d.apiClient.SecretCreate(ctx, secret) -} - -func (d *DryRunClient) SecretRemove(ctx context.Context, id string) error { - return d.apiClient.SecretRemove(ctx, id) -} - -func (d *DryRunClient) SecretInspectWithRaw(ctx context.Context, name string) (swarm.Secret, []byte, error) { - return d.apiClient.SecretInspectWithRaw(ctx, name) -} - -func (d *DryRunClient) SecretUpdate(ctx context.Context, id string, version swarm.Version, secret swarm.SecretSpec) error { - return d.apiClient.SecretUpdate(ctx, id, version, secret) -} - -func (d *DryRunClient) Events(ctx context.Context, options events.ListOptions) (<-chan events.Message, <-chan error) { - return d.apiClient.Events(ctx, options) -} - -func (d *DryRunClient) Info(ctx context.Context) (system.Info, error) { - return d.apiClient.Info(ctx) -} - -func (d *DryRunClient) RegistryLogin(ctx context.Context, auth registry.AuthConfig) (registry.AuthenticateOKBody, error) { - return d.apiClient.RegistryLogin(ctx, auth) -} - -func (d *DryRunClient) DiskUsage(ctx context.Context, options moby.DiskUsageOptions) (moby.DiskUsage, error) { - return d.apiClient.DiskUsage(ctx, options) -} - -func (d *DryRunClient) Ping(ctx context.Context) (moby.Ping, error) { - return d.apiClient.Ping(ctx) -} - -func (d *DryRunClient) VolumeInspect(ctx context.Context, volumeID string) (volume.Volume, error) { - return d.apiClient.VolumeInspect(ctx, volumeID) -} - -func (d *DryRunClient) VolumeInspectWithRaw(ctx context.Context, volumeID string) (volume.Volume, []byte, error) { - return d.apiClient.VolumeInspectWithRaw(ctx, volumeID) -} - -func (d *DryRunClient) VolumeList(ctx context.Context, opts volume.ListOptions) (volume.ListResponse, error) { - return d.apiClient.VolumeList(ctx, opts) -} - -func (d *DryRunClient) VolumesPrune(ctx context.Context, pruneFilter filters.Args) (volume.PruneReport, error) { - return d.apiClient.VolumesPrune(ctx, pruneFilter) -} - -func (d *DryRunClient) VolumeUpdate(ctx context.Context, volumeID string, version swarm.Version, options volume.UpdateOptions) error { - return d.apiClient.VolumeUpdate(ctx, volumeID, version, options) -} - -func (d *DryRunClient) ClientVersion() string { - return d.apiClient.ClientVersion() -} - -func (d *DryRunClient) DaemonHost() string { - return d.apiClient.DaemonHost() -} - -func (d *DryRunClient) HTTPClient() *http.Client { - return d.apiClient.HTTPClient() -} - -func (d *DryRunClient) ServerVersion(ctx context.Context) (moby.Version, error) { - return d.apiClient.ServerVersion(ctx) -} - -func (d *DryRunClient) NegotiateAPIVersion(ctx context.Context) { - d.apiClient.NegotiateAPIVersion(ctx) -} - -func (d *DryRunClient) NegotiateAPIVersionPing(ping moby.Ping) { - d.apiClient.NegotiateAPIVersionPing(ping) -} - -func (d *DryRunClient) DialHijack(ctx context.Context, url, proto string, meta map[string][]string) (net.Conn, error) { - return d.apiClient.DialHijack(ctx, url, proto, meta) -} - -func (d *DryRunClient) Dialer() func(context.Context) (net.Conn, error) { - return d.apiClient.Dialer() -} - -func (d *DryRunClient) Close() error { - return d.apiClient.Close() -} - -func (d *DryRunClient) CheckpointCreate(ctx context.Context, container string, options checkpoint.CreateOptions) error { - return d.apiClient.CheckpointCreate(ctx, container, options) -} - -func (d *DryRunClient) CheckpointDelete(ctx context.Context, container string, options checkpoint.DeleteOptions) error { - return d.apiClient.CheckpointDelete(ctx, container, options) -} - -func (d *DryRunClient) CheckpointList(ctx context.Context, container string, options checkpoint.ListOptions) ([]checkpoint.Summary, error) { - return d.apiClient.CheckpointList(ctx, container, options) -} diff --git a/pkg/e2e/assert.go b/pkg/e2e/assert.go deleted file mode 100644 index 25547203dbe..00000000000 --- a/pkg/e2e/assert.go +++ /dev/null @@ -1,45 +0,0 @@ -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "encoding/json" - "strings" - "testing" - - "github.com/stretchr/testify/require" -) - -// RequireServiceState ensures that the container is in the expected state -// (running or exited). -func RequireServiceState(t testing.TB, cli *CLI, service string, state string) { - t.Helper() - psRes := cli.RunDockerComposeCmd(t, "ps", "--all", "--format=json", service) - var svc map[string]any - require.NoError(t, json.Unmarshal([]byte(psRes.Stdout()), &svc), - "Invalid `compose ps` JSON: command output: %s", - psRes.Combined()) - - require.Equal(t, service, svc["Service"], - "Found ps output for unexpected service") - require.Equalf(t, - strings.ToLower(state), - strings.ToLower(svc["State"].(string)), - "Service %q (%s) not in expected state", - service, svc["Name"], - ) -} diff --git a/pkg/e2e/bridge_test.go b/pkg/e2e/bridge_test.go deleted file mode 100644 index c4c99b8d292..00000000000 --- a/pkg/e2e/bridge_test.go +++ /dev/null @@ -1,61 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "path/filepath" - "strings" - "testing" - - "gotest.tools/v3/assert" -) - -func TestConvertAndTransformList(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "bridge" - const bridgeImageVersion = "v0.0.3" - tmpDir := t.TempDir() - - t.Run("kubernetes manifests", func(t *testing.T) { - kubedir := filepath.Join(tmpDir, "kubernetes") - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/bridge/compose.yaml", "--project-name", projectName, "bridge", "convert", - "--output", kubedir, "--transformation", fmt.Sprintf("docker/compose-bridge-kubernetes:%s", bridgeImageVersion)) - assert.NilError(t, res.Error) - assert.Equal(t, res.ExitCode, 0) - res = c.RunCmd(t, "diff", "-r", kubedir, "./fixtures/bridge/expected-kubernetes") - assert.NilError(t, res.Error, res.Combined()) - }) - - t.Run("helm charts", func(t *testing.T) { - helmDir := filepath.Join(tmpDir, "helm") - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/bridge/compose.yaml", "--project-name", projectName, "bridge", "convert", - "--output", helmDir, "--transformation", fmt.Sprintf("docker/compose-bridge-helm:%s", bridgeImageVersion)) - assert.NilError(t, res.Error) - assert.Equal(t, res.ExitCode, 0) - res = c.RunCmd(t, "diff", "-r", helmDir, "./fixtures/bridge/expected-helm") - assert.NilError(t, res.Error, res.Combined()) - }) - - t.Run("list transformers images", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "bridge", "transformations", - "ls") - assert.Assert(t, strings.Contains(res.Stdout(), "docker/compose-bridge-helm"), res.Combined()) - assert.Assert(t, strings.Contains(res.Stdout(), "docker/compose-bridge-kubernetes"), res.Combined()) - }) -} diff --git a/pkg/e2e/build_test.go b/pkg/e2e/build_test.go deleted file mode 100644 index 1fc3ac87667..00000000000 --- a/pkg/e2e/build_test.go +++ /dev/null @@ -1,641 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "net/http" - "os" - "regexp" - "runtime" - "strconv" - "strings" - "testing" - "time" - - "github.com/stretchr/testify/require" - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" - "gotest.tools/v3/poll" -) - -func TestLocalComposeBuild(t *testing.T) { - for _, env := range []string{"DOCKER_BUILDKIT=0", "DOCKER_BUILDKIT=1"} { - c := NewCLI(t, WithEnv(strings.Split(env, ",")...)) - - t.Run(env+" build named and unnamed images", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "-f", "build-test-nginx") - c.RunDockerOrExitError(t, "rmi", "-f", "custom-nginx") - - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "build") - - res.Assert(t, icmd.Expected{Out: "COPY static /usr/share/nginx/html"}) - c.RunDockerCmd(t, "image", "inspect", "build-test-nginx") - c.RunDockerCmd(t, "image", "inspect", "custom-nginx") - }) - - t.Run(env+" build with build-arg", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "-f", "build-test-nginx") - c.RunDockerOrExitError(t, "rmi", "-f", "custom-nginx") - - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "build", "--build-arg", "FOO=BAR") - - res := c.RunDockerCmd(t, "image", "inspect", "build-test-nginx") - res.Assert(t, icmd.Expected{Out: `"FOO": "BAR"`}) - }) - - t.Run(env+" build with build-arg set by env", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "-f", "build-test-nginx") - c.RunDockerOrExitError(t, "rmi", "-f", "custom-nginx") - - icmd.RunCmd(c.NewDockerComposeCmd(t, - "--project-directory", - "fixtures/build-test", - "build", - "--build-arg", - "FOO"), - func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "FOO=BAR") - }).Assert(t, icmd.Success) - - res := c.RunDockerCmd(t, "image", "inspect", "build-test-nginx") - res.Assert(t, icmd.Expected{Out: `"FOO": "BAR"`}) - }) - - t.Run(env+" build with multiple build-args ", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "-f", "multi-args-multiargs") - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/build-test/multi-args", "build") - - icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "DOCKER_BUILDKIT=0") - }) - - res := c.RunDockerCmd(t, "image", "inspect", "multi-args-multiargs") - res.Assert(t, icmd.Expected{Out: `"RESULT": "SUCCESS"`}) - }) - - t.Run(env+" build as part of up", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "-f", "build-test-nginx") - c.RunDockerOrExitError(t, "rmi", "-f", "custom-nginx") - - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "up", "-d") - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "down") - }) - - res.Assert(t, icmd.Expected{Out: "COPY static /usr/share/nginx/html"}) - res.Assert(t, icmd.Expected{Out: "COPY static2 /usr/share/nginx/html"}) - - output := HTTPGetWithRetry(t, "http://localhost:8070", http.StatusOK, 2*time.Second, 20*time.Second) - assert.Assert(t, strings.Contains(output, "Hello from Nginx container")) - - c.RunDockerCmd(t, "image", "inspect", "build-test-nginx") - c.RunDockerCmd(t, "image", "inspect", "custom-nginx") - }) - - t.Run(env+" no rebuild when up again", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "up", "-d") - - assert.Assert(t, !strings.Contains(res.Stdout(), "COPY static")) - }) - - t.Run(env+" rebuild when up --build", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "up", "-d", "--build") - - res.Assert(t, icmd.Expected{Out: "COPY static /usr/share/nginx/html"}) - res.Assert(t, icmd.Expected{Out: "COPY static2 /usr/share/nginx/html"}) - }) - - t.Run(env+" build --push ignored for unnamed images", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "build", "--push", "nginx") - assert.Assert(t, !strings.Contains(res.Stdout(), "failed to push"), res.Stdout()) - }) - - t.Run(env+" build --quiet", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "build", "--quiet") - res.Assert(t, icmd.Expected{Out: ""}) - }) - - t.Run(env+" cleanup build project", func(t *testing.T) { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "down") - c.RunDockerOrExitError(t, "rmi", "-f", "build-test-nginx") - c.RunDockerOrExitError(t, "rmi", "-f", "custom-nginx") - }) - } -} - -func TestBuildSSH(t *testing.T) { - if runtime.GOOS == "windows" { - t.Skip("Running on Windows. Skipping...") - } - c := NewParallelCLI(t) - - t.Run("build failed with ssh default value", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test", "build", "--ssh", "") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "invalid empty ssh agent socket: make sure SSH_AUTH_SOCK is set", - }) - }) - - t.Run("build succeed with ssh from Compose file", func(t *testing.T) { - c.RunDockerOrExitError(t, "rmi", "build-test-ssh") - - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/ssh", "build") - c.RunDockerCmd(t, "image", "inspect", "build-test-ssh") - }) - - t.Run("build succeed with ssh from CLI", func(t *testing.T) { - c.RunDockerOrExitError(t, "rmi", "build-test-ssh") - - c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/ssh/compose-without-ssh.yaml", "--project-directory", - "fixtures/build-test/ssh", "build", "--no-cache", "--ssh", "fake-ssh=./fixtures/build-test/ssh/fake_rsa") - c.RunDockerCmd(t, "image", "inspect", "build-test-ssh") - }) - - /* - FIXME disabled waiting for https://github.com/moby/buildkit/issues/5558 - t.Run("build failed with wrong ssh key id from CLI", func(t *testing.T) { - c.RunDockerOrExitError(t, "rmi", "build-test-ssh") - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/build-test/ssh/compose-without-ssh.yaml", - "--project-directory", "fixtures/build-test/ssh", "build", "--no-cache", "--ssh", - "wrong-ssh=./fixtures/build-test/ssh/fake_rsa") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "unset ssh forward key fake-ssh", - }) - }) - */ - - t.Run("build succeed as part of up with ssh from Compose file", func(t *testing.T) { - c.RunDockerOrExitError(t, "rmi", "build-test-ssh") - - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/ssh", "up", "-d", "--build") - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/ssh", "down") - }) - c.RunDockerCmd(t, "image", "inspect", "build-test-ssh") - }) -} - -func TestBuildSecrets(t *testing.T) { - if runtime.GOOS == "windows" { - t.Skip("skipping test on windows") - } - c := NewParallelCLI(t) - - t.Run("build with secrets", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "build-test-secret") - - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/build-test/secrets", "build") - - res := icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "SOME_SECRET=bar") - }) - - res.Assert(t, icmd.Success) - }) -} - -func TestBuildTags(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("build with tags", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "build-test-tags") - - c.RunDockerComposeCmd(t, "--project-directory", "./fixtures/build-test/tags", "build", "--no-cache") - - res := c.RunDockerCmd(t, "image", "inspect", "build-test-tags") - expectedOutput := `"RepoTags": [ - "docker/build-test-tags:1.0.0", - "build-test-tags:latest", - "other-image-name:v1.0.0" - ], -` - res.Assert(t, icmd.Expected{Out: expectedOutput}) - }) -} - -func TestBuildImageDependencies(t *testing.T) { - doTest := func(t *testing.T, cli *CLI, args ...string) { - resetState := func() { - cli.RunDockerComposeCmd(t, "down", "--rmi=all", "-t=0") - res := cli.RunDockerOrExitError(t, "image", "rm", "build-dependencies-service") - if res.Error != nil { - require.Contains(t, res.Stderr(), `No such image: build-dependencies-service`) - } - } - resetState() - t.Cleanup(resetState) - - // the image should NOT exist now - res := cli.RunDockerOrExitError(t, "image", "inspect", "build-dependencies-service") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "No such image: build-dependencies-service", - }) - - res = cli.RunDockerComposeCmd(t, args...) - t.Log(res.Combined()) - - res = cli.RunDockerCmd(t, - "image", "inspect", "--format={{ index .RepoTags 0 }}", - "build-dependencies-service") - res.Assert(t, icmd.Expected{Out: "build-dependencies-service:latest"}) - - res = cli.RunDockerComposeCmd(t, "down", "-t0", "--rmi=all", "--remove-orphans") - t.Log(res.Combined()) - - res = cli.RunDockerOrExitError(t, "image", "inspect", "build-dependencies-service") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "No such image: build-dependencies-service", - }) - } - - t.Run("ClassicBuilder", func(t *testing.T) { - cli := NewCLI(t, WithEnv( - "DOCKER_BUILDKIT=0", - "COMPOSE_FILE=./fixtures/build-dependencies/classic.yaml", - )) - doTest(t, cli, "build") - doTest(t, cli, "build", "--with-dependencies", "service") - }) - - t.Run("Bake by additional contexts", func(t *testing.T) { - cli := NewCLI(t, WithEnv( - "DOCKER_BUILDKIT=1", "COMPOSE_BAKE=1", - "COMPOSE_FILE=./fixtures/build-dependencies/compose.yaml", - )) - doTest(t, cli, "--verbose", "build") - doTest(t, cli, "--verbose", "build", "service") - doTest(t, cli, "--verbose", "up", "--build", "service") - }) -} - -func TestBuildPlatformsWithCorrectBuildxConfig(t *testing.T) { - if runtime.GOOS == "windows" { - t.Skip("Running on Windows. Skipping...") - } - c := NewParallelCLI(t) - - // declare builder - result := c.RunDockerCmd(t, "buildx", "create", "--name", "build-platform", "--use", "--bootstrap") - assert.NilError(t, result.Error) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/platforms", "down") - _ = c.RunDockerCmd(t, "buildx", "rm", "-f", "build-platform") - }) - - t.Run("platform not supported by builder", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", - "-f", "fixtures/build-test/platforms/compose-unsupported-platform.yml", "build") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "no match for platform", - }) - }) - - t.Run("multi-arch build ok", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", "build") - assert.NilError(t, res.Error, res.Stderr()) - res.Assert(t, icmd.Expected{Out: "I am building for linux/arm64"}) - res.Assert(t, icmd.Expected{Out: "I am building for linux/amd64"}) - }) - - t.Run("multi-arch multi service builds ok", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", - "-f", "fixtures/build-test/platforms/compose-multiple-platform-builds.yaml", "build") - assert.NilError(t, res.Error, res.Stderr()) - res.Assert(t, icmd.Expected{Out: "I'm Service A and I am building for linux/arm64"}) - res.Assert(t, icmd.Expected{Out: "I'm Service A and I am building for linux/amd64"}) - res.Assert(t, icmd.Expected{Out: "I'm Service B and I am building for linux/arm64"}) - res.Assert(t, icmd.Expected{Out: "I'm Service B and I am building for linux/amd64"}) - res.Assert(t, icmd.Expected{Out: "I'm Service C and I am building for linux/arm64"}) - res.Assert(t, icmd.Expected{Out: "I'm Service C and I am building for linux/amd64"}) - }) - - t.Run("multi-arch up --build", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", "up", "--build", "--menu=false") - assert.NilError(t, res.Error, res.Stderr()) - res.Assert(t, icmd.Expected{Out: "platforms-1 exited with code 0"}) - }) - - t.Run("use DOCKER_DEFAULT_PLATFORM value when up --build", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/build-test/platforms", "up", "--build", "--menu=false") - res := icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "DOCKER_DEFAULT_PLATFORM=linux/amd64") - }) - assert.NilError(t, res.Error, res.Stderr()) - res.Assert(t, icmd.Expected{Out: "I am building for linux/amd64"}) - assert.Assert(t, !strings.Contains(res.Stdout(), "I am building for linux/arm64")) - }) - - t.Run("use service platform value when no build platforms defined ", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", - "-f", "fixtures/build-test/platforms/compose-service-platform-and-no-build-platforms.yaml", "build") - assert.NilError(t, res.Error, res.Stderr()) - res.Assert(t, icmd.Expected{Out: "I am building for linux/386"}) - }) -} - -func TestBuildPrivileged(t *testing.T) { - c := NewParallelCLI(t) - - // declare builder - result := c.RunDockerCmd(t, "buildx", "create", "--name", "build-privileged", "--use", "--bootstrap", "--buildkitd-flags", - `'--allow-insecure-entitlement=security.insecure'`) - assert.NilError(t, result.Error) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/privileged", "down") - _ = c.RunDockerCmd(t, "buildx", "rm", "-f", "build-privileged") - }) - - t.Run("use build privileged mode to run insecure build command", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/privileged", "build") - capEffRe := regexp.MustCompile("CapEff:\t([0-9a-f]+)") - matches := capEffRe.FindStringSubmatch(res.Stdout()) - assert.Equal(t, 2, len(matches), "Did not match CapEff in output, matches: %v", matches) - - capEff, err := strconv.ParseUint(matches[1], 16, 64) - assert.NilError(t, err, "Parsing CapEff: %s", matches[1]) - - // NOTE: can't use constant from x/sys/unix or tests won't compile on macOS/Windows - // #define CAP_SYS_ADMIN 21 - // https://github.com/torvalds/linux/blob/v6.1/include/uapi/linux/capability.h#L278 - const capSysAdmin = 0x15 - if capEff&capSysAdmin != capSysAdmin { - t.Fatalf("CapEff %s is missing CAP_SYS_ADMIN", matches[1]) - } - }) -} - -func TestBuildPlatformsStandardErrors(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("no platform support with Classic Builder", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/build-test/platforms", "build") - - res := icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "DOCKER_BUILDKIT=0") - }) - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "the classic builder doesn't support multi-arch build, set DOCKER_BUILDKIT=1 to use BuildKit", - }) - }) - - t.Run("builder does not support multi-arch", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", "build") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "Multi-platform build is not supported for the docker driver.", - }) - }) - - t.Run("service platform not defined in platforms build section", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", - "-f", "fixtures/build-test/platforms/compose-service-platform-not-in-build-platforms.yaml", "build") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: `service.build.platforms MUST include service.platform "linux/riscv64"`, - }) - }) - - t.Run("DOCKER_DEFAULT_PLATFORM value not defined in platforms build section", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/build-test/platforms", "build") - res := icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "DOCKER_DEFAULT_PLATFORM=windows/amd64") - }) - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: `service "platforms" build.platforms does not support value set by DOCKER_DEFAULT_PLATFORM: windows/amd64`, - }) - }) - - t.Run("no privileged support with Classic Builder", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/build-test/privileged", "build") - - res := icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "DOCKER_BUILDKIT=0") - }) - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "the classic builder doesn't support privileged mode, set DOCKER_BUILDKIT=1 to use BuildKit", - }) - }) -} - -func TestBuildBuilder(t *testing.T) { - c := NewParallelCLI(t) - builderName := "build-with-builder" - // declare builder - result := c.RunDockerCmd(t, "buildx", "create", "--name", builderName, "--use", "--bootstrap") - assert.NilError(t, result.Error) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/", "down") - _ = c.RunDockerCmd(t, "buildx", "rm", "-f", builderName) - }) - - t.Run("use specific builder to run build command", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test", "build", "--builder", builderName) - assert.NilError(t, res.Error, res.Stderr()) - }) - - t.Run("error when using specific builder to run build command", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test", "build", "--builder", "unknown-builder") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: fmt.Sprintf(`no builder %q found`, "unknown-builder"), - }) - }) -} - -func TestBuildEntitlements(t *testing.T) { - c := NewParallelCLI(t) - - // declare builder - result := c.RunDockerCmd(t, "buildx", "create", "--name", "build-insecure", "--use", "--bootstrap", "--buildkitd-flags", - `'--allow-insecure-entitlement=security.insecure'`) - assert.NilError(t, result.Error) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/entitlements", "down") - _ = c.RunDockerCmd(t, "buildx", "rm", "-f", "build-insecure") - }) - - t.Run("use build privileged mode to run insecure build command", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/entitlements", "build") - capEffRe := regexp.MustCompile("CapEff:\t([0-9a-f]+)") - matches := capEffRe.FindStringSubmatch(res.Stdout()) - assert.Equal(t, 2, len(matches), "Did not match CapEff in output, matches: %v", matches) - - capEff, err := strconv.ParseUint(matches[1], 16, 64) - assert.NilError(t, err, "Parsing CapEff: %s", matches[1]) - - // NOTE: can't use constant from x/sys/unix or tests won't compile on macOS/Windows - // #define CAP_SYS_ADMIN 21 - // https://github.com/torvalds/linux/blob/v6.1/include/uapi/linux/capability.h#L278 - const capSysAdmin = 0x15 - if capEff&capSysAdmin != capSysAdmin { - t.Fatalf("CapEff %s is missing CAP_SYS_ADMIN", matches[1]) - } - }) -} - -func TestBuildDependsOn(t *testing.T) { - c := NewParallelCLI(t) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/build-dependencies/compose-depends_on.yaml", "down", "--rmi=local") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/build-dependencies/compose-depends_on.yaml", "--progress=plain", "up", "test2") - out := res.Combined() - assert.Check(t, strings.Contains(out, "test1 Built")) -} - -func TestBuildSubset(t *testing.T) { - c := NewParallelCLI(t) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/subset/compose.yaml", "down", "--rmi=local") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/subset/compose.yaml", "build", "main") - out := res.Combined() - assert.Check(t, strings.Contains(out, "main Built")) -} - -func TestBuildDependentImage(t *testing.T) { - c := NewParallelCLI(t) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/dependencies/compose.yaml", "down", "--rmi=local") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/dependencies/compose.yaml", "build", "firstbuild") - out := res.Combined() - assert.Check(t, strings.Contains(out, "firstbuild Built")) - - res = c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/dependencies/compose.yaml", "build", "secondbuild") - out = res.Combined() - assert.Check(t, strings.Contains(out, "secondbuild Built")) -} - -func TestBuildSubDependencies(t *testing.T) { - c := NewParallelCLI(t) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/sub-dependencies/compose.yaml", "down", "--rmi=local") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/sub-dependencies/compose.yaml", "build", "main") - out := res.Combined() - assert.Check(t, strings.Contains(out, "main Built")) - - res = c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/sub-dependencies/compose.yaml", "up", "--build", "main") - out = res.Combined() - assert.Check(t, strings.Contains(out, "main Built")) -} - -func TestBuildLongOutputLine(t *testing.T) { - c := NewParallelCLI(t) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/long-output-line/compose.yaml", "down", "--rmi=local") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/long-output-line/compose.yaml", "build", "long-line") - out := res.Combined() - assert.Check(t, strings.Contains(out, "long-line Built")) - - res = c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/long-output-line/compose.yaml", "up", "--build", "long-line") - out = res.Combined() - assert.Check(t, strings.Contains(out, "long-line Built")) -} - -func TestBuildDependentImageWithProfile(t *testing.T) { - c := NewParallelCLI(t) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/profiles/compose.yaml", "down", "--rmi=local") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/profiles/compose.yaml", "build", "secret-build-test") - out := res.Combined() - assert.Check(t, strings.Contains(out, "secret-build-test Built")) -} - -func TestBuildTLS(t *testing.T) { - t.Helper() - - c := NewParallelCLI(t) - const dindBuilder = "e2e-dind-builder" - tmp := t.TempDir() - - t.Cleanup(func() { - c.RunDockerCmd(t, "rm", "-f", dindBuilder) - c.RunDockerCmd(t, "context", "rm", dindBuilder) - }) - - c.RunDockerCmd(t, "run", "--name", dindBuilder, "--privileged", "-p", "2376:2376", "-d", "docker:dind") - - poll.WaitOn(t, func(_ poll.LogT) poll.Result { - res := c.RunDockerCmd(t, "logs", dindBuilder) - if strings.Contains(res.Combined(), "API listen on [::]:2376") { - return poll.Success() - } - return poll.Continue("waiting for Docker daemon to be running") - }, poll.WithTimeout(10*time.Second)) - - time.Sleep(1 * time.Second) // wait for dind setup - c.RunDockerCmd(t, "cp", dindBuilder+":/certs/client", tmp) - - c.RunDockerCmd(t, "context", "create", dindBuilder, "--docker", - fmt.Sprintf("host=tcp://localhost:2376,ca=%s/client/ca.pem,cert=%s/client/cert.pem,key=%s/client/key.pem,skip-tls-verify=1", tmp, tmp, tmp)) - - cmd := c.NewDockerComposeCmd(t, "-f", "fixtures/build-test/minimal/compose.yaml", "build") - cmd.Env = append(cmd.Env, "DOCKER_CONTEXT="+dindBuilder) - cmd.Stdout = os.Stdout - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{Err: "Built"}) -} - -func TestBuildEscaped(t *testing.T) { - c := NewParallelCLI(t) - - res := c.RunDockerComposeCmd(t, "--project-directory", "./fixtures/build-test/escaped", "build", "--no-cache", "foo") - res.Assert(t, icmd.Expected{Out: "foo is ${bar}"}) - - res = c.RunDockerComposeCmd(t, "--project-directory", "./fixtures/build-test/escaped", "build", "--no-cache", "echo") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "--project-directory", "./fixtures/build-test/escaped", "build", "--no-cache", "arg") - res.Assert(t, icmd.Success) -} diff --git a/pkg/e2e/cancel_test.go b/pkg/e2e/cancel_test.go deleted file mode 100644 index 64f3ff609a9..00000000000 --- a/pkg/e2e/cancel_test.go +++ /dev/null @@ -1,99 +0,0 @@ -//go:build !windows - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "context" - "fmt" - "os/exec" - "strings" - "syscall" - "testing" - "time" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" - - "github.com/docker/compose/v5/pkg/utils" -) - -func TestComposeCancel(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("metrics on cancel Compose build", func(t *testing.T) { - const buildProjectPath = "fixtures/build-infinite/compose.yaml" - - ctx, cancel := context.WithCancel(t.Context()) - defer cancel() - - // require a separate groupID from the process running tests, in order to simulate ctrl+C from a terminal. - // sending kill signal - var stdout, stderr utils.SafeBuffer - cmd, err := StartWithNewGroupID( - ctx, - c.NewDockerComposeCmd(t, "-f", buildProjectPath, "build", "--progress", "plain"), - &stdout, - &stderr, - ) - assert.NilError(t, err) - processDone := make(chan error, 1) - go func() { - defer close(processDone) - processDone <- cmd.Wait() - }() - - c.WaitForCondition(t, func() (bool, string) { - out := stdout.String() - errors := stderr.String() - return strings.Contains(out, - "RUN sleep infinity"), fmt.Sprintf("'RUN sleep infinity' not found in : \n%s\nStderr: \n%s\n", out, - errors) - }, 30*time.Second, 1*time.Second) - - // simulate Ctrl-C : send signal to processGroup, children will have same groupId by default - err = syscall.Kill(-cmd.Process.Pid, syscall.SIGINT) - assert.NilError(t, err) - - select { - case <-ctx.Done(): - t.Fatal("test context canceled") - case err := <-processDone: - // TODO(milas): Compose should really not return exit code 130 here, - // this is an old hack for the compose-cli wrapper - assert.Error(t, err, "exit status 130", - "STDOUT:\n%s\nSTDERR:\n%s\n", stdout.String(), stderr.String()) - case <-time.After(10 * time.Second): - t.Fatal("timeout waiting for Compose exit") - } - }) -} - -func StartWithNewGroupID(ctx context.Context, command icmd.Cmd, stdout *utils.SafeBuffer, stderr *utils.SafeBuffer) (*exec.Cmd, error) { - cmd := exec.CommandContext(ctx, command.Command[0], command.Command[1:]...) - cmd.Env = command.Env - cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} - if stdout != nil { - cmd.Stdout = stdout - } - if stderr != nil { - cmd.Stderr = stderr - } - err := cmd.Start() - return cmd, err -} diff --git a/pkg/e2e/cascade_test.go b/pkg/e2e/cascade_test.go deleted file mode 100644 index 5bb23e1db54..00000000000 --- a/pkg/e2e/cascade_test.go +++ /dev/null @@ -1,55 +0,0 @@ -//go:build !windows - -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - - "gotest.tools/v3/assert" -) - -func TestCascadeStop(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-cascade-stop" - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/cascade/compose.yaml", "--project-name", projectName, - "up", "--abort-on-container-exit") - assert.Assert(t, strings.Contains(res.Combined(), "exit-1 exited with code 0"), res.Combined()) - // no --exit-code-from, so this is not an error - assert.Equal(t, res.ExitCode, 0) -} - -func TestCascadeFail(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-cascade-fail" - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "./fixtures/cascade/compose.yaml", "--project-name", projectName, - "up", "--abort-on-container-failure") - assert.Assert(t, strings.Contains(res.Combined(), "exit-1 exited with code 0"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "fail-1 exited with code 111"), res.Combined()) - // failing exit code should be propagated - assert.Equal(t, res.ExitCode, 111) -} diff --git a/pkg/e2e/commit_test.go b/pkg/e2e/commit_test.go deleted file mode 100644 index 0daf130545b..00000000000 --- a/pkg/e2e/commit_test.go +++ /dev/null @@ -1,93 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" -) - -func TestCommit(t *testing.T) { - const projectName = "e2e-commit-service" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/commit/compose.yaml", "--project-name", projectName, "up", "-d", "service") - - c.RunDockerComposeCmd( - t, - "--project-name", - projectName, - "commit", - "-a", - "John Hannibal Smith <hannibal@a-team.com>", - "-c", - "ENV DEBUG=true", - "-m", - "sample commit", - "service", - "service:latest", - ) -} - -func TestCommitWithReplicas(t *testing.T) { - const projectName = "e2e-commit-service-with-replicas" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/commit/compose.yaml", "--project-name", projectName, "up", "-d", "service-with-replicas") - - c.RunDockerComposeCmd( - t, - "--project-name", - projectName, - "commit", - "-a", - "John Hannibal Smith <hannibal@a-team.com>", - "-c", - "ENV DEBUG=true", - "-m", - "sample commit", - "--index=1", - "service-with-replicas", - "service-with-replicas:1", - ) - c.RunDockerComposeCmd( - t, - "--project-name", - projectName, - "commit", - "-a", - "John Hannibal Smith <hannibal@a-team.com>", - "-c", - "ENV DEBUG=true", - "-m", - "sample commit", - "--index=2", - "service-with-replicas", - "service-with-replicas:2", - ) -} diff --git a/pkg/e2e/compose_environment_test.go b/pkg/e2e/compose_environment_test.go deleted file mode 100644 index 62d6c9ebc5e..00000000000 --- a/pkg/e2e/compose_environment_test.go +++ /dev/null @@ -1,245 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestEnvPriority(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("up", func(t *testing.T) { - c.RunDockerOrExitError(t, "rmi", "env-compose-priority") - c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose-with-env.yaml", - "up", "-d", "--build") - }) - - // Full options activated - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From OS Environment) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("compose file priority", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose-with-env.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - cmd.Env = append(cmd.Env, "WHEREAMI=shell") - res := icmd.RunCmd(cmd) - assert.Equal(t, strings.TrimSpace(res.Stdout()), "shell") - }) - - // Full options activated - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("compose file priority", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose-with-env.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override", - "run", "--rm", "-e", "WHEREAMI=shell", "env-compose-priority") - res := icmd.RunCmd(cmd) - assert.Equal(t, strings.TrimSpace(res.Stdout()), "shell") - }) - - // No Compose file, all other options - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From OS Environment) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("shell priority", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - cmd.Env = append(cmd.Env, "WHEREAMI=shell") - res := icmd.RunCmd(cmd) - assert.Equal(t, strings.TrimSpace(res.Stdout()), "shell") - }) - - // No Compose file, all other options with env variable from OS environment - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From environment) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("shell priority file with default value", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override.with.default", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - cmd.Env = append(cmd.Env, "WHEREAMI=shell") - res := icmd.RunCmd(cmd) - assert.Equal(t, strings.TrimSpace(res.Stdout()), "shell") - }) - - // No Compose file, all other options with env variable from OS environment - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From environment default value from file in --env-file) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("shell priority implicitly set", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override.with.default", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - res := icmd.RunCmd(cmd) - assert.Equal(t, strings.TrimSpace(res.Stdout()), "EnvFileDefaultValue") - }) - - // No Compose file, all other options with env variable from OS environment - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From environment default value from file in COMPOSE_ENV_FILES) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("shell priority from COMPOSE_ENV_FILES variable", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - cmd.Env = append(cmd.Env, "COMPOSE_ENV_FILES=./fixtures/environment/env-priority/.env.override.with.default") - res := icmd.RunCmd(cmd) - stdout := res.Stdout() - assert.Equal(t, strings.TrimSpace(stdout), "EnvFileDefaultValue") - }) - - // No Compose file and env variable pass to the run command - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("shell priority from run command", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override", - "run", "--rm", "-e", "WHEREAMI=shell-run", "env-compose-priority") - assert.Equal(t, strings.TrimSpace(res.Stdout()), "shell-run") - }) - - // No Compose file & no env variable but override env file - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From environment patched by .env as a default --env-file value) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("override env file from compose", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose-with-env-file.yaml", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - assert.Equal(t, strings.TrimSpace(res.Stdout()), "Env File") - }) - - // No Compose file & no env variable but override by default env file - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From environment patched by --env-file value) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("override env file", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - assert.Equal(t, strings.TrimSpace(res.Stdout()), "override") - }) - - // No Compose file & no env variable but override env file - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From environment patched by --env-file value) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("env file", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - assert.Equal(t, strings.TrimSpace(res.Stdout()), "Env File") - }) - - // No Compose file & no env variable, using an empty override env file - // 1. Command Line (docker compose run --env <KEY[=VAL]>) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive <-- Result expected - // 5. Variable is not defined - t.Run("use Dockerfile", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.empty", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - assert.Equal(t, strings.TrimSpace(res.Stdout()), "Dockerfile") - }) - - t.Run("down", func(t *testing.T) { - c.RunDockerComposeCmd(t, "--project-name", "env-priority", "down") - }) -} - -func TestEnvInterpolation(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("shell priority from run command", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-interpolation/compose.yaml", "config") - cmd.Env = append(cmd.Env, "WHEREAMI=shell") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{Out: `IMAGE: default_env:shell`}) - }) - - t.Run("shell priority from run command using default value fallback", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-interpolation-default-value/compose.yaml", "config"). - Assert(t, icmd.Expected{Out: `IMAGE: default_env:EnvFileDefaultValue`}) - }) -} - -func TestCommentsInEnvFile(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("comments in env files", func(t *testing.T) { - c.RunDockerOrExitError(t, "rmi", "env-file-comments") - - c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-file-comments/compose.yaml", "up", "-d", "--build") - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-file-comments/compose.yaml", - "run", "--rm", "-e", "COMMENT", "-e", "NO_COMMENT", "env-file-comments") - - res.Assert(t, icmd.Expected{Out: `COMMENT=1234`}) - res.Assert(t, icmd.Expected{Out: `NO_COMMENT=1234#5`}) - - c.RunDockerComposeCmd(t, "--project-name", "env-file-comments", "down", "--rmi", "all") - }) -} - -func TestUnsetEnv(t *testing.T) { - c := NewParallelCLI(t) - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", "empty-variable", "down", "--rmi", "all") - }) - - t.Run("override env variable", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/empty-variable/compose.yaml", "build") - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/empty-variable/compose.yaml", - "run", "-e", "EMPTY=hello", "--rm", "empty-variable") - res.Assert(t, icmd.Expected{Out: `=hello=`}) - }) - - t.Run("unset env variable", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/empty-variable/compose.yaml", - "run", "--rm", "empty-variable") - res.Assert(t, icmd.Expected{Out: `==`}) - }) -} diff --git a/pkg/e2e/compose_exec_test.go b/pkg/e2e/compose_exec_test.go deleted file mode 100644 index 92b7e7ac74b..00000000000 --- a/pkg/e2e/compose_exec_test.go +++ /dev/null @@ -1,94 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestLocalComposeExec(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "compose-e2e-exec" - - cmdArgs := func(cmd string, args ...string) []string { - ret := []string{"--project-directory", "fixtures/simple-composefile", "--project-name", projectName, cmd} - ret = append(ret, args...) - return ret - } - - cleanup := func() { - c.RunDockerComposeCmd(t, cmdArgs("down", "--timeout=0")...) - } - cleanup() - t.Cleanup(cleanup) - - c.RunDockerComposeCmd(t, cmdArgs("up", "-d")...) - - t.Run("exec true", func(t *testing.T) { - c.RunDockerComposeCmd(t, cmdArgs("exec", "simple", "/bin/true")...) - }) - - t.Run("exec false", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, cmdArgs("exec", "simple", "/bin/false")...) - res.Assert(t, icmd.Expected{ExitCode: 1}) - }) - - t.Run("exec with env set", func(t *testing.T) { - res := icmd.RunCmd(c.NewDockerComposeCmd(t, cmdArgs("exec", "-e", "FOO", "simple", "/usr/bin/env")...), - func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "FOO=BAR") - }) - res.Assert(t, icmd.Expected{Out: "FOO=BAR"}) - }) - - t.Run("exec without env set", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, cmdArgs("exec", "-e", "FOO", "simple", "/usr/bin/env")...) - assert.Check(t, !strings.Contains(res.Stdout(), "FOO="), res.Combined()) - }) -} - -func TestLocalComposeExecOneOff(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "compose-e2e-exec-one-off" - defer c.cleanupWithDown(t, projectName) - cmdArgs := func(cmd string, args ...string) []string { - ret := []string{"--project-directory", "fixtures/simple-composefile", "--project-name", projectName, cmd} - ret = append(ret, args...) - return ret - } - - c.RunDockerComposeCmd(t, cmdArgs("run", "-d", "simple")...) - - t.Run("exec in one-off container", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, cmdArgs("exec", "-e", "FOO", "simple", "/usr/bin/env")...) - assert.Check(t, !strings.Contains(res.Stdout(), "FOO="), res.Combined()) - }) - - t.Run("exec with index", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, cmdArgs("exec", "--index", "1", "-e", "FOO", "simple", "/usr/bin/env")...) - res.Assert(t, icmd.Expected{ExitCode: 1, Err: "service \"simple\" is not running container #1"}) - }) - cmdResult := c.RunDockerCmd(t, "ps", "-q", "--filter", "label=com.docker.compose.project=compose-e2e-exec-one-off").Stdout() - containerIDs := strings.Split(cmdResult, "\n") - _ = c.RunDockerOrExitError(t, append([]string{"stop"}, containerIDs...)...) -} diff --git a/pkg/e2e/compose_run_build_once_test.go b/pkg/e2e/compose_run_build_once_test.go deleted file mode 100644 index f9726bb3b31..00000000000 --- a/pkg/e2e/compose_run_build_once_test.go +++ /dev/null @@ -1,100 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "crypto/rand" - "encoding/hex" - "fmt" - "regexp" - "strings" - "testing" - - "gotest.tools/v3/assert" -) - -// TestRunBuildOnce tests that services with pull_policy: build are only built once -// when using 'docker compose run', even when they are dependencies. -// This addresses a bug where dependencies were built twice: once in startDependencies -// and once in ensureImagesExists. -func TestRunBuildOnce(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("dependency with pull_policy build is built only once", func(t *testing.T) { - projectName := randomProjectName("build-once") - _ = c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once.yaml", "down", "--rmi", "local", "--remove-orphans", "-v") - res := c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once.yaml", "--verbose", "run", "--build", "--rm", "curl") - - output := res.Stdout() - - nginxBuilds := countServiceBuilds(output, projectName, "nginx") - - assert.Equal(t, nginxBuilds, 1, "nginx should build once, built %d times\nOutput:\n%s", nginxBuilds, output) - assert.Assert(t, strings.Contains(res.Stdout(), "curl service")) - - c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once.yaml", "down", "--remove-orphans") - }) - - t.Run("nested dependencies build only once each", func(t *testing.T) { - projectName := randomProjectName("build-nested") - _ = c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once-nested.yaml", "down", "--rmi", "local", "--remove-orphans", "-v") - res := c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once-nested.yaml", "--verbose", "run", "--build", "--rm", "app") - - output := res.Stdout() - - dbBuilds := countServiceBuilds(output, projectName, "db") - apiBuilds := countServiceBuilds(output, projectName, "api") - appBuilds := countServiceBuilds(output, projectName, "app") - - assert.Equal(t, dbBuilds, 1, "db should build once, built %d times\nOutput:\n%s", dbBuilds, output) - assert.Equal(t, apiBuilds, 1, "api should build once, built %d times\nOutput:\n%s", apiBuilds, output) - assert.Equal(t, appBuilds, 1, "app should build once, built %d times\nOutput:\n%s", appBuilds, output) - assert.Assert(t, strings.Contains(output, "App running")) - - c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once-nested.yaml", "down", "--rmi", "local", "--remove-orphans", "-v") - }) - - t.Run("service with no dependencies builds once", func(t *testing.T) { - projectName := randomProjectName("build-simple") - _ = c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once-no-deps.yaml", "down", "--rmi", "local", "--remove-orphans") - res := c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once-no-deps.yaml", "run", "--build", "--rm", "simple") - - output := res.Stdout() - - simpleBuilds := countServiceBuilds(output, projectName, "simple") - - assert.Equal(t, simpleBuilds, 1, "simple should build once, built %d times\nOutput:\n%s", simpleBuilds, output) - assert.Assert(t, strings.Contains(res.Stdout(), "Simple service")) - - c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once-no-deps.yaml", "down", "--remove-orphans") - }) -} - -// countServiceBuilds counts how many times a service was built by matching -// the "naming to *{projectName}-{serviceName}* done" pattern in the output -func countServiceBuilds(output, projectName, serviceName string) int { - pattern := regexp.MustCompile(`naming to .*` + regexp.QuoteMeta(projectName) + `-` + regexp.QuoteMeta(serviceName) + `.* done`) - return len(pattern.FindAllString(output, -1)) -} - -// randomProjectName generates a unique project name for parallel test execution -// Format: prefix-<8 random hex chars> (e.g., "build-once-3f4a9b2c") -func randomProjectName(prefix string) string { - b := make([]byte, 4) // 4 bytes = 8 hex chars - rand.Read(b) //nolint:errcheck - return fmt.Sprintf("%s-%s", prefix, hex.EncodeToString(b)) -} diff --git a/pkg/e2e/compose_run_test.go b/pkg/e2e/compose_run_test.go deleted file mode 100644 index 7ee2313aa2b..00000000000 --- a/pkg/e2e/compose_run_test.go +++ /dev/null @@ -1,281 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "os" - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestLocalComposeRun(t *testing.T) { - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "run-test") - - t.Run("compose run", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "back") - lines := Lines(res.Stdout()) - assert.Equal(t, lines[len(lines)-1], "Hello there!!", res.Stdout()) - assert.Assert(t, !strings.Contains(res.Combined(), "orphan")) - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "back", "echo", - "Hello one more time") - lines = Lines(res.Stdout()) - assert.Equal(t, lines[len(lines)-1], "Hello one more time", res.Stdout()) - assert.Assert(t, strings.Contains(res.Combined(), "orphan")) - }) - - t.Run("check run container exited", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps", "--all") - lines := Lines(res.Stdout()) - var runContainerID string - var truncatedSlug string - for _, line := range lines { - fields := strings.Fields(line) - containerID := fields[len(fields)-1] - assert.Assert(t, !strings.HasPrefix(containerID, "run-test-front")) - if strings.HasPrefix(containerID, "run-test-back") { - // only the one-off container for back service - assert.Assert(t, strings.HasPrefix(containerID, "run-test-back-run-"), containerID) - truncatedSlug = strings.Replace(containerID, "run-test-back-run-", "", 1) - runContainerID = containerID - } - if strings.HasPrefix(containerID, "run-test-db-1") { - assert.Assert(t, strings.Contains(line, "Up"), line) - } - } - assert.Assert(t, runContainerID != "") - res = c.RunDockerCmd(t, "inspect", runContainerID) - res.Assert(t, icmd.Expected{Out: ` "Status": "exited"`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.project": "run-test"`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.oneoff": "True",`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.slug": "` + truncatedSlug}) - }) - - t.Run("compose run --rm", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "--rm", "back", "echo", - "Hello again") - lines := Lines(res.Stdout()) - assert.Equal(t, lines[len(lines)-1], "Hello again", res.Stdout()) - - res = c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, strings.Contains(res.Stdout(), "run-test-back"), res.Stdout()) - }) - - t.Run("down", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "down", "--remove-orphans") - res := c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, !strings.Contains(res.Stdout(), "run-test"), res.Stdout()) - }) - - t.Run("compose run --volumes", func(t *testing.T) { - wd, err := os.Getwd() - assert.NilError(t, err) - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "--volumes", wd+":/foo", - "back", "/bin/sh", "-c", "ls /foo") - res.Assert(t, icmd.Expected{Out: "compose_run_test.go"}) - - res = c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, strings.Contains(res.Stdout(), "run-test-back"), res.Stdout()) - }) - - t.Run("compose run --publish", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/ports.yaml", "run", "--publish", "8081:80", "-d", "back", - "/bin/sh", "-c", "sleep 1") - res := c.RunDockerCmd(t, "ps") - assert.Assert(t, strings.Contains(res.Stdout(), "8081->80/tcp"), res.Stdout()) - assert.Assert(t, !strings.Contains(res.Stdout(), "8082->80/tcp"), res.Stdout()) - }) - - t.Run("compose run --service-ports", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/ports.yaml", "run", "--service-ports", "-d", "back", - "/bin/sh", "-c", "sleep 1") - res := c.RunDockerCmd(t, "ps") - assert.Assert(t, strings.Contains(res.Stdout(), "8082->80/tcp"), res.Stdout()) - }) - - t.Run("compose run orphan", func(t *testing.T) { - // Use different compose files to get an orphan container - c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/orphan.yaml", "run", "simple") - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "back", "echo", "Hello") - assert.Assert(t, strings.Contains(res.Combined(), "orphan")) - - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "back", "echo", "Hello") - res = icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "COMPOSE_IGNORE_ORPHANS=True") - }) - assert.Assert(t, !strings.Contains(res.Combined(), "orphan")) - }) - - t.Run("down", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "down") - icmd.RunCmd(cmd, func(c *icmd.Cmd) { - c.Env = append(c.Env, "COMPOSE_REMOVE_ORPHANS=True") - }) - res := c.RunDockerCmd(t, "ps", "--all") - - assert.Assert(t, !strings.Contains(res.Stdout(), "run-test"), res.Stdout()) - }) - - t.Run("run starts only container and dependencies", func(t *testing.T) { - // ensure that even if another service is up run does not start it: https://github.com/docker/compose/issues/9459 - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/deps.yaml", "up", "service_b", "--menu=false") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/deps.yaml", "run", "service_a") - assert.Assert(t, strings.Contains(res.Combined(), "shared_dep"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "service_b"), res.Combined()) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/deps.yaml", "down", "--remove-orphans") - }) - - t.Run("run without dependencies", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/deps.yaml", "run", "--no-deps", "service_a") - assert.Assert(t, !strings.Contains(res.Combined(), "shared_dep"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "service_b"), res.Combined()) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/deps.yaml", "down", "--remove-orphans") - }) - - t.Run("run with not required dependency", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/dependencies/deps-not-required.yaml", "run", "foo") - assert.Assert(t, strings.Contains(res.Combined(), "foo"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "bar"), res.Combined()) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/dependencies/deps-not-required.yaml", "down", "--remove-orphans") - }) - - t.Run("--quiet-pull", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/quiet-pull.yaml", "down", "--remove-orphans", "--rmi", "all") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/quiet-pull.yaml", "run", "--quiet-pull", "backend") - assert.Assert(t, !strings.Contains(res.Combined(), "Pull complete"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "Pulled"), res.Combined()) - }) - - t.Run("COMPOSE_PROGRESS quiet", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/quiet-pull.yaml", "down", "--remove-orphans", "--rmi", "all") - res.Assert(t, icmd.Success) - - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/run-test/quiet-pull.yaml", "run", "backend") - res = icmd.RunCmd(cmd, func(c *icmd.Cmd) { - c.Env = append(c.Env, "COMPOSE_PROGRESS=quiet") - }) - assert.Assert(t, !strings.Contains(res.Combined(), "Pull complete"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "Pulled"), res.Combined()) - }) - - t.Run("--pull", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/pull.yaml", "down", "--remove-orphans", "--rmi", "all") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/pull.yaml", "run", "--pull", "always", "backend") - assert.Assert(t, strings.Contains(res.Combined(), "Image nginx Pulling"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "Image nginx Pulled"), res.Combined()) - }) - - t.Run("compose run --env-from-file", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "--env-from-file", "./fixtures/run-test/run.env", - "front", "env") - res.Assert(t, icmd.Expected{Out: "FOO=BAR"}) - }) - - t.Run("compose run -rm with stop signal", func(t *testing.T) { - projectName := "run-test" - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "-f", "./fixtures/ps-test/compose.yaml", "run", "--rm", "-d", "nginx") - res.Assert(t, icmd.Success) - - res = c.RunDockerCmd(t, "ps", "--quiet", "--filter", "name=run-test-nginx") - containerID := strings.TrimSpace(res.Stdout()) - - res = c.RunDockerCmd(t, "stop", containerID) - res.Assert(t, icmd.Success) - res = c.RunDockerCmd(t, "ps", "--all", "--filter", "name=run-test-nginx", "--format", "'{{.Names}}'") - assert.Assert(t, !strings.Contains(res.Stdout(), "run-test-nginx"), res.Stdout()) - }) - - t.Run("compose run --env", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "--env", "FOO=BAR", - "front", "env") - res.Assert(t, icmd.Expected{Out: "FOO=BAR"}) - }) - - t.Run("compose run --build", func(t *testing.T) { - c.cleanupWithDown(t, "run-test", "--rmi=local") - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "build", "echo", "hello world") - res.Assert(t, icmd.Expected{Out: "hello world"}) - }) - - t.Run("compose run with piped input detection", func(t *testing.T) { - if composeStandaloneMode { - t.Skip("Skipping test compose with piped input detection in standalone mode") - } - // Test that piped input is properly detected and TTY is automatically disabled - // This tests the logic added in run.go that checks dockerCli.In().IsTerminal() - cmd := c.NewCmd("sh", "-c", "echo 'piped-content' | docker compose -f ./fixtures/run-test/piped-test.yaml run --rm piped-test") - res := icmd.RunCmd(cmd) - - res.Assert(t, icmd.Expected{Out: "piped-content"}) - res.Assert(t, icmd.Success) - }) - - t.Run("compose run piped input should not allocate TTY", func(t *testing.T) { - if composeStandaloneMode { - t.Skip("Skipping test compose with piped input detection in standalone mode") - } - // Test that when stdin is piped, the container correctly detects no TTY - // This verifies that the automatic noTty=true setting works correctly - cmd := c.NewCmd("sh", "-c", "echo '' | docker compose -f ./fixtures/run-test/piped-test.yaml run --rm tty-test") - res := icmd.RunCmd(cmd) - - res.Assert(t, icmd.Expected{Out: "No TTY detected"}) - res.Assert(t, icmd.Success) - }) - - t.Run("compose run piped input with explicit --tty should fail", func(t *testing.T) { - if composeStandaloneMode { - t.Skip("Skipping test compose with piped input detection in standalone mode") - } - // Test that explicitly requesting TTY with piped input fails with proper error message - // This should trigger the "input device is not a TTY" error - cmd := c.NewCmd("sh", "-c", "echo 'test' | docker compose -f ./fixtures/run-test/piped-test.yaml run --rm --tty piped-test") - res := icmd.RunCmd(cmd) - - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "the input device is not a TTY", - }) - }) - - t.Run("compose run piped input with --no-TTY=false should fail", func(t *testing.T) { - if composeStandaloneMode { - t.Skip("Skipping test compose with piped input detection in standalone mode") - } - // Test that explicitly disabling --no-TTY (i.e., requesting TTY) with piped input fails - // This should also trigger the "input device is not a TTY" error - cmd := c.NewCmd("sh", "-c", "echo 'test' | docker compose -f ./fixtures/run-test/piped-test.yaml run --rm --no-TTY=false piped-test") - res := icmd.RunCmd(cmd) - - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "the input device is not a TTY", - }) - }) -} diff --git a/pkg/e2e/compose_test.go b/pkg/e2e/compose_test.go deleted file mode 100644 index 3b5a7341a70..00000000000 --- a/pkg/e2e/compose_test.go +++ /dev/null @@ -1,408 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "net/http" - "os" - "path/filepath" - "strings" - "testing" - "time" - - testify "github.com/stretchr/testify/assert" - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestLocalComposeUp(t *testing.T) { - // this test shares a fixture with TestCompatibility and can't run at the same time - c := NewCLI(t) - - const projectName = "compose-e2e-demo" - - t.Run("up", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/sentences/compose.yaml", "--project-name", projectName, "up", "-d") - }) - - t.Run("check accessing running app", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-p", projectName, "ps") - res.Assert(t, icmd.Expected{Out: `web`}) - - endpoint := "http://localhost:90" - output := HTTPGetWithRetry(t, endpoint+"/words/noun", http.StatusOK, 2*time.Second, 20*time.Second) - assert.Assert(t, strings.Contains(output, `"word":`)) - - res = c.RunDockerCmd(t, "network", "ls") - res.Assert(t, icmd.Expected{Out: projectName + "_default"}) - }) - - t.Run("top", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-p", projectName, "top") - output := res.Stdout() - head := []string{"UID", "PID", "PPID", "C", "STIME", "TTY", "TIME", "CMD"} - for _, h := range head { - assert.Assert(t, strings.Contains(output, h), output) - } - assert.Assert(t, strings.Contains(output, `java -Xmx8m -Xms8m -jar /app/words.jar`), output) - assert.Assert(t, strings.Contains(output, `/dispatcher`), output) - }) - - t.Run("check compose labels", func(t *testing.T) { - res := c.RunDockerCmd(t, "inspect", projectName+"-web-1") - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.container-number": "1"`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.project": "compose-e2e-demo"`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.oneoff": "False",`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.config-hash":`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.project.config_files":`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.project.working_dir":`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.service": "web"`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.version":`}) - - res = c.RunDockerCmd(t, "network", "inspect", projectName+"_default") - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.network": "default"`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.project": `}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.version": `}) - }) - - t.Run("check user labels", func(t *testing.T) { - res := c.RunDockerCmd(t, "inspect", projectName+"-web-1") - res.Assert(t, icmd.Expected{Out: `"my-label": "test"`}) - }) - - t.Run("check healthcheck output", func(t *testing.T) { - c.WaitForCmdResult(t, c.NewDockerComposeCmd(t, "-p", projectName, "ps", "--format", "json"), - IsHealthy(projectName+"-web-1"), - 5*time.Second, 1*time.Second) - - res := c.RunDockerComposeCmd(t, "-p", projectName, "ps") - assertServiceStatus(t, projectName, "web", "(healthy)", res.Stdout()) - }) - - t.Run("images", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-p", projectName, "images") - res.Assert(t, icmd.Expected{Out: `compose-e2e-demo-db-1 gtardif/sentences-db latest`}) - res.Assert(t, icmd.Expected{Out: `compose-e2e-demo-web-1 gtardif/sentences-web latest`}) - res.Assert(t, icmd.Expected{Out: `compose-e2e-demo-words-1 gtardif/sentences-api latest`}) - }) - - t.Run("down SERVICE", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "web") - - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "ps") - assert.Assert(t, !strings.Contains(res.Combined(), "compose-e2e-demo-web-1"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "compose-e2e-demo-db-1"), res.Combined()) - }) - - t.Run("down", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - t.Run("check containers after down", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, !strings.Contains(res.Combined(), projectName), res.Combined()) - }) - - t.Run("check networks after down", func(t *testing.T) { - res := c.RunDockerCmd(t, "network", "ls") - assert.Assert(t, !strings.Contains(res.Combined(), projectName), res.Combined()) - }) -} - -func TestDownComposefileInParentFolder(t *testing.T) { - c := NewParallelCLI(t) - - tmpFolder, err := os.MkdirTemp("fixtures/simple-composefile", "test-tmp") - assert.NilError(t, err) - defer os.Remove(tmpFolder) //nolint:errcheck - projectName := filepath.Base(tmpFolder) - - res := c.RunDockerComposeCmd(t, "--project-directory", tmpFolder, "up", "-d") - res.Assert(t, icmd.Expected{Err: "Started", ExitCode: 0}) - - res = c.RunDockerComposeCmd(t, "-p", projectName, "down") - res.Assert(t, icmd.Expected{Err: "Removed", ExitCode: 0}) -} - -func TestAttachRestart(t *testing.T) { - t.Skip("Skipping test until we can fix it") - - if _, ok := os.LookupEnv("CI"); ok { - t.Skip("Skipping test on CI... flaky") - } - c := NewParallelCLI(t) - - cmd := c.NewDockerComposeCmd(t, "--ansi=never", "--project-directory", "./fixtures/attach-restart", "up") - res := icmd.StartCmd(cmd) - defer c.RunDockerComposeCmd(t, "-p", "attach-restart", "down") - - c.WaitForCondition(t, func() (bool, string) { - debug := res.Combined() - return strings.Count(res.Stdout(), - "failing-1 exited with code 1") == 3, fmt.Sprintf("'failing-1 exited with code 1' not found 3 times in : \n%s\n", - debug) - }, 4*time.Minute, 2*time.Second) - - assert.Equal(t, strings.Count(res.Stdout(), "failing-1 | world"), 3, res.Combined()) -} - -func TestInitContainer(t *testing.T) { - c := NewParallelCLI(t) - - res := c.RunDockerComposeCmd(t, "--ansi=never", "--project-directory", "./fixtures/init-container", "up", "--menu=false") - defer c.RunDockerComposeCmd(t, "-p", "init-container", "down") - testify.Regexp(t, "foo-1 | hello(?m:.*)bar-1 | world", res.Stdout()) -} - -func TestRm(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "compose-e2e-rm" - - t.Run("up", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-composefile/compose.yaml", "-p", projectName, "up", "-d") - }) - - t.Run("rm --stop --force simple", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-composefile/compose.yaml", "-p", projectName, "rm", - "--stop", "--force", "simple") - res.Assert(t, icmd.Expected{Err: "Removed", ExitCode: 0}) - }) - - t.Run("check containers after rm", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, !strings.Contains(res.Combined(), projectName+"-simple"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), projectName+"-another"), res.Combined()) - }) - - t.Run("up (again)", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-composefile/compose.yaml", "-p", projectName, "up", "-d") - }) - - t.Run("rm ---stop --force <none>", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-composefile/compose.yaml", "-p", projectName, "rm", - "--stop", "--force") - res.Assert(t, icmd.Expected{ExitCode: 0}) - }) - - t.Run("check containers after rm", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, !strings.Contains(res.Combined(), projectName+"-simple"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), projectName+"-another"), res.Combined()) - }) - - t.Run("down", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-p", projectName, "down") - }) -} - -func TestCompatibility(t *testing.T) { - // this test shares a fixture with TestLocalComposeUp and can't run at the same time - c := NewCLI(t) - - const projectName = "compose-e2e-compatibility" - - t.Run("up", func(t *testing.T) { - c.RunDockerComposeCmd(t, "--compatibility", "-f", "./fixtures/sentences/compose.yaml", "--project-name", - projectName, "up", "-d") - }) - - t.Run("check container names", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps", "--format", "{{.Names}}") - res.Assert(t, icmd.Expected{Out: "compose-e2e-compatibility_web_1"}) - res.Assert(t, icmd.Expected{Out: "compose-e2e-compatibility_words_1"}) - res.Assert(t, icmd.Expected{Out: "compose-e2e-compatibility_db_1"}) - }) - - t.Run("down", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-p", projectName, "down") - }) -} - -func TestConfig(t *testing.T) { - const projectName = "compose-e2e-config" - c := NewParallelCLI(t) - - wd, err := os.Getwd() - assert.NilError(t, err) - - t.Run("up", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-build-test/compose.yaml", "-p", projectName, "config") - res.Assert(t, icmd.Expected{Out: fmt.Sprintf(`name: %s -services: - nginx: - build: - context: %s - dockerfile: Dockerfile - networks: - default: null -networks: - default: - name: compose-e2e-config_default -`, projectName, filepath.Join(wd, "fixtures", "simple-build-test", "nginx-build")), ExitCode: 0}) - }) -} - -func TestConfigInterpolate(t *testing.T) { - const projectName = "compose-e2e-config-interpolate" - c := NewParallelCLI(t) - - wd, err := os.Getwd() - assert.NilError(t, err) - - t.Run("config", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-build-test/compose-interpolate.yaml", "-p", projectName, "config", "--no-interpolate") - res.Assert(t, icmd.Expected{Out: fmt.Sprintf(`name: %s -networks: - default: - name: compose-e2e-config-interpolate_default -services: - nginx: - build: - context: %s - dockerfile: ${MYVAR} - networks: - default: null -`, projectName, filepath.Join(wd, "fixtures", "simple-build-test", "nginx-build")), ExitCode: 0}) - }) -} - -func TestStopWithDependenciesAttached(t *testing.T) { - const projectName = "compose-e2e-stop-with-deps" - c := NewParallelCLI(t, WithEnv("COMMAND=echo hello")) - - cleanup := func() { - c.RunDockerComposeCmd(t, "-p", projectName, "down", "--remove-orphans", "--timeout=0") - } - cleanup() - t.Cleanup(cleanup) - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/dependencies/compose.yaml", "-p", projectName, "up", "--attach-dependencies", "foo", "--menu=false") - res.Assert(t, icmd.Expected{Out: "exited with code 0"}) -} - -func TestRemoveOrphaned(t *testing.T) { - const projectName = "compose-e2e-remove-orphaned" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "-p", projectName, "down", "--remove-orphans", "--timeout=0") - } - cleanup() - t.Cleanup(cleanup) - - // run stack - c.RunDockerComposeCmd(t, "-f", "./fixtures/sentences/compose.yaml", "-p", projectName, "up", "-d") - - // down "web" service with orphaned removed - c.RunDockerComposeCmd(t, "-f", "./fixtures/sentences/compose.yaml", "-p", projectName, "down", "--remove-orphans", "web") - - // check "words" service has not been considered orphaned - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/sentences/compose.yaml", "-p", projectName, "ps", "--format", "{{.Name}}") - res.Assert(t, icmd.Expected{Out: fmt.Sprintf("%s-words-1", projectName)}) -} - -func TestComposeFileSetByDotEnv(t *testing.T) { - c := NewCLI(t) - defer c.cleanupWithDown(t, "dotenv") - - cmd := c.NewDockerComposeCmd(t, "config") - cmd.Dir = filepath.Join(".", "fixtures", "dotenv") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ - ExitCode: 0, - Out: "image: test:latest", - }) - res.Assert(t, icmd.Expected{ - Out: "image: enabled:profile", - }) -} - -func TestComposeFileSetByProjectDirectory(t *testing.T) { - c := NewCLI(t) - defer c.cleanupWithDown(t, "dotenv") - - dir := filepath.Join(".", "fixtures", "dotenv", "development") - cmd := c.NewDockerComposeCmd(t, "--project-directory", dir, "config") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ - ExitCode: 0, - Out: "image: backend:latest", - }) -} - -func TestComposeFileSetByEnvFile(t *testing.T) { - c := NewCLI(t) - defer c.cleanupWithDown(t, "dotenv") - - dotEnv, err := os.CreateTemp(t.TempDir(), ".env") - assert.NilError(t, err) - err = os.WriteFile(dotEnv.Name(), []byte(` -COMPOSE_FILE=fixtures/dotenv/development/compose.yaml -IMAGE_NAME=test -IMAGE_TAG=latest -COMPOSE_PROFILES=test -`), 0o700) - assert.NilError(t, err) - - cmd := c.NewDockerComposeCmd(t, "--env-file", dotEnv.Name(), "config") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ - Out: "image: test:latest", - }) - res.Assert(t, icmd.Expected{ - Out: "image: enabled:profile", - }) -} - -func TestNestedDotEnv(t *testing.T) { - c := NewCLI(t) - defer c.cleanupWithDown(t, "nested") - - cmd := c.NewDockerComposeCmd(t, "run", "echo") - cmd.Dir = filepath.Join(".", "fixtures", "nested") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ - ExitCode: 0, - Out: "root win=root", - }) - - cmd = c.NewDockerComposeCmd(t, "run", "echo") - cmd.Dir = filepath.Join(".", "fixtures", "nested", "sub") - defer c.cleanupWithDown(t, "nested") - res = icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ - ExitCode: 0, - Out: "root sub win=sub", - }) -} - -func TestUnnecessaryResources(t *testing.T) { - const projectName = "compose-e2e-unnecessary-resources" - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, projectName) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "./fixtures/external/compose.yaml", "-p", projectName, "up", "-d") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "network foo_bar declared as external, but could not be found", - }) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/external/compose.yaml", "-p", projectName, "up", "-d", "test") - // Should not fail as missing external network is not used -} diff --git a/pkg/e2e/compose_up_test.go b/pkg/e2e/compose_up_test.go deleted file mode 100644 index b00cb3c1861..00000000000 --- a/pkg/e2e/compose_up_test.go +++ /dev/null @@ -1,114 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - "time" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestUpWait(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-deps-wait" - - timeout := time.After(30 * time.Second) - done := make(chan bool) - go func() { - //nolint:nolintlint,testifylint // helper asserts inside goroutine; acceptable in this e2e test - res := c.RunDockerComposeCmd(t, "-f", "fixtures/dependencies/deps-completed-successfully.yaml", "--project-name", projectName, "up", "--wait", "-d") - assert.Assert(t, strings.Contains(res.Combined(), "e2e-deps-wait-oneshot-1"), res.Combined()) - done <- true - }() - - select { - case <-timeout: - t.Fatal("test did not finish in time") - case <-done: - break - } - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") -} - -func TestUpExitCodeFrom(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-exit-code-from" - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/start-fail/start-depends_on-long-lived.yaml", "--project-name", projectName, "up", "--menu=false", "--exit-code-from=failure", "failure") - res.Assert(t, icmd.Expected{ExitCode: 42}) - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--remove-orphans") -} - -func TestUpExitCodeFromContainerKilled(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-exit-code-from-kill" - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/start-fail/start-depends_on-long-lived.yaml", "--project-name", projectName, "up", "--menu=false", "--exit-code-from=test") - res.Assert(t, icmd.Expected{ExitCode: 143}) - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--remove-orphans") -} - -func TestPortRange(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-port-range" - - reset := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--remove-orphans", "--timeout=0") - } - reset() - t.Cleanup(reset) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/port-range/compose.yaml", "--project-name", projectName, "up", "-d") - res.Assert(t, icmd.Success) -} - -func TestStdoutStderr(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-stdout-stderr" - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/stdout-stderr/compose.yaml", "--project-name", projectName, "up", "--menu=false") - res.Assert(t, icmd.Expected{Out: "log to stdout", Err: "log to stderr"}) - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--remove-orphans") -} - -func TestLoggingDriver(t *testing.T) { - c := NewCLI(t) - const projectName = "e2e-logging-driver" - defer c.cleanupWithDown(t, projectName) - - host := "HOST=127.0.0.1" - res := c.RunDockerCmd(t, "info", "-f", "{{.OperatingSystem}}") - os := res.Stdout() - if strings.TrimSpace(os) == "Docker Desktop" { - host = "HOST=host.docker.internal" - } - - cmd := c.NewDockerComposeCmd(t, "-f", "fixtures/logging-driver/compose.yaml", "--project-name", projectName, "up", "-d") - cmd.Env = append(cmd.Env, host, "BAR=foo") - icmd.RunCmd(cmd).Assert(t, icmd.Success) - - cmd = c.NewDockerComposeCmd(t, "-f", "fixtures/logging-driver/compose.yaml", "--project-name", projectName, "up", "-d") - cmd.Env = append(cmd.Env, host, "BAR=zot") - icmd.RunCmd(cmd).Assert(t, icmd.Success) -} diff --git a/pkg/e2e/config_test.go b/pkg/e2e/config_test.go deleted file mode 100644 index 15d3e3d932a..00000000000 --- a/pkg/e2e/config_test.go +++ /dev/null @@ -1,70 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" - - "gotest.tools/v3/icmd" -) - -func TestLocalComposeConfig(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "compose-e2e-config" - - t.Run("yaml", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/config/compose.yaml", "--project-name", projectName, "config") - res.Assert(t, icmd.Expected{Out: ` - ports: - - mode: ingress - target: 80 - published: "8080" - protocol: tcp`}) - }) - - t.Run("json", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/config/compose.yaml", "--project-name", projectName, "config", "--format", "json") - res.Assert(t, icmd.Expected{Out: `"published": "8080"`}) - }) - - t.Run("--no-interpolate", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/config/compose.yaml", "--project-name", projectName, "config", "--no-interpolate") - res.Assert(t, icmd.Expected{Out: `- ${PORT:-8080}:80`}) - }) - - t.Run("--variables --format json", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/config/compose.yaml", "--project-name", projectName, "config", "--variables", "--format", "json") - res.Assert(t, icmd.Expected{Out: `{ - "PORT": { - "Name": "PORT", - "DefaultValue": "8080", - "PresenceValue": "", - "Required": false - } -}`}) - }) - - t.Run("--variables --format yaml", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/config/compose.yaml", "--project-name", projectName, "config", "--variables", "--format", "yaml") - res.Assert(t, icmd.Expected{Out: `PORT: - name: PORT - defaultvalue: "8080" - presencevalue: "" - required: false`}) - }) -} diff --git a/pkg/e2e/configs_test.go b/pkg/e2e/configs_test.go deleted file mode 100644 index c7d86f1c765..00000000000 --- a/pkg/e2e/configs_test.go +++ /dev/null @@ -1,57 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" - - "gotest.tools/v3/icmd" -) - -func TestConfigFromEnv(t *testing.T) { - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "configs") - - t.Run("config from file", func(t *testing.T) { - res := icmd.RunCmd(c.NewDockerComposeCmd(t, "-f", "./fixtures/configs/compose.yaml", "run", "from_file")) - res.Assert(t, icmd.Expected{Out: "This is my config file"}) - }) - - t.Run("config from env", func(t *testing.T) { - res := icmd.RunCmd(c.NewDockerComposeCmd(t, "-f", "./fixtures/configs/compose.yaml", "run", "from_env"), - func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "CONFIG=config") - }) - res.Assert(t, icmd.Expected{Out: "config"}) - }) - - t.Run("config inlined", func(t *testing.T) { - res := icmd.RunCmd(c.NewDockerComposeCmd(t, "-f", "./fixtures/configs/compose.yaml", "run", "inlined"), - func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "CONFIG=config") - }) - res.Assert(t, icmd.Expected{Out: "This is my config"}) - }) - - t.Run("custom target", func(t *testing.T) { - res := icmd.RunCmd(c.NewDockerComposeCmd(t, "-f", "./fixtures/configs/compose.yaml", "run", "target"), - func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "CONFIG=config") - }) - res.Assert(t, icmd.Expected{Out: "This is my config"}) - }) -} diff --git a/pkg/e2e/container_name_test.go b/pkg/e2e/container_name_test.go deleted file mode 100644 index 99de738c237..00000000000 --- a/pkg/e2e/container_name_test.go +++ /dev/null @@ -1,43 +0,0 @@ -//go:build !windows - -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" - - "gotest.tools/v3/icmd" -) - -func TestUpContainerNameConflict(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-container_name_conflict" - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/container_name/compose.yaml", "--project-name", projectName, "up") - res.Assert(t, icmd.Expected{ExitCode: 1, Err: `container name "test" is already in use`}) - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - c.RunDockerComposeCmd(t, "-f", "fixtures/container_name/compose.yaml", "--project-name", projectName, "up", "test") - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - c.RunDockerComposeCmd(t, "-f", "fixtures/container_name/compose.yaml", "--project-name", projectName, "up", "another_test") -} diff --git a/pkg/e2e/cp_test.go b/pkg/e2e/cp_test.go deleted file mode 100644 index 4b58c8b6d26..00000000000 --- a/pkg/e2e/cp_test.go +++ /dev/null @@ -1,114 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "os" - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestCopy(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "copy_e2e" - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "--project-name", projectName, "down") - - os.Remove("./fixtures/cp-test/from-default.txt") //nolint:errcheck - os.Remove("./fixtures/cp-test/from-indexed.txt") //nolint:errcheck - os.RemoveAll("./fixtures/cp-test/cp-folder2") //nolint:errcheck - }) - - t.Run("start service", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "--project-name", projectName, "up", - "--scale", "nginx=5", "-d") - }) - - t.Run("make sure service is running", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-p", projectName, "ps") - assertServiceStatus(t, projectName, "nginx", "Up", res.Stdout()) - }) - - t.Run("copy to container copies the file to the all containers by default", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "-p", projectName, "cp", - "./fixtures/cp-test/cp-me.txt", "nginx:/tmp/default.txt") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - output := c.RunDockerCmd(t, "exec", projectName+"-nginx-1", "cat", "/tmp/default.txt").Stdout() - assert.Assert(t, strings.Contains(output, `hello world`), output) - - output = c.RunDockerCmd(t, "exec", projectName+"-nginx-2", "cat", "/tmp/default.txt").Stdout() - assert.Assert(t, strings.Contains(output, `hello world`), output) - - output = c.RunDockerCmd(t, "exec", projectName+"-nginx-3", "cat", "/tmp/default.txt").Stdout() - assert.Assert(t, strings.Contains(output, `hello world`), output) - }) - - t.Run("copy to container with a given index copies the file to the given container", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "-p", projectName, "cp", "--index=3", - "./fixtures/cp-test/cp-me.txt", "nginx:/tmp/indexed.txt") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - output := c.RunDockerCmd(t, "exec", projectName+"-nginx-3", "cat", "/tmp/indexed.txt").Stdout() - assert.Assert(t, strings.Contains(output, `hello world`), output) - - res = c.RunDockerOrExitError(t, "exec", projectName+"-nginx-2", "cat", "/tmp/indexed.txt") - res.Assert(t, icmd.Expected{ExitCode: 1}) - }) - - t.Run("copy from a container copies the file to the host from the first container by default", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "-p", projectName, "cp", - "nginx:/tmp/default.txt", "./fixtures/cp-test/from-default.txt") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - data, err := os.ReadFile("./fixtures/cp-test/from-default.txt") - assert.NilError(t, err) - assert.Equal(t, `hello world`, string(data)) - }) - - t.Run("copy from a container with a given index copies the file to host", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "-p", projectName, "cp", "--index=3", - "nginx:/tmp/indexed.txt", "./fixtures/cp-test/from-indexed.txt") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - data, err := os.ReadFile("./fixtures/cp-test/from-indexed.txt") - assert.NilError(t, err) - assert.Equal(t, `hello world`, string(data)) - }) - - t.Run("copy to and from a container also work with folder", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "-p", projectName, "cp", - "./fixtures/cp-test/cp-folder", "nginx:/tmp") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - output := c.RunDockerCmd(t, "exec", projectName+"-nginx-1", "cat", "/tmp/cp-folder/cp-me.txt").Stdout() - assert.Assert(t, strings.Contains(output, `hello world from folder`), output) - - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "-p", projectName, "cp", - "nginx:/tmp/cp-folder", "./fixtures/cp-test/cp-folder2") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - data, err := os.ReadFile("./fixtures/cp-test/cp-folder2/cp-me.txt") - assert.NilError(t, err) - assert.Equal(t, `hello world from folder`, string(data)) - }) -} diff --git a/pkg/e2e/e2e_config_plugin.go b/pkg/e2e/e2e_config_plugin.go deleted file mode 100644 index 84ca6eabcd9..00000000000 --- a/pkg/e2e/e2e_config_plugin.go +++ /dev/null @@ -1,21 +0,0 @@ -//go:build !standalone - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -const composeStandaloneMode = false diff --git a/pkg/e2e/e2e_config_standalone.go b/pkg/e2e/e2e_config_standalone.go deleted file mode 100644 index a4c4fe62943..00000000000 --- a/pkg/e2e/e2e_config_standalone.go +++ /dev/null @@ -1,21 +0,0 @@ -//go:build standalone - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -const composeStandaloneMode = true diff --git a/pkg/e2e/env_file_test.go b/pkg/e2e/env_file_test.go deleted file mode 100644 index d9ea1d4c2b6..00000000000 --- a/pkg/e2e/env_file_test.go +++ /dev/null @@ -1,53 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestRawEnvFile(t *testing.T) { - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "dotenv") - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/dotenv/raw.yaml", "run", "test") - assert.Equal(t, strings.TrimSpace(res.Stdout()), "'{\"key\": \"value\"}'") -} - -func TestUnusedMissingEnvFile(t *testing.T) { - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "unused_dotenv") - - c.RunDockerComposeCmd(t, "-f", "./fixtures/env_file/compose.yaml", "up", "-d", "serviceA") - - // Runtime operations should work even with missing env file - c.RunDockerComposeCmd(t, "-f", "./fixtures/env_file/compose.yaml", "ps") - c.RunDockerComposeCmd(t, "-f", "./fixtures/env_file/compose.yaml", "logs") - c.RunDockerComposeCmd(t, "-f", "./fixtures/env_file/compose.yaml", "down") -} - -func TestRunEnvFile(t *testing.T) { - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "run_dotenv") - - res := c.RunDockerComposeCmd(t, "--project-directory", "./fixtures/env_file", "run", "serviceC", "env") - res.Assert(t, icmd.Expected{Out: "FOO=BAR"}) -} diff --git a/pkg/e2e/exec_test.go b/pkg/e2e/exec_test.go deleted file mode 100644 index 16643a0f8a0..00000000000 --- a/pkg/e2e/exec_test.go +++ /dev/null @@ -1,47 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" - - "gotest.tools/v3/icmd" -) - -func TestExec(t *testing.T) { - const projectName = "e2e-exec" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/exec/compose.yaml", "--project-name", projectName, "run", "-d", "test", "cat") - - res := c.RunDockerComposeCmdNoCheck(t, "--project-name", projectName, "exec", "--index=1", "test", "ps") - res.Assert(t, icmd.Expected{Err: "service \"test\" is not running container #1", ExitCode: 1}) - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "exec", "test", "ps") - res.Assert(t, icmd.Expected{Out: "cat"}) // one-off container was selected - - c.RunDockerComposeCmd(t, "-f", "./fixtures/exec/compose.yaml", "--project-name", projectName, "up", "-d") - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "exec", "test", "ps") - res.Assert(t, icmd.Expected{Out: "tail"}) // service container was selected -} diff --git a/pkg/e2e/export_test.go b/pkg/e2e/export_test.go deleted file mode 100644 index baa0dc5b94c..00000000000 --- a/pkg/e2e/export_test.go +++ /dev/null @@ -1,50 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" -) - -func TestExport(t *testing.T) { - const projectName = "e2e-export-service" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/export/compose.yaml", "--project-name", projectName, "up", "-d", "service") - c.RunDockerComposeCmd(t, "--project-name", projectName, "export", "-o", "service.tar", "service") -} - -func TestExportWithReplicas(t *testing.T) { - const projectName = "e2e-export-service-with-replicas" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/export/compose.yaml", "--project-name", projectName, "up", "-d", "service-with-replicas") - c.RunDockerComposeCmd(t, "--project-name", projectName, "export", "-o", "r1.tar", "--index=1", "service-with-replicas") - c.RunDockerComposeCmd(t, "--project-name", projectName, "export", "-o", "r2.tar", "--index=2", "service-with-replicas") -} diff --git a/pkg/e2e/expose_test.go b/pkg/e2e/expose_test.go deleted file mode 100644 index 438f7b54900..00000000000 --- a/pkg/e2e/expose_test.go +++ /dev/null @@ -1,48 +0,0 @@ -//go:build !windows - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "os" - "path/filepath" - "testing" - - "gotest.tools/v3/assert" -) - -// see https://github.com/docker/compose/issues/13378 -func TestExposeRange(t *testing.T) { - c := NewParallelCLI(t) - - f := filepath.Join(t.TempDir(), "compose.yaml") - err := os.WriteFile(f, []byte(` -name: test-expose-range -services: - test: - image: alpine - expose: - - "9091-9092" -`), 0o644) - assert.NilError(t, err) - - t.Cleanup(func() { - c.cleanupWithDown(t, "test-expose-range") - }) - c.RunDockerComposeCmd(t, "-f", f, "up") -} diff --git a/pkg/e2e/fixtures/attach-restart/compose.yaml b/pkg/e2e/fixtures/attach-restart/compose.yaml deleted file mode 100644 index d92143677fd..00000000000 --- a/pkg/e2e/fixtures/attach-restart/compose.yaml +++ /dev/null @@ -1,8 +0,0 @@ -services: - failing: - image: alpine - command: sh -c "sleep 0.1 && echo world && /bin/false" - deploy: - restart_policy: - condition: "on-failure" - max_attempts: 2 diff --git a/pkg/e2e/fixtures/bridge/Dockerfile b/pkg/e2e/fixtures/bridge/Dockerfile deleted file mode 100644 index 4cdd9857779..00000000000 --- a/pkg/e2e/fixtures/bridge/Dockerfile +++ /dev/null @@ -1,18 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine -ENV ENV_FROM_DOCKERFILE=1 -EXPOSE 8081 -CMD ["echo", "Hello from Dockerfile"] diff --git a/pkg/e2e/fixtures/bridge/compose.yaml b/pkg/e2e/fixtures/bridge/compose.yaml deleted file mode 100644 index 4fbd9bd94cf..00000000000 --- a/pkg/e2e/fixtures/bridge/compose.yaml +++ /dev/null @@ -1,31 +0,0 @@ -services: - serviceA: - image: alpine - build: . - ports: - - 80:8080 - networks: - - private-network - configs: - - source: my-config - target: /etc/my-config1.txt - serviceB: - image: alpine - build: . - ports: - - 8081:8082 - secrets: - - my-secrets - networks: - - private-network - - public-network -configs: - my-config: - file: my-config.txt -secrets: - my-secrets: - file: not-so-secret.txt -networks: - private-network: - internal: true - public-network: {} diff --git a/pkg/e2e/fixtures/bridge/expected-helm/Chart.yaml b/pkg/e2e/fixtures/bridge/expected-helm/Chart.yaml deleted file mode 100755 index 44a00001138..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/Chart.yaml +++ /dev/null @@ -1,12 +0,0 @@ -#! Chart.yaml -apiVersion: v2 -name: bridge -version: 0.0.1 -# kubeVersion: >= 1.29.1 -description: A generated Helm Chart for bridge generated via compose-bridge. -type: application -keywords: - - bridge -appVersion: 'v0.0.1' -sources: -annotations: diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/0-bridge-namespace.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/0-bridge-namespace.yaml deleted file mode 100755 index 953ebe7bb12..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/0-bridge-namespace.yaml +++ /dev/null @@ -1,8 +0,0 @@ -#! 0-bridge-namespace.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Namespace -metadata: - name: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/bridge-configs.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/bridge-configs.yaml deleted file mode 100755 index 48e8b0cf6ac..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/bridge-configs.yaml +++ /dev/null @@ -1,12 +0,0 @@ -#! bridge-configs.yaml -# Generated code, do not edit -apiVersion: v1 -kind: ConfigMap -metadata: - name: {{ .Values.projectName }} - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge -data: - my-config: | - My config file diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/my-secrets-secret.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/my-secrets-secret.yaml deleted file mode 100755 index 63659713ba7..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/my-secrets-secret.yaml +++ /dev/null @@ -1,13 +0,0 @@ -#! my-secrets-secret.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Secret -metadata: - name: my-secrets - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.secret: my-secrets -data: - my-secrets: bm90LXNlY3JldA== -type: Opaque diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/private-network-network-policy.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/private-network-network-policy.yaml deleted file mode 100755 index 0300049be68..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/private-network-network-policy.yaml +++ /dev/null @@ -1,24 +0,0 @@ -#! private-network-network-policy.yaml -# Generated code, do not edit -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: private-network-network-policy - namespace: {{ .Values.namespace }} -spec: - podSelector: - matchLabels: - com.docker.compose.network.private-network: "true" - policyTypes: - - Ingress - - Egress - ingress: - - from: - - podSelector: - matchLabels: - com.docker.compose.network.private-network: "true" - egress: - - to: - - podSelector: - matchLabels: - com.docker.compose.network.private-network: "true" diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/public-network-network-policy.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/public-network-network-policy.yaml deleted file mode 100755 index da042b3e8c1..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/public-network-network-policy.yaml +++ /dev/null @@ -1,24 +0,0 @@ -#! public-network-network-policy.yaml -# Generated code, do not edit -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: public-network-network-policy - namespace: {{ .Values.namespace }} -spec: - podSelector: - matchLabels: - com.docker.compose.network.public-network: "true" - policyTypes: - - Ingress - - Egress - ingress: - - from: - - podSelector: - matchLabels: - com.docker.compose.network.public-network: "true" - egress: - - to: - - podSelector: - matchLabels: - com.docker.compose.network.public-network: "true" diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-deployment.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-deployment.yaml deleted file mode 100755 index afef74e8bad..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-deployment.yaml +++ /dev/null @@ -1,49 +0,0 @@ -#! serviceA-deployment.yaml -# Generated code, do not edit -apiVersion: apps/v1 -kind: Deployment -metadata: - name: servicea - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - app.kubernetes.io/managed-by: Helm -spec: - replicas: {{ .Values.deployment.defaultReplicas }} - selector: - matchLabels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - strategy: - type: {{ .Values.deployment.strategy }} - template: - metadata: - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - com.docker.compose.network.private-network: "true" - spec: - containers: - - name: servicea - image: {{ .Values.serviceA.image }} - imagePullPolicy: {{ .Values.serviceA.imagePullPolicy }} - resources: - limits: - cpu: {{ .Values.resources.defaultCpuLimit }} - memory: {{ .Values.resources.defaultMemoryLimit }} - ports: - - name: servicea-8080 - containerPort: 8080 - volumeMounts: - - name: etc-my-config1-txt - mountPath: /etc/my-config1.txt - subPath: my-config - readOnly: true - volumes: - - name: etc-my-config1-txt - configMap: - name: {{ .Values.projectName }} - items: - - key: my-config - path: my-config diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-expose.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-expose.yaml deleted file mode 100755 index 5d733bd2245..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-expose.yaml +++ /dev/null @@ -1,19 +0,0 @@ -#! serviceA-expose.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: servicea - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - app.kubernetes.io/managed-by: Helm -spec: - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - ports: - - name: servicea-8080 - port: 8080 - targetPort: servicea-8080 diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-service.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-service.yaml deleted file mode 100755 index 2138281ba93..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-service.yaml +++ /dev/null @@ -1,25 +0,0 @@ -# check if there is at least one published port - -#! serviceA-service.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: servicea-published - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - app.kubernetes.io/managed-by: Helm -spec: - type: {{ .Values.service.type }} - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - ports: - - name: servicea-80 - port: 80 - protocol: TCP - targetPort: servicea-8080 - -# check if there is at least one published port diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-deployment.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-deployment.yaml deleted file mode 100755 index 7ea9d998f7f..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-deployment.yaml +++ /dev/null @@ -1,50 +0,0 @@ -#! serviceB-deployment.yaml -# Generated code, do not edit -apiVersion: apps/v1 -kind: Deployment -metadata: - name: serviceb - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - app.kubernetes.io/managed-by: Helm -spec: - replicas: {{ .Values.deployment.defaultReplicas }} - selector: - matchLabels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - strategy: - type: {{ .Values.deployment.strategy }} - template: - metadata: - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - com.docker.compose.network.private-network: "true" - com.docker.compose.network.public-network: "true" - spec: - containers: - - name: serviceb - image: {{ .Values.serviceB.image }} - imagePullPolicy: {{ .Values.serviceB.imagePullPolicy }} - resources: - limits: - cpu: {{ .Values.resources.defaultCpuLimit }} - memory: {{ .Values.resources.defaultMemoryLimit }} - ports: - - name: serviceb-8082 - containerPort: 8082 - volumeMounts: - - name: run-secrets-my-secrets - mountPath: /run/secrets/my-secrets - subPath: my-secrets - readOnly: true - volumes: - - name: run-secrets-my-secrets - secret: - secretName: my-secrets - items: - - key: my-secrets - path: my-secrets diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-expose.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-expose.yaml deleted file mode 100755 index f413254dca0..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-expose.yaml +++ /dev/null @@ -1,19 +0,0 @@ -#! serviceB-expose.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: serviceb - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - app.kubernetes.io/managed-by: Helm -spec: - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - ports: - - name: serviceb-8082 - port: 8082 - targetPort: serviceb-8082 diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-service.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-service.yaml deleted file mode 100755 index 6860f3d2804..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-service.yaml +++ /dev/null @@ -1,21 +0,0 @@ -#! serviceB-service.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: serviceb-published - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - app.kubernetes.io/managed-by: Helm -spec: - type: {{ .Values.service.type }} - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - ports: - - name: serviceb-8081 - port: 8081 - protocol: TCP - targetPort: serviceb-8082 diff --git a/pkg/e2e/fixtures/bridge/expected-helm/values.yaml b/pkg/e2e/fixtures/bridge/expected-helm/values.yaml deleted file mode 100755 index 78315927666..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/values.yaml +++ /dev/null @@ -1,30 +0,0 @@ -#! values.yaml -# Project Name -projectName: bridge -# Namespace -namespace: bridge -# Default deployment settings -deployment: - strategy: Recreate - defaultReplicas: 1 -# Default resource limits -resources: - defaultCpuLimit: "100m" - defaultMemoryLimit: "512Mi" -# Service settings -service: - type: LoadBalancer -# Storage settings -storage: - defaultStorageClass: "hostpath" - defaultSize: "100Mi" - defaultAccessMode: "ReadWriteOnce" -# Services variables -serviceA: - image: alpine - imagePullPolicy: IfNotPresent -serviceB: - image: alpine - imagePullPolicy: IfNotPresent - -# You can apply the same logic to loop on networks, volumes, secrets and configs... diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/0-bridge-namespace.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/0-bridge-namespace.yaml deleted file mode 100755 index 40e4b0e23f4..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/0-bridge-namespace.yaml +++ /dev/null @@ -1,8 +0,0 @@ -#! 0-bridge-namespace.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Namespace -metadata: - name: bridge - labels: - com.docker.compose.project: bridge diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/bridge-configs.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/bridge-configs.yaml deleted file mode 100755 index 822d2e076ef..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/bridge-configs.yaml +++ /dev/null @@ -1,12 +0,0 @@ -#! bridge-configs.yaml -# Generated code, do not edit -apiVersion: v1 -kind: ConfigMap -metadata: - name: bridge - namespace: bridge - labels: - com.docker.compose.project: bridge -data: - my-config: | - My config file diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/kustomization.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/kustomization.yaml deleted file mode 100755 index ff8428feae2..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/kustomization.yaml +++ /dev/null @@ -1,16 +0,0 @@ -#! kustomization.yaml -# Generated code, do not edit -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization -resources: - - 0-bridge-namespace.yaml - - bridge-configs.yaml - - my-secrets-secret.yaml - - private-network-network-policy.yaml - - public-network-network-policy.yaml - - serviceA-deployment.yaml - - serviceA-expose.yaml - - serviceA-service.yaml - - serviceB-deployment.yaml - - serviceB-expose.yaml - - serviceB-service.yaml diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/my-secrets-secret.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/my-secrets-secret.yaml deleted file mode 100755 index 559eba6a26e..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/my-secrets-secret.yaml +++ /dev/null @@ -1,13 +0,0 @@ -#! my-secrets-secret.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Secret -metadata: - name: my-secrets - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.secret: my-secrets -data: - my-secrets: bm90LXNlY3JldA== -type: Opaque diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/private-network-network-policy.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/private-network-network-policy.yaml deleted file mode 100755 index 3f59b22dd9d..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/private-network-network-policy.yaml +++ /dev/null @@ -1,24 +0,0 @@ -#! private-network-network-policy.yaml -# Generated code, do not edit -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: private-network-network-policy - namespace: bridge -spec: - podSelector: - matchLabels: - com.docker.compose.network.private-network: "true" - policyTypes: - - Ingress - - Egress - ingress: - - from: - - podSelector: - matchLabels: - com.docker.compose.network.private-network: "true" - egress: - - to: - - podSelector: - matchLabels: - com.docker.compose.network.private-network: "true" diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/public-network-network-policy.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/public-network-network-policy.yaml deleted file mode 100755 index 04913d4b9a7..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/public-network-network-policy.yaml +++ /dev/null @@ -1,24 +0,0 @@ -#! public-network-network-policy.yaml -# Generated code, do not edit -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: public-network-network-policy - namespace: bridge -spec: - podSelector: - matchLabels: - com.docker.compose.network.public-network: "true" - policyTypes: - - Ingress - - Egress - ingress: - - from: - - podSelector: - matchLabels: - com.docker.compose.network.public-network: "true" - egress: - - to: - - podSelector: - matchLabels: - com.docker.compose.network.public-network: "true" diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-deployment.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-deployment.yaml deleted file mode 100755 index 0779cf56268..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-deployment.yaml +++ /dev/null @@ -1,44 +0,0 @@ -#! serviceA-deployment.yaml -# Generated code, do not edit -apiVersion: apps/v1 -kind: Deployment -metadata: - name: servicea - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA -spec: - replicas: 1 - selector: - matchLabels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - strategy: - type: Recreate - template: - metadata: - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - com.docker.compose.network.private-network: "true" - spec: - containers: - - name: servicea - image: alpine - imagePullPolicy: IfNotPresent - ports: - - name: servicea-8080 - containerPort: 8080 - volumeMounts: - - name: etc-my-config1-txt - mountPath: /etc/my-config1.txt - subPath: my-config - readOnly: true - volumes: - - name: etc-my-config1-txt - configMap: - name: bridge - items: - - key: my-config - path: my-config diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-expose.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-expose.yaml deleted file mode 100755 index d0bd013ecff..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-expose.yaml +++ /dev/null @@ -1,18 +0,0 @@ -#! serviceA-expose.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: servicea - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA -spec: - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - ports: - - name: servicea-8080 - port: 8080 - targetPort: servicea-8080 diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-service.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-service.yaml deleted file mode 100755 index 628cf04189c..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-service.yaml +++ /dev/null @@ -1,23 +0,0 @@ -# check if there is at least one published port - -#! serviceA-service.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: servicea-published - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA -spec: - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - ports: - - name: servicea-80 - port: 80 - protocol: TCP - targetPort: servicea-8080 - -# check if there is at least one published port diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-deployment.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-deployment.yaml deleted file mode 100755 index 191720c2014..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-deployment.yaml +++ /dev/null @@ -1,45 +0,0 @@ -#! serviceB-deployment.yaml -# Generated code, do not edit -apiVersion: apps/v1 -kind: Deployment -metadata: - name: serviceb - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB -spec: - replicas: 1 - selector: - matchLabels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - strategy: - type: Recreate - template: - metadata: - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - com.docker.compose.network.private-network: "true" - com.docker.compose.network.public-network: "true" - spec: - containers: - - name: serviceb - image: alpine - imagePullPolicy: IfNotPresent - ports: - - name: serviceb-8082 - containerPort: 8082 - volumeMounts: - - name: run-secrets-my-secrets - mountPath: /run/secrets/my-secrets - subPath: my-secrets - readOnly: true - volumes: - - name: run-secrets-my-secrets - secret: - secretName: my-secrets - items: - - key: my-secrets - path: my-secrets diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-expose.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-expose.yaml deleted file mode 100755 index 2025868991d..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-expose.yaml +++ /dev/null @@ -1,18 +0,0 @@ -#! serviceB-expose.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: serviceb - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB -spec: - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - ports: - - name: serviceb-8082 - port: 8082 - targetPort: serviceb-8082 diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-service.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-service.yaml deleted file mode 100755 index 94104185871..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-service.yaml +++ /dev/null @@ -1,19 +0,0 @@ -#! serviceB-service.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: serviceb-published - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB -spec: - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - ports: - - name: serviceb-8081 - port: 8081 - protocol: TCP - targetPort: serviceb-8082 diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/kustomization.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/kustomization.yaml deleted file mode 100755 index a192e45f0fe..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/kustomization.yaml +++ /dev/null @@ -1,9 +0,0 @@ -#! kustomization.yaml -# Generated code, do not edit -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization -resources: - - ../../base -patches: - - path: serviceA-service.yaml - - path: serviceB-service.yaml diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/serviceA-service.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/serviceA-service.yaml deleted file mode 100755 index 6453b5adba3..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/serviceA-service.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# check if there is at least one published port - -#! serviceA-service.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: servicea-published - namespace: bridge -spec: - type: LoadBalancer - -# check if there is at least one published port diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/serviceB-service.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/serviceB-service.yaml deleted file mode 100755 index f21b674336b..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/serviceB-service.yaml +++ /dev/null @@ -1,9 +0,0 @@ -#! serviceB-service.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: serviceb-published - namespace: bridge -spec: - type: LoadBalancer diff --git a/pkg/e2e/fixtures/bridge/my-config.txt b/pkg/e2e/fixtures/bridge/my-config.txt deleted file mode 100644 index 24d11e40bb8..00000000000 --- a/pkg/e2e/fixtures/bridge/my-config.txt +++ /dev/null @@ -1 +0,0 @@ -My config file \ No newline at end of file diff --git a/pkg/e2e/fixtures/bridge/not-so-secret.txt b/pkg/e2e/fixtures/bridge/not-so-secret.txt deleted file mode 100644 index 4e76a78aebb..00000000000 --- a/pkg/e2e/fixtures/bridge/not-so-secret.txt +++ /dev/null @@ -1 +0,0 @@ -not-secret \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-dependencies/base.dockerfile b/pkg/e2e/fixtures/build-dependencies/base.dockerfile deleted file mode 100644 index 9dce0b74f41..00000000000 --- a/pkg/e2e/fixtures/build-dependencies/base.dockerfile +++ /dev/null @@ -1,19 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine - -COPY hello.txt /hello.txt - -CMD [ "/bin/true" ] diff --git a/pkg/e2e/fixtures/build-dependencies/classic.yaml b/pkg/e2e/fixtures/build-dependencies/classic.yaml deleted file mode 100644 index b0dbbaad0a1..00000000000 --- a/pkg/e2e/fixtures/build-dependencies/classic.yaml +++ /dev/null @@ -1,14 +0,0 @@ -services: - base: - image: base - init: true - build: - context: . - dockerfile: base.dockerfile - service: - init: true - depends_on: - - base - build: - context: . - dockerfile: service.dockerfile diff --git a/pkg/e2e/fixtures/build-dependencies/compose-depends_on.yaml b/pkg/e2e/fixtures/build-dependencies/compose-depends_on.yaml deleted file mode 100644 index 90b2beaef18..00000000000 --- a/pkg/e2e/fixtures/build-dependencies/compose-depends_on.yaml +++ /dev/null @@ -1,15 +0,0 @@ -services: - test1: - pull_policy: build - build: - dockerfile_inline: FROM alpine - command: - - echo - - "test 1 success" - test2: - image: alpine - depends_on: - - test1 - command: - - echo - - "test 2 success" \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-dependencies/compose.yaml b/pkg/e2e/fixtures/build-dependencies/compose.yaml deleted file mode 100644 index 952a7199ef8..00000000000 --- a/pkg/e2e/fixtures/build-dependencies/compose.yaml +++ /dev/null @@ -1,13 +0,0 @@ -services: - base: - init: true - build: - context: . - dockerfile: base.dockerfile - service: - init: true - build: - context: . - additional_contexts: - base: "service:base" - dockerfile: service.dockerfile diff --git a/pkg/e2e/fixtures/build-dependencies/hello.txt b/pkg/e2e/fixtures/build-dependencies/hello.txt deleted file mode 100644 index 810e7ba64ac..00000000000 --- a/pkg/e2e/fixtures/build-dependencies/hello.txt +++ /dev/null @@ -1 +0,0 @@ -this file was copied from base -> service diff --git a/pkg/e2e/fixtures/build-dependencies/service.dockerfile b/pkg/e2e/fixtures/build-dependencies/service.dockerfile deleted file mode 100644 index 8c710b57321..00000000000 --- a/pkg/e2e/fixtures/build-dependencies/service.dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM base - -CMD [ "cat", "/hello.txt" ] diff --git a/pkg/e2e/fixtures/build-infinite/compose.yaml b/pkg/e2e/fixtures/build-infinite/compose.yaml deleted file mode 100644 index cdc1e169357..00000000000 --- a/pkg/e2e/fixtures/build-infinite/compose.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - service1: - build: service1 \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-infinite/service1/Dockerfile b/pkg/e2e/fixtures/build-infinite/service1/Dockerfile deleted file mode 100644 index 3fd64e7b655..00000000000 --- a/pkg/e2e/fixtures/build-infinite/service1/Dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM busybox - -RUN sleep infinity \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/compose.yaml b/pkg/e2e/fixtures/build-test/compose.yaml deleted file mode 100644 index 2db602a2d8f..00000000000 --- a/pkg/e2e/fixtures/build-test/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - nginx: - build: nginx-build - ports: - - 8070:80 - - nginx2: - build: nginx-build2 - image: custom-nginx diff --git a/pkg/e2e/fixtures/build-test/dependencies/compose.yaml b/pkg/e2e/fixtures/build-test/dependencies/compose.yaml deleted file mode 100644 index eb5de31f943..00000000000 --- a/pkg/e2e/fixtures/build-test/dependencies/compose.yaml +++ /dev/null @@ -1,26 +0,0 @@ -services: - firstbuild: - build: - dockerfile_inline: | - FROM alpine - additional_contexts: - dep1: service:dep1 - entrypoint: ["echo", "Hello from firstbuild"] - depends_on: - - dep1 - - secondbuild: - build: - dockerfile_inline: | - FROM alpine - additional_contexts: - dep1: service:dep1 - entrypoint: ["echo", "Hello from secondbuild"] - depends_on: - - dep1 - - dep1: - build: - dockerfile_inline: | - FROM alpine - entrypoint: ["echo", "Hello from dep1"] \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/entitlements/Dockerfile b/pkg/e2e/fixtures/build-test/entitlements/Dockerfile deleted file mode 100644 index a242eb52d4b..00000000000 --- a/pkg/e2e/fixtures/build-test/entitlements/Dockerfile +++ /dev/null @@ -1,19 +0,0 @@ -# syntax = docker/dockerfile:experimental - - -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine -RUN --security=insecure cat /proc/self/status | grep CapEff diff --git a/pkg/e2e/fixtures/build-test/entitlements/compose.yaml b/pkg/e2e/fixtures/build-test/entitlements/compose.yaml deleted file mode 100644 index 403529e6477..00000000000 --- a/pkg/e2e/fixtures/build-test/entitlements/compose.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - privileged-service: - build: - context: . - entitlements: - - security.insecure - diff --git a/pkg/e2e/fixtures/build-test/escaped/Dockerfile b/pkg/e2e/fixtures/build-test/escaped/Dockerfile deleted file mode 100644 index dd507f4fffd..00000000000 --- a/pkg/e2e/fixtures/build-test/escaped/Dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine -ARG foo -RUN echo foo is $foo diff --git a/pkg/e2e/fixtures/build-test/escaped/compose.yaml b/pkg/e2e/fixtures/build-test/escaped/compose.yaml deleted file mode 100644 index 2d0077b9e63..00000000000 --- a/pkg/e2e/fixtures/build-test/escaped/compose.yaml +++ /dev/null @@ -1,23 +0,0 @@ -services: - foo: - build: - context: . - args: - foo: $${bar} - - echo: - build: - dockerfile_inline: | - FROM bash - RUN <<'EOF' - echo $(seq 10) - EOF - - arg: - build: - args: - BOOL: "true" - dockerfile_inline: | - FROM alpine:latest - ARG BOOL - RUN /bin/$${BOOL} diff --git a/pkg/e2e/fixtures/build-test/long-output-line/Dockerfile b/pkg/e2e/fixtures/build-test/long-output-line/Dockerfile deleted file mode 100644 index 5227a491434..00000000000 --- a/pkg/e2e/fixtures/build-test/long-output-line/Dockerfile +++ /dev/null @@ -1,49 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -FROM alpine -# We generate warnings *on purpose* to bloat the JSON output of bake -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/long-output-line/compose.yaml b/pkg/e2e/fixtures/build-test/long-output-line/compose.yaml deleted file mode 100644 index d0e2a4fffdc..00000000000 --- a/pkg/e2e/fixtures/build-test/long-output-line/compose.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - long-line: - build: - context: . - dockerfile: Dockerfile diff --git a/pkg/e2e/fixtures/build-test/minimal/Dockerfile b/pkg/e2e/fixtures/build-test/minimal/Dockerfile deleted file mode 100644 index 968515a22ed..00000000000 --- a/pkg/e2e/fixtures/build-test/minimal/Dockerfile +++ /dev/null @@ -1,16 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM scratch -COPY . . \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/minimal/compose.yaml b/pkg/e2e/fixtures/build-test/minimal/compose.yaml deleted file mode 100644 index 8362c60b900..00000000000 --- a/pkg/e2e/fixtures/build-test/minimal/compose.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - test: - build: . \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/multi-args/Dockerfile b/pkg/e2e/fixtures/build-test/multi-args/Dockerfile deleted file mode 100644 index b2c18cdd664..00000000000 --- a/pkg/e2e/fixtures/build-test/multi-args/Dockerfile +++ /dev/null @@ -1,19 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -ARG IMAGE=666 -ARG TAG=666 - -FROM ${IMAGE}:${TAG} -RUN echo "SUCCESS" diff --git a/pkg/e2e/fixtures/build-test/multi-args/compose.yaml b/pkg/e2e/fixtures/build-test/multi-args/compose.yaml deleted file mode 100644 index 8bedfd6384e..00000000000 --- a/pkg/e2e/fixtures/build-test/multi-args/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - multiargs: - build: - context: . - args: - IMAGE: alpine - TAG: latest - labels: - - RESULT=SUCCESS diff --git a/pkg/e2e/fixtures/build-test/nginx-build/Dockerfile b/pkg/e2e/fixtures/build-test/nginx-build/Dockerfile deleted file mode 100644 index dd79c0e4a31..00000000000 --- a/pkg/e2e/fixtures/build-test/nginx-build/Dockerfile +++ /dev/null @@ -1,19 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM nginx:alpine - -ARG FOO -LABEL FOO=$FOO -COPY static /usr/share/nginx/html diff --git a/pkg/e2e/fixtures/build-test/nginx-build/static/index.html b/pkg/e2e/fixtures/build-test/nginx-build/static/index.html deleted file mode 100644 index 914e3406cb9..00000000000 --- a/pkg/e2e/fixtures/build-test/nginx-build/static/index.html +++ /dev/null @@ -1,10 +0,0 @@ -<!doctype html> -<html lang="en"> -<head> - <meta charset="utf-8">

diff --git a/.dockerignore b/.dockerignore index e660fd93d31..c1323a91826 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1 +1,14 @@ -bin/ +*.egg-info +.coverage +.git +.github +.tox +build +binaries +coverage-html +docs/_site +*venv +.tox +**/__pycache__ +*.pyc +Jenkinsfile diff --git a/.gitattributes b/.gitattributes deleted file mode 100644 index 9c91b79966e..00000000000 --- a/.gitattributes +++ /dev/null @@ -1,2 +0,0 @@ -core.autocrlf false -*.golden text eol=lf diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 209b8217f30..85ab9015f3b 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1,2 +1,6 @@ -# global rules -* @docker/compose-maintainers \ No newline at end of file +# GitHub code owners +# See https://help.github.com/articles/about-codeowners/ +# +# KEEP THIS FILE SORTED. Order is important. Last match takes precedence. + +* @aiordache @ndeloof @rumpl @ulyssessouza diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 00000000000..2f3012f6135 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,63 @@ +--- +name: Bug report +about: Report a bug encountered while using docker-compose +title: '' +labels: kind/bug +assignees: '' + +--- + + + +## Description of the issue + +## Context information (for bug reports) + +**Output of `docker-compose version`** +``` +(paste here) +``` + +**Output of `docker version`** +``` +(paste here) +``` + +**Output of `docker-compose config`** +(Make sure to add the relevant `-f` and other flags) +``` +(paste here) +``` + + +## Steps to reproduce the issue + +1. +2. +3. + +### Observed result + +### Expected result + +### Stacktrace / full error message + +``` +(paste here) +``` + +## Additional information + +OS version / distribution, `docker-compose` install method, etc. diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml deleted file mode 100644 index 37b546967dd..00000000000 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ /dev/null @@ -1,55 +0,0 @@ -name: 🐞 Bug -description: File a bug/issue -title: "[BUG] " -labels: ['status/0-triage', 'kind/bug'] -body: - - type: textarea - attributes: - label: Description - description: | - Briefly describe the problem you are having. - - Include both the current behavior (what you are seeing) as well as what you expected to happen. - validations: - required: true - - type: markdown - attributes: - value: | - [Docker Swarm](https://www.mirantis.com/software/swarm/) uses a distinct compose file parser and - as such doesn't support some of the recent features of Docker Compose. Please contact Mirantis - if you need assistance with compose file support in Docker Swarm. - - type: textarea - attributes: - label: Steps To Reproduce - description: Steps to reproduce the behavior. - placeholder: | - 1. In this environment... - 2. With this config... - 3. Run '...' - 4. See error... - validations: - required: false - - type: textarea - attributes: - label: Compose Version - description: | - Paste output of `docker compose version` and `docker-compose version`. - render: Text - validations: - required: false - - type: textarea - attributes: - label: Docker Environment - description: Paste output of `docker info`. - render: Text - validations: - required: false - - type: textarea - attributes: - label: Anything else? - description: | - Links? References? Anything that will give us more context about the issue you are encountering! - - Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in. - validations: - required: false diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml deleted file mode 100644 index cc4b65bf24d..00000000000 --- a/.github/ISSUE_TEMPLATE/config.yml +++ /dev/null @@ -1,11 +0,0 @@ -blank_issues_enabled: true -contact_links: - - name: Docker Community Slack - url: https://dockr.ly/slack - about: 'Use the #docker-compose channel' - - name: Docker Support Forums - url: https://forums.docker.com/c/open-source-projects/compose/15 - about: 'Use the "Open Source Projects > Compose" category' - - name: 'Ask on Stack Overflow' - url: https://stackoverflow.com/questions/tagged/docker-compose - about: 'Use the [docker-compose] tag when creating new questions' diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 00000000000..603d34c38ab --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,32 @@ +--- +name: Feature request +about: Suggest an idea to improve Compose +title: '' +labels: kind/feature +assignees: '' + +--- + + + +**Is your feature request related to a problem? Please describe.** +A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] + +**Describe the solution you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Additional context** +Add any other context or screenshots about the feature request here. diff --git a/.github/ISSUE_TEMPLATE/feature_request.yaml b/.github/ISSUE_TEMPLATE/feature_request.yaml deleted file mode 100644 index 677a1684fc0..00000000000 --- a/.github/ISSUE_TEMPLATE/feature_request.yaml +++ /dev/null @@ -1,13 +0,0 @@ -name: Feature request -description: Missing functionality? Come tell us about it! -labels: - - kind/feature - - status/0-triage -body: - - type: textarea - id: description - attributes: - label: Description - description: What is the feature you want to see? - validations: - required: true diff --git a/.github/ISSUE_TEMPLATE/question-about-using-compose.md b/.github/ISSUE_TEMPLATE/question-about-using-compose.md new file mode 100644 index 00000000000..ccb4e9b33bb --- /dev/null +++ b/.github/ISSUE_TEMPLATE/question-about-using-compose.md @@ -0,0 +1,12 @@ +--- +name: Question about using Compose +about: This is not the appropriate channel +title: '' +labels: kind/question +assignees: '' + +--- + +Please post on our forums: https://forums.docker.com for questions about using `docker-compose`. + +Posts that are not a bug report or a feature/enhancement request will not be addressed on this issue tracker. diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md deleted file mode 100644 index 00e87ff8eaa..00000000000 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ /dev/null @@ -1,6 +0,0 @@ -**What I did** - -**Related issue** - - -**(not mandatory) A picture of a cute animal, if possible in relation to what you did** diff --git a/.github/SECURITY.md b/.github/SECURITY.md deleted file mode 100644 index 88f7528d524..00000000000 --- a/.github/SECURITY.md +++ /dev/null @@ -1,44 +0,0 @@ -# Security Policy - -The maintainers of Docker Compose take security seriously. If you discover -a security issue, please bring it to their attention right away! - -## Reporting a Vulnerability - -Please **DO NOT** file a public issue, instead send your report privately -to [security@docker.com](mailto:security@docker.com). - -Reporter(s) can expect a response within 72 hours, acknowledging the issue was -received. - -## Review Process - -After receiving the report, an initial triage and technical analysis is -performed to confirm the report and determine its scope. We may request -additional information in this stage of the process. - -Once a reviewer has confirmed the relevance of the report, a draft security -advisory will be created on GitHub. The draft advisory will be used to discuss -the issue with maintainers, the reporter(s), and where applicable, other -affected parties under embargo. - -If the vulnerability is accepted, a timeline for developing a patch, public -disclosure, and patch release will be determined. If there is an embargo period -on public disclosure before the patch release, the reporter(s) are expected to -participate in the discussion of the timeline and abide by agreed upon dates -for public disclosure. - -## Accreditation - -Security reports are greatly appreciated and we will publicly thank you, -although we will keep your name confidential if you request it. We also like to -send gifts - if you're into swag, make sure to let us know. We do not currently -offer a paid security bounty program at this time. - -## Supported Versions - -This project docs not provide long-term supported versions, and only the current -release and `main` branch are actively maintained. Docker Compose v1, and the -corresponding [v1 branch](https://github.com/docker/compose/tree/v1) reached -EOL and are no longer supported. - diff --git a/.github/dependabot.yml b/.github/dependabot.yml deleted file mode 100644 index 3810add71fc..00000000000 --- a/.github/dependabot.yml +++ /dev/null @@ -1,23 +0,0 @@ -version: 2 -updates: - - package-ecosystem: gomod - directory: / - schedule: - interval: daily - ignore: - # docker + moby deps require coordination - - dependency-name: "github.com/docker/buildx" - # buildx is still 0.x - update-types: ["version-update:semver-minor"] - - dependency-name: "github.com/moby/buildkit" - # buildkit is still 0.x - update-types: [ "version-update:semver-minor" ] - - dependency-name: "github.com/docker/cli" - update-types: ["version-update:semver-major"] - - dependency-name: "github.com/docker/docker" - update-types: ["version-update:semver-major"] - - dependency-name: "github.com/containerd/containerd" - # containerd major/minor must be kept in sync with moby - update-types: [ "version-update:semver-major", "version-update:semver-minor" ] - # OTEL dependencies should be upgraded in sync with engine, cli, buildkit and buildx projects - - dependency-name: "go.opentelemetry.io/*" diff --git a/.github/stale.yml b/.github/stale.yml index c14cb12918a..6de76aef987 100644 --- a/.github/stale.yml +++ b/.github/stale.yml @@ -1,7 +1,7 @@ # Configuration for probot-stale - https://github.com/probot/stale # Number of days of inactivity before an Issue or Pull Request becomes stale -daysUntilStale: 90 +daysUntilStale: 180 # Number of days of inactivity before an Issue or Pull Request with the stale label is closed. # Set to false to disable. If disabled, issues still need to be closed manually, but will remain marked as stale. @@ -12,7 +12,7 @@ onlyLabels: [] # Issues or Pull Requests with these labels will never be considered stale. Set to `[]` to disable exemptLabels: - - "kind/feature" + - kind/feature # Set to true to ignore issues in a project (defaults to false) exemptProjects: false @@ -56,4 +56,4 @@ only: issues # issues: # exemptLabels: -# - confirmed \ No newline at end of file +# - confirmed diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml deleted file mode 100644 index 3c6ef149d23..00000000000 --- a/.github/workflows/ci.yml +++ /dev/null @@ -1,338 +0,0 @@ -name: ci - -concurrency: - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -on: - push: - branches: - - 'main' - tags: - - 'v*' - pull_request: - workflow_dispatch: - inputs: - debug_enabled: - description: 'To run with tmate enter "debug_enabled"' - required: false - default: "false" - -permissions: - contents: read # to fetch code (actions/checkout) - -jobs: - validate: - runs-on: ubuntu-latest - strategy: - fail-fast: false - matrix: - target: - - lint - - validate-go-mod - - validate-headers - - validate-docs - steps: - - - name: Checkout - uses: actions/checkout@v4 - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v3 - - - name: Run - run: | - make ${{ matrix.target }} - - binary: - uses: docker/github-builder/.github/workflows/bake.yml@v1 - permissions: - contents: read # same as global permission - id-token: write # for signing attestation(s) with GitHub OIDC Token - with: - runner: amd64 - artifact-name: compose - artifact-upload: true - cache: true - cache-scope: binary - target: release - output: local - sbom: true - sign: ${{ github.event_name != 'pull_request' }} - - binary-finalize: - runs-on: ubuntu-latest - needs: - - binary - steps: - - - name: Download artifacts - uses: actions/download-artifact@v7 - with: - path: /tmp/compose-output - name: ${{ needs.binary.outputs.artifact-name }} - - - name: Rename provenance and sbom - run: | - for pdir in /tmp/compose-output/*/; do - ( - cd "$pdir" - binname=$(find . -name 'docker-compose-*') - filename=$(basename "${binname%.exe}") - mv "provenance.json" "${filename}.provenance.json" - mv "sbom-binary.spdx.json" "${filename}.sbom.json" - find . -name 'sbom*.json' -exec rm {} \; - if [ -f "provenance.sigstore.json" ]; then - mv "provenance.sigstore.json" "${filename}.sigstore.json" - fi - ) - done - mkdir -p "./bin/release" - mv /tmp/compose-output/**/* "./bin/release/" - - - name: Create checksum file - working-directory: ./bin/release - run: | - find . -type f -print0 | sort -z | xargs -r0 shasum -a 256 -b | sed 's# \*\./# *#' > $RUNNER_TEMP/checksums.txt - shasum -a 256 -U -c $RUNNER_TEMP/checksums.txt - mv $RUNNER_TEMP/checksums.txt . - cat checksums.txt | while read sum file; do - if [[ "${file#\*}" == docker-compose-* && "${file#\*}" != *.provenance.json && "${file#\*}" != *.sbom.json && "${file#\*}" != *.sigstore.json ]]; then - echo "$sum $file" > ${file#\*}.sha256 - fi - done - - - name: Upload artifacts - uses: actions/upload-artifact@v6 - with: - name: release - path: ./bin/release/* - if-no-files-found: error - - bin-image-test: - if: github.event_name == 'pull_request' - uses: docker/github-builder/.github/workflows/bake.yml@v1 - with: - runner: amd64 - target: image-cross - cache: true - cache-scope: bin-image-test - output: image - push: false - sbom: true - set-meta-labels: true - meta-images: | - compose-bin - meta-tags: | - type=ref,event=pr - meta-bake-target: meta-helper - - test: - runs-on: ubuntu-latest - steps: - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v3 - - - name: Test - uses: docker/bake-action@v6 - with: - targets: test - set: | - *.cache-from=type=gha,scope=test - *.cache-to=type=gha,scope=test - - - name: Gather coverage data - uses: actions/upload-artifact@v4 - with: - name: coverage-data-unit - path: bin/coverage/unit/ - if-no-files-found: error - - - name: Unit Test Summary - uses: test-summary/action@v2 - with: - paths: bin/coverage/unit/report.xml - if: always() - - e2e: - runs-on: ubuntu-latest - name: e2e (${{ matrix.mode }}, ${{ matrix.channel }}) - strategy: - fail-fast: false - matrix: - include: - # current stable - - mode: plugin - engine: 29 - channel: stable - - mode: standalone - engine: 29 - channel: stable - - # old stable (latest major - 1) - - mode: plugin - engine: 28 - channel: oldstable - - mode: standalone - engine: 28 - channel: oldstable - steps: - - name: Prepare - run: | - mode=${{ matrix.mode }} - engine=${{ matrix.engine }} - echo "MODE_ENGINE_PAIR=${mode}-${engine}" >> $GITHUB_ENV - - - name: Checkout - uses: actions/checkout@v4 - - - name: Install Docker ${{ matrix.engine }} - run: | - sudo systemctl stop docker.service - sudo apt-get purge docker-ce docker-ce-cli containerd.io docker-compose-plugin docker-ce-rootless-extras docker-buildx-plugin - sudo apt-get install curl - curl -fsSL https://test.docker.com -o get-docker.sh - sudo sh ./get-docker.sh --version ${{ matrix.engine }} - - - name: Check Docker Version - run: docker --version - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v3 - - - name: Set up Docker Model - run: | - sudo apt-get install docker-model-plugin - docker model version - - - name: Set up Go - uses: actions/setup-go@v6 - with: - go-version-file: '.go-version' - check-latest: true - cache: true - - - name: Build example provider - run: make example-provider - - - name: Build - uses: docker/bake-action@v6 - with: - source: . - targets: binary-with-coverage - set: | - *.cache-from=type=gha,scope=binary-linux-amd64 - *.cache-from=type=gha,scope=binary-e2e-${{ matrix.mode }} - *.cache-to=type=gha,scope=binary-e2e-${{ matrix.mode }},mode=max - env: - BUILD_TAGS: e2e - - - name: Setup tmate session - if: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.debug_enabled }} - uses: mxschmitt/action-tmate@8b4e4ac71822ed7e0ad5fb3d1c33483e9e8fb270 # v3.11 - with: - limit-access-to-actor: true - github-token: ${{ secrets.GITHUB_TOKEN }} - - - name: Test plugin mode - if: ${{ matrix.mode == 'plugin' }} - run: | - rm -rf ./bin/coverage/e2e - mkdir -p ./bin/coverage/e2e - make e2e-compose GOCOVERDIR=bin/coverage/e2e TEST_FLAGS="-v" - - - name: Gather coverage data - if: ${{ matrix.mode == 'plugin' }} - uses: actions/upload-artifact@v4 - with: - name: coverage-data-e2e-${{ env.MODE_ENGINE_PAIR }} - path: bin/coverage/e2e/ - if-no-files-found: error - - - name: Test standalone mode - if: ${{ matrix.mode == 'standalone' }} - run: | - rm -f /usr/local/bin/docker-compose - cp bin/build/docker-compose /usr/local/bin - make e2e-compose-standalone - - - name: e2e Test Summary - uses: test-summary/action@v2 - with: - paths: /tmp/report/report.xml - if: always() - - coverage: - runs-on: ubuntu-latest - needs: - - test - - e2e - steps: - # codecov won't process the report without the source code available - - name: Checkout - uses: actions/checkout@v4 - - name: Set up Go - uses: actions/setup-go@v6 - with: - go-version-file: '.go-version' - check-latest: true - - name: Download unit test coverage - uses: actions/download-artifact@v4 - with: - name: coverage-data-unit - path: coverage/unit - merge-multiple: true - - name: Download E2E test coverage - uses: actions/download-artifact@v4 - with: - pattern: coverage-data-e2e-* - path: coverage/e2e - merge-multiple: true - - name: Merge coverage reports - run: | - go tool covdata textfmt -i=./coverage/unit,./coverage/e2e -o ./coverage.txt - - name: Store coverage report in GitHub Actions - uses: actions/upload-artifact@v4 - with: - name: go-covdata-txt - path: ./coverage.txt - if-no-files-found: error - - name: Upload coverage to Codecov - uses: codecov/codecov-action@v5 - with: - files: ./coverage.txt - - release: - permissions: - contents: write # to create a release (ncipollo/release-action) - runs-on: ubuntu-latest - needs: - - binary-finalize - steps: - - - name: Checkout - uses: actions/checkout@v4 - - - name: Download artifacts - uses: actions/download-artifact@v7 - with: - path: ./bin/release - name: release - - - name: List artifacts - run: | - tree -nh ./bin/release - - - name: Check artifacts - run: | - find bin/release -type f -exec file -e ascii -- {} + - - - name: GitHub Release - if: startsWith(github.ref, 'refs/tags/v') - uses: ncipollo/release-action@58ae73b360456532aafd58ee170c045abbeaee37 # v1.10.0 - with: - artifacts: ./bin/release/* - generateReleaseNotes: true - draft: true - token: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/docs-upstream.yml b/.github/workflows/docs-upstream.yml deleted file mode 100644 index 214c88381fd..00000000000 --- a/.github/workflows/docs-upstream.yml +++ /dev/null @@ -1,51 +0,0 @@ -# this workflow runs the remote validate bake target from docker/docs -# to check if yaml reference docs used in this repo are valid -name: docs-upstream - -# Default to 'contents: read', which grants actions to read commits. -# -# If any permission is set, any permission not included in the list is -# implicitly set to "none". -# -# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions -permissions: - contents: read - -concurrency: - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -on: - push: - branches: - - 'main' - - 'v[0-9]*' - paths: - - '.github/workflows/docs-upstream.yml' - - 'docs/**' - pull_request: - paths: - - '.github/workflows/docs-upstream.yml' - - 'docs/**' - -jobs: - docs-yaml: - runs-on: ubuntu-latest - steps: - - - name: Checkout - uses: actions/checkout@v4 - - - name: Upload reference YAML docs - uses: actions/upload-artifact@v4 - with: - name: docs-yaml - path: docs/reference - retention-days: 1 - - validate: - uses: docker/docs/.github/workflows/validate-upstream.yml@main - needs: - - docs-yaml - with: - module-name: docker/compose diff --git a/.github/workflows/merge.yml b/.github/workflows/merge.yml deleted file mode 100644 index 1e0d37fd24f..00000000000 --- a/.github/workflows/merge.yml +++ /dev/null @@ -1,141 +0,0 @@ -name: merge - -concurrency: - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -on: - push: - branches: - - 'main' - tags: - - 'v*' - -permissions: - contents: read # to fetch code (actions/checkout) - -env: - REPO_SLUG: "docker/compose-bin" - -jobs: - e2e: - name: Build and test - runs-on: ${{ matrix.os }} - timeout-minutes: 15 - strategy: - fail-fast: false - matrix: - os: [desktop-windows, desktop-macos, desktop-m1] - # mode: [plugin, standalone] - mode: [plugin] - env: - GO111MODULE: "on" - steps: - - uses: actions/checkout@v4 - - - uses: actions/setup-go@v6 - with: - go-version-file: '.go-version' - cache: true - check-latest: true - - - name: List Docker resources on machine - run: | - docker ps --all - docker volume ls - docker network ls - docker image ls - - name: Remove Docker resources on machine - continue-on-error: true - run: | - docker kill $(docker ps -q) - docker rm -f $(docker ps -aq) - docker volume rm -f $(docker volume ls -q) - docker ps --all - - - name: Unit tests - run: make test - - - name: Build binaries - run: | - make - - name: Check arch of go compose binary - run: | - file ./bin/build/docker-compose - if: ${{ !contains(matrix.os, 'desktop-windows') }} - - - name: Test plugin mode - if: ${{ matrix.mode == 'plugin' }} - run: | - make e2e-compose - - - name: Test standalone mode - if: ${{ matrix.mode == 'standalone' }} - run: | - make e2e-compose-standalone - - bin-image-prepare: - runs-on: ubuntu-24.04 - outputs: - repo-slug: ${{ env.REPO_SLUG }} - steps: - # FIXME: can't use env object in reusable workflow inputs: https://github.com/orgs/community/discussions/26671 - - run: echo "Exposing env vars for reusable workflow" - - bin-image: - uses: docker/github-builder/.github/workflows/bake.yml@v1 - needs: - - bin-image-prepare - permissions: - contents: read # same as global permission - id-token: write # for signing attestation(s) with GitHub OIDC Token - with: - runner: amd64 - target: image-cross - cache: true - cache-scope: bin-image - output: image - push: ${{ github.event_name != 'pull_request' }} - sbom: true - set-meta-labels: true - meta-images: | - ${{ needs.bin-image-prepare.outputs.repo-slug }} - meta-tags: | - type=ref,event=tag - type=edge - meta-bake-target: meta-helper - secrets: - registry-auths: | - - registry: docker.io - username: ${{ secrets.DOCKERPUBLICBOT_USERNAME }} - password: ${{ secrets.DOCKERPUBLICBOT_WRITE_PAT }} - - desktop-edge-test: - runs-on: ubuntu-latest - needs: bin-image - steps: - - - name: Generate Token - id: generate_token - uses: actions/create-github-app-token@v1 - with: - app-id: ${{ vars.DOCKERDESKTOP_APP_ID }} - private-key: ${{ secrets.DOCKERDESKTOP_APP_PRIVATEKEY }} - owner: docker - repositories: | - ${{ secrets.DOCKERDESKTOP_REPO }} - - - name: Trigger Docker Desktop e2e with edge version - uses: actions/github-script@v7 - with: - github-token: ${{ steps.generate_token.outputs.token }} - script: | - await github.rest.actions.createWorkflowDispatch({ - owner: 'docker', - repo: '${{ secrets.DOCKERDESKTOP_REPO }}', - workflow_id: 'compose-edge-integration.yml', - ref: 'main', - inputs: { - "image-tag": "${{ env.REPO_SLUG }}:edge" - } - }) diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml deleted file mode 100644 index b8f0e5e2500..00000000000 --- a/.github/workflows/scorecards.yml +++ /dev/null @@ -1,63 +0,0 @@ -name: Scorecards supply-chain security -on: - # Only the default branch is supported. - branch_protection_rule: - schedule: - - cron: '44 9 * * 4' - push: - branches: [ "main" ] - -jobs: - analysis: - name: Scorecards analysis - runs-on: ubuntu-latest - permissions: - # Needed to upload the results to code-scanning dashboard. - security-events: write - # Used to receive a badge. - id-token: write - # read permissions to all the other objects - actions: read - attestations: read - checks: read - contents: read - deployments: read - issues: read - discussions: read - packages: read - pages: read - pull-requests: read - statuses: read - - steps: - - name: "Checkout code" - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # tag=v4.4.2 - with: - persist-credentials: false - - - name: "Run analysis" - uses: ossf/scorecard-action@62b2cac7ed8198b15735ed49ab1e5cf35480ba46 # tag=v2.4.0 - with: - results_file: results.sarif - results_format: sarif - - # Publish the results for public repositories to enable scorecard badges. For more details, see - # https://github.com/ossf/scorecard-action#publishing-results. - # For private repositories, `publish_results` will automatically be set to `false`, regardless - # of the value entered here. - publish_results: true - - # Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF - # format to the repository Actions tab. - - name: "Upload artifact" - uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # tag=v4.5.0 - with: - name: SARIF file - path: results.sarif - retention-days: 5 - - # Upload the results to GitHub's code scanning dashboard. - - name: "Upload to code-scanning" - uses: github/codeql-action/upload-sarif@3096afedf9873361b2b2f65e1445b13272c83eb8 # tag=v2.20.00 - with: - sarif_file: results.sarif diff --git a/.github/workflows/stale.yml b/.github/workflows/stale.yml deleted file mode 100644 index 2a747ee1c25..00000000000 --- a/.github/workflows/stale.yml +++ /dev/null @@ -1,33 +0,0 @@ -name: 'Close stale issues' - -# Default to 'contents: read', which grants actions to read commits. -# -# If any permission is set, any permission not included in the list is -# implicitly set to "none". -# -# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions -permissions: - contents: read - -on: - schedule: - - cron: '0 0 * * 0,3' # at midnight UTC every Sunday and Wednesday -jobs: - stale: - runs-on: ubuntu-latest - permissions: - issues: write - pull-requests: write - steps: - - uses: actions/stale@v9 - with: - repo-token: ${{ secrets.GITHUB_TOKEN }} - stale-issue-message: > - This issue has been automatically marked as stale because it has not had - recent activity. It will be closed if no further activity occurs. Thank you - for your contributions. - days-before-issue-stale: 150 # marks stale after 5 months - days-before-issue-close: 30 # closes 1 month after being marked with no action - stale-issue-label: "stale" - exempt-issue-labels: "kind/feature,kind/enhancement" - \ No newline at end of file diff --git a/.gitignore b/.gitignore index f01544f105f..79888274847 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,18 @@ -bin/ -/.vscode/ -coverage.out -covdatafiles/ +*.egg-info +*.pyc +*.swo +*.swp +.cache +.coverage* .DS_Store -pkg/e2e/*.tar +.idea + +/.tox +/binaries +/build +/compose/GITSHA +/coverage-html +/dist +/docs/_site +/README.rst +/*venv diff --git a/.go-version b/.go-version deleted file mode 100644 index 5759850c03e..00000000000 --- a/.go-version +++ /dev/null @@ -1 +0,0 @@ -1.25.7 \ No newline at end of file diff --git a/.golangci.yml b/.golangci.yml deleted file mode 100644 index 7d75550fea0..00000000000 --- a/.golangci.yml +++ /dev/null @@ -1,110 +0,0 @@ -version: "2" -run: - concurrency: 2 -linters: - default: none - enable: - - copyloopvar - - depguard - - errcheck - - errorlint - - forbidigo - - gocritic - - gocyclo - - gomodguard - - govet - - ineffassign - - lll - - misspell - - nakedret - - nolintlint - - revive - - staticcheck - - testifylint - - unconvert - - unparam - - unused - settings: - depguard: - rules: - all: - deny: - - pkg: io/ioutil - desc: io/ioutil package has been deprecated - - pkg: github.com/docker/docker/errdefs - desc: use github.com/containerd/errdefs instead. - - pkg: golang.org/x/exp/maps - desc: use stdlib maps package - - pkg: golang.org/x/exp/slices - desc: use stdlib slices package - - pkg: gopkg.in/yaml.v2 - desc: compose-go uses yaml.v3 - forbidigo: - analyze-types: true - forbid: - - pattern: 'context\.Background' - pkg: '^context$' - msg: "in tests, use t.Context() instead of context.Background()" - - pattern: 'context\.TODO' - pkg: '^context$' - msg: "in tests, use t.Context() instead of context.TODO()" - gocritic: - disabled-checks: - - paramTypeCombine - - unnamedResult - - whyNoLint - enabled-tags: - - diagnostic - - opinionated - - style - gocyclo: - min-complexity: 16 - gomodguard: - blocked: - modules: - - github.com/pkg/errors: - recommendations: - - errors - - fmt - versions: - - github.com/distribution/distribution: - reason: use distribution/reference - - gotest.tools: - version: < 3.0.0 - reason: deprecated, pre-modules version - lll: - line-length: 200 - revive: - rules: - - name: package-comments - disabled: true - exclusions: - generated: lax - paths: - - third_party$ - - builtin$ - - examples$ - rules: - - path-except: '_test\.go' - linters: - - forbidigo -issues: - max-issues-per-linter: 0 - max-same-issues: 0 -formatters: - enable: - - gci - - gofumpt - exclusions: - generated: lax - paths: - - third_party$ - - builtin$ - - examples$ - settings: - gci: - sections: - - standard - - default - - localmodule - custom-order: true # make the section order the same as the order of "sections". diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 00000000000..45f6f6fcf1f --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,28 @@ +- repo: git://github.com/pre-commit/pre-commit-hooks + sha: 'v0.9.1' + hooks: + - id: check-added-large-files + - id: check-docstring-first + - id: check-merge-conflict + - id: check-yaml + - id: check-json + - id: debug-statements + - id: end-of-file-fixer + - id: flake8 + - id: name-tests-test + exclude: 'tests/(integration/testcases\.py|helpers\.py)' + - id: requirements-txt-fixer + - id: trailing-whitespace +- repo: git://github.com/asottile/reorder_python_imports + sha: v1.3.4 + hooks: + - id: reorder-python-imports + language_version: 'python3.9' + args: + - --py3-plus +- repo: https://github.com/asottile/pyupgrade + rev: v2.1.0 + hooks: + - id: pyupgrade + args: + - --py3-plus diff --git a/BUILDING.md b/BUILDING.md deleted file mode 100644 index e9861f08140..00000000000 --- a/BUILDING.md +++ /dev/null @@ -1,95 +0,0 @@ - -### Prerequisites - -* Windows: - * [Docker Desktop](https://docs.docker.com/desktop/setup/install/windows-install/) - * make - * go (see [go.mod](go.mod) for minimum version) -* macOS: - * [Docker Desktop](https://docs.docker.com/desktop/setup/install/mac-install/) - * make - * go (see [go.mod](go.mod) for minimum version) -* Linux: - * [Docker 20.10 or later](https://docs.docker.com/engine/install/) - * make - * go (see [go.mod](go.mod) for minimum version) - -### Building the CLI - -Once you have the prerequisites installed, you can build the CLI using: - -```console -make -``` - -This will output a `docker-compose` CLI plugin for your host machine in -`./bin/build`. - -You can statically cross compile the CLI for Windows, macOS, and Linux using the -`cross` target. - -### Unit tests - -To run all of the unit tests, run: - -```console -make test -``` - -If you need to update a golden file simply do `go test ./... -test.update-golden`. - -### End-to-end tests -To run e2e tests, the Compose CLI binary needs to be built. All the commands to run e2e tests propose a version -with the prefix `build-and-e2e` to first build the CLI before executing tests. - -Note that this requires a local Docker Engine to be running. - -#### Whole end-to-end tests suite - -To execute both CLI and standalone e2e tests, run : - -```console -make e2e -``` - -Or if you need to build the CLI, run: -```console -make build-and-e2e -``` - -#### Plugin end-to-end tests suite - -To execute CLI plugin e2e tests, run : - -```console -make e2e-compose -``` - -Or if you need to build the CLI, run: -```console -make build-and-e2e-compose -``` - -#### Standalone end-to-end tests suite - -To execute the standalone CLI e2e tests, run : - -```console -make e2e-compose-standalone -``` - -Or if you need to build the CLI, run: - -```console -make build-and-e2e-compose-standalone -``` - -## Releases - -To create a new release: -* Check that the CI is green on the main branch for the commit you want to release -* Run the release GitHub Actions workflow with a tag of form vx.y.z following existing tags. - -This will automatically create a new tag, release and make binaries for -Windows, macOS, and Linux available for download on the -[releases page](https://github.com/docker/compose/releases). diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 00000000000..3a9078d27d6 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,2515 @@ +Change log +========== + +1.27.4 (2020-09-24) +------------------- + +### Bugs + +- Remove path checks for bind mounts + +- Fix port rendering to output long form syntax for non-v1 + +- Add protocol to the docker socket address + +1.27.3 (2020-09-16) +------------------- + +### Bugs + +- Merge `max_replicas_per_node` on `docker-compose config` + +- Fix `depends_on` serialization on `docker-compose config` + +- Fix scaling when some containers are not running on `docker-compose up` + +- Enable relative paths for `driver_opts.device` for `local` driver + +- Allow strings for `cpus` fields + +1.27.2 (2020-09-10) +------------------- + +### Bugs + +- Fix bug on `docker-compose run` container attach + +1.27.1 (2020-09-10) +------------------- + +### Bugs + +- Fix `docker-compose run` when `service.scale` is specified + +- Allow `driver` property for external networks as temporary workaround for swarm network propagation issue + +- Pin new internal schema version to `3.9` as the default + +- Preserve the version when configured in the compose file + +1.27.0 (2020-09-07) +------------------- + +### Features + +- Merge 2.x and 3.x compose formats and align with COMPOSE_SPEC schema + +- Implement service mode for ipc + +- Pass `COMPOSE_PROJECT_NAME` environment variable in container mode + +- Make run behave in the same way as up + +- Use `docker build` on `docker-compose run` when `COMPOSE_DOCKER_CLI_BUILD` environment variable is set + +- Use docker-py default API version for engine queries (`auto`) + +- Parse `network_mode` on build + +### Bugs + +- Ignore build context path validation when building is not required + +- Fix float to bytes conversion via docker-py bump to 4.3.1 + +- Fix scale bug when deploy section is set + +- Fix `docker-py` bump in `setup.py` + +- Fix experimental build failure detection + +- Fix context propagation to docker cli + +### Miscellaneous + +- Drop support for Python 2.7 + +- Bump `docker-py` to 4.3.1 + +- Bump `tox` to 3.19.0 + +- Bump `virtualenv` to 20.0.30 + +- Add script for docs syncronization + +1.26.2 (2020-07-02) +------------------- + +### Bugs + +- Enforce `docker-py` 4.2.2 as minimum version when installing with pip + +1.26.1 (2020-06-30) +------------------- + +### Features + +- Bump `docker-py` from 4.2.1 to 4.2.2 + +### Bugs + +- Enforce `docker-py` 4.2.1 as minimum version when installing with pip + +- Fix context load for non-docker endpoints + +1.26.0 (2020-06-03) +------------------- + +### Features + +- Add `docker context` support + +- Add missing test dependency `ddt` to `setup.py` + +- Add `--attach-dependencies` to command `up` for attaching to dependencies + +- Allow compatibility option with `COMPOSE_COMPATIBILITY` environment variable + +- Bump `Pytest` to 5.3.4 and add refactor compatibility with new version + +- Bump `OpenSSL` from 1.1.1f to 1.1.1g + +- Bump `docker-py` from 4.2.0 to 4.2.1 + +### Bugs + +- Properly escape values coming from env_files + +- Sync compose-schemas with upstream (docker/cli) + +- Remove `None` entries on exec command + +- Add `python-dotenv` to delegate `.env` file processing + +- Don't adjust output on terminal width when piped into another command + +- Show an error message when `version` attribute is malformed + +- Fix HTTPS connection when DOCKER_HOST is remote + +1.25.5 (2020-02-04) +------------------- + +### Features + +- Bump OpenSSL from 1.1.1d to 1.1.1f + +- Add 3.8 compose version + +1.25.4 (2020-01-23) +------------------- + +### Bugfixes + +- Fix CI script to enforce the minimal MacOS version to 10.11 + +- Fix docker-compose exec for keys with no value + +1.25.3 (2020-01-23) +------------------- + +### Bugfixes + +- Fix CI script to enforce the compilation with Python3 + +- Fix binary's sha256 in the release page + +1.25.2 (2020-01-20) +------------------- + +### Features + +- Allow compatibility option with `COMPOSE_COMPATIBILITY` environment variable + +- Bump PyInstaller from 3.5 to 3.6 + +- Bump pysocks from 1.6.7 to 1.7.1 + +- Bump websocket-client from 0.32.0 to 0.57.0 + +- Bump urllib3 from 1.24.2 to 1.25.7 + +- Bump jsonschema from 3.0.1 to 3.2.0 + +- Bump PyYAML from 4.2b1 to 5.3 + +- Bump certifi from 2017.4.17 to 2019.11.28 + +- Bump coverage from 4.5.4 to 5.0.3 + +- Bump paramiko from 2.6.0 to 2.7.1 + +- Bump cached-property from 1.3.0 to 1.5.1 + +- Bump minor Linux and MacOSX dependencies + +### Bugfixes + +- Validate version format on formats 2+ + +- Assume infinite terminal width when not running in a terminal + +1.25.1 (2020-01-06) +------------------- + +### Features + +- Bump `pytest-cov` 2.8.1 + +- Bump `flake8` 3.7.9 + +- Bump `coverage` 4.5.4 + +### Bugfixes + +- Decode APIError explanation to unicode before usage on start and create of a container + +- Reports when images that cannot be pulled and must be built + +- Discard label `com.docker.compose.filepaths` having None as value. Typically, when coming from stdin + +- Added OSX binary as a directory to solve slow start up time caused by MacOS Catalina binary scan + +- Passed in HOME env-var in container mode (running with `script/run/run.sh`) + +- Reverted behavior of "only pull images that we can't build" and replace by a warning informing the image we can't pull and must be built + + +1.25.0 (2019-11-18) +------------------- + +### Features + +- Set no-colors to true if CLICOLOR env variable is set to 0 + +- Add working dir, config files and env file in service labels + +- Add dependencies for ARM build + +- Add BuildKit support, use `DOCKER_BUILDKIT=1` and `COMPOSE_DOCKER_CLI_BUILD=1` + +- Bump paramiko to 2.6.0 + +- Add working dir, config files and env file in service labels + +- Add tag `docker-compose:latest` + +- Add `docker-compose:<version>-alpine` image/tag + +- Add `docker-compose:<version>-debian` image/tag + +- Bumped `docker-py` 4.1.0 + +- Supports `requests` up to 2.22.0 version + +- Drops empty tag on `build:cache_from` + +- `Dockerfile` now generates `libmusl` binaries for alpine + +- Only pull images that can't be built + +- Attribute `scale` can now accept `0` as a value + +- Added `--quiet` build flag + +- Added `--no-interpolate` to `docker-compose config` + +- Bump OpenSSL for macOS build (`1.1.0j` to `1.1.1c`) + +- Added `--no-rm` to `build` command + +- Added support for `credential_spec` + +- Resolve digests without pulling image + +- Upgrade `pyyaml` to `4.2b1` + +- Lowered severity to `warning` if `down` tries to remove nonexisting image + +- Use improved API fields for project events when possible + +- Update `setup.py` for modern `pypi/setuptools` and remove `pandoc` dependencies + +- Removed `Dockerfile.armhf` which is no longer needed + +### Bugfixes + +- Make container service color deterministic, remove red from chosen colors + +- Fix non ascii chars error. Python2 only + +- Format image size as decimal to be align with Docker CLI + +- Use Python Posix support to get tty size + +- Fix same file 'extends' optimization + +- Use python POSIX support to get tty size + +- Format image size as decimal to be align with Docker CLI + +- Fixed stdin_open + +- Fixed `--remove-orphans` when used with `up --no-start` + +- Fixed `docker-compose ps --all` + +- Fixed `depends_on` dependency recreation behavior + +- Fixed bash completion for `build --memory` + +- Fixed misleading warning concerning env vars when performing an `exec` command + +- Fixed failure check in parallel_execute_watch + +- Fixed race condition after pulling image + +- Fixed error on duplicate mount points + +- Fixed merge on networks section + +- Always connect Compose container to `stdin` + +- Fixed the presentation of failed services on 'docker-compose start' when containers are not available + +1.24.1 (2019-06-24) +------------------- + +### Bugfixes + +- Fixed acceptance tests + +1.24.0 (2019-03-28) +------------------- + +### Features + +- Added support for connecting to the Docker Engine using the `ssh` protocol. + +- Added a `--all` flag to `docker-compose ps` to include stopped one-off containers + in the command's output. + +- Add bash completion for `ps --all|-a` + +- Support for credential_spec + +- Add `--parallel` to `docker build`'s options in `bash` and `zsh` completion + +### Bugfixes + +- Fixed a bug where some valid credential helpers weren't properly handled by Compose + when attempting to pull images from private registries. + +- Fixed an issue where the output of `docker-compose start` before containers were created + was misleading + +- To match the Docker CLI behavior and to avoid confusing issues, Compose will no longer + accept whitespace in variable names sourced from environment files. + +- Compose will now report a configuration error if a service attempts to declare + duplicate mount points in the volumes section. + +- Fixed an issue with the containerized version of Compose that prevented users from + writing to stdin during interactive sessions started by `run` or `exec`. + +- One-off containers started by `run` no longer adopt the restart policy of the service, + and are instead set to never restart. + +- Fixed an issue that caused some container events to not appear in the output of + the `docker-compose events` command. + +- Missing images will no longer stop the execution of `docker-compose down` commands + (a warning will be displayed instead). + +- Force `virtualenv` version for macOS CI + +- Fix merging of compose files when network has `None` config + +- Fix `CTRL+C` issues by enabling `bootloader_ignore_signals` in `pyinstaller` + +- Bump `docker-py` version to `3.7.2` to fix SSH and proxy config issues + +- Fix release script and some typos on release documentation + +1.23.2 (2018-11-28) +------------------- + +### Bugfixes + +- Reverted a 1.23.0 change that appended random strings to container names + created by `docker-compose up`, causing addressability issues. + Note: Containers created by `docker-compose run` will continue to use + randomly generated names to avoid collisions during parallel runs. + +- Fixed an issue where some `dockerfile` paths would fail unexpectedly when + attempting to build on Windows. + +- Fixed a bug where build context URLs would fail to build on Windows. + +- Fixed a bug that caused `run` and `exec` commands to fail for some otherwise + accepted values of the `--host` parameter. + +- Fixed an issue where overrides for the `storage_opt` and `isolation` keys in + service definitions weren't properly applied. + +- Fixed a bug where some invalid Compose files would raise an uncaught + exception during validation. + +1.23.1 (2018-11-01) +------------------- + +### Bugfixes + +- Fixed a bug where working with containers created with a previous (< 1.23.0) + version of Compose would cause unexpected crashes + +- Fixed an issue where the behavior of the `--project-directory` flag would + vary depending on which subcommand was being used. + +1.23.0 (2018-10-30) +------------------- + +### Important note + +The default naming scheme for containers created by Compose in this version +has changed from `<project>_<service>_<index>` to +`<project>_<service>_<index>_<slug>`, where `<slug>` is a randomly-generated +hexadecimal string. Please make sure to update scripts relying on the old +naming scheme accordingly before upgrading. + +### Features + +- Logs for containers restarting after a crash will now appear in the output + of the `up` and `logs` commands. + +- Added `--hash` option to the `docker-compose config` command, allowing users + to print a hash string for each service's configuration to facilitate rolling + updates. + +- Added `--parallel` flag to the `docker-compose build` command, allowing + Compose to build up to 5 images simultaneously. + +- Output for the `pull` command now reports status / progress even when pulling + multiple images in parallel. + +- For images with multiple names, Compose will now attempt to match the one + present in the service configuration in the output of the `images` command. + +### Bugfixes + +- Parallel `run` commands for the same service will no longer fail due to name + collisions. + +- Fixed an issue where paths longer than 260 characters on Windows clients would + cause `docker-compose build` to fail. + +- Fixed a bug where attempting to mount `/var/run/docker.sock` with + Docker Desktop for Windows would result in failure. + +- The `--project-directory` option is now used by Compose to determine where to + look for the `.env` file. + +- `docker-compose build` no longer fails when attempting to pull an image with + credentials provided by the gcloud credential helper. + +- Fixed the `--exit-code-from` option in `docker-compose up` to always report + the actual exit code even when the watched container isn't the cause of the + exit. + +- Fixed an issue that would prevent recreating a service in some cases where + a volume would be mapped to the same mountpoint as a volume declared inside + the image's Dockerfile. + +- Fixed a bug that caused hash configuration with multiple networks to be + inconsistent, causing some services to be unnecessarily restarted. + +- Fixed a bug that would cause failures with variable substitution for services + with a name containing one or more dot characters + +- Fixed a pipe handling issue when using the containerized version of Compose. + +- Fixed a bug causing `external: false` entries in the Compose file to be + printed as `external: true` in the output of `docker-compose config` + +- Fixed a bug where issuing a `docker-compose pull` command on services + without a defined image key would cause Compose to crash + +- Volumes and binds are now mounted in the order they're declared in the + service definition + +### Miscellaneous + +- The `zsh` completion script has been updated with new options, and no + longer suggests container names where service names are expected. + +1.22.0 (2018-07-17) +------------------- + +### Features + +#### Compose format version 3.7 + +- Introduced version 3.7 of the `docker-compose.yml` specification. + This version requires Docker Engine 18.06.0 or above. + +- Added support for `rollback_config` in the deploy configuration + +- Added support for the `init` parameter in service configurations + +- Added support for extension fields in service, network, volume, secret, + and config configurations + +#### Compose format version 2.4 + +- Added support for extension fields in service, network, + and volume configurations + +### Bugfixes + +- Fixed a bug that prevented deployment with some Compose files when + `DOCKER_DEFAULT_PLATFORM` was set + +- Compose will no longer try to create containers or volumes with + invalid starting characters + +- Fixed several bugs that prevented Compose commands from working properly + with containers created with an older version of Compose + +- Fixed an issue with the output of `docker-compose config` with the + `--compatibility-mode` flag enabled when the source file contains + attachable networks + +- Fixed a bug that prevented the `gcloud` credential store from working + properly when used with the Compose binary on UNIX + +- Fixed a bug that caused connection errors when trying to operate + over a non-HTTPS TCP connection on Windows + +- Fixed a bug that caused builds to fail on Windows if the Dockerfile + was located in a subdirectory of the build context + +- Fixed an issue that prevented proper parsing of UTF-8 BOM encoded + Compose files on Windows + +- Fixed an issue with handling of the double-wildcard (`**`) pattern in `.dockerignore` files when using `docker-compose build` + +- Fixed a bug that caused auth values in legacy `.dockercfg` files to be ignored +- `docker-compose build` will no longer attempt to create image names starting with an invalid character + +1.21.2 (2018-05-03) +------------------- + +### Bugfixes + +- Fixed a bug where the ip_range attribute in IPAM configs was prevented + from passing validation + +1.21.1 (2018-04-27) +------------------- + +### Bugfixes + +- In 1.21.0, we introduced a change to how project names are sanitized for + internal use in resource names. This caused issues when manipulating an + existing, deployed application whose name had changed as a result. + This release properly detects resources using "legacy" naming conventions. + +- Fixed an issue where specifying an in-context Dockerfile using an absolute + path would fail despite being valid. + +- Fixed a bug where IPAM option changes were incorrectly detected, preventing + redeployments. + +- Validation of v2 files now properly checks the structure of IPAM configs. + +- Improved support for credentials stores on Windows to include binaries using + extensions other than `.exe`. The list of valid extensions is determined by + the contents of the `PATHEXT` environment variable. + +- Fixed a bug where Compose would generate invalid binds containing duplicate + elements with some v3.2 files, triggering errors at the Engine level during + deployment. + +1.21.0 (2018-04-10) +------------------- + +### New features + +#### Compose file version 2.4 + +- Introduced version 2.4 of the `docker-compose.yml` specification. + This version requires Docker Engine 17.12.0 or above. + +- Added support for the `platform` parameter in service definitions. + If supplied, the parameter is also used when performing build for the + service. + +#### Compose file version 2.2 and up + +- Added support for the `cpu_rt_period` and `cpu_rt_runtime` parameters + in service definitions (2.x only). + +#### Compose file version 2.1 and up + +- Added support for the `cpu_period` parameter in service definitions + (2.x only). + +- Added support for the `isolation` parameter in service build configurations. + Additionally, the `isolation` parameter is used for builds as well if no + `build.isolation` parameter is defined. (2.x only) + +#### All formats + +- Added support for the `--workdir` flag in `docker-compose exec`. + +- Added support for the `--compress` flag in `docker-compose build`. + +- `docker-compose pull` is now performed in parallel by default. You can + opt out using the `--no-parallel` flag. The `--parallel` flag is now + deprecated and will be removed in a future version. + +- Dashes and underscores in project names are no longer stripped out. + +- `docker-compose build` now supports the use of Dockerfile from outside + the build context. + +### Bugfixes + +- Compose now checks that the volume's configuration matches the remote + volume, and errors out if a mismatch is detected. + +- Fixed a bug that caused Compose to raise unexpected errors when attempting + to create several one-off containers in parallel. + +- Fixed a bug with argument parsing when using `docker-machine config` to + generate TLS flags for `exec` and `run` commands. + +- Fixed a bug where variable substitution with an empty default value + (e.g. `${VAR:-}`) would print an incorrect warning. + +- Improved resilience when encoding of the Compose file doesn't match the + system's. Users are encouraged to use UTF-8 when possible. + +- Fixed a bug where external overlay networks in Swarm would be incorrectly + recognized as inexistent by Compose, interrupting otherwise valid + operations. + +1.20.1 (2018-03-21) +------------------- + +### Bugfixes + +- Fixed an issue where `docker-compose build` would error out if the + build context contained directory symlinks + +1.20.0 (2018-03-20) +------------------- + +### New features + +#### Compose file version 3.6 + +- Introduced version 3.6 of the `docker-compose.yml` specification. + This version requires Docker Engine 18.02.0 or above. + +- Added support for the `tmpfs.size` property in volume mappings + +#### Compose file version 3.2 and up + +- The `--build-arg` option can now be used without specifying a service + in `docker-compose build` + +#### Compose file version 2.3 + +- Added support for `device_cgroup_rules` in service definitions + +- Added support for the `tmpfs.size` property in long-form volume mappings + +- The `--build-arg` option can now be used without specifying a service + in `docker-compose build` + +#### All formats + +- Added a `--log-level` option to the top-level `docker-compose` command. + Accepted values are `debug`, `info`, `warning`, `error`, `critical`. + Default log level is `info` + +- `docker-compose run` now allows users to unset the container's entrypoint + +- Proxy configuration found in the `~/.docker/config.json` file now populates + environment and build args for containers created by Compose + +- Added the `--use-aliases` flag to `docker-compose run`, indicating that + network aliases declared in the service's config should be used for the + running container + +- Added the `--include-deps` flag to `docker-compose pull` + +- `docker-compose run` now kills and removes the running container upon + receiving `SIGHUP` + +- `docker-compose ps` now shows the containers' health status if available + +- Added the long-form `--detach` option to the `exec`, `run` and `up` + commands + +### Bugfixes + +- Fixed `.dockerignore` handling, notably with regard to absolute paths + and last-line precedence rules + +- Fixed an issue where Compose would make costly DNS lookups when connecting + to the Engine when using Docker For Mac + +- Fixed a bug introduced in 1.19.0 which caused the default certificate path + to not be honored by Compose + +- Fixed a bug where Compose would incorrectly check whether a symlink's + destination was accessible when part of a build context + +- Fixed a bug where `.dockerignore` files containing lines of whitespace + caused Compose to error out on Windows + +- Fixed a bug where `--tls*` and `--host` options wouldn't be properly honored + for interactive `run` and `exec` commands + +- A `seccomp:<filepath>` entry in the `security_opt` config now correctly + sends the contents of the file to the engine + +- ANSI output for `up` and `down` operations should no longer affect the wrong + lines + +- Improved support for non-unicode locales + +- Fixed a crash occurring on Windows when the user's home directory name + contained non-ASCII characters + +- Fixed a bug occurring during builds caused by files with a negative `mtime` + values in the build context + +- Fixed an encoding bug when streaming build progress + +1.19.0 (2018-02-07) +------------------- + +### Breaking changes + +- On UNIX platforms, interactive `run` and `exec` commands now require + the `docker` CLI to be installed on the client by default. To revert + to the previous behavior, users may set the `COMPOSE_INTERACTIVE_NO_CLI` + environment variable. + +### New features + +#### Compose file version 3.x + +- The output of the `config` command should now merge `deploy` options from + several Compose files in a more accurate manner + +#### Compose file version 2.3 + +- Added support for the `runtime` option in service definitions + +#### Compose file version 2.1 and up + +- Added support for the `${VAR:?err}` and `${VAR?err}` variable interpolation + syntax to indicate mandatory variables + +#### Compose file version 2.x + +- Added `priority` key to service network mappings, allowing the user to + define in which order the specified service will connect to each network + +#### All formats + +- Added `--renew-anon-volumes` (shorthand `-V`) to the `up` command, + preventing Compose from recovering volume data from previous containers for + anonymous volumes + +- Added limit for number of simultaneous parallel operations, which should + prevent accidental resource exhaustion of the server. Default is 64 and + can be configured using the `COMPOSE_PARALLEL_LIMIT` environment variable + +- Added `--always-recreate-deps` flag to the `up` command to force recreating + dependent services along with the dependency owner + +- Added `COMPOSE_IGNORE_ORPHANS` environment variable to forgo orphan + container detection and suppress warnings + +- Added `COMPOSE_FORCE_WINDOWS_HOST` environment variable to force Compose + to parse volume definitions as if the Docker host was a Windows system, + even if Compose itself is currently running on UNIX + +- Bash completion should now be able to better differentiate between running, + stopped and paused services + +### Bugfixes + +- Fixed a bug that would cause the `build` command to report a connection + error when the build context contained unreadable files or FIFO objects. + These file types will now be handled appropriately + +- Fixed various issues around interactive `run`/`exec` sessions. + +- Fixed a bug where setting TLS options with environment and CLI flags + simultaneously would result in part of the configuration being ignored + +- Fixed a bug where the DOCKER_TLS_VERIFY environment variable was being + ignored by Compose + +- Fixed a bug where the `-d` and `--timeout` flags in `up` were erroneously + marked as incompatible + +- Fixed a bug where the recreation of a service would break if the image + associated with the previous container had been removed + +- Fixed a bug where updating a mount's target would break Compose when + trying to recreate the associated service + +- Fixed a bug where `tmpfs` volumes declared using the extended syntax in + Compose files using version 3.2 would be erroneously created as anonymous + volumes instead + +- Fixed a bug where type conversion errors would print a stacktrace instead + of exiting gracefully + +- Fixed some errors related to unicode handling + +- Dependent services no longer get recreated along with the dependency owner + if their configuration hasn't changed + +- Added better validation of `labels` fields in Compose files. Label values + containing scalar types (number, boolean) now get automatically converted + to strings + +1.18.0 (2017-12-15) +------------------- + +### New features + +#### Compose file version 3.5 + +- Introduced version 3.5 of the `docker-compose.yml` specification. + This version requires Docker Engine 17.06.0 or above + +- Added support for the `shm_size` parameter in build configurations + +- Added support for the `isolation` parameter in service definitions + +- Added support for custom names for network, secret and config definitions + +#### Compose file version 2.3 + +- Added support for `extra_hosts` in build configuration + +- Added support for the [long syntax](https://docs.docker.com/compose/compose-file/#long-syntax-3) for volume entries, as previously introduced in the 3.2 format. + Note that using this syntax will create [mounts](https://docs.docker.com/engine/admin/volumes/bind-mounts/) instead of volumes. + +#### Compose file version 2.1 and up + +- Added support for the `oom_kill_disable` parameter in service definitions + (2.x only) + +- Added support for custom names for network definitions (2.x only) + + +#### All formats + +- Values interpolated from the environment will now be converted to the + proper type when used in non-string fields. + +- Added support for `--label` in `docker-compose run` + +- Added support for `--timeout` in `docker-compose down` + +- Added support for `--memory` in `docker-compose build` + +- Setting `stop_grace_period` in service definitions now also sets the + container's `stop_timeout` + +### Bugfixes + +- Fixed an issue where Compose was still handling service hostname according + to legacy engine behavior, causing hostnames containing dots to be cut up + +- Fixed a bug where the `X-Y:Z` syntax for ports was considered invalid + by Compose + +- Fixed an issue with CLI logging causing duplicate messages and inelegant + output to occur + +- Fixed an issue that caused `stop_grace_period` to be ignored when using + multiple Compose files + +- Fixed a bug that caused `docker-compose images` to crash when using + untagged images + +- Fixed a bug where the valid `${VAR:-}` syntax would cause Compose to + error out + +- Fixed a bug where `env_file` entries using an UTF-8 BOM were being read + incorrectly + +- Fixed a bug where missing secret files would generate an empty directory + in their place + +- Fixed character encoding issues in the CLI's error handlers + +- Added validation for the `test` field in healthchecks + +- Added validation for the `subnet` field in IPAM configurations + +- Added validation for `volumes` properties when using the long syntax in + service definitions + +- The CLI now explicit prevents using `-d` and `--timeout` together + in `docker-compose up` + +1.17.1 (2017-11-08) +------------------ + +### Bugfixes + +- Fixed a bug that would prevent creating new containers when using + container labels in the list format as part of the service's definition. + +1.17.0 (2017-11-02) +------------------- + +### New features + +#### Compose file version 3.4 + +- Introduced version 3.4 of the `docker-compose.yml` specification. + This version requires to be used with Docker Engine 17.06.0 or above. + +- Added support for `cache_from`, `network` and `target` options in build + configurations + +- Added support for the `order` parameter in the `update_config` section + +- Added support for setting a custom name in volume definitions using + the `name` parameter + +#### Compose file version 2.3 + +- Added support for `shm_size` option in build configuration + +#### Compose file version 2.x + +- Added support for extension fields (`x-*`). Also available for v3.4 files + +#### All formats + +- Added new `--no-start` to the `up` command, allowing users to create all + resources (networks, volumes, containers) without starting services. + The `create` command is deprecated in favor of this new option + +### Bugfixes + +- Fixed a bug where `extra_hosts` values would be overridden by extension + files instead of merging together + +- Fixed a bug where the validation for v3.2 files would prevent using the + `consistency` field in service volume definitions + +- Fixed a bug that would cause a crash when configuration fields expecting + unique items would contain duplicates + +- Fixed a bug where mount overrides with a different mode would create a + duplicate entry instead of overriding the original entry + +- Fixed a bug where build labels declared as a list wouldn't be properly + parsed + +- Fixed a bug where the output of `docker-compose config` would be invalid + for some versions if the file contained custom-named external volumes + +- Improved error handling when issuing a build command on Windows using an + unsupported file version + +- Fixed an issue where networks with identical names would sometimes be + created when running `up` commands concurrently. + +1.16.1 (2017-09-01) +------------------- + +### Bugfixes + +- Fixed bug that prevented using `extra_hosts` in several configuration files. + +1.16.0 (2017-08-31) +------------------- + +### New features + +#### Compose file version 2.3 + +- Introduced version 2.3 of the `docker-compose.yml` specification. + This version requires to be used with Docker Engine 17.06.0 or above. + +- Added support for the `target` parameter in build configurations + +- Added support for the `start_period` parameter in healthcheck + configurations + +#### Compose file version 2.x + +- Added support for the `blkio_config` parameter in service definitions + +- Added support for setting a custom name in volume definitions using + the `name` parameter (not available for version 2.0) + +#### All formats + +- Added new CLI flag `--no-ansi` to suppress ANSI control characters in + output + +### Bugfixes + +- Fixed a bug where nested `extends` instructions weren't resolved + properly, causing "file not found" errors + +- Fixed several issues with `.dockerignore` parsing + +- Fixed issues where logs of TTY-enabled services were being printed + incorrectly and causing `MemoryError` exceptions + +- Fixed a bug where printing application logs would sometimes be interrupted + by a `UnicodeEncodeError` exception on Python 3 + +- The `$` character in the output of `docker-compose config` is now + properly escaped + +- Fixed a bug where running `docker-compose top` would sometimes fail + with an uncaught exception + +- Fixed a bug where `docker-compose pull` with the `--parallel` flag + would return a `0` exit code when failing + +- Fixed an issue where keys in `deploy.resources` were not being validated + +- Fixed an issue where the `logging` options in the output of + `docker-compose config` would be set to `null`, an invalid value + +- Fixed the output of the `docker-compose images` command when an image + would come from a private repository using an explicit port number + +- Fixed the output of `docker-compose config` when a port definition used + `0` as the value for the published port + +1.15.0 (2017-07-26) +------------------- + +### New features + +#### Compose file version 2.2 + +- Added support for the `network` parameter in build configurations. + +#### Compose file version 2.1 and up + +- The `pid` option in a service's definition now supports a `service:<name>` + value. + +- Added support for the `storage_opt` parameter in in service definitions. + This option is not available for the v3 format + +#### All formats + +- Added `--quiet` flag to `docker-compose pull`, suppressing progress output + +- Some improvements to CLI output + +### Bugfixes + +- Volumes specified through the `--volume` flag of `docker-compose run` now + complement volumes declared in the service's definition instead of replacing + them + +- Fixed a bug where using multiple Compose files would unset the scale value + defined inside the Compose file. + +- Fixed an issue where the `credHelpers` entries in the `config.json` file + were not being honored by Compose + +- Fixed a bug where using multiple Compose files with port declarations + would cause failures in Python 3 environments + +- Fixed a bug where some proxy-related options present in the user's + environment would prevent Compose from running + +- Fixed an issue where the output of `docker-compose config` would be invalid + if the original file used `Y` or `N` values + +- Fixed an issue preventing `up` operations on a previously created stack on + Windows Engine. + +1.14.0 (2017-06-19) +------------------- + +### New features + +#### Compose file version 3.3 + +- Introduced version 3.3 of the `docker-compose.yml` specification. + This version requires to be used with Docker Engine 17.06.0 or above. + Note: the `credential_spec` and `configs` keys only apply to Swarm services + and will be ignored by Compose + +#### Compose file version 2.2 + +- Added the following parameters in service definitions: `cpu_count`, + `cpu_percent`, `cpus` + +#### Compose file version 2.1 + +- Added support for build labels. This feature is also available in the + 2.2 and 3.3 formats. + +#### All formats + +- Added shorthand `-u` for `--user` flag in `docker-compose exec` + +- Differences in labels between the Compose file and remote network + will now print a warning instead of preventing redeployment. + +### Bugfixes + +- Fixed a bug where service's dependencies were being rescaled to their + default scale when running a `docker-compose run` command + +- Fixed a bug where `docker-compose rm` with the `--stop` flag was not + behaving properly when provided with a list of services to remove + +- Fixed a bug where `cache_from` in the build section would be ignored when + using more than one Compose file. + +- Fixed a bug that prevented binding the same port to different IPs when + using more than one Compose file. + +- Fixed a bug where override files would not be picked up by Compose if they + had the `.yaml` extension + +- Fixed a bug on Windows Engine where networks would be incorrectly flagged + for recreation + +- Fixed a bug where services declaring ports would cause crashes on some + versions of Python 3 + +- Fixed a bug where the output of `docker-compose config` would sometimes + contain invalid port definitions + +1.13.0 (2017-05-02) +------------------- + +### Breaking changes + +- `docker-compose up` now resets a service's scaling to its default value. + You can use the newly introduced `--scale` option to specify a custom + scale value + +### New features + +#### Compose file version 2.2 + +- Introduced version 2.2 of the `docker-compose.yml` specification. This + version requires to be used with Docker Engine 1.13.0 or above + +- Added support for `init` in service definitions. + +- Added support for `scale` in service definitions. The configuration's value + can be overridden using the `--scale` flag in `docker-compose up`. + Please note that the `scale` command is disabled for this file format + +#### Compose file version 2.x + +- Added support for `options` in the `ipam` section of network definitions + +### Bugfixes + +- Fixed a bug where paths provided to compose via the `-f` option were not + being resolved properly + +- Fixed a bug where the `ext_ip::target_port` notation in the ports section + was incorrectly marked as invalid + +- Fixed an issue where the `exec` command would sometimes not return control + to the terminal when using the `-d` flag + +- Fixed a bug where secrets were missing from the output of the `config` + command for v3.2 files + +- Fixed an issue where `docker-compose` would hang if no internet connection + was available + +- Fixed an issue where paths containing unicode characters passed via the `-f` + flag were causing Compose to crash + +- Fixed an issue where the output of `docker-compose config` would be invalid + if the Compose file contained external secrets + +- Fixed a bug where using `--exit-code-from` with `up` would fail if Compose + was installed in a Python 3 environment + +- Fixed a bug where recreating containers using a combination of `tmpfs` and + `volumes` would result in an invalid config state + + +1.12.0 (2017-04-04) +------------------- + +### New features + +#### Compose file version 3.2 + +- Introduced version 3.2 of the `docker-compose.yml` specification + +- Added support for `cache_from` in the `build` section of services + +- Added support for the new expanded ports syntax in service definitions + +- Added support for the new expanded volumes syntax in service definitions + +#### Compose file version 2.1 + +- Added support for `pids_limit` in service definitions + +#### Compose file version 2.0 and up + +- Added `--volumes` option to `docker-compose config` that lists named + volumes declared for that project + +- Added support for `mem_reservation` in service definitions (2.x only) + +- Added support for `dns_opt` in service definitions (2.x only) + +#### All formats + +- Added a new `docker-compose images` command that lists images used by + the current project's containers + +- Added a `--stop` (shorthand `-s`) option to `docker-compose rm` that stops + the running containers before removing them + +- Added a `--resolve-image-digests` option to `docker-compose config` that + pins the image version for each service to a permanent digest + +- Added a `--exit-code-from SERVICE` option to `docker-compose up`. When + used, `docker-compose` will exit on any container's exit with the code + corresponding to the specified service's exit code + +- Added a `--parallel` option to `docker-compose pull` that enables images + for multiple services to be pulled simultaneously + +- Added a `--build-arg` option to `docker-compose build` + +- Added a `--volume <volume_mapping>` (shorthand `-v`) option to + `docker-compose run` to declare runtime volumes to be mounted + +- Added a `--project-directory PATH` option to `docker-compose` that will + affect path resolution for the project + +- When using `--abort-on-container-exit` in `docker-compose up`, the exit + code for the container that caused the abort will be the exit code of + the `docker-compose up` command + +- Users can now configure which path separator character they want to use + to separate the `COMPOSE_FILE` environment value using the + `COMPOSE_PATH_SEPARATOR` environment variable + +- Added support for port range to single port in port mappings + (e.g. `8000-8010:80`) + +### Bugfixes + +- `docker-compose run --rm` now removes anonymous volumes after execution, + matching the behavior of `docker run --rm`. + +- Fixed a bug where override files containing port lists would cause a + TypeError to be raised + +- Fixed a bug where the `deploy` key would be missing from the output of + `docker-compose config` + +- Fixed a bug where scaling services up or down would sometimes re-use + obsolete containers + +- Fixed a bug where the output of `docker-compose config` would be invalid + if the project declared anonymous volumes + +- Variable interpolation now properly occurs in the `secrets` section of + the Compose file + +- The `secrets` section now properly appears in the output of + `docker-compose config` + +- Fixed a bug where changes to some networks properties would not be + detected against previously created networks + +- Fixed a bug where `docker-compose` would crash when trying to write into + a closed pipe + +- Fixed an issue where Compose would not pick up on the value of + COMPOSE_TLS_VERSION when used in combination with command-line TLS flags + +1.11.2 (2017-02-17) +------------------- + +### Bugfixes + +- Fixed a bug that was preventing secrets configuration from being + loaded properly + +- Fixed a bug where the `docker-compose config` command would fail + if the config file contained secrets definitions + +- Fixed an issue where Compose on some linux distributions would + pick up and load an outdated version of the requests library + +- Fixed an issue where socket-type files inside a build folder + would cause `docker-compose` to crash when trying to build that + service + +- Fixed an issue where recursive wildcard patterns `**` were not being + recognized in `.dockerignore` files. + +1.11.1 (2017-02-09) +------------------- + +### Bugfixes + +- Fixed a bug where the 3.1 file format was not being recognized as valid + by the Compose parser + +1.11.0 (2017-02-08) +------------------- + +### New Features + +#### Compose file version 3.1 + +- Introduced version 3.1 of the `docker-compose.yml` specification. This + version requires Docker Engine 1.13.0 or above. It introduces support + for secrets. See the documentation for more information + +#### Compose file version 2.0 and up + +- Introduced the `docker-compose top` command that displays processes running + for the different services managed by Compose. + +### Bugfixes + +- Fixed a bug where extending a service defining a healthcheck dictionary + would cause `docker-compose` to error out. + +- Fixed an issue where the `pid` entry in a service definition was being + ignored when using multiple Compose files. + +1.10.1 (2017-02-01) +------------------ + +### Bugfixes + +- Fixed an issue where presence of older versions of the docker-py + package would cause unexpected crashes while running Compose + +- Fixed an issue where healthcheck dependencies would be lost when + using multiple compose files for a project + +- Fixed a few issues that made the output of the `config` command + invalid + +- Fixed an issue where adding volume labels to v3 Compose files would + result in an error + +- Fixed an issue on Windows where build context paths containing unicode + characters were being improperly encoded + +- Fixed a bug where Compose would occasionally crash while streaming logs + when containers would stop or restart + +1.10.0 (2017-01-18) +------------------- + +### New Features + +#### Compose file version 3.0 + +- Introduced version 3.0 of the `docker-compose.yml` specification. This + version requires to be used with Docker Engine 1.13 or above and is + specifically designed to work with the `docker stack` commands. + +#### Compose file version 2.1 and up + +- Healthcheck configuration can now be done in the service definition using + the `healthcheck` parameter + +- Containers dependencies can now be set up to wait on positive healthchecks + when declared using `depends_on`. See the documentation for the updated + syntax. + **Note:** This feature will not be ported to version 3 Compose files. + +- Added support for the `sysctls` parameter in service definitions + +- Added support for the `userns_mode` parameter in service definitions + +- Compose now adds identifying labels to networks and volumes it creates + +#### Compose file version 2.0 and up + +- Added support for the `stop_grace_period` option in service definitions. + +### Bugfixes + +- Colored output now works properly on Windows. + +- Fixed a bug where docker-compose run would fail to set up link aliases + in interactive mode on Windows. + +- Networks created by Compose are now always made attachable + (Compose files v2.1 and up). + +- Fixed a bug where falsy values of `COMPOSE_CONVERT_WINDOWS_PATHS` + (`0`, `false`, empty value) were being interpreted as true. + +- Fixed a bug where forward slashes in some .dockerignore patterns weren't + being parsed correctly on Windows + + +1.9.0 (2016-11-16) +----------------- + +**Breaking changes** + +- When using Compose with Docker Toolbox/Machine on Windows, volume paths are + no longer converted from `C:\Users` to `/c/Users`-style by default. To + re-enable this conversion so that your volumes keep working, set the + environment variable `COMPOSE_CONVERT_WINDOWS_PATHS=1`. Users of + Docker for Windows are not affected and do not need to set the variable. + +New Features + +- Interactive mode for `docker-compose run` and `docker-compose exec` is + now supported on Windows platforms. Please note that the `docker` binary + is required to be present on the system for this feature to work. + +- Introduced version 2.1 of the `docker-compose.yml` specification. This + version requires to be used with Docker Engine 1.12 or above. + - Added support for setting volume labels and network labels in + `docker-compose.yml`. + - Added support for the `isolation` parameter in service definitions. + - Added support for link-local IPs in the service networks definitions. + - Added support for shell-style inline defaults in variable interpolation. + The supported forms are `${FOO-default}` (fall back if FOO is unset) and + `${FOO:-default}` (fall back if FOO is unset or empty). + +- Added support for the `group_add` and `oom_score_adj` parameters in + service definitions. + +- Added support for the `internal` and `enable_ipv6` parameters in network + definitions. + +- Compose now defaults to using the `npipe` protocol on Windows. + +- Overriding a `logging` configuration will now properly merge the `options` + mappings if the `driver` values do not conflict. + +Bug Fixes + +- Fixed several bugs related to `npipe` protocol support on Windows. + +- Fixed an issue with Windows paths being incorrectly converted when + using Docker on Windows Server. + +- Fixed a bug where an empty `restart` value would sometimes result in an + exception being raised. + +- Fixed an issue where service logs containing unicode characters would + sometimes cause an error to occur. + +- Fixed a bug where unicode values in environment variables would sometimes + raise a unicode exception when retrieved. + +- Fixed an issue where Compose would incorrectly detect a configuration + mismatch for overlay networks. + + +1.8.1 (2016-09-22) +----------------- + +Bug Fixes + +- Fixed a bug where users using a credentials store were not able + to access their private images. + +- Fixed a bug where users using identity tokens to authenticate + were not able to access their private images. + +- Fixed a bug where an `HttpHeaders` entry in the docker configuration + file would cause Compose to crash when trying to build an image. + +- Fixed a few bugs related to the handling of Windows paths in volume + binding declarations. + +- Fixed a bug where Compose would sometimes crash while trying to + read a streaming response from the engine. + +- Fixed an issue where Compose would crash when encountering an API error + while streaming container logs. + +- Fixed an issue where Compose would erroneously try to output logs from + drivers not handled by the Engine's API. + +- Fixed a bug where options from the `docker-machine config` command would + not be properly interpreted by Compose. + +- Fixed a bug where the connection to the Docker Engine would + sometimes fail when running a large number of services simultaneously. + +- Fixed an issue where Compose would sometimes print a misleading + suggestion message when running the `bundle` command. + +- Fixed a bug where connection errors would not be handled properly by + Compose during the project initialization phase. + +- Fixed a bug where a misleading error would appear when encountering + a connection timeout. + + +1.8.0 (2016-06-14) +----------------- + +**Breaking Changes** + +- As announced in 1.7.0, `docker-compose rm` now removes containers + created by `docker-compose run` by default. + +- Setting `entrypoint` on a service now empties out any default + command that was set on the image (i.e. any `CMD` instruction in the + Dockerfile used to build it). This makes it consistent with + the `--entrypoint` flag to `docker run`. + +New Features + +- Added `docker-compose bundle`, a command that builds a bundle file + to be consumed by the new *Docker Stack* commands in Docker 1.12. + +- Added `docker-compose push`, a command that pushes service images + to a registry. + +- Compose now supports specifying a custom TLS version for + interaction with the Docker Engine using the `COMPOSE_TLS_VERSION` + environment variable. + +Bug Fixes + +- Fixed a bug where Compose would erroneously try to read `.env` + at the project's root when it is a directory. + +- `docker-compose run -e VAR` now passes `VAR` through from the shell + to the container, as with `docker run -e VAR`. + +- Improved config merging when multiple compose files are involved + for several service sub-keys. + +- Fixed a bug where volume mappings containing Windows drives would + sometimes be parsed incorrectly. + +- Fixed a bug in Windows environment where volume mappings of the + host's root directory would be parsed incorrectly. + +- Fixed a bug where `docker-compose config` would output an invalid + Compose file if external networks were specified. + +- Fixed an issue where unset buildargs would be assigned a string + containing `'None'` instead of the expected empty value. + +- Fixed a bug where yes/no prompts on Windows would not show before + receiving input. + +- Fixed a bug where trying to `docker-compose exec` on Windows + without the `-d` option would exit with a stacktrace. This will + still fail for the time being, but should do so gracefully. + +- Fixed a bug where errors during `docker-compose up` would show + an unrelated stacktrace at the end of the process. + +- `docker-compose create` and `docker-compose start` show more + descriptive error messages when something goes wrong. + + +1.7.1 (2016-05-04) +----------------- + +Bug Fixes + +- Fixed a bug where the output of `docker-compose config` for v1 files + would be an invalid configuration file. + +- Fixed a bug where `docker-compose config` would not check the validity + of links. + +- Fixed an issue where `docker-compose help` would not output a list of + available commands and generic options as expected. + +- Fixed an issue where filtering by service when using `docker-compose logs` + would not apply for newly created services. + +- Fixed a bug where unchanged services would sometimes be recreated in + in the up phase when using Compose with Python 3. + +- Fixed an issue where API errors encountered during the up phase would + not be recognized as a failure state by Compose. + +- Fixed a bug where Compose would raise a NameError because of an undefined + exception name on non-Windows platforms. + +- Fixed a bug where the wrong version of `docker-py` would sometimes be + installed alongside Compose. + +- Fixed a bug where the host value output by `docker-machine config default` + would not be recognized as valid options by the `docker-compose` + command line. + +- Fixed an issue where Compose would sometimes exit unexpectedly while + reading events broadcasted by a Swarm cluster. + +- Corrected a statement in the docs about the location of the `.env` file, + which is indeed read from the current directory, instead of in the same + location as the Compose file. + + +1.7.0 (2016-04-13) +------------------ + +**Breaking Changes** + +- `docker-compose logs` no longer follows log output by default. It now + matches the behaviour of `docker logs` and exits after the current logs + are printed. Use `-f` to get the old default behaviour. + +- Booleans are no longer allows as values for mappings in the Compose file + (for keys `environment`, `labels` and `extra_hosts`). Previously this + was a warning. Boolean values should be quoted so they become string values. + +New Features + +- Compose now looks for a `.env` file in the directory where it's run and + reads any environment variables defined inside, if they're not already + set in the shell environment. This lets you easily set defaults for + variables used in the Compose file, or for any of the `COMPOSE_*` or + `DOCKER_*` variables. + +- Added a `--remove-orphans` flag to both `docker-compose up` and + `docker-compose down` to remove containers for services that were removed + from the Compose file. + +- Added a `--all` flag to `docker-compose rm` to include containers created + by `docker-compose run`. This will become the default behavior in the next + version of Compose. + +- Added support for all the same TLS configuration flags used by the `docker` + client: `--tls`, `--tlscert`, `--tlskey`, etc. + +- Compose files now support the `tmpfs` and `shm_size` options. + +- Added the `--workdir` flag to `docker-compose run` + +- `docker-compose logs` now shows logs for new containers that are created + after it starts. + +- The `COMPOSE_FILE` environment variable can now contain multiple files, + separated by the host system's standard path separator (`:` on Mac/Linux, + `;` on Windows). + +- You can now specify a static IP address when connecting a service to a + network with the `ipv4_address` and `ipv6_address` options. + +- Added `--follow`, `--timestamp`, and `--tail` flags to the + `docker-compose logs` command. + +- `docker-compose up`, and `docker-compose start` will now start containers + in parallel where possible. + +- `docker-compose stop` now stops containers in reverse dependency order + instead of all at once. + +- Added the `--build` flag to `docker-compose up` to force it to build a new + image. It now shows a warning if an image is automatically built when the + flag is not used. + +- Added the `docker-compose exec` command for executing a process in a running + container. + + +Bug Fixes + +- `docker-compose down` now removes containers created by + `docker-compose run`. + +- A more appropriate error is shown when a timeout is hit during `up` when + using a tty. + +- Fixed a bug in `docker-compose down` where it would abort if some resources + had already been removed. + +- Fixed a bug where changes to network aliases would not trigger a service + to be recreated. + +- Fix a bug where a log message was printed about creating a new volume + when it already existed. + +- Fixed a bug where interrupting `up` would not always shut down containers. + +- Fixed a bug where `log_opt` and `log_driver` were not properly carried over + when extending services in the v1 Compose file format. + +- Fixed a bug where empty values for build args would cause file validation + to fail. + +1.6.2 (2016-02-23) +------------------ + +- Fixed a bug where connecting to a TLS-enabled Docker Engine would fail with + a certificate verification error. + +1.6.1 (2016-02-23) +------------------ + +Bug Fixes + +- Fixed a bug where recreating a container multiple times would cause the + new container to be started without the previous volumes. + +- Fixed a bug where Compose would set the value of unset environment variables + to an empty string, instead of a key without a value. + +- Provide a better error message when Compose requires a more recent version + of the Docker API. + +- Add a missing config field `network.aliases` which allows setting a network + scoped alias for a service. + +- Fixed a bug where `run` would not start services listed in `depends_on`. + +- Fixed a bug where `networks` and `network_mode` where not merged when using + extends or multiple Compose files. + +- Fixed a bug with service aliases where the short container id alias was + only contained 10 characters, instead of the 12 characters used in previous + versions. + +- Added a missing log message when creating a new named volume. + +- Fixed a bug where `build.args` was not merged when using `extends` or + multiple Compose files. + +- Fixed some bugs with config validation when null values or incorrect types + were used instead of a mapping. + +- Fixed a bug where a `build` section without a `context` would show a stack + trace instead of a helpful validation message. + +- Improved compatibility with swarm by only setting a container affinity to + the previous instance of a services' container when the service uses an + anonymous container volume. Previously the affinity was always set on all + containers. + +- Fixed the validation of some `driver_opts` would cause an error if a number + was used instead of a string. + +- Some improvements to the `run.sh` script used by the Compose container install + option. + +- Fixed a bug with `up --abort-on-container-exit` where Compose would exit, + but would not stop other containers. + +- Corrected the warning message that is printed when a boolean value is used + as a value in a mapping. + + +1.6.0 (2016-01-15) +------------------ + +Major Features: + +- Compose 1.6 introduces a new format for `docker-compose.yml` which lets + you define networks and volumes in the Compose file as well as services. It + also makes a few changes to the structure of some configuration options. + + You don't have to use it - your existing Compose files will run on Compose + 1.6 exactly as they do today. + + Check the upgrade guide for full details: + https://docs.docker.com/compose/compose-file#upgrading + +- Support for networking has exited experimental status and is the recommended + way to enable communication between containers. + + If you use the new file format, your app will use networking. If you aren't + ready yet, just leave your Compose file as it is and it'll continue to work + just the same. + + By default, you don't have to configure any networks. In fact, using + networking with Compose involves even less configuration than using links. + Consult the networking guide for how to use it: + https://docs.docker.com/compose/networking + + The experimental flags `--x-networking` and `--x-network-driver`, introduced + in Compose 1.5, have been removed. + +- You can now pass arguments to a build if you're using the new file format: + + build: + context: . + args: + buildno: 1 + +- You can now specify both a `build` and an `image` key if you're using the + new file format. `docker-compose build` will build the image and tag it with + the name you've specified, while `docker-compose pull` will attempt to pull + it. + +- There's a new `events` command for monitoring container events from + the application, much like `docker events`. This is a good primitive for + building tools on top of Compose for performing actions when particular + things happen, such as containers starting and stopping. + +- There's a new `depends_on` option for specifying dependencies between + services. This enforces the order of startup, and ensures that when you run + `docker-compose up SERVICE` on a service with dependencies, those are started + as well. + +New Features: + +- Added a new command `config` which validates and prints the Compose + configuration after interpolating variables, resolving relative paths, and + merging multiple files and `extends`. + +- Added a new command `create` for creating containers without starting them. + +- Added a new command `down` to stop and remove all the resources created by + `up` in a single command. + +- Added support for the `cpu_quota` configuration option. + +- Added support for the `stop_signal` configuration option. + +- Commands `start`, `restart`, `pause`, and `unpause` now exit with an + error status code if no containers were modified. + +- Added a new `--abort-on-container-exit` flag to `up` which causes `up` to + stop all container and exit once the first container exits. + +- Removed support for `FIG_FILE`, `FIG_PROJECT_NAME`, and no longer reads + `fig.yml` as a default Compose file location. + +- Removed the `migrate-to-labels` command. + +- Removed the `--allow-insecure-ssl` flag. + + +Bug Fixes: + +- Fixed a validation bug that prevented the use of a range of ports in + the `expose` field. + +- Fixed a validation bug that prevented the use of arrays in the `entrypoint` + field if they contained duplicate entries. + +- Fixed a bug that caused `ulimits` to be ignored when used with `extends`. + +- Fixed a bug that prevented ipv6 addresses in `extra_hosts`. + +- Fixed a bug that caused `extends` to be ignored when included from + multiple Compose files. + +- Fixed an incorrect warning when a container volume was defined in + the Compose file. + +- Fixed a bug that prevented the force shutdown behaviour of `up` and + `logs`. + +- Fixed a bug that caused `None` to be printed as the network driver name + when the default network driver was used. + +- Fixed a bug where using the string form of `dns` or `dns_search` would + cause an error. + +- Fixed a bug where a container would be reported as "Up" when it was + in the restarting state. + +- Fixed a confusing error message when DOCKER_CERT_PATH was not set properly. + +- Fixed a bug where attaching to a container would fail if it was using a + non-standard logging driver (or none at all). + + +1.5.2 (2015-12-03) +------------------ + +- Fixed a bug which broke the use of `environment` and `env_file` with + `extends`, and caused environment keys without values to have a `None` + value, instead of a value from the host environment. + +- Fixed a regression in 1.5.1 that caused a warning about volumes to be + raised incorrectly when containers were recreated. + +- Fixed a bug which prevented building a `Dockerfile` that used `ADD <url>` + +- Fixed a bug with `docker-compose restart` which prevented it from + starting stopped containers. + +- Fixed handling of SIGTERM and SIGINT to properly stop containers + +- Add support for using a url as the value of `build` + +- Improved the validation of the `expose` option + + +1.5.1 (2015-11-12) +------------------ + +- Add the `--force-rm` option to `build`. + +- Add the `ulimit` option for services in the Compose file. + +- Fixed a bug where `up` would error with "service needs to be built" if + a service changed from using `image` to using `build`. + +- Fixed a bug that would cause incorrect output of parallel operations + on some terminals. + +- Fixed a bug that prevented a container from being recreated when the + mode of a `volumes_from` was changed. + +- Fixed a regression in 1.5.0 where non-utf-8 unicode characters would cause + `up` or `logs` to crash. + +- Fixed a regression in 1.5.0 where Compose would use a success exit status + code when a command fails due to an HTTP timeout communicating with the + docker daemon. + +- Fixed a regression in 1.5.0 where `name` was being accepted as a valid + service option which would override the actual name of the service. + +- When using `--x-networking` Compose no longer sets the hostname to the + container name. + +- When using `--x-networking` Compose will only create the default network + if at least one container is using the network. + +- When printings logs during `up` or `logs`, flush the output buffer after + each line to prevent buffering issues from hiding logs. + +- Recreate a container if one of its dependencies is being created. + Previously a container was only recreated if it's dependencies already + existed, but were being recreated as well. + +- Add a warning when a `volume` in the Compose file is being ignored + and masked by a container volume from a previous container. + +- Improve the output of `pull` when run without a tty. + +- When using multiple Compose files, validate each before attempting to merge + them together. Previously invalid files would result in not helpful errors. + +- Allow dashes in keys in the `environment` service option. + +- Improve validation error messages by including the filename as part of the + error message. + + +1.5.0 (2015-11-03) +------------------ + +**Breaking changes:** + +With the introduction of variable substitution support in the Compose file, any +Compose file that uses an environment variable (`$VAR` or `${VAR}`) in the `command:` +or `entrypoint:` field will break. + +Previously these values were interpolated inside the container, with a value +from the container environment. In Compose 1.5.0, the values will be +interpolated on the host, with a value from the host environment. + +To migrate a Compose file to 1.5.0, escape the variables with an extra `$` +(ex: `$$VAR` or `$${VAR}`). See +https://github.com/docker/compose/blob/8cc8e61/docs/compose-file.md#variable-substitution + +Major features: + +- Compose is now available for Windows. + +- Environment variables can be used in the Compose file. See + https://github.com/docker/compose/blob/8cc8e61/docs/compose-file.md#variable-substitution + +- Multiple compose files can be specified, allowing you to override + settings in the default Compose file. See + https://github.com/docker/compose/blob/8cc8e61/docs/reference/docker-compose.md + for more details. + +- Compose now produces better error messages when a file contains + invalid configuration. + +- `up` now waits for all services to exit before shutting down, + rather than shutting down as soon as one container exits. + +- Experimental support for the new docker networking system can be + enabled with the `--x-networking` flag. Read more here: + https://github.com/docker/docker/blob/8fee1c20/docs/userguide/dockernetworks.md + +New features: + +- You can now optionally pass a mode to `volumes_from`, e.g. + `volumes_from: ["servicename:ro"]`. + +- Since Docker now lets you create volumes with names, you can refer to those + volumes by name in `docker-compose.yml`. For example, + `volumes: ["mydatavolume:/data"]` will mount the volume named + `mydatavolume` at the path `/data` inside the container. + + If the first component of an entry in `volumes` starts with a `.`, `/` or + `~`, it is treated as a path and expansion of relative paths is performed as + necessary. Otherwise, it is treated as a volume name and passed straight + through to Docker. + + Read more on named volumes and volume drivers here: + https://github.com/docker/docker/blob/244d9c33/docs/userguide/dockervolumes.md + +- `docker-compose build --pull` instructs Compose to pull the base image for + each Dockerfile before building. + +- `docker-compose pull --ignore-pull-failures` instructs Compose to continue + if it fails to pull a single service's image, rather than aborting. + +- You can now specify an IPC namespace in `docker-compose.yml` with the `ipc` + option. + +- Containers created by `docker-compose run` can now be named with the + `--name` flag. + +- If you install Compose with pip or use it as a library, it now works with + Python 3. + +- `image` now supports image digests (in addition to ids and tags), e.g. + `image: "busybox@sha256:38a203e1986cf79639cfb9b2e1d6e773de84002feea2d4eb006b52004ee8502d"` + +- `ports` now supports ranges of ports, e.g. + + ports: + - "3000-3005" + - "9000-9001:8000-8001" + +- `docker-compose run` now supports a `-p|--publish` parameter, much like + `docker run -p`, for publishing specific ports to the host. + +- `docker-compose pause` and `docker-compose unpause` have been implemented, + analogous to `docker pause` and `docker unpause`. + +- When using `extends` to copy configuration from another service in the same + Compose file, you can omit the `file` option. + +- Compose can be installed and run as a Docker image. This is an experimental + feature. + +Bug fixes: + +- All values for the `log_driver` option which are supported by the Docker + daemon are now supported by Compose. + +- `docker-compose build` can now be run successfully against a Swarm cluster. + + +1.4.2 (2015-09-22) +------------------ + +- Fixed a regression in the 1.4.1 release that would cause `docker-compose up` + without the `-d` option to exit immediately. + +1.4.1 (2015-09-10) +------------------ + +The following bugs have been fixed: + +- Some configuration changes (notably changes to `links`, `volumes_from`, and + `net`) were not properly triggering a container recreate as part of + `docker-compose up`. +- `docker-compose up <service>` was showing logs for all services instead of + just the specified services. +- Containers with custom container names were showing up in logs as + `service_number` instead of their custom container name. +- When scaling a service sometimes containers would be recreated even when + the configuration had not changed. + + +1.4.0 (2015-08-04) +------------------ + +- By default, `docker-compose up` now only recreates containers for services whose configuration has changed since they were created. This should result in a dramatic speed-up for many applications. + + The experimental `--x-smart-recreate` flag which introduced this feature in Compose 1.3.0 has been removed, and a `--force-recreate` flag has been added for when you want to recreate everything. + +- Several of Compose's commands - `scale`, `stop`, `kill` and `rm` - now perform actions on multiple containers in parallel, rather than in sequence, which will run much faster on larger applications. + +- You can now specify a custom name for a service's container with `container_name`. Because Docker container names must be unique, this means you can't scale the service beyond one container. + +- You no longer have to specify a `file` option when using `extends` - it will default to the current file. + +- Service names can now contain dots, dashes and underscores. + +- Compose can now read YAML configuration from standard input, rather than from a file, by specifying `-` as the filename. This makes it easier to generate configuration dynamically: + + $ echo 'redis: {"image": "redis"}' | docker-compose --file - up + +- There's a new `docker-compose version` command which prints extended information about Compose's bundled dependencies. + +- `docker-compose.yml` now supports `log_opt` as well as `log_driver`, allowing you to pass extra configuration to a service's logging driver. + +- `docker-compose.yml` now supports `memswap_limit`, similar to `docker run --memory-swap`. + +- When mounting volumes with the `volumes` option, you can now pass in any mode supported by the daemon, not just `:ro` or `:rw`. For example, SELinux users can pass `:z` or `:Z`. + +- You can now specify a custom volume driver with the `volume_driver` option in `docker-compose.yml`, much like `docker run --volume-driver`. + +- A bug has been fixed where Compose would fail to pull images from private registries serving plain (unsecured) HTTP. The `--allow-insecure-ssl` flag, which was previously used to work around this issue, has been deprecated and now has no effect. + +- A bug has been fixed where `docker-compose build` would fail if the build depended on a private Hub image or an image from a private registry. + +- A bug has been fixed where Compose would crash if there were containers which the Docker daemon had not finished removing. + +- Two bugs have been fixed where Compose would sometimes fail with a "Duplicate bind mount" error, or fail to attach volumes to a container, if there was a volume path specified in `docker-compose.yml` with a trailing slash. + +Thanks @mnowster, @dnephin, @ekristen, @funkyfuture, @jeffk and @lukemarsden! + +1.3.3 (2015-07-15) +------------------ + +Two regressions have been fixed: + +- When stopping containers gracefully, Compose was setting the timeout to 0, effectively forcing a SIGKILL every time. +- Compose would sometimes crash depending on the formatting of container data returned from the Docker API. + +1.3.2 (2015-07-14) +------------------ + +The following bugs have been fixed: + +- When there were one-off containers created by running `docker-compose run` on an older version of Compose, `docker-compose run` would fail with a name collision. Compose now shows an error if you have leftover containers of this type lying around, and tells you how to remove them. +- Compose was not reading Docker authentication config files created in the new location, `~/docker/config.json`, and authentication against private registries would therefore fail. +- When a container had a pseudo-TTY attached, its output in `docker-compose up` would be truncated. +- `docker-compose up --x-smart-recreate` would sometimes fail when an image tag was updated. +- `docker-compose up` would sometimes create two containers with the same numeric suffix. +- `docker-compose rm` and `docker-compose ps` would sometimes list services that aren't part of the current project (though no containers were erroneously removed). +- Some `docker-compose` commands would not show an error if invalid service names were passed in. + +Thanks @dano, @josephpage, @kevinsimper, @lieryan, @phemmer, @soulrebel and @sschepens! + +1.3.1 (2015-06-21) +------------------ + +The following bugs have been fixed: + +- `docker-compose build` would always attempt to pull the base image before building. +- `docker-compose help migrate-to-labels` failed with an error. +- If no network mode was specified, Compose would set it to "bridge", rather than allowing the Docker daemon to use its configured default network mode. + +1.3.0 (2015-06-18) +------------------ + +Firstly, two important notes: + +- **This release contains breaking changes, and you will need to either remove or migrate your existing containers before running your app** - see the [upgrading section of the install docs](https://github.com/docker/compose/blob/1.3.0rc1/docs/install.md#upgrading) for details. + +- Compose now requires Docker 1.6.0 or later. + +We've done a lot of work in this release to remove hacks and make Compose more stable: + +- Compose now uses container labels, rather than names, to keep track of containers. This makes Compose both faster and easier to integrate with your own tools. + +- Compose no longer uses "intermediate containers" when recreating containers for a service. This makes `docker-compose up` less complex and more resilient to failure. + +There are some new features: + +- `docker-compose up` has an **experimental** new behaviour: it will only recreate containers for services whose configuration has changed in `docker-compose.yml`. This will eventually become the default, but for now you can take it for a spin: + + $ docker-compose up --x-smart-recreate + +- When invoked in a subdirectory of a project, `docker-compose` will now climb up through parent directories until it finds a `docker-compose.yml`. + +Several new configuration keys have been added to `docker-compose.yml`: + +- `dockerfile`, like `docker build --file`, lets you specify an alternate Dockerfile to use with `build`. +- `labels`, like `docker run --labels`, lets you add custom metadata to containers. +- `extra_hosts`, like `docker run --add-host`, lets you add entries to a container's `/etc/hosts` file. +- `pid: host`, like `docker run --pid=host`, lets you reuse the same PID namespace as the host machine. +- `cpuset`, like `docker run --cpuset-cpus`, lets you specify which CPUs to allow execution in. +- `read_only`, like `docker run --read-only`, lets you mount a container's filesystem as read-only. +- `security_opt`, like `docker run --security-opt`, lets you specify [security options](https://docs.docker.com/engine/reference/run/#security-configuration). +- `log_driver`, like `docker run --log-driver`, lets you specify a [log driver](https://docs.docker.com/engine/reference/run/#logging-drivers-log-driver). + +Many bugs have been fixed, including the following: + +- The output of `docker-compose run` was sometimes truncated, especially when running under Jenkins. +- A service's volumes would sometimes not update after volume configuration was changed in `docker-compose.yml`. +- Authenticating against third-party registries would sometimes fail. +- `docker-compose run --rm` would fail to remove the container if the service had a `restart` policy in place. +- `docker-compose scale` would refuse to scale a service beyond 1 container if it exposed a specific port number on the host. +- Compose would refuse to create multiple volume entries with the same host path. + +Thanks @ahromis, @albers, @aleksandr-vin, @antoineco, @ccverak, @chernjie, @dnephin, @edmorley, @fordhurley, @josephpage, @KyleJamesWalker, @lsowen, @mchasal, @noironetworks, @sdake, @sdurrheimer, @sherter, @stephenlawrence, @thaJeztah, @thieman, @turtlemonvh, @twhiteman, @vdemeester, @xuxinkun and @zwily! + +1.2.0 (2015-04-16) +------------------ + +- `docker-compose.yml` now supports an `extends` option, which enables a service to inherit configuration from another service in another configuration file. This is really good for sharing common configuration between apps, or for configuring the same app for different environments. Here's the [documentation](https://github.com/docker/compose/blob/master/docs/yml.md#extends). + +- When using Compose with a Swarm cluster, containers that depend on one another will be co-scheduled on the same node. This means that most Compose apps will now work out of the box, as long as they don't use `build`. + +- Repeated invocations of `docker-compose up` when using Compose with a Swarm cluster now work reliably. + +- Directories passed to `build`, filenames passed to `env_file` and volume host paths passed to `volumes` are now treated as relative to the *directory of the configuration file*, not the directory that `docker-compose` is being run in. In the majority of cases, those are the same, but if you use the `-f|--file` argument to specify a configuration file in another directory, **this is a breaking change**. + +- A service can now share another service's network namespace with `net: container:<service>`. + +- `volumes_from` and `net: container:<service>` entries are taken into account when resolving dependencies, so `docker-compose up <service>` will correctly start all dependencies of `<service>`. + +- `docker-compose run` now accepts a `--user` argument to specify a user to run the command as, just like `docker run`. + +- The `up`, `stop` and `restart` commands now accept a `--timeout` (or `-t`) argument to specify how long to wait when attempting to gracefully stop containers, just like `docker stop`. + +- `docker-compose rm` now accepts `-f` as a shorthand for `--force`, just like `docker rm`. + +Thanks, @abesto, @albers, @alunduil, @dnephin, @funkyfuture, @gilclark, @IanVS, @KingsleyKelly, @knutwalker, @thaJeztah and @vmalloc! + +1.1.0 (2015-02-25) +------------------ + +Fig has been renamed to Docker Compose, or just Compose for short. This has several implications for you: + +- The command you type is now `docker-compose`, not `fig`. +- You should rename your fig.yml to docker-compose.yml. +- If you’re installing via PyPI, the package is now `docker-compose`, so install it with `pip install docker-compose`. + +Besides that, there’s a lot of new stuff in this release: + +- We’ve made a few small changes to ensure that Compose will work with Swarm, Docker’s new clustering tool (https://github.com/docker/swarm). Eventually you'll be able to point Compose at a Swarm cluster instead of a standalone Docker host and it’ll run your containers on the cluster with no extra work from you. As Swarm is still developing, integration is rough and lots of Compose features don't work yet. + +- `docker-compose run` now has a `--service-ports` flag for exposing ports on the given service. This is useful for e.g. running your webapp with an interactive debugger. + +- You can now link to containers outside your app with the `external_links` option in docker-compose.yml. + +- You can now prevent `docker-compose up` from automatically building images with the `--no-build` option. This will make fewer API calls and run faster. + +- If you don’t specify a tag when using the `image` key, Compose will default to the `latest` tag, rather than pulling all tags. + +- `docker-compose kill` now supports the `-s` flag, allowing you to specify the exact signal you want to send to a service’s containers. + +- docker-compose.yml now has an `env_file` key, analogous to `docker run --env-file`, letting you specify multiple environment variables in a separate file. This is great if you have a lot of them, or if you want to keep sensitive information out of version control. + +- docker-compose.yml now supports the `dns_search`, `cap_add`, `cap_drop`, `cpu_shares` and `restart` options, analogous to `docker run`’s `--dns-search`, `--cap-add`, `--cap-drop`, `--cpu-shares` and `--restart` options. + +- Compose now ships with Bash tab completion - see the installation and usage docs at https://github.com/docker/compose/blob/1.1.0/docs/completion.md + +- A number of bugs have been fixed - see the milestone for details: https://github.com/docker/compose/issues?q=milestone%3A1.1.0+ + +Thanks @dnephin, @squebe, @jbalonso, @raulcd, @benlangfield, @albers, @ggtools, @bersace, @dtenenba, @petercv, @drewkett, @TFenby, @paulRbr, @Aigeruth and @salehe! + +1.0.1 (2014-11-04) +------------------ + + - Added an `--allow-insecure-ssl` option to allow `fig up`, `fig run` and `fig pull` to pull from insecure registries. + - Fixed `fig run` not showing output in Jenkins. + - Fixed a bug where Fig couldn't build Dockerfiles with ADD statements pointing at URLs. + +1.0.0 (2014-10-16) +------------------ + +The highlights: + + - [Fig has joined Docker.](https://www.orchardup.com/blog/orchard-is-joining-docker) Fig will continue to be maintained, but we'll also be incorporating the best bits of Fig into Docker itself. + + This means the GitHub repository has moved to [https://github.com/docker/fig](https://github.com/docker/fig) and our IRC channel is now #docker-fig on Freenode. + + - Fig can be used with the [official Docker OS X installer](https://docs.docker.com/installation/mac/). Boot2Docker will mount the home directory from your host machine so volumes work as expected. + + - Fig supports Docker 1.3. + + - It is now possible to connect to the Docker daemon using TLS by using the `DOCKER_CERT_PATH` and `DOCKER_TLS_VERIFY` environment variables. + + - There is a new `fig port` command which outputs the host port binding of a service, in a similar way to `docker port`. + + - There is a new `fig pull` command which pulls the latest images for a service. + + - There is a new `fig restart` command which restarts a service's containers. + + - Fig creates multiple containers in service by appending a number to the service name (e.g. `db_1`, `db_2`, etc). As a convenience, Fig will now give the first container an alias of the service name (e.g. `db`). + + This link alias is also a valid hostname and added to `/etc/hosts` so you can connect to linked services using their hostname. For example, instead of resolving the environment variables `DB_PORT_5432_TCP_ADDR` and `DB_PORT_5432_TCP_PORT`, you could just use the hostname `db` and port `5432` directly. + + - Volume definitions now support `ro` mode, expanding `~` and expanding environment variables. + + - `.dockerignore` is supported when building. + + - The project name can be set with the `FIG_PROJECT_NAME` environment variable. + + - The `--env` and `--entrypoint` options have been added to `fig run`. + + - The Fig binary for Linux is now linked against an older version of glibc so it works on CentOS 6 and Debian Wheezy. + +Other things: + + - `fig ps` now works on Jenkins and makes fewer API calls to the Docker daemon. + - `--verbose` displays more useful debugging output. + - When starting a service where `volumes_from` points to a service without any containers running, that service will now be started. + - Lots of docs improvements. Notably, environment variables are documented and official repositories are used throughout. + +Thanks @dnephin, @d11wtq, @marksteve, @rubbish, @jbalonso, @timfreund, @alunduil, @mieciu, @shuron, @moss, @suzaku and @chmouel! Whew. + +0.5.2 (2014-07-28) +------------------ + + - Added a `--no-cache` option to `fig build`, which bypasses the cache just like `docker build --no-cache`. + - Fixed the `dns:` fig.yml option, which was causing fig to error out. + - Fixed a bug where fig couldn't start under Python 2.6. + - Fixed a log-streaming bug that occasionally caused fig to exit. + +Thanks @dnephin and @marksteve! + + +0.5.1 (2014-07-11) +------------------ + + - If a service has a command defined, `fig run [service]` with no further arguments will run it. + - The project name now defaults to the directory containing fig.yml, not the current working directory (if they're different) + - `volumes_from` now works properly with containers as well as services + - Fixed a race condition when recreating containers in `fig up` + +Thanks @ryanbrainard and @d11wtq! + + +0.5.0 (2014-07-11) +------------------ + + - Fig now starts links when you run `fig run` or `fig up`. + + For example, if you have a `web` service which depends on a `db` service, `fig run web ...` will start the `db` service. + + - Environment variables can now be resolved from the environment that Fig is running in. Just specify it as a blank variable in your `fig.yml` and, if set, it'll be resolved: + ``` + environment: + RACK_ENV: development + SESSION_SECRET: + ``` + + - `volumes_from` is now supported in `fig.yml`. All of the volumes from the specified services and containers will be mounted: + + ``` + volumes_from: + - service_name + - container_name + ``` + + - A host address can now be specified in `ports`: + + ``` + ports: + - "0.0.0.0:8000:8000" + - "127.0.0.1:8001:8001" + ``` + + - The `net` and `workdir` options are now supported in `fig.yml`. + - The `hostname` option now works in the same way as the Docker CLI, splitting out into a `domainname` option. + - TTY behaviour is far more robust, and resizes are supported correctly. + - Load YAML files safely. + +Thanks to @d11wtq, @ryanbrainard, @rail44, @j0hnsmith, @binarin, @Elemecca, @mozz100 and @marksteve for their help with this release! + + +0.4.2 (2014-06-18) +------------------ + + - Fix various encoding errors when using `fig run`, `fig up` and `fig build`. + +0.4.1 (2014-05-08) +------------------ + + - Add support for Docker 0.11.0. (Thanks @marksteve!) + - Make project name configurable. (Thanks @jefmathiot!) + - Return correct exit code from `fig run`. + +0.4.0 (2014-04-29) +------------------ + + - Support Docker 0.9 and 0.10 + - Display progress bars correctly when pulling images (no more ski slopes) + - `fig up` now stops all services when any container exits + - Added support for the `privileged` config option in fig.yml (thanks @kvz!) + - Shortened and aligned log prefixes in `fig up` output + - Only containers started with `fig run` link back to their own service + - Handle UTF-8 correctly when streaming `fig build/run/up` output (thanks @mauvm and @shanejonas!) + - Error message improvements + +0.3.2 (2014-03-05) +------------------ + + - Added an `--rm` option to `fig run`. (Thanks @marksteve!) + - Added an `expose` option to `fig.yml`. + +0.3.1 (2014-03-04) +------------------ + + - Added contribution instructions. (Thanks @kvz!) + - Fixed `fig rm` throwing an error. + - Fixed a bug in `fig ps` on Docker 0.8.1 when there is a container with no command. + +0.3.0 (2014-03-03) +------------------ + + - We now ship binaries for OS X and Linux. No more having to install with Pip! + - Add `-f` flag to specify alternate `fig.yml` files + - Add support for custom link names + - Fix a bug where recreating would sometimes hang + - Update docker-py to support Docker 0.8.0. + - Various documentation improvements + - Various error message improvements + +Thanks @marksteve, @Gazler and @teozkr! + +0.2.2 (2014-02-17) +------------------ + + - Resolve dependencies using Cormen/Tarjan topological sort + - Fix `fig up` not printing log output + - Stop containers in reverse order to starting + - Fix scale command not binding ports + +Thanks to @barnybug and @dustinlacewell for their work on this release. + +0.2.1 (2014-02-04) +------------------ + + - General improvements to error reporting (#77, #79) + +0.2.0 (2014-01-31) +------------------ + + - Link services to themselves so run commands can access the running service. (#67) + - Much better documentation. + - Make service dependency resolution more reliable. (#48) + - Load Fig configurations with a `.yaml` extension. (#58) + +Big thanks to @cameronmaske, @mrchrisadams and @damianmoore for their help with this release. + +0.1.4 (2014-01-27) +------------------ + + - Add a link alias without the project name. This makes the environment variables a little shorter: `REDIS_1_PORT_6379_TCP_ADDR`. (#54) + +0.1.3 (2014-01-23) +------------------ + + - Fix ports sometimes being configured incorrectly. (#46) + - Fix log output sometimes not displaying. (#47) + +0.1.2 (2014-01-22) +------------------ + + - Add `-T` option to `fig run` to disable pseudo-TTY. (#34) + - Fix `fig up` requiring the ubuntu image to be pulled to recreate containers. (#33) Thanks @cameronmaske! + - Improve reliability, fix arrow keys and fix a race condition in `fig run`. (#34, #39, #40) + +0.1.1 (2014-01-17) +------------------ + + - Fix bug where ports were not exposed correctly (#29). Thanks @dustinlacewell! + +0.1.0 (2014-01-16) +------------------ + + - Containers are recreated on each `fig up`, ensuring config is up-to-date with `fig.yml` (#2) + - Add `fig scale` command (#9) + - Use `DOCKER_HOST` environment variable to find Docker daemon, for consistency with the official Docker client (was previously `DOCKER_URL`) (#19) + - Truncate long commands in `fig ps` (#18) + - Fill out CLI help banners for commands (#15, #16) + - Show a friendlier error when `fig.yml` is missing (#4) + - Fix bug with `fig build` logging (#3) + - Fix bug where builds would time out if a step took a long time without generating output (#6) + - Fix bug where streaming container output over the Unix socket raised an error (#7) + +Big thanks to @tomstuart, @EnTeQuAk, @schickling, @aronasorman and @GeoffreyPlitt. + +0.0.2 (2014-01-02) +------------------ + + - Improve documentation + - Try to connect to Docker on `tcp://localdocker:4243` and a UNIX socket in addition to `localhost`. + - Improve `fig up` behaviour + - Add confirmation prompt to `fig rm` + - Add `fig build` command + +0.0.1 (2013-12-20) +------------------ + +Initial release. diff --git a/CHANGES.md b/CHANGES.md new file mode 120000 index 00000000000..83b694704ba --- /dev/null +++ b/CHANGES.md @@ -0,0 +1 @@ +CHANGELOG.md \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9b80b8abc45..5bf7cb1318c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,339 +1,76 @@ -# Contributing to Docker +# Contributing to Compose -Want to hack on Docker? Awesome! We have a contributor's guide that explains -[setting up a Docker development environment and the contribution -process](https://docs.docker.com/contribute/). +Compose is a part of the Docker project, and follows the same rules and +principles. Take a read of [Docker's contributing guidelines](https://github.com/docker/docker/blob/master/CONTRIBUTING.md) +to get an overview. -This page contains information about reporting issues as well as some tips and -guidelines useful to experienced open source contributors. Finally, make sure -you read our [community guidelines](#docker-community-guidelines) before you -start participating. +## TL;DR -## Topics +Pull requests will need: -- [Contributing to Docker](#contributing-to-docker) - - [Topics](#topics) - - [Reporting security issues](#reporting-security-issues) - - [Reporting other issues](#reporting-other-issues) - - [Quick contribution tips and guidelines](#quick-contribution-tips-and-guidelines) - - [Pull requests are always welcome](#pull-requests-are-always-welcome) - - [Talking to other Docker users and contributors](#talking-to-other-docker-users-and-contributors) - - [Conventions](#conventions) - - [Merge approval](#merge-approval) - - [Sign your work](#sign-your-work) - - [How can I become a maintainer?](#how-can-i-become-a-maintainer) - - [Docker community guidelines](#docker-community-guidelines) - - [Coding Style](#coding-style) + - Tests + - Documentation + - [To be signed off](https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work) + - A logical series of [well written commits](https://github.com/alphagov/styleguides/blob/master/git.md) -## Reporting security issues +## Development environment -The Docker maintainers take security seriously. If you discover a security -issue, please bring it to their attention right away! +If you're looking contribute to Compose +but you're new to the project or maybe even to Python, here are the steps +that should get you started. -Please **DO NOT** file a public issue, instead, send your report privately to -[security@docker.com](mailto:security@docker.com). +1. Fork [https://github.com/docker/compose](https://github.com/docker/compose) + to your username. +2. Clone your forked repository locally `git clone git@github.com:yourusername/compose.git`. +3. You must [configure a remote](https://help.github.com/articles/configuring-a-remote-for-a-fork/) for your fork so that you can [sync changes you make](https://help.github.com/articles/syncing-a-fork/) with the original repository. +4. Enter the local directory `cd compose`. +5. Set up a development environment by running `python setup.py develop`. This + will install the dependencies and set up a symlink from your `docker-compose` + executable to the checkout of the repository. When you now run + `docker-compose` from anywhere on your machine, it will run your development + version of Compose. -Security reports are greatly appreciated and we will publicly thank you for them. -We also like to send gifts—if you're into Docker swag, make sure to let -us know. We currently do not offer a paid security bounty program but are not -ruling it out in the future. +## Install pre-commit hooks +This step is optional, but recommended. Pre-commit hooks will run style checks +and in some cases fix style issues for you, when you commit code. -## Reporting other issues +Install the git pre-commit hooks using [tox](https://tox.readthedocs.io) by +running `tox -e pre-commit` or by following the +[pre-commit install guide](http://pre-commit.com/#install). -A great way to contribute to the project is to send a detailed report when you -encounter an issue. We always appreciate a well-written, thorough bug report, -and will thank you for it! +To run the style checks at any time run `tox -e pre-commit`. -Check that [our issue database](https://github.com/docker/compose/labels/Docker%20Compose%20V2) -doesn't already include that problem or suggestion before submitting an issue. -If you find a match, you can use the "subscribe" button to get notified of -updates. Do *not* leave random "+1" or "I have this too" comments, as they -only clutter the discussion, and don't help to resolve it. However, if you -have ways to reproduce the issue or have additional information that may help -resolve the issue, please leave a comment. +## Submitting a pull request -When reporting issues, always include: +See Docker's [basic contribution workflow](https://docs.docker.com/v17.06/opensource/code/#code-contribution-workflow) for a guide on how to submit a pull request for code. -* The output of `docker version`. -* The output of `docker context show`. -* The output of `docker info`. +## Documentation changes -Also, include the steps required to reproduce the problem if possible and -applicable. This information will help us review and fix your issue faster. -When sending lengthy log files, consider posting them as a gist -(https://gist.github.com). -Don't forget to remove sensitive data from your log files before posting (you -can replace those parts with "REDACTED"). +Issues and pull requests to update the documentation should be submitted to the [docs repo](https://github.com/docker/docker.github.io). You can learn more about contributing to the documentation [here](https://docs.docker.com/opensource/#how-to-contribute-to-the-docs). -_Note:_ -Maintainers might request additional information to diagnose an issue, -if initial reporter doesn't answer within a reasonable delay (a few weeks), -issue will be closed. +## Running the test suite -## Quick contribution tips and guidelines +Use the test script to run linting checks and then the full test suite against +different Python interpreters: -This section gives the experienced contributor some tips and guidelines. + $ script/test/default -### Pull requests are always welcome +Tests are run against a Docker daemon inside a container, so that we can test +against multiple Docker versions. By default they'll run against only the latest +Docker version - set the `DOCKER_VERSIONS` environment variable to "all" to run +against all supported versions: -Not sure if that typo is worth a pull request? Found a bug and know how to fix -it? Do it! We will appreciate it. Any significant change, like adding a backend, -should be documented as -[a GitHub issue](https://github.com/docker/compose/issues) -before anybody starts working on it. + $ DOCKER_VERSIONS=all script/test/default -We are always thrilled to receive pull requests. We do our best to process them -quickly. If your pull request is not accepted on the first try, -don't get discouraged! +Arguments to `script/test/default` are passed through to the `tox` executable, so +you can specify a test directory, file, module, class or method: -### Talking to other Docker users and contributors + $ script/test/default tests/unit + $ script/test/default tests/unit/cli_test.py + $ script/test/default tests/unit/config/config_test.py::ConfigTest + $ script/test/default tests/unit/config/config_test.py::ConfigTest::test_load -<table class="tg"> - <col width="45%"> - <col width="65%"> - <tr> - <td>Community Slack</td> - <td> - The Docker Community has a dedicated Slack chat to discuss features and issues. You can sign-up <a href="https://www.docker.com/community/" target="_blank">with this link</a>. - </td> - </tr> - <tr> - <td>Forums</td> - <td> - A public forum for users to discuss questions and explore current design patterns and - best practices about Docker and related projects in the Docker Ecosystem. To participate, - just log in with your Docker Hub account on <a href="https://forums.docker.com" target="_blank">https://forums.docker.com</a>. - </td> - </tr> - <tr> - <td>Twitter</td> - <td> - You can follow <a href="https://twitter.com/docker/" target="_blank">Docker's Twitter feed</a> - to get updates on our products. You can also tweet us questions or just - share blogs or stories. - </td> - </tr> - <tr> - <td>Stack Overflow</td> - <td> - Stack Overflow has over 17000 Docker questions listed. We regularly - monitor <a href="https://stackoverflow.com/questions/tagged/docker" target="_blank">Docker questions</a> - and so do many other knowledgeable Docker users. - </td> - </tr> -</table> +## Finding things to work on - -### Conventions - -Fork the repository and make changes on your fork in a feature branch: - -- If it's a bug fix branch, name it XXXX-something where XXXX is the number of - the issue. -- If it's a feature branch, create an enhancement issue to announce - your intentions, and name it XXXX-something where XXXX is the number of the - issue. - -Submit unit tests for your changes. Go has a great test framework built in; use -it! Take a look at existing tests for inspiration. Also, end-to-end tests are -available. Run the full test suite, both unit tests and e2e tests on your -branch before submitting a pull request. See [BUILDING.md](BUILDING.md) for -instructions to build and run tests. - -Write clean code. Universally formatted code promotes ease of writing, reading, -and maintenance. Always run `gofmt -s -w file.go` on each changed file before -committing your changes. Most editors have plug-ins that do this automatically. - -Pull request descriptions should be as clear as possible and include a reference -to all the issues that they address. - -Commit messages must start with a capitalized and short summary (max. 50 chars) -written in the imperative, followed by an optional, more detailed explanatory -text which is separated from the summary by an empty line. - -Code review comments may be added to your pull request. Discuss, then make the -suggested modifications and push additional commits to your feature branch. Post -a comment after pushing. New commits show up in the pull request automatically, -but the reviewers are notified only when you comment. - -Pull requests must be cleanly rebased on top of the base branch without multiple branches -mixed into the PR. - -**Git tip**: If your PR no longer merges cleanly, use `rebase master` in your -feature branch to update your pull request rather than `merge master`. - -Before you make a pull request, squash your commits into logical units of work -using `git rebase -i` and `git push -f`. A logical unit of work is a consistent -set of patches that should be reviewed together: for example, upgrading the -version of a vendored dependency and taking advantage of its now available new -feature constitute two separate units of work. Implementing a new function and -calling it in another file constitute a single logical unit of work. The very -high majority of submissions should have a single commit, so if in doubt: squash -down to one. - -After every commit, make sure the test suite passes. Include documentation -changes in the same pull request so that a revert would remove all traces of -the feature or fix. - -Include an issue reference like `Closes #XXXX` or `Fixes #XXXX` in the pull -request description that closes an issue. Including references automatically -closes the issue on a merge. - -Please do not add yourself to the `AUTHORS` file, as it is regenerated regularly -from the Git history. - -Please see the [Coding Style](#coding-style) for further guidelines. - -### Merge approval - -Docker maintainers use LGTM (Looks Good To Me) in comments on the code review to -indicate acceptance. - -A change requires at least 2 LGTMs from the maintainers of each -component affected. - -For more details, see the [MAINTAINERS](MAINTAINERS) page. - -### Sign your work - -The sign-off is a simple line at the end of the explanation for the patch. Your -signature certifies that you wrote the patch or otherwise have the right to pass -it on as an open-source patch. The rules are pretty simple: if you can certify -the below (from [developercertificate.org](https://developercertificate.org/)): - -``` -Developer Certificate of Origin -Version 1.1 - -Copyright (C) 2004, 2006 The Linux Foundation and its contributors. -660 York Street, Suite 102, -San Francisco, CA 94110 USA - -Everyone is permitted to copy and distribute verbatim copies of this -license document, but changing it is not allowed. - -Developer's Certificate of Origin 1.1 - -By making a contribution to this project, I certify that: - -(a) The contribution was created in whole or in part by me and I - have the right to submit it under the open source license - indicated in the file; or - -(b) The contribution is based upon previous work that, to the best - of my knowledge, is covered under an appropriate open source - license and I have the right under that license to submit that - work with modifications, whether created in whole or in part - by me, under the same open source license (unless I am - permitted to submit under a different license), as indicated - in the file; or - -(c) The contribution was provided directly to me by some other - person who certified (a), (b) or (c) and I have not modified - it. - -(d) I understand and agree that this project and the contribution - are public and that a record of the contribution (including all - personal information I submit with it, including my sign-off) is - maintained indefinitely and may be redistributed consistent with - this project or the open source license(s) involved. -``` - -Then you just add a line to every git commit message: - - Signed-off-by: Joe Smith <joe.smith@email.com> - -Use your real name (sorry, no pseudonyms or anonymous contributions.) - -If you set your `user.name` and `user.email` git configs, you can sign your -commit automatically with `git commit -s`. - -### How can I become a maintainer? - -The procedures for adding new maintainers are explained in the global -[MAINTAINERS](https://github.com/docker/opensource/blob/main/MAINTAINERS) -file in the -[https://github.com/docker/opensource/](https://github.com/docker/opensource/) -repository. - -Don't forget: being a maintainer is a time investment. Make sure you -will have time to make yourself available. You don't have to be a -maintainer to make a difference on the project! - -## Docker community guidelines - -We want to keep the Docker community awesome, growing and collaborative. We need -your help to keep it that way. To help with this we've come up with some general -guidelines for the community as a whole: - -* Be nice: Be courteous, respectful and polite to fellow community members: - no regional, racial, gender or other abuse will be tolerated. We like - nice people way better than mean ones! - -* Encourage diversity and participation: Make everyone in our community feel - welcome, regardless of their background and the extent of their - contributions, and do everything possible to encourage participation in - our community. - -* Keep it legal: Basically, don't get us in trouble. Share only content that - you own, do not share private or sensitive information, and don't break - the law. - -* Stay on topic: Make sure that you are posting to the correct channel and - avoid off-topic discussions. Remember when you update an issue or respond - to an email you are potentially sending it to a large number of people. Please - consider this before you update. Also, remember that nobody likes spam. - -* Don't send emails to the maintainers: There's no need to send emails to the - maintainers to ask them to investigate an issue or to take a look at a - pull request. Instead of sending an email, GitHub mentions should be - used to ping maintainers to review a pull request, a proposal or an - issue. - -## Coding Style - -Unless explicitly stated, we follow all coding guidelines from the Go -community. While some of these standards may seem arbitrary, they somehow seem -to result in a solid, consistent codebase. - -It is possible that the code base does not currently comply with these -guidelines. We are not looking for a massive PR that fixes this, since that -goes against the spirit of the guidelines. All new contributors should make their -best effort to clean up and make the code base better than they left it. -Obviously, apply your best judgement. Remember, the goal here is to make the -code base easier for humans to navigate and understand. Always keep that in -mind when nudging others to comply. - -The rules: - -1. All code should be formatted with `gofmt -s`. -2. All code should pass the default levels of - [`golint`](https://github.com/golang/lint). -3. All code should follow the guidelines covered in [Effective - Go](https://go.dev/doc/effective_go) and [Go Code Review - Comments](https://go.dev/wiki/CodeReviewComments). -4. Include code comments. Tell us the why, the history and the context. -5. Document _all_ declarations and methods, even private ones. Declare - expectations, caveats and anything else that may be important. If a type - gets exported, having the comments already there will ensure it's ready. -6. Variable name length should be proportional to its context and no longer. - `noCommaALongVariableNameLikeThisIsNotMoreClearWhenASimpleCommentWouldDo`. - In practice, short methods will have short variable names and globals will - have longer names. -7. No underscores in package names. If you need a compound name, step back, - and re-examine why you need a compound name. If you still think you need a - compound name, lose the underscore. -8. No utils or helpers packages. If a function is not general enough to - warrant its own package, it has not been written generally enough to be a - part of a util package. Just leave it unexported and well-documented. -9. All tests should run with `go test` and outside tooling should not be - required. No, we don't need another unit testing framework. Assertion - packages are acceptable if they provide _real_ incremental value. -10. Even though we call these "rules" above, they are actually just - guidelines. Since you've read all the rules, you now know that. - -If you are having trouble getting into the mood of idiomatic Go, we recommend -reading through [Effective Go](https://go.dev/doc/effective_go). The -[Go Blog](https://go.dev/blog/) is also a great resource. Drinking the -kool-aid is a lot easier than going thirsty. +[Issues marked with the `exp/beginner` label](https://github.com/docker/compose/issues?q=is%3Aopen+is%3Aissue+label%3Aexp%2Fbeginner) are a good starting point for people looking to make their first contribution to the project. diff --git a/Dockerfile b/Dockerfile index 7ab1c8a18dd..a4fc34a92c1 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,197 +1,100 @@ -# syntax=docker/dockerfile:1 +ARG DOCKER_VERSION=19.03 +ARG PYTHON_VERSION=3.9.0 +ARG BUILD_ALPINE_VERSION=3.12 +ARG BUILD_CENTOS_VERSION=7 +ARG BUILD_DEBIAN_VERSION=slim-buster -# Copyright 2020 Docker Compose CLI authors +ARG RUNTIME_ALPINE_VERSION=3.12 +ARG RUNTIME_CENTOS_VERSION=7 +ARG RUNTIME_DEBIAN_VERSION=buster-slim -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at +ARG DISTRO=alpine -# http://www.apache.org/licenses/LICENSE-2.0 +FROM docker:${DOCKER_VERSION} AS docker-cli -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -ARG GO_VERSION=1.25.7 -ARG XX_VERSION=1.9.0 -ARG GOLANGCI_LINT_VERSION=v2.8.0 -ARG ADDLICENSE_VERSION=v1.0.0 - -ARG BUILD_TAGS="e2e" -ARG DOCS_FORMATS="md,yaml" -ARG LICENSE_FILES=".*$Dockerfile\|Makefile\|\.go\|\.hcl\|\.sh$" - -# xx is a helper for cross-compilation -FROM --platform=${BUILDPLATFORM} tonistiigi/xx:${XX_VERSION} AS xx - -# osxcross contains the MacOSX cross toolchain for xx -FROM crazymax/osxcross:15.5-alpine AS osxcross - -FROM golangci/golangci-lint:${GOLANGCI_LINT_VERSION}-alpine AS golangci-lint -FROM ghcr.io/google/addlicense:${ADDLICENSE_VERSION} AS addlicense - -FROM --platform=${BUILDPLATFORM} golang:${GO_VERSION}-alpine3.22 AS base -COPY --from=xx / / +FROM python:${PYTHON_VERSION}-alpine${BUILD_ALPINE_VERSION} AS build-alpine RUN apk add --no-cache \ - clang \ - docker \ - file \ - findutils \ - git \ - make \ - protoc \ - protobuf-dev -WORKDIR /src -ENV CGO_ENABLED=0 - -FROM base AS build-base -COPY go.* . -RUN --mount=type=cache,target=/go/pkg/mod \ - --mount=type=cache,target=/root/.cache/go-build \ - go mod download - -FROM build-base AS vendored -RUN --mount=type=bind,target=.,rw \ - --mount=type=cache,target=/go/pkg/mod \ - go mod tidy && mkdir /out && cp go.mod go.sum /out - -FROM scratch AS vendor-update -COPY --from=vendored /out / - -FROM vendored AS vendor-validate -RUN --mount=type=bind,target=.,rw <<EOT - set -e - git add -A - cp -rf /out/* . - diff=$(git status --porcelain -- go.mod go.sum) - if [ -n "$diff" ]; then - echo >&2 'ERROR: Vendor result differs. Please vendor your package with "make go-mod-tidy"' - echo "$diff" - exit 1 - fi -EOT - -FROM build-base AS build -ARG BUILD_TAGS -ARG BUILD_FLAGS -ARG TARGETPLATFORM -RUN --mount=type=bind,target=. \ - --mount=type=cache,target=/root/.cache \ - --mount=type=cache,target=/go/pkg/mod \ - --mount=type=bind,from=osxcross,src=/osxsdk,target=/xx-sdk \ - xx-go --wrap && \ - if [ "$(xx-info os)" == "darwin" ]; then export CGO_ENABLED=1; export BUILD_TAGS=fsnotify,$BUILD_TAGS; fi && \ - make build GO_BUILDTAGS="$BUILD_TAGS" DESTDIR=/out && \ - xx-verify --static /out/docker-compose - -FROM build-base AS lint -ARG BUILD_TAGS -ENV GOLANGCI_LINT_CACHE=/cache/golangci-lint -RUN --mount=type=bind,target=. \ - --mount=type=cache,target=/root/.cache \ - --mount=type=cache,target=/go/pkg/mod \ - --mount=type=cache,target=/cache/golangci-lint \ - --mount=from=golangci-lint,source=/usr/bin/golangci-lint,target=/usr/bin/golangci-lint \ - golangci-lint cache status && \ - golangci-lint run --build-tags "$BUILD_TAGS" ./... - -FROM build-base AS test -ARG CGO_ENABLED=0 -ARG BUILD_TAGS -RUN --mount=type=bind,target=. \ - --mount=type=cache,target=/root/.cache \ - --mount=type=cache,target=/go/pkg/mod \ - rm -rf /tmp/coverage && \ - mkdir -p /tmp/coverage && \ - rm -rf /tmp/report && \ - mkdir -p /tmp/report && \ - go run gotest.tools/gotestsum@latest --format testname --junitfile "/tmp/report/report.xml" -- -tags "$BUILD_TAGS" -v -cover -covermode=atomic $(go list $(TAGS) ./... | grep -vE 'e2e') -args -test.gocoverdir="/tmp/coverage" && \ - go tool covdata percent -i=/tmp/coverage - -FROM scratch AS test-coverage -COPY --from=test --link /tmp/coverage / -COPY --from=test --link /tmp/report / - -FROM base AS license-set -ARG LICENSE_FILES -RUN --mount=type=bind,target=.,rw \ - --mount=from=addlicense,source=/app/addlicense,target=/usr/bin/addlicense \ - find . -regex "${LICENSE_FILES}" | xargs addlicense -c 'Docker Compose CLI' -l apache && \ - mkdir /out && \ - find . -regex "${LICENSE_FILES}" | cpio -pdm /out - -FROM scratch AS license-update -COPY --from=set /out / - -FROM base AS license-validate -ARG LICENSE_FILES -RUN --mount=type=bind,target=. \ - --mount=from=addlicense,source=/app/addlicense,target=/usr/bin/addlicense \ - find . -regex "${LICENSE_FILES}" | xargs addlicense -check -c 'Docker Compose CLI' -l apache -ignore validate -ignore testdata -ignore resolvepath -v - -FROM base AS docsgen -WORKDIR /src -RUN --mount=target=. \ - --mount=target=/root/.cache,type=cache \ - --mount=type=cache,target=/go/pkg/mod \ - go build -o /out/docsgen ./docs/yaml/main/generate.go - -FROM --platform=${BUILDPLATFORM} alpine AS docs-build -RUN apk add --no-cache rsync git -WORKDIR /src -COPY --from=docsgen /out/docsgen /usr/bin -ARG DOCS_FORMATS -RUN --mount=target=/context \ - --mount=target=.,type=tmpfs <<EOT - set -e - rsync -a /context/. . - docsgen --formats "$DOCS_FORMATS" --source "docs/reference" - mkdir /out - cp -r docs/reference /out -EOT - -FROM scratch AS docs-update -COPY --from=docs-build /out /out - -FROM docs-build AS docs-validate -RUN --mount=target=/context \ - --mount=target=.,type=tmpfs <<EOT - set -e - rsync -a /context/. . - git add -A - rm -rf docs/reference/* - cp -rf /out/* ./docs/ - if [ -n "$(git status --porcelain -- docs/reference)" ]; then - echo >&2 'ERROR: Docs result differs. Please update with "make docs"' - git status --porcelain -- docs/reference - exit 1 - fi -EOT - -FROM scratch AS binary-unix -COPY --link --from=build /out/docker-compose / -FROM binary-unix AS binary-darwin -FROM binary-unix AS binary-linux -FROM scratch AS binary-windows -COPY --link --from=build /out/docker-compose /docker-compose.exe -FROM binary-$TARGETOS AS binary -# enable scanning for this stage -ARG BUILDKIT_SBOM_SCAN_STAGE=true - -FROM --platform=$BUILDPLATFORM alpine AS releaser -WORKDIR /work -ARG TARGETOS + bash \ + build-base \ + ca-certificates \ + curl \ + gcc \ + git \ + libc-dev \ + libffi-dev \ + libgcc \ + make \ + musl-dev \ + openssl \ + openssl-dev \ + zlib-dev +ENV BUILD_BOOTLOADER=1 + +FROM python:${PYTHON_VERSION}-${BUILD_DEBIAN_VERSION} AS build-debian +RUN apt-get update && apt-get install --no-install-recommends -y \ + curl \ + gcc \ + git \ + libc-dev \ + libffi-dev \ + libgcc-8-dev \ + libssl-dev \ + make \ + openssl \ + zlib1g-dev + +FROM centos:${BUILD_CENTOS_VERSION} AS build-centos +RUN yum install -y \ + gcc \ + git \ + libffi-devel \ + make \ + openssl \ + openssl-devel +WORKDIR /tmp/python3/ +ARG PYTHON_VERSION +RUN curl -L https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz | tar xzf - \ + && cd Python-${PYTHON_VERSION} \ + && ./configure --enable-optimizations --enable-shared --prefix=/usr LDFLAGS="-Wl,-rpath /usr/lib" \ + && make altinstall +RUN alternatives --install /usr/bin/python python /usr/bin/python2.7 50 +RUN alternatives --install /usr/bin/python python /usr/bin/python$(echo "${PYTHON_VERSION%.*}") 60 +RUN curl https://bootstrap.pypa.io/get-pip.py | python - + +FROM build-${DISTRO} AS build +ENTRYPOINT ["sh", "/usr/local/bin/docker-compose-entrypoint.sh"] +WORKDIR /code/ +COPY docker-compose-entrypoint.sh /usr/local/bin/ +COPY --from=docker-cli /usr/local/bin/docker /usr/local/bin/docker +RUN pip install \ + virtualenv==20.2.2 \ + tox==3.20.1 +COPY requirements-dev.txt . +COPY requirements-indirect.txt . +COPY requirements.txt . +RUN pip install -r requirements.txt -r requirements-indirect.txt -r requirements-dev.txt +COPY .pre-commit-config.yaml . +COPY tox.ini . +COPY setup.py . +COPY README.md . +COPY compose compose/ +RUN tox --notest +COPY . . +ARG GIT_COMMIT=unknown +ENV DOCKER_COMPOSE_GITSHA=$GIT_COMMIT +RUN script/build/linux-entrypoint + +FROM scratch AS bin ARG TARGETARCH -ARG TARGETVARIANT -RUN --mount=from=binary \ - mkdir -p /out && \ - # TODO: should just use standard arch - TARGETARCH=$([ "$TARGETARCH" = "amd64" ] && echo "x86_64" || echo "$TARGETARCH"); \ - TARGETARCH=$([ "$TARGETARCH" = "arm64" ] && echo "aarch64" || echo "$TARGETARCH"); \ - cp docker-compose* "/out/docker-compose-${TARGETOS}-${TARGETARCH}${TARGETVARIANT}$(ls docker-compose* | sed -e 's/^docker-compose//')" - -FROM scratch AS release -COPY --from=releaser /out/ / +ARG TARGETOS +COPY --from=build /usr/local/bin/docker-compose /docker-compose-${TARGETOS}-${TARGETARCH} + +FROM alpine:${RUNTIME_ALPINE_VERSION} AS runtime-alpine +FROM debian:${RUNTIME_DEBIAN_VERSION} AS runtime-debian +FROM centos:${RUNTIME_CENTOS_VERSION} AS runtime-centos +FROM runtime-${DISTRO} AS runtime +COPY docker-compose-entrypoint.sh /usr/local/bin/ +ENTRYPOINT ["sh", "/usr/local/bin/docker-compose-entrypoint.sh"] +COPY --from=docker-cli /usr/local/bin/docker /usr/local/bin/docker +COPY --from=build /usr/local/bin/docker-compose /usr/local/bin/docker-compose diff --git a/Jenkinsfile b/Jenkinsfile new file mode 100644 index 00000000000..40629216a38 --- /dev/null +++ b/Jenkinsfile @@ -0,0 +1,115 @@ +#!groovy + +def dockerVersions = ['19.03.13'] +def baseImages = ['alpine', 'debian'] +def pythonVersions = ['py39'] + +pipeline { + agent none + + options { + skipDefaultCheckout(true) + buildDiscarder(logRotator(daysToKeepStr: '30')) + timeout(time: 2, unit: 'HOURS') + timestamps() + } + environment { + DOCKER_BUILDKIT="1" + } + + stages { + stage('Build test images') { + // TODO use declarative 1.5.0 `matrix` once available on CI + parallel { + stage('alpine') { + agent { + label 'ubuntu && amd64 && !zfs' + } + steps { + buildImage('alpine') + } + } + stage('debian') { + agent { + label 'ubuntu && amd64 && !zfs' + } + steps { + buildImage('debian') + } + } + } + } + stage('Test') { + steps { + // TODO use declarative 1.5.0 `matrix` once available on CI + script { + def testMatrix = [:] + baseImages.each { baseImage -> + dockerVersions.each { dockerVersion -> + pythonVersions.each { pythonVersion -> + testMatrix["${baseImage}_${dockerVersion}_${pythonVersion}"] = runTests(dockerVersion, pythonVersion, baseImage) + } + } + } + + parallel testMatrix + } + } + } + } +} + + +def buildImage(baseImage) { + def scmvar = checkout(scm) + def imageName = "dockerbuildbot/compose:${baseImage}-${scmvar.GIT_COMMIT}" + image = docker.image(imageName) + + withDockerRegistry(credentialsId:'dockerbuildbot-index.docker.io') { + try { + image.pull() + } catch (Exception exc) { + ansiColor('xterm') { + sh """docker build -t ${imageName} \\ + --target build \\ + --build-arg DISTRO="${baseImage}" \\ + --build-arg GIT_COMMIT="${scmvar.GIT_COMMIT}" \\ + .\\ + """ + sh "docker push ${imageName}" + } + echo "${imageName}" + return imageName + } + } +} + +def runTests(dockerVersion, pythonVersion, baseImage) { + return { + stage("python=${pythonVersion} docker=${dockerVersion} ${baseImage}") { + node("ubuntu && amd64 && !zfs") { + def scmvar = checkout(scm) + def imageName = "dockerbuildbot/compose:${baseImage}-${scmvar.GIT_COMMIT}" + def storageDriver = sh(script: "docker info -f \'{{.Driver}}\'", returnStdout: true).trim() + echo "Using local system's storage driver: ${storageDriver}" + withDockerRegistry(credentialsId:'dockerbuildbot-index.docker.io') { + sh """docker run \\ + -t \\ + --rm \\ + --privileged \\ + --volume="\$(pwd)/.git:/code/.git" \\ + --volume="/var/run/docker.sock:/var/run/docker.sock" \\ + -e "TAG=${imageName}" \\ + -e "STORAGE_DRIVER=${storageDriver}" \\ + -e "DOCKER_VERSIONS=${dockerVersion}" \\ + -e "BUILD_NUMBER=${env.BUILD_NUMBER}" \\ + -e "PY_TEST_VERSIONS=${pythonVersion}" \\ + --entrypoint="script/test/ci" \\ + ${imageName} \\ + --verbose + """ + } + } + } + } +} diff --git a/LICENSE b/LICENSE index 7a4a3ea2424..27448585ad4 100644 --- a/LICENSE +++ b/LICENSE @@ -176,18 +176,7 @@ END OF TERMS AND CONDITIONS - APPENDIX: How to apply the Apache License to your work. - - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "[]" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. - - Copyright [yyyy] [name of copyright owner] + Copyright 2014 Docker, Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -199,4 +188,4 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and - limitations under the License. \ No newline at end of file + limitations under the License. diff --git a/MAINTAINERS b/MAINTAINERS new file mode 100644 index 00000000000..7e178147e84 --- /dev/null +++ b/MAINTAINERS @@ -0,0 +1,105 @@ +# Compose maintainers file +# +# This file describes who runs the docker/compose project and how. +# This is a living document - if you see something out of date or missing, speak up! +# +# It is structured to be consumable by both humans and programs. +# To extract its contents programmatically, use any TOML-compliant parser. +# +# This file is compiled into the MAINTAINERS file in docker/opensource. +# +[Org] + [Org."Core maintainers"] + people = [ + "aiordache", + "ndeloof", + "rumpl", + "ulyssessouza", + ] + [Org.Alumni] + people = [ + # Aanand Prasad is one of the two creators of the fig project + # which later went on to become docker-compose, and a longtime + # maintainer responsible for several keystone features + "aanand", + # Ben Firshman is also one of the fig creators and contributed + # heavily to the project's design and UX as well as the + # day-to-day maintenance + "bfirsh", + # Mazz Mosley made significant contributions to the project + # in 2015 with solid bugfixes and improved error handling + # among them + "mnowster", + # Daniel Nephin is one of the longest-running maintainers on + # the Compose project, and has contributed several major features + # including muti-file support, variable interpolation, secrets + # emulation and many more + "dnephin", + + "shin-", + "mefyl", + "mnottale", + ] + +[people] + +# A reference list of all people associated with the project. +# All other sections should refer to people by their canonical key +# in the people section. + + # ADD YOURSELF HERE IN ALPHABETICAL ORDER + + [people.aanand] + Name = "Aanand Prasad" + Email = "aanand.prasad@gmail.com" + GitHub = "aanand" + + [people.aiordache] + Name = "Anca Iordache" + Email = "anca.iordache@docker.com" + GitHub = "aiordache" + + [people.bfirsh] + Name = "Ben Firshman" + Email = "ben@firshman.co.uk" + GitHub = "bfirsh" + + [people.dnephin] + Name = "Daniel Nephin" + Email = "dnephin@gmail.com" + GitHub = "dnephin" + + [people.mefyl] + Name = "Quentin Hocquet" + Email = "quentin.hocquet@docker.com" + GitHub = "mefyl" + + [people.mnottale] + Name = "Matthieu Nottale" + Email = "matthieu.nottale@docker.com" + GitHub = "mnottale" + + [people.mnowster] + Name = "Mazz Mosley" + Email = "mazz@houseofmnowster.com" + GitHub = "mnowster" + + [people.ndeloof] + Name = "Nicolas De Loof" + Email = "nicolas.deloof@gmail.com" + GitHub = "ndeloof" + + [people.rumpl] + Name = "Djordje Lukic" + Email = "djordje.lukic@docker.com" + GitHub = "rumpl" + + [people.shin-] + Name = "Joffrey F" + Email = "f.joffrey@gmail.com" + GitHub = "shin-" + + [people.ulyssessouza] + Name = "Ulysses Domiciano Souza" + Email = "ulysses.souza@docker.com" + GitHub = "ulyssessouza" diff --git a/MANIFEST.in b/MANIFEST.in new file mode 100644 index 00000000000..313b4e00814 --- /dev/null +++ b/MANIFEST.in @@ -0,0 +1,15 @@ +include Dockerfile +include LICENSE +include requirements-indirect.txt +include requirements.txt +include requirements-dev.txt +include tox.ini +include *.md +include README.md +include compose/config/*.json +include compose/GITSHA +recursive-include contrib/completion * +recursive-include tests * +global-exclude *.pyc +global-exclude *.pyo +global-exclude *.un~ diff --git a/Makefile b/Makefile index a29eebe7739..0a7a5c366b4 100644 --- a/Makefile +++ b/Makefile @@ -1,162 +1,57 @@ -# Copyright 2020 Docker Compose CLI authors +TAG = "docker-compose:alpine-$(shell git rev-parse --short HEAD)" +GIT_VOLUME = "--volume=$(shell pwd)/.git:/code/.git" -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at +DOCKERFILE ?="Dockerfile" +DOCKER_BUILD_TARGET ?="build" -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -PKG := github.com/docker/compose/v5 -VERSION ?= $(shell git describe --match 'v[0-9]*' --dirty='.m' --always --tags) - -GO_LDFLAGS ?= -w -X ${PKG}/internal.Version=${VERSION} -GO_BUILDTAGS ?= e2e -DRIVE_PREFIX?= -ifeq ($(OS),Windows_NT) - DETECTED_OS = Windows - DRIVE_PREFIX=C: -else - DETECTED_OS = $(shell uname -s) -endif - -ifeq ($(DETECTED_OS),Windows) - BINARY_EXT=.exe -endif - -BUILD_FLAGS?= -TEST_FLAGS?= -E2E_TEST?= -ifneq ($(E2E_TEST),) - TEST_FLAGS:=$(TEST_FLAGS) -run '$(E2E_TEST)' +UNAME_S := $(shell uname -s) +ifeq ($(UNAME_S),Linux) + BUILD_SCRIPT = linux endif - -EXCLUDE_E2E_TESTS?= -ifneq ($(EXCLUDE_E2E_TESTS),) - TEST_FLAGS:=$(TEST_FLAGS) -skip '$(EXCLUDE_E2E_TESTS)' +ifeq ($(UNAME_S),Darwin) + BUILD_SCRIPT = osx endif -BUILDX_CMD ?= docker buildx - -# DESTDIR overrides the output path for binaries and other artifacts -# this is used by docker/docker-ce-packaging for the apt/rpm builds, -# so it's important that the resulting binary ends up EXACTLY at the -# path $DESTDIR/docker-compose when specified. -# -# See https://github.com/docker/docker-ce-packaging/blob/e43fbd37e48fde49d907b9195f23b13537521b94/rpm/SPECS/docker-compose-plugin.spec#L47 -# -# By default, all artifacts go to subdirectories under ./bin/ in the -# repo root, e.g. ./bin/build, ./bin/coverage, ./bin/release. -DESTDIR ?= - -all: build - -.PHONY: build ## Build the compose cli-plugin -build: - GO111MODULE=on go build $(BUILD_FLAGS) -trimpath -tags "$(GO_BUILDTAGS)" -ldflags "$(GO_LDFLAGS)" -o "$(or $(DESTDIR),./bin/build)/docker-compose$(BINARY_EXT)" ./cmd - -.PHONY: binary -binary: - BUILD_TAGS="$(GO_BUILDTAGS)" $(BUILDX_CMD) bake binary - -.PHONY: binary-with-coverage -binary-with-coverage: - BUILD_TAGS="$(GO_BUILDTAGS)" $(BUILDX_CMD) bake binary-with-coverage - -.PHONY: install -install: binary - mkdir -p ~/.docker/cli-plugins - install $(or $(DESTDIR),./bin/build)/docker-compose ~/.docker/cli-plugins/docker-compose +COMPOSE_SPEC_SCHEMA_PATH = "compose/config/compose_spec.json" +COMPOSE_SPEC_RAW_URL = "https://raw.githubusercontent.com/compose-spec/compose-spec/master/schema/compose-spec.json" -.PHONY: e2e-compose -e2e-compose: example-provider ## Run end to end local tests in plugin mode. Set E2E_TEST=TestName to run a single test - go run gotest.tools/gotestsum@latest --format testname --junitfile "/tmp/report/report.xml" -- -v $(TEST_FLAGS) -count=1 ./pkg/e2e +all: cli -.PHONY: e2e-compose-standalone -e2e-compose-standalone: ## Run End to end local tests in standalone mode. Set E2E_TEST=TestName to run a single test - go run gotest.tools/gotestsum@latest --format testname --junitfile "/tmp/report/report.xml" -- $(TEST_FLAGS) -v -count=1 -parallel=1 --tags=standalone ./pkg/e2e +cli: download-compose-spec ## Compile the cli + ./script/build/$(BUILD_SCRIPT) -.PHONY: build-and-e2e-compose -build-and-e2e-compose: build e2e-compose ## Compile the compose cli-plugin and run end to end local tests in plugin mode. Set E2E_TEST=TestName to run a single test +download-compose-spec: ## Download the compose-spec schema from it's repo + curl -so $(COMPOSE_SPEC_SCHEMA_PATH) $(COMPOSE_SPEC_RAW_URL) -.PHONY: build-and-e2e-compose-standalone -build-and-e2e-compose-standalone: build e2e-compose-standalone ## Compile the compose cli-plugin and run End to end local tests in standalone mode. Set E2E_TEST=TestName to run a single test - -.PHONY: example-provider -example-provider: ## build example provider for e2e tests - go build -o bin/build/example-provider docs/examples/provider.go - -.PHONY: mocks -mocks: - mockgen --version >/dev/null 2>&1 || go install go.uber.org/mock/mockgen@v0.4.0 - mockgen -destination pkg/mocks/mock_docker_cli.go -package mocks github.com/docker/cli/cli/command Cli - mockgen -destination pkg/mocks/mock_docker_api.go -package mocks github.com/docker/docker/client APIClient - mockgen -destination pkg/mocks/mock_docker_compose_api.go -package mocks -source=./pkg/api/api.go Service - -.PHONY: e2e -e2e: e2e-compose e2e-compose-standalone ## Run end to end local tests in both modes. Set E2E_TEST=TestName to run a single test - -.PHONY: build-and-e2e -build-and-e2e: build e2e-compose e2e-compose-standalone ## Compile the compose cli-plugin and run end to end local tests in both modes. Set E2E_TEST=TestName to run a single test - -.PHONY: cross -cross: ## Compile the CLI for linux, darwin and windows - $(BUILDX_CMD) bake binary-cross - -.PHONY: test -test: ## Run unit tests - $(BUILDX_CMD) bake test - -.PHONY: cache-clear cache-clear: ## Clear the builder cache - $(BUILDX_CMD) prune --force --filter type=exec.cachemount --filter=unused-for=24h + @docker builder prune --force --filter type=exec.cachemount --filter=unused-for=24h -.PHONY: lint -lint: ## run linter(s) - $(BUILDX_CMD) bake lint +base-image: ## Builds base image + docker build -f $(DOCKERFILE) -t $(TAG) --target $(DOCKER_BUILD_TARGET) . -.PHONY: fmt -fmt: - gofumpt --version >/dev/null 2>&1 || go install mvdan.cc/gofumpt@latest - gofumpt -w . +lint: base-image ## Run linter + docker run --rm \ + --tty \ + $(GIT_VOLUME) \ + $(TAG) \ + tox -e pre-commit -.PHONY: docs -docs: ## generate documentation - $(eval $@_TMP_OUT := $(shell mktemp -d -t compose-output.XXXXXXXXXX)) - $(BUILDX_CMD) bake --set "*.output=type=local,dest=$($@_TMP_OUT)" docs-update - rm -rf ./docs/internal - cp -R "$(DRIVE_PREFIX)$($@_TMP_OUT)"/out/* ./docs/ - rm -rf "$(DRIVE_PREFIX)$($@_TMP_OUT)"/* +test-unit: base-image ## Run tests + docker run --rm \ + --tty \ + $(GIT_VOLUME) \ + $(TAG) \ + pytest -v tests/unit/ -.PHONY: validate-docs -validate-docs: ## validate the doc does not change - $(BUILDX_CMD) bake docs-validate +test: ## Run all tests + ./script/test/default -.PHONY: check-dependencies -check-dependencies: ## check dependency updates - go list -u -m -f '{{if not .Indirect}}{{if .Update}}{{.}}{{end}}{{end}}' all - -.PHONY: validate-headers -validate-headers: ## Check license header for all files - $(BUILDX_CMD) bake license-validate - -.PHONY: go-mod-tidy -go-mod-tidy: ## Run go mod tidy in a container and output resulting go.mod and go.sum - $(BUILDX_CMD) bake vendor-update - -.PHONY: validate-go-mod -validate-go-mod: ## Validate go.mod and go.sum are up-to-date - $(BUILDX_CMD) bake vendor-validate - -validate: validate-go-mod validate-headers validate-docs ## Validate sources - -pre-commit: validate check-dependencies lint build test e2e-compose +pre-commit: lint test-unit cli help: ## Show help @echo Please specify a build target. The choices are: @grep -E '^[0-9a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' + +FORCE: + +.PHONY: all cli download-compose-spec cache-clear base-image lint test-unit test pre-commit help diff --git a/NOTICE b/NOTICE deleted file mode 100644 index dffae91878a..00000000000 --- a/NOTICE +++ /dev/null @@ -1,4 +0,0 @@ -Docker Compose V2 -Copyright 2020 Docker Compose authors - -This product includes software developed at Docker, Inc. (https://www.docker.com). diff --git a/README.md b/README.md index 254376f9e4b..d0d23d8af60 100644 --- a/README.md +++ b/README.md @@ -1,41 +1,30 @@ -# Table of Contents -- [Docker Compose](#docker-compose) -- [Where to get Docker Compose](#where-to-get-docker-compose) - + [Windows and macOS](#windows-and-macos) - + [Linux](#linux) -- [Quick Start](#quick-start) -- [Contributing](#contributing) -- [Legacy](#legacy) - -# Docker Compose - -[![GitHub release](https://img.shields.io/github/v/release/docker/compose.svg?style=flat-square)](https://github.com/docker/compose/releases/latest) -[![PkgGoDev](https://img.shields.io/badge/go.dev-docs-007d9c?style=flat-square&logo=go&logoColor=white)](https://pkg.go.dev/github.com/docker/compose/v5) -[![Build Status](https://img.shields.io/github/actions/workflow/status/docker/compose/ci.yml?label=ci&logo=github&style=flat-square)](https://github.com/docker/compose/actions?query=workflow%3Aci) -[![Go Report Card](https://goreportcard.com/badge/github.com/docker/compose/v5?style=flat-square)](https://goreportcard.com/report/github.com/docker/compose/v5) -[![Codecov](https://codecov.io/gh/docker/compose/branch/main/graph/badge.svg?token=HP3K4Y4ctu)](https://codecov.io/gh/docker/compose) -[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/docker/compose/badge)](https://api.securityscorecards.dev/projects/github.com/docker/compose) +Docker Compose +============== +[![Build Status](https://ci-next.docker.com/public/buildStatus/icon?job=compose/master)](https://ci-next.docker.com/public/job/compose/job/master/) + ![Docker Compose](logo.png?raw=true "Docker Compose Logo") Docker Compose is a tool for running multi-container applications on Docker defined using the [Compose file format](https://compose-spec.io). -A Compose file is used to define how one or more containers that make up +A Compose file is used to define how the one or more containers that make up your application are configured. Once you have a Compose file, you can create and start your application with a -single command: `docker compose up`. +single command: `docker-compose up`. -> **Note**: About Docker Swarm -> Docker Swarm used to rely on the legacy compose file format but did not adopt the compose specification -> so is missing some of the recent enhancements in the compose syntax. After -> [acquisition by Mirantis](https://www.mirantis.com/software/swarm/) swarm isn't maintained by Docker Inc, and -> as such some Docker Compose features aren't accessible to swarm users. +Compose files can be used to deploy applications locally, or to the cloud on +[Amazon ECS](https://aws.amazon.com/ecs) or +[Microsoft ACI](https://azure.microsoft.com/services/container-instances/) using +the Docker CLI. You can read more about how to do this: +- [Compose for Amazon ECS](https://docs.docker.com/engine/context/ecs-integration/) +- [Compose for Microsoft ACI](https://docs.docker.com/engine/context/aci-integration/) -# Where to get Docker Compose +Where to get Docker Compose +---------------------------- ### Windows and macOS Docker Compose is included in -[Docker Desktop](https://www.docker.com/products/docker-desktop/) +[Docker Desktop](https://www.docker.com/products/docker-desktop) for Windows and macOS. ### Linux @@ -43,25 +32,25 @@ for Windows and macOS. You can download Docker Compose binaries from the [release page](https://github.com/docker/compose/releases) on this repository. -Rename the relevant binary for your OS to `docker-compose` and copy it to `$HOME/.docker/cli-plugins` - -Or copy it into one of these folders to install it system-wide: +### Using pip -* `/usr/local/lib/docker/cli-plugins` OR `/usr/local/libexec/docker/cli-plugins` -* `/usr/lib/docker/cli-plugins` OR `/usr/libexec/docker/cli-plugins` +If your platform is not supported, you can download Docker Compose using `pip`: -(might require making the downloaded file executable with `chmod +x`) +```console +pip install docker-compose +``` +> **Note:** Docker Compose requires Python 3.6 or later. Quick Start ----------- -Using Docker Compose is a three-step process: +Using Docker Compose is basically a three-step process: 1. Define your app's environment with a `Dockerfile` so it can be reproduced anywhere. -2. Define the services that make up your app in `compose.yaml` so +2. Define the services that make up your app in `docker-compose.yml` so they can be run together in an isolated environment. -3. Lastly, run `docker compose up` and Compose will start and run your entire +3. Lastly, run `docker-compose up` and Compose will start and run your entire app. A Compose file looks like this: @@ -78,16 +67,22 @@ services: image: redis ``` +You can find examples of Compose applications in our +[Awesome Compose repository](https://github.com/docker/awesome-compose). + +For more information about the Compose format, see the +[Compose file reference](https://docs.docker.com/compose/compose-file/). + Contributing ------------ Want to help develop Docker Compose? Check out our -[contributing documentation](CONTRIBUTING.md). +[contributing documentation](https://github.com/docker/compose/blob/master/CONTRIBUTING.md). If you find an issue, please report it on the [issue tracker](https://github.com/docker/compose/issues/new/choose). -Legacy -------------- +Releasing +--------- -The Python version of Compose is available under the `v1` [branch](https://github.com/docker/compose/tree/v1). +Releases are built by maintainers, following an outline of the [release process](https://github.com/docker/compose/blob/master/project/RELEASE-PROCESS.md). diff --git a/Release.Jenkinsfile b/Release.Jenkinsfile new file mode 100644 index 00000000000..38cc53433fc --- /dev/null +++ b/Release.Jenkinsfile @@ -0,0 +1,310 @@ +#!groovy + +def dockerVersions = ['19.03.13', '18.09.9'] +def baseImages = ['alpine', 'debian'] +def pythonVersions = ['py39'] + +pipeline { + agent none + + options { + skipDefaultCheckout(true) + buildDiscarder(logRotator(daysToKeepStr: '30')) + timeout(time: 2, unit: 'HOURS') + timestamps() + } + environment { + DOCKER_BUILDKIT="1" + } + + stages { + stage('Build test images') { + // TODO use declarative 1.5.0 `matrix` once available on CI + parallel { + stage('alpine') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + buildImage('alpine') + } + } + stage('debian') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + buildImage('debian') + } + } + } + } + stage('Test') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + // TODO use declarative 1.5.0 `matrix` once available on CI + script { + def testMatrix = [:] + baseImages.each { baseImage -> + dockerVersions.each { dockerVersion -> + pythonVersions.each { pythonVersion -> + testMatrix["${baseImage}_${dockerVersion}_${pythonVersion}"] = runTests(dockerVersion, pythonVersion, baseImage) + } + } + } + + parallel testMatrix + } + } + } + stage('Generate Changelog') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + checkout scm + withCredentials([string(credentialsId: 'github-compose-release-test-token', variable: 'GITHUB_TOKEN')]) { + sh "./script/release/generate_changelog.sh" + } + archiveArtifacts artifacts: 'CHANGELOG.md' + stash( name: "changelog", includes: 'CHANGELOG.md' ) + } + } + stage('Package') { + parallel { + stage('macosx binary') { + agent { + label 'mac-python' + } + environment { + DEPLOYMENT_TARGET="10.11" + } + steps { + checkout scm + sh './script/setup/osx' + sh 'tox -e py39 -- tests/unit' + sh './script/build/osx' + dir ('dist') { + checksum('docker-compose-Darwin-x86_64') + checksum('docker-compose-Darwin-x86_64.tgz') + } + archiveArtifacts artifacts: 'dist/*', fingerprint: true + dir("dist") { + stash name: "bin-darwin" + } + } + } + stage('linux binary') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + checkout scm + sh ' ./script/build/linux' + dir ('dist') { + checksum('docker-compose-Linux-x86_64') + } + archiveArtifacts artifacts: 'dist/*', fingerprint: true + dir("dist") { + stash name: "bin-linux" + } + } + } + stage('windows binary') { + agent { + label 'windows-python' + } + environment { + PATH = "C:\\Python39;C:\\Python39\\Scripts;$PATH" + } + steps { + checkout scm + bat 'tox.exe -e py39 -- tests/unit' + powershell '.\\script\\build\\windows.ps1' + dir ('dist') { + checksum('docker-compose-Windows-x86_64.exe') + } + archiveArtifacts artifacts: 'dist/*', fingerprint: true + dir("dist") { + stash name: "bin-win" + } + } + } + stage('alpine image') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + buildRuntimeImage('alpine') + } + } + stage('debian image') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + buildRuntimeImage('debian') + } + } + } + } + stage('Release') { + when { + buildingTag() + } + parallel { + stage('Pushing images') { + agent { + label 'linux && docker && ubuntu-2004' + } + steps { + pushRuntimeImage('alpine') + pushRuntimeImage('debian') + } + } + stage('Creating Github Release') { + agent { + label 'linux && docker && ubuntu-2004' + } + environment { + GITHUB_TOKEN = credentials('github-release-token') + } + steps { + checkout scm + sh 'mkdir -p dist' + dir("dist") { + unstash "bin-darwin" + unstash "bin-linux" + unstash "bin-win" + unstash "changelog" + sh(""" + curl -SfL https://github.com/github/hub/releases/download/v2.13.0/hub-linux-amd64-2.13.0.tgz | tar xzv --wildcards 'hub-*/bin/hub' --strip=2 + ./hub release create --draft --prerelease=${env.TAG_NAME !=~ /v[0-9\.]+/} \\ + -a docker-compose-Darwin-x86_64 \\ + -a docker-compose-Darwin-x86_64.sha256 \\ + -a docker-compose-Darwin-x86_64.tgz \\ + -a docker-compose-Darwin-x86_64.tgz.sha256 \\ + -a docker-compose-Linux-x86_64 \\ + -a docker-compose-Linux-x86_64.sha256 \\ + -a docker-compose-Windows-x86_64.exe \\ + -a docker-compose-Windows-x86_64.exe.sha256 \\ + -a ../script/run/run.sh \\ + -F CHANGELOG.md \${TAG_NAME} + """) + } + } + } + stage('Publishing Python packages') { + agent { + label 'linux && docker && ubuntu-2004' + } + environment { + PYPIRC = credentials('pypirc-docker-dsg-cibot') + } + steps { + checkout scm + sh """ + rm -rf build/ dist/ + pip3 install wheel + python3 setup.py sdist bdist_wheel + pip3 install twine + ~/.local/bin/twine upload --config-file ${PYPIRC} ./dist/docker-compose-*.tar.gz ./dist/docker_compose-*-py2.py3-none-any.whl + """ + } + } + } + } + } +} + + +def buildImage(baseImage) { + def scmvar = checkout(scm) + def imageName = "dockerbuildbot/compose:${baseImage}-${scmvar.GIT_COMMIT}" + image = docker.image(imageName) + + withDockerRegistry(credentialsId:'dockerbuildbot-index.docker.io') { + try { + image.pull() + } catch (Exception exc) { + ansiColor('xterm') { + sh """docker build -t ${imageName} \\ + --target build \\ + --build-arg DISTRO="${baseImage}" \\ + --build-arg GIT_COMMIT="${scmvar.GIT_COMMIT}" \\ + .\\ + """ + sh "docker push ${imageName}" + } + echo "${imageName}" + return imageName + } + } +} + +def runTests(dockerVersion, pythonVersion, baseImage) { + return { + stage("python=${pythonVersion} docker=${dockerVersion} ${baseImage}") { + node("linux && docker && ubuntu-2004") { + def scmvar = checkout(scm) + def imageName = "dockerbuildbot/compose:${baseImage}-${scmvar.GIT_COMMIT}" + def storageDriver = sh(script: "docker info -f \'{{.Driver}}\'", returnStdout: true).trim() + echo "Using local system's storage driver: ${storageDriver}" + withDockerRegistry(credentialsId:'dockerbuildbot-index.docker.io') { + sh """docker run \\ + -t \\ + --rm \\ + --privileged \\ + --volume="\$(pwd)/.git:/code/.git" \\ + --volume="/var/run/docker.sock:/var/run/docker.sock" \\ + -e "TAG=${imageName}" \\ + -e "STORAGE_DRIVER=${storageDriver}" \\ + -e "DOCKER_VERSIONS=${dockerVersion}" \\ + -e "BUILD_NUMBER=${env.BUILD_NUMBER}" \\ + -e "PY_TEST_VERSIONS=${pythonVersion}" \\ + --entrypoint="script/test/ci" \\ + ${imageName} \\ + --verbose + """ + } + } + } + } +} + +def buildRuntimeImage(baseImage) { + scmvar = checkout scm + def imageName = "docker/compose:${baseImage}-${env.BRANCH_NAME}" + ansiColor('xterm') { + sh """docker build -t ${imageName} \\ + --build-arg DISTRO="${baseImage}" \\ + --build-arg GIT_COMMIT="${scmvar.GIT_COMMIT.take(7)}" \\ + . + """ + } + sh "mkdir -p dist" + sh "docker save ${imageName} -o dist/docker-compose-${baseImage}.tar" + stash name: "compose-${baseImage}", includes: "dist/docker-compose-${baseImage}.tar" +} + +def pushRuntimeImage(baseImage) { + unstash "compose-${baseImage}" + sh "docker load -i dist/docker-compose-${baseImage}.tar" + withDockerRegistry(credentialsId: 'dockerhub-dockerdsgcibot') { + sh "docker push docker/compose:${baseImage}-${env.TAG_NAME}" + if (baseImage == "alpine" && env.TAG_NAME != null) { + sh "docker tag docker/compose:alpine-${env.TAG_NAME} docker/compose:${env.TAG_NAME}" + sh "docker push docker/compose:${env.TAG_NAME}" + } + } +} + +def checksum(filepath) { + if (isUnix()) { + sh "openssl sha256 -r -out ${filepath}.sha256 ${filepath}" + } else { + powershell "(Get-FileHash -Path ${filepath} -Algorithm SHA256 | % hash).ToLower() + ' *${filepath}' | Out-File -encoding ascii ${filepath}.sha256" + } +} diff --git a/SWARM.md b/SWARM.md new file mode 100644 index 00000000000..c6f378a9a34 --- /dev/null +++ b/SWARM.md @@ -0,0 +1 @@ +This file has moved to: https://docs.docker.com/compose/swarm/ diff --git a/bin/docker-compose b/bin/docker-compose new file mode 100755 index 00000000000..5976e1d4aa5 --- /dev/null +++ b/bin/docker-compose @@ -0,0 +1,3 @@ +#!/usr/bin/env python +from compose.cli.main import main +main() diff --git a/cmd/cmdtrace/cmd_span.go b/cmd/cmdtrace/cmd_span.go deleted file mode 100644 index cce3e5db3f3..00000000000 --- a/cmd/cmdtrace/cmd_span.go +++ /dev/null @@ -1,148 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package cmdtrace - -import ( - "context" - "errors" - "fmt" - "sort" - "strings" - "time" - - dockercli "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - flag "github.com/spf13/pflag" - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/codes" - "go.opentelemetry.io/otel/trace" - - commands "github.com/docker/compose/v5/cmd/compose" - "github.com/docker/compose/v5/internal/tracing" -) - -// Setup should be called as part of the command's PersistentPreRunE -// as soon as possible after initializing the dockerCli. -// -// It initializes the tracer for the CLI using both auto-detection -// from the Docker context metadata as well as standard OTEL_ env -// vars, creates a root span for the command, and wraps the actual -// command invocation to ensure the span is properly finalized and -// exported before exit. -func Setup(cmd *cobra.Command, dockerCli command.Cli, args []string) error { - tracingShutdown, err := tracing.InitTracing(dockerCli) - if err != nil { - return fmt.Errorf("initializing tracing: %w", err) - } - - ctx := cmd.Context() - ctx, cmdSpan := otel.Tracer("").Start( - ctx, - "cli/"+strings.Join(commandName(cmd), "-"), - ) - cmdSpan.SetAttributes( - attribute.StringSlice("cli.flags", getFlags(cmd.Flags())), - attribute.Bool("cli.isatty", dockerCli.In().IsTerminal()), - ) - - cmd.SetContext(ctx) - wrapRunE(cmd, cmdSpan, tracingShutdown) - return nil -} - -// wrapRunE injects a wrapper function around the command's actual RunE (or Run) -// method. This is necessary to capture the command result for reporting as well -// as flushing any spans before exit. -// -// Unfortunately, PersistentPostRun(E) can't be used for this purpose because it -// only runs if RunE does _not_ return an error, but this should run unconditionally. -func wrapRunE(c *cobra.Command, cmdSpan trace.Span, tracingShutdown tracing.ShutdownFunc) { - origRunE := c.RunE - if origRunE == nil { - origRun := c.Run - //nolint:unparam // wrapper function for RunE, always returns nil by design - origRunE = func(cmd *cobra.Command, args []string) error { - origRun(cmd, args) - return nil - } - c.Run = nil - } - - c.RunE = func(cmd *cobra.Command, args []string) error { - cmdErr := origRunE(cmd, args) - if cmdSpan != nil { - if cmdErr != nil && !errors.Is(cmdErr, context.Canceled) { - // default exit code is 1 if a more descriptive error - // wasn't returned - exitCode := 1 - var statusErr dockercli.StatusError - if errors.As(cmdErr, &statusErr) { - exitCode = statusErr.StatusCode - } - cmdSpan.SetStatus(codes.Error, "CLI command returned error") - cmdSpan.RecordError(cmdErr, trace.WithAttributes( - attribute.Int("exit_code", exitCode), - )) - - } else { - cmdSpan.SetStatus(codes.Ok, "") - } - cmdSpan.End() - } - if tracingShutdown != nil { - // use background for root context because the cmd's context might have - // been canceled already - ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) - defer cancel() - // TODO(milas): add an env var to enable logging from the - // OTel components for debugging purposes - _ = tracingShutdown(ctx) - } - return cmdErr - } -} - -// commandName returns the path components for a given command, -// in reverse alphabetical order for consistent usage metrics. -// -// The root Compose command and anything before (i.e. "docker") -// are not included. -// -// For example: -// - docker compose alpha watch -> [watch, alpha] -// - docker-compose up -> [up] -func commandName(cmd *cobra.Command) []string { - var name []string - for c := cmd; c != nil; c = c.Parent() { - if c.Name() == commands.PluginName { - break - } - name = append(name, c.Name()) - } - sort.Sort(sort.Reverse(sort.StringSlice(name))) - return name -} - -func getFlags(fs *flag.FlagSet) []string { - var result []string - fs.Visit(func(flag *flag.Flag) { - result = append(result, flag.Name) - }) - return result -} diff --git a/cmd/cmdtrace/cmd_span_test.go b/cmd/cmdtrace/cmd_span_test.go deleted file mode 100644 index 27becd3c363..00000000000 --- a/cmd/cmdtrace/cmd_span_test.go +++ /dev/null @@ -1,113 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package cmdtrace - -import ( - "reflect" - "testing" - - "github.com/spf13/cobra" - flag "github.com/spf13/pflag" - - commands "github.com/docker/compose/v5/cmd/compose" -) - -func TestGetFlags(t *testing.T) { - // Initialize flagSet with flags - fs := flag.NewFlagSet("up", flag.ContinueOnError) - var ( - detach string - timeout string - ) - fs.StringVar(&detach, "detach", "d", "") - fs.StringVar(&timeout, "timeout", "t", "") - _ = fs.Set("detach", "detach") - _ = fs.Set("timeout", "timeout") - - tests := []struct { - name string - input *flag.FlagSet - expected []string - }{ - { - name: "NoFlags", - input: flag.NewFlagSet("NoFlags", flag.ContinueOnError), - expected: nil, - }, - { - name: "Flags", - input: fs, - expected: []string{"detach", "timeout"}, - }, - } - - for _, test := range tests { - t.Run(test.name, func(t *testing.T) { - result := getFlags(test.input) - if !reflect.DeepEqual(result, test.expected) { - t.Errorf("Expected %v, but got %v", test.expected, result) - } - }) - } -} - -func TestCommandName(t *testing.T) { - tests := []struct { - name string - setupCmd func() *cobra.Command - want []string - }{ - { - name: "docker compose alpha watch -> [watch, alpha]", - setupCmd: func() *cobra.Command { - dockerCmd := &cobra.Command{Use: "docker"} - composeCmd := &cobra.Command{Use: commands.PluginName} - alphaCmd := &cobra.Command{Use: "alpha"} - watchCmd := &cobra.Command{Use: "watch"} - - dockerCmd.AddCommand(composeCmd) - composeCmd.AddCommand(alphaCmd) - alphaCmd.AddCommand(watchCmd) - - return watchCmd - }, - want: []string{"watch", "alpha"}, - }, - { - name: "docker-compose up -> [up]", - setupCmd: func() *cobra.Command { - dockerComposeCmd := &cobra.Command{Use: commands.PluginName} - upCmd := &cobra.Command{Use: "up"} - - dockerComposeCmd.AddCommand(upCmd) - - return upCmd - }, - want: []string{"up"}, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - cmd := tt.setupCmd() - got := commandName(cmd) - if !reflect.DeepEqual(got, tt.want) { - t.Errorf("commandName() = %v, want %v", got, tt.want) - } - }) - } -} diff --git a/cmd/compatibility/convert.go b/cmd/compatibility/convert.go deleted file mode 100644 index 78d9b8303c5..00000000000 --- a/cmd/compatibility/convert.go +++ /dev/null @@ -1,116 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compatibility - -import ( - "fmt" - "os" - "strings" - - "github.com/docker/compose/v5/cmd/compose" -) - -func getCompletionCommands() []string { - return []string{ - "__complete", - "__completeNoDesc", - } -} - -func getBoolFlags() []string { - return []string{ - "--debug", "-D", - "--verbose", - "--tls", - "--tlsverify", - } -} - -func getStringFlags() []string { - return []string{ - "--tlscacert", - "--tlscert", - "--tlskey", - "--host", "-H", - "--context", - "--log-level", - } -} - -// Convert transforms standalone docker-compose args into CLI plugin compliant ones -func Convert(args []string) []string { - var rootFlags []string - command := []string{compose.PluginName} - l := len(args) -ARGS: - for i := 0; i < l; i++ { - arg := args[i] - if contains(getCompletionCommands(), arg) { - command = append([]string{arg}, command...) - continue - } - if arg != "" && arg[0] != '-' { - command = append(command, args[i:]...) - break - } - - switch arg { - case "--verbose": - arg = "--debug" - case "-h": - // docker cli has deprecated -h to avoid ambiguity with -H, while docker-compose still support it - arg = "--help" - case "--version", "-v": - // redirect --version pseudo-command to actual command - arg = "version" - } - - if contains(getBoolFlags(), arg) { - rootFlags = append(rootFlags, arg) - continue - } - for _, flag := range getStringFlags() { - if arg == flag { - i++ - if i >= l { - fmt.Fprintf(os.Stderr, "flag needs an argument: '%s'\n", arg) - os.Exit(1) - } - rootFlags = append(rootFlags, arg, args[i]) - continue ARGS - } - if strings.HasPrefix(arg, flag) { - _, val, found := strings.Cut(arg, "=") - if found { - rootFlags = append(rootFlags, flag, val) - continue ARGS - } - } - } - command = append(command, arg) - } - return append(rootFlags, command...) -} - -func contains(array []string, needle string) bool { - for _, val := range array { - if val == needle { - return true - } - } - return false -} diff --git a/cmd/compatibility/convert_test.go b/cmd/compatibility/convert_test.go deleted file mode 100644 index ae01665e92a..00000000000 --- a/cmd/compatibility/convert_test.go +++ /dev/null @@ -1,132 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compatibility - -import ( - "errors" - "os" - "os/exec" - "testing" - - "gotest.tools/v3/assert" -) - -func Test_convert(t *testing.T) { - tests := []struct { - name string - args []string - want []string - wantErr bool - }{ - { - name: "compose only", - args: []string{"up"}, - want: []string{"compose", "up"}, - }, - { - name: "with context", - args: []string{"--context", "foo", "-f", "compose.yaml", "up"}, - want: []string{"--context", "foo", "compose", "-f", "compose.yaml", "up"}, - }, - { - name: "with context arg", - args: []string{"--context=foo", "-f", "compose.yaml", "up"}, - want: []string{"--context", "foo", "compose", "-f", "compose.yaml", "up"}, - }, - { - name: "with host", - args: []string{"--host", "tcp://1.2.3.4", "up"}, - want: []string{"--host", "tcp://1.2.3.4", "compose", "up"}, - }, - { - name: "compose --verbose", - args: []string{"--verbose"}, - want: []string{"--debug", "compose"}, - }, - { - name: "compose --version", - args: []string{"--version"}, - want: []string{"compose", "version"}, - }, - { - name: "compose -v", - args: []string{"-v"}, - want: []string{"compose", "version"}, - }, - { - name: "help", - args: []string{"-h"}, - want: []string{"compose", "--help"}, - }, - { - name: "issues/1962", - args: []string{"psql", "-h", "postgres"}, - want: []string{"compose", "psql", "-h", "postgres"}, // -h should not be converted to --help - }, - { - name: "issues/8648", - args: []string{"exec", "mongo", "mongo", "--host", "mongo"}, - want: []string{"compose", "exec", "mongo", "mongo", "--host", "mongo"}, // --host is passed to exec - }, - { - name: "issues/12", - args: []string{"--log-level", "INFO", "up"}, - want: []string{"--log-level", "INFO", "compose", "up"}, - }, - { - name: "empty string argument", - args: []string{"--project-directory", "", "ps"}, - want: []string{"compose", "--project-directory", "", "ps"}, - }, - { - name: "compose as project name", - args: []string{"--project-name", "compose", "down", "--remove-orphans"}, - want: []string{"compose", "--project-name", "compose", "down", "--remove-orphans"}, - }, - { - name: "completion command", - args: []string{"__complete", "up"}, - want: []string{"__complete", "compose", "up"}, - }, - { - name: "string flag without argument", - args: []string{"--log-level"}, - wantErr: true, - }, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - if tt.wantErr { - if os.Getenv("BE_CRASHER") == "1" { - Convert(tt.args) - return - } - cmd := exec.Command(os.Args[0], "-test.run=^"+t.Name()+"$") - cmd.Env = append(os.Environ(), "BE_CRASHER=1") - err := cmd.Run() - var e *exec.ExitError - if errors.As(err, &e) && !e.Success() { - return - } - t.Fatalf("process ran with err %v, want exit status 1", err) - } else { - got := Convert(tt.args) - assert.DeepEqual(t, tt.want, got) - } - }) - } -} diff --git a/cmd/compose/alpha.go b/cmd/compose/alpha.go deleted file mode 100644 index 8acc969ca30..00000000000 --- a/cmd/compose/alpha.go +++ /dev/null @@ -1,37 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" -) - -// alphaCommand groups all experimental subcommands -func alphaCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - cmd := &cobra.Command{ - Short: "Experimental commands", - Use: "alpha [COMMAND]", - Hidden: true, - Annotations: map[string]string{ - "experimentalCLI": "true", - }, - } - cmd.AddCommand( - vizCommand(p, dockerCli, backendOptions), - publishCommand(p, dockerCli, backendOptions), - generateCommand(p, dockerCli, backendOptions), - ) - return cmd -} diff --git a/cmd/compose/attach.go b/cmd/compose/attach.go deleted file mode 100644 index 20f23fca20f..00000000000 --- a/cmd/compose/attach.go +++ /dev/null @@ -1,86 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type attachOpts struct { - *composeOptions - - service string - index int - - detachKeys string - noStdin bool - proxy bool -} - -func attachCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := attachOpts{ - composeOptions: &composeOptions{ - ProjectOptions: p, - }, - } - runCmd := &cobra.Command{ - Use: "attach [OPTIONS] SERVICE", - Short: "Attach local standard input, output, and error streams to a service's running container", - Args: cobra.MinimumNArgs(1), - PreRunE: Adapt(func(ctx context.Context, args []string) error { - opts.service = args[0] - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runAttach(ctx, dockerCli, backendOptions, opts) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - runCmd.Flags().IntVar(&opts.index, "index", 0, "index of the container if service has multiple replicas.") - runCmd.Flags().StringVarP(&opts.detachKeys, "detach-keys", "", "", "Override the key sequence for detaching from a container.") - - runCmd.Flags().BoolVar(&opts.noStdin, "no-stdin", false, "Do not attach STDIN") - runCmd.Flags().BoolVar(&opts.proxy, "sig-proxy", true, "Proxy all received signals to the process") - return runCmd -} - -func runAttach(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts attachOpts) error { - projectName, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - attachOpts := api.AttachOptions{ - Service: opts.service, - Index: opts.index, - DetachKeys: opts.detachKeys, - NoStdin: opts.noStdin, - Proxy: opts.proxy, - } - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Attach(ctx, projectName, attachOpts) -} diff --git a/cmd/compose/bridge.go b/cmd/compose/bridge.go deleted file mode 100644 index fcc780844bd..00000000000 --- a/cmd/compose/bridge.go +++ /dev/null @@ -1,155 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - - "github.com/distribution/reference" - "github.com/docker/cli/cli/command" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/pkg/stringid" - "github.com/docker/go-units" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/bridge" - "github.com/docker/compose/v5/pkg/compose" -) - -func bridgeCommand(p *ProjectOptions, dockerCli command.Cli) *cobra.Command { - cmd := &cobra.Command{ - Use: "bridge CMD [OPTIONS]", - Short: "Convert compose files into another model", - TraverseChildren: true, - } - cmd.AddCommand( - convertCommand(p, dockerCli), - transformersCommand(dockerCli), - ) - return cmd -} - -func convertCommand(p *ProjectOptions, dockerCli command.Cli) *cobra.Command { - convertOpts := bridge.ConvertOptions{} - cmd := &cobra.Command{ - Use: "convert", - Short: "Convert compose files to Kubernetes manifests, Helm charts, or another model", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runConvert(ctx, dockerCli, p, convertOpts) - }), - } - flags := cmd.Flags() - flags.StringVarP(&convertOpts.Output, "output", "o", "out", "The output directory for the Kubernetes resources") - flags.StringArrayVarP(&convertOpts.Transformations, "transformation", "t", nil, "Transformation to apply to compose model (default: docker/compose-bridge-kubernetes)") - flags.StringVar(&convertOpts.Templates, "templates", "", "Directory containing transformation templates") - return cmd -} - -func runConvert(ctx context.Context, dockerCli command.Cli, p *ProjectOptions, opts bridge.ConvertOptions) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, _, err := p.ToProject(ctx, dockerCli, backend, nil) - if err != nil { - return err - } - return bridge.Convert(ctx, dockerCli, project, opts) -} - -func transformersCommand(dockerCli command.Cli) *cobra.Command { - cmd := &cobra.Command{ - Use: "transformations CMD [OPTIONS]", - Short: "Manage transformation images", - } - cmd.AddCommand( - listTransformersCommand(dockerCli), - createTransformerCommand(dockerCli), - ) - return cmd -} - -func listTransformersCommand(dockerCli command.Cli) *cobra.Command { - options := lsOptions{} - cmd := &cobra.Command{ - Use: "list", - Aliases: []string{"ls"}, - Short: "List available transformations", - RunE: Adapt(func(ctx context.Context, args []string) error { - transformers, err := bridge.ListTransformers(ctx, dockerCli) - if err != nil { - return err - } - return displayTransformer(dockerCli, transformers, options) - }), - } - cmd.Flags().StringVar(&options.Format, "format", "table", "Format the output. Values: [table | json]") - cmd.Flags().BoolVarP(&options.Quiet, "quiet", "q", false, "Only display transformer names") - return cmd -} - -func displayTransformer(dockerCli command.Cli, transformers []image.Summary, options lsOptions) error { - if options.Quiet { - for _, t := range transformers { - if len(t.RepoTags) > 0 { - _, _ = fmt.Fprintln(dockerCli.Out(), t.RepoTags[0]) - } else { - _, _ = fmt.Fprintln(dockerCli.Out(), t.ID) - } - } - return nil - } - return formatter.Print(transformers, options.Format, dockerCli.Out(), - func(w io.Writer) { - for _, img := range transformers { - id := stringid.TruncateID(img.ID) - size := units.HumanSizeWithPrecision(float64(img.Size), 3) - repo, tag := "<none>", "<none>" - if len(img.RepoTags) > 0 { - ref, err := reference.ParseDockerRef(img.RepoTags[0]) - if err == nil { - // ParseDockerRef will reject a local image ID - repo = reference.FamiliarName(ref) - if tagged, ok := ref.(reference.Tagged); ok { - tag = tagged.Tag() - } - } - } - - _, _ = fmt.Fprintf(w, "%s\t%s\t%s\t%s\n", id, repo, tag, size) - } - }, - "IMAGE ID", "REPO", "TAGS", "SIZE") -} - -func createTransformerCommand(dockerCli command.Cli) *cobra.Command { - var opts bridge.CreateTransformerOptions - cmd := &cobra.Command{ - Use: "create [OPTION] PATH", - Short: "Create a new transformation", - RunE: Adapt(func(ctx context.Context, args []string) error { - opts.Dest = args[0] - return bridge.CreateTransformer(ctx, dockerCli, opts) - }), - } - cmd.Flags().StringVarP(&opts.From, "from", "f", "", "Existing transformation to copy (default: docker/compose-bridge-kubernetes)") - return cmd -} diff --git a/cmd/compose/build.go b/cmd/compose/build.go deleted file mode 100644 index 996cf4d9ef0..00000000000 --- a/cmd/compose/build.go +++ /dev/null @@ -1,178 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - "strings" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - cliopts "github.com/docker/cli/opts" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/display" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type buildOptions struct { - *ProjectOptions - quiet bool - pull bool - push bool - args []string - noCache bool - memory cliopts.MemBytes - ssh string - builder string - deps bool - print bool - check bool - sbom string - provenance string -} - -func (opts buildOptions) toAPIBuildOptions(services []string) (api.BuildOptions, error) { - var SSHKeys []types.SSHKey - if opts.ssh != "" { - id, path, found := strings.Cut(opts.ssh, "=") - if !found && id != "default" { - return api.BuildOptions{}, fmt.Errorf("invalid ssh key %q", opts.ssh) - } - SSHKeys = append(SSHKeys, types.SSHKey{ - ID: id, - Path: path, - }) - } - builderName := opts.builder - if builderName == "" { - builderName = os.Getenv("BUILDX_BUILDER") - } - - uiMode := display.Mode - if uiMode == display.ModeJSON { - uiMode = "rawjson" - } - - return api.BuildOptions{ - Pull: opts.pull, - Push: opts.push, - Progress: uiMode, - Args: types.NewMappingWithEquals(opts.args), - NoCache: opts.noCache, - Quiet: opts.quiet, - Services: services, - Deps: opts.deps, - Memory: int64(opts.memory), - Print: opts.print, - Check: opts.check, - SSHs: SSHKeys, - Builder: builderName, - SBOM: opts.sbom, - Provenance: opts.provenance, - }, nil -} - -func buildCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := buildOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "build [OPTIONS] [SERVICE...]", - Short: "Build or rebuild services", - PreRunE: Adapt(func(ctx context.Context, args []string) error { - if opts.quiet { - display.Mode = display.ModeQuiet - devnull, err := os.Open(os.DevNull) - if err != nil { - return err - } - os.Stdout = devnull - } - return nil - }), - RunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - if cmd.Flags().Changed("ssh") && opts.ssh == "" { - opts.ssh = "default" - } - if cmd.Flags().Changed("progress") && opts.ssh == "" { - fmt.Fprint(os.Stderr, "--progress is a global compose flag, better use `docker compose --progress xx build ...\n") - } - return runBuild(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.BoolVar(&opts.push, "push", false, "Push service images") - flags.BoolVarP(&opts.quiet, "quiet", "q", false, "Suppress the build output") - flags.BoolVar(&opts.pull, "pull", false, "Always attempt to pull a newer version of the image") - flags.StringArrayVar(&opts.args, "build-arg", []string{}, "Set build-time variables for services") - flags.StringVar(&opts.ssh, "ssh", "", "Set SSH authentications used when building service images. (use 'default' for using your default SSH Agent)") - flags.StringVar(&opts.builder, "builder", "", "Set builder to use") - flags.BoolVar(&opts.deps, "with-dependencies", false, "Also build dependencies (transitively)") - flags.StringVar(&opts.provenance, "provenance", "", `Add a provenance attestation`) - flags.StringVar(&opts.sbom, "sbom", "", `Add a SBOM attestation`) - - flags.Bool("parallel", true, "Build images in parallel. DEPRECATED") - flags.MarkHidden("parallel") //nolint:errcheck - flags.Bool("compress", true, "Compress the build context using gzip. DEPRECATED") - flags.MarkHidden("compress") //nolint:errcheck - flags.Bool("force-rm", true, "Always remove intermediate containers. DEPRECATED") - flags.MarkHidden("force-rm") //nolint:errcheck - flags.BoolVar(&opts.noCache, "no-cache", false, "Do not use cache when building the image") - flags.Bool("no-rm", false, "Do not remove intermediate containers after a successful build. DEPRECATED") - flags.MarkHidden("no-rm") //nolint:errcheck - flags.VarP(&opts.memory, "memory", "m", "Set memory limit for the build container. Not supported by BuildKit.") - flags.StringVar(&p.Progress, "progress", "", fmt.Sprintf(`Set type of ui output (%s)`, strings.Join(printerModes, ", "))) - flags.MarkHidden("progress") //nolint:errcheck - flags.BoolVar(&opts.print, "print", false, "Print equivalent bake file") - flags.BoolVar(&opts.check, "check", false, "Check build configuration") - - return cmd -} - -func runBuild(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts buildOptions, services []string) error { - if opts.print { - backendOptions.Add(compose.WithEventProcessor(display.Quiet())) - } - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - opts.All = true // do not drop resources as build may involve some dependencies by additional_contexts - project, _, err := opts.ToProject(ctx, dockerCli, backend, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - if err := applyPlatforms(project, false); err != nil { - return err - } - - apiBuildOptions, err := opts.toAPIBuildOptions(services) - if err != nil { - return err - } - apiBuildOptions.Attestations = true - - return backend.Build(ctx, project, apiBuildOptions) -} diff --git a/cmd/compose/commit.go b/cmd/compose/commit.go deleted file mode 100644 index 730deb2e561..00000000000 --- a/cmd/compose/commit.go +++ /dev/null @@ -1,97 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/opts" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type commitOptions struct { - *ProjectOptions - - service string - reference string - - pause bool - comment string - author string - changes opts.ListOpts - - index int -} - -func commitCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - options := commitOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "commit [OPTIONS] SERVICE [REPOSITORY[:TAG]]", - Short: "Create a new image from a service container's changes", - Args: cobra.RangeArgs(1, 2), - PreRunE: Adapt(func(ctx context.Context, args []string) error { - options.service = args[0] - if len(args) > 1 { - options.reference = args[1] - } - - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runCommit(ctx, dockerCli, backendOptions, options) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - flags := cmd.Flags() - flags.IntVar(&options.index, "index", 0, "index of the container if service has multiple replicas.") - - flags.BoolVarP(&options.pause, "pause", "p", true, "Pause container during commit") - flags.StringVarP(&options.comment, "message", "m", "", "Commit message") - flags.StringVarP(&options.author, "author", "a", "", `Author (e.g., "John Hannibal Smith <hannibal@a-team.com>")`) - options.changes = opts.NewListOpts(nil) - flags.VarP(&options.changes, "change", "c", "Apply Dockerfile instruction to the created image") - - return cmd -} - -func runCommit(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, options commitOptions) error { - projectName, err := options.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Commit(ctx, projectName, api.CommitOptions{ - Service: options.service, - Reference: options.reference, - Pause: options.pause, - Comment: options.comment, - Author: options.author, - Changes: options.changes, - Index: options.index, - }) -} diff --git a/cmd/compose/completion.go b/cmd/compose/completion.go deleted file mode 100644 index 0d6c9fe4d40..00000000000 --- a/cmd/compose/completion.go +++ /dev/null @@ -1,119 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "sort" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -// validArgsFn defines a completion func to be returned to fetch completion options -type validArgsFn func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) - -func noCompletion() validArgsFn { - return func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - return []string{}, cobra.ShellCompDirectiveNoSpace - } -} - -func completeServiceNames(dockerCli command.Cli, p *ProjectOptions) validArgsFn { - return func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - p.Offline = true - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return nil, cobra.ShellCompDirectiveNoFileComp - } - - project, _, err := p.ToProject(cmd.Context(), dockerCli, backend, nil) - if err != nil { - return nil, cobra.ShellCompDirectiveNoFileComp - } - var values []string - serviceNames := append(project.ServiceNames(), project.DisabledServiceNames()...) - for _, s := range serviceNames { - if toComplete == "" || strings.HasPrefix(s, toComplete) { - values = append(values, s) - } - } - return values, cobra.ShellCompDirectiveNoFileComp - } -} - -func completeProjectNames(dockerCli command.Cli, backendOptions *BackendOptions) func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - return func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return nil, cobra.ShellCompDirectiveError - } - - list, err := backend.List(cmd.Context(), api.ListOptions{ - All: true, - }) - if err != nil { - return nil, cobra.ShellCompDirectiveError - } - var values []string - for _, stack := range list { - if strings.HasPrefix(stack.Name, toComplete) { - values = append(values, stack.Name) - } - } - return values, cobra.ShellCompDirectiveNoFileComp - } -} - -func completeProfileNames(dockerCli command.Cli, p *ProjectOptions) validArgsFn { - return func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - p.Offline = true - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return nil, cobra.ShellCompDirectiveNoFileComp - } - - project, _, err := p.ToProject(cmd.Context(), dockerCli, backend, nil) - if err != nil { - return nil, cobra.ShellCompDirectiveNoFileComp - } - - allProfileNames := project.AllServices().GetProfiles() - sort.Strings(allProfileNames) - - var values []string - for _, profileName := range allProfileNames { - if strings.HasPrefix(profileName, toComplete) { - values = append(values, profileName) - } - } - return values, cobra.ShellCompDirectiveNoFileComp - } -} - -func completeScaleArgs(cli command.Cli, p *ProjectOptions) cobra.CompletionFunc { - return func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - completions, directive := completeServiceNames(cli, p)(cmd, args, toComplete) - for i, completion := range completions { - completions[i] = completion + "=" - } - return completions, directive - } -} diff --git a/cmd/compose/compose.go b/cmd/compose/compose.go deleted file mode 100644 index 4adc460de19..00000000000 --- a/cmd/compose/compose.go +++ /dev/null @@ -1,724 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "encoding/json" - "errors" - "fmt" - "io" - "os" - "os/signal" - "path/filepath" - "strconv" - "strings" - "syscall" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/dotenv" - "github.com/compose-spec/compose-go/v2/loader" - composepaths "github.com/compose-spec/compose-go/v2/paths" - "github.com/compose-spec/compose-go/v2/types" - composegoutils "github.com/compose-spec/compose-go/v2/utils" - "github.com/docker/buildx/util/logutil" - dockercli "github.com/docker/cli/cli" - "github.com/docker/cli/cli-plugins/metadata" - "github.com/docker/cli/cli/command" - "github.com/docker/cli/pkg/kvfile" - "github.com/morikuni/aec" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/cmd/display" - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" - "github.com/docker/compose/v5/pkg/remote" - "github.com/docker/compose/v5/pkg/utils" -) - -const ( - // ComposeParallelLimit set the limit running concurrent operation on docker engine - ComposeParallelLimit = "COMPOSE_PARALLEL_LIMIT" - // ComposeProjectName define the project name to be used, instead of guessing from parent directory - ComposeProjectName = "COMPOSE_PROJECT_NAME" - // ComposeCompatibility try to mimic compose v1 as much as possible - ComposeCompatibility = api.ComposeCompatibility - // ComposeRemoveOrphans remove "orphaned" containers, i.e. containers tagged for current project but not declared as service - ComposeRemoveOrphans = "COMPOSE_REMOVE_ORPHANS" - // ComposeIgnoreOrphans ignore "orphaned" containers - ComposeIgnoreOrphans = "COMPOSE_IGNORE_ORPHANS" - // ComposeEnvFiles defines the env files to use if --env-file isn't used - ComposeEnvFiles = "COMPOSE_ENV_FILES" - // ComposeMenu defines if the navigation menu should be rendered. Can be also set via --menu - ComposeMenu = "COMPOSE_MENU" - // ComposeProgress defines type of progress output, if --progress isn't used - ComposeProgress = "COMPOSE_PROGRESS" -) - -// rawEnv load a dot env file using docker/cli key=value parser, without attempt to interpolate or evaluate values -func rawEnv(r io.Reader, filename string, vars map[string]string, lookup func(key string) (string, bool)) error { - lines, err := kvfile.ParseFromReader(r, lookup) - if err != nil { - return fmt.Errorf("failed to parse env_file %s: %w", filename, err) - } - for _, line := range lines { - key, value, _ := strings.Cut(line, "=") - vars[key] = value - } - return nil -} - -var stdioToStdout bool - -func init() { - // compose evaluates env file values for interpolation - // `raw` format allows to load env_file with the same parser used by docker run --env-file - dotenv.RegisterFormat("raw", rawEnv) - - if v, ok := os.LookupEnv("COMPOSE_STATUS_STDOUT"); ok { - stdioToStdout, _ = strconv.ParseBool(v) - } -} - -// Command defines a compose CLI command as a func with args -type Command func(context.Context, []string) error - -// CobraCommand defines a cobra command function -type CobraCommand func(context.Context, *cobra.Command, []string) error - -// AdaptCmd adapt a CobraCommand func to cobra library -func AdaptCmd(fn CobraCommand) func(cmd *cobra.Command, args []string) error { - return func(cmd *cobra.Command, args []string) error { - ctx, cancel := context.WithCancel(cmd.Context()) - - s := make(chan os.Signal, 1) - signal.Notify(s, syscall.SIGTERM, syscall.SIGINT) - go func() { - <-s - cancel() - signal.Stop(s) - close(s) - }() - - err := fn(ctx, cmd, args) - if api.IsErrCanceled(err) || errors.Is(ctx.Err(), context.Canceled) { - err = dockercli.StatusError{ - StatusCode: 130, - } - } - if display.Mode == display.ModeJSON { - err = makeJSONError(err) - } - return err - } -} - -// Adapt a Command func to cobra library -func Adapt(fn Command) func(cmd *cobra.Command, args []string) error { - return AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - return fn(ctx, args) - }) -} - -type ProjectOptions struct { - ProjectName string - Profiles []string - ConfigPaths []string - WorkDir string - ProjectDir string - EnvFiles []string - Compatibility bool - Progress string - Offline bool - All bool - insecureRegistries []string -} - -// ProjectFunc does stuff within a types.Project -type ProjectFunc func(ctx context.Context, project *types.Project) error - -// ProjectServicesFunc does stuff within a types.Project and a selection of services -type ProjectServicesFunc func(ctx context.Context, project *types.Project, services []string) error - -// WithProject creates a cobra run command from a ProjectFunc based on configured project options and selected services -func (o *ProjectOptions) WithProject(fn ProjectFunc, dockerCli command.Cli) func(cmd *cobra.Command, args []string) error { - return o.WithServices(dockerCli, func(ctx context.Context, project *types.Project, services []string) error { - return fn(ctx, project) - }) -} - -// WithServices creates a cobra run command from a ProjectFunc based on configured project options and selected services -func (o *ProjectOptions) WithServices(dockerCli command.Cli, fn ProjectServicesFunc) func(cmd *cobra.Command, args []string) error { - return Adapt(func(ctx context.Context, services []string) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, metrics, err := o.ToProject(ctx, dockerCli, backend, services, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - ctx = context.WithValue(ctx, tracing.MetricsKey{}, metrics) - - project, err = project.WithServicesEnvironmentResolved(true) - if err != nil { - return err - } - - return fn(ctx, project, services) - }) -} - -type jsonErrorData struct { - Error bool `json:"error,omitempty"` - Message string `json:"message,omitempty"` -} - -func errorAsJSON(message string) string { - errorMessage := &jsonErrorData{ - Error: true, - Message: message, - } - marshal, err := json.Marshal(errorMessage) - if err == nil { - return string(marshal) - } else { - return message - } -} - -func makeJSONError(err error) error { - if err == nil { - return nil - } - var statusErr dockercli.StatusError - if errors.As(err, &statusErr) { - return dockercli.StatusError{ - StatusCode: statusErr.StatusCode, - Status: errorAsJSON(statusErr.Status), - } - } - return fmt.Errorf("%s", errorAsJSON(err.Error())) -} - -func (o *ProjectOptions) addProjectFlags(f *pflag.FlagSet) { - f.StringArrayVar(&o.Profiles, "profile", []string{}, "Specify a profile to enable") - f.StringVarP(&o.ProjectName, "project-name", "p", "", "Project name") - f.StringArrayVarP(&o.ConfigPaths, "file", "f", []string{}, "Compose configuration files") - f.StringArrayVar(&o.insecureRegistries, "insecure-registry", []string{}, "Use insecure registry to pull Compose OCI artifacts. Doesn't apply to images") - _ = f.MarkHidden("insecure-registry") - f.StringArrayVar(&o.EnvFiles, "env-file", defaultStringArrayVar(ComposeEnvFiles), "Specify an alternate environment file") - f.StringVar(&o.ProjectDir, "project-directory", "", "Specify an alternate working directory\n(default: the path of the, first specified, Compose file)") - f.StringVar(&o.WorkDir, "workdir", "", "DEPRECATED! USE --project-directory INSTEAD.\nSpecify an alternate working directory\n(default: the path of the, first specified, Compose file)") - f.BoolVar(&o.Compatibility, "compatibility", false, "Run compose in backward compatibility mode") - f.StringVar(&o.Progress, "progress", os.Getenv(ComposeProgress), fmt.Sprintf(`Set type of progress output (%s)`, strings.Join(printerModes, ", "))) - f.BoolVar(&o.All, "all-resources", false, "Include all resources, even those not used by services") - _ = f.MarkHidden("workdir") -} - -// get default value for a command line flag that is set by a coma-separated value in environment variable -func defaultStringArrayVar(env string) []string { - return strings.FieldsFunc(os.Getenv(env), func(c rune) bool { - return c == ',' - }) -} - -func (o *ProjectOptions) projectOrName(ctx context.Context, dockerCli command.Cli, services ...string) (*types.Project, string, error) { - name := o.ProjectName - var project *types.Project - if len(o.ConfigPaths) > 0 || o.ProjectName == "" { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return nil, "", err - } - - p, _, err := o.ToProject(ctx, dockerCli, backend, services, cli.WithDiscardEnvFile, cli.WithoutEnvironmentResolution) - if err != nil { - envProjectName := os.Getenv(ComposeProjectName) - if envProjectName != "" { - return nil, envProjectName, nil - } - return nil, "", err - } - project = p - name = p.Name - } - return project, name, nil -} - -func (o *ProjectOptions) toProjectName(ctx context.Context, dockerCli command.Cli) (string, error) { - if o.ProjectName != "" { - return o.ProjectName, nil - } - - envProjectName := os.Getenv(ComposeProjectName) - if envProjectName != "" { - return envProjectName, nil - } - - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return "", err - } - - project, _, err := o.ToProject(ctx, dockerCli, backend, nil) - if err != nil { - return "", err - } - return project.Name, nil -} - -func (o *ProjectOptions) ToModel(ctx context.Context, dockerCli command.Cli, services []string, po ...cli.ProjectOptionsFn) (map[string]any, error) { - remotes := o.remoteLoaders(dockerCli) - for _, r := range remotes { - po = append(po, cli.WithResourceLoader(r)) - } - - options, err := o.toProjectOptions(po...) - if err != nil { - return nil, err - } - - if o.Compatibility || utils.StringToBool(options.Environment[ComposeCompatibility]) { - api.Separator = "_" - } - - return options.LoadModel(ctx) -} - -// ToProject loads a Compose project using the LoadProject API. -// Accepts optional cli.ProjectOptionsFn to control loader behavior. -func (o *ProjectOptions) ToProject(ctx context.Context, dockerCli command.Cli, backend api.Compose, services []string, po ...cli.ProjectOptionsFn) (*types.Project, tracing.Metrics, error) { - var metrics tracing.Metrics - remotes := o.remoteLoaders(dockerCli) - - // Setup metrics listener to collect project data - metricsListener := func(event string, metadata map[string]any) { - switch event { - case "extends": - metrics.CountExtends++ - case "include": - paths := metadata["path"].(types.StringList) - for _, path := range paths { - var isRemote bool - for _, r := range remotes { - if r.Accept(path) { - isRemote = true - break - } - } - if isRemote { - metrics.CountIncludesRemote++ - } else { - metrics.CountIncludesLocal++ - } - } - } - } - - loadOpts := api.ProjectLoadOptions{ - ProjectName: o.ProjectName, - ConfigPaths: o.ConfigPaths, - WorkingDir: o.ProjectDir, - EnvFiles: o.EnvFiles, - Profiles: o.Profiles, - Services: services, - Offline: o.Offline, - All: o.All, - Compatibility: o.Compatibility, - ProjectOptionsFns: po, - LoadListeners: []api.LoadListener{metricsListener}, - OCI: api.OCIOptions{ - InsecureRegistries: o.insecureRegistries, - }, - } - - project, err := backend.LoadProject(ctx, loadOpts) - if err != nil { - return nil, metrics, err - } - - return project, metrics, nil -} - -func (o *ProjectOptions) remoteLoaders(dockerCli command.Cli) []loader.ResourceLoader { - if o.Offline { - return nil - } - git := remote.NewGitRemoteLoader(dockerCli, o.Offline) - oci := remote.NewOCIRemoteLoader(dockerCli, o.Offline, api.OCIOptions{}) - return []loader.ResourceLoader{git, oci} -} - -func (o *ProjectOptions) toProjectOptions(po ...cli.ProjectOptionsFn) (*cli.ProjectOptions, error) { - opts := []cli.ProjectOptionsFn{ - cli.WithWorkingDirectory(o.ProjectDir), - // First apply os.Environment, always win - cli.WithOsEnv, - } - - if _, present := os.LookupEnv("PWD"); !present { - if pwd, err := os.Getwd(); err != nil { - return nil, err - } else { - opts = append(opts, cli.WithEnv([]string{"PWD=" + pwd})) - } - } - - opts = append(opts, - // Load PWD/.env if present and no explicit --env-file has been set - cli.WithEnvFiles(o.EnvFiles...), - // read dot env file to populate project environment - cli.WithDotEnv, - // get compose file path set by COMPOSE_FILE - cli.WithConfigFileEnv, - // if none was selected, get default compose.yaml file from current dir or parent folder - cli.WithDefaultConfigPath, - // .. and then, a project directory != PWD maybe has been set so let's load .env file - cli.WithEnvFiles(o.EnvFiles...), //nolint:gocritic // intentionally applying cli.WithEnvFiles twice. - cli.WithDotEnv, //nolint:gocritic // intentionally applying cli.WithDotEnv twice. - // eventually COMPOSE_PROFILES should have been set - cli.WithDefaultProfiles(o.Profiles...), - cli.WithName(o.ProjectName), - ) - - return cli.NewProjectOptions(o.ConfigPaths, append(po, opts...)...) -} - -// PluginName is the name of the plugin -const PluginName = "compose" - -// RunningAsStandalone detects when running as a standalone program -func RunningAsStandalone() bool { - return len(os.Args) < 2 || os.Args[1] != metadata.MetadataSubcommandName && os.Args[1] != PluginName -} - -type BackendOptions struct { - Options []compose.Option -} - -func (o *BackendOptions) Add(option compose.Option) { - o.Options = append(o.Options, option) -} - -// RootCommand returns the compose command with its child commands -func RootCommand(dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { //nolint:gocyclo - // filter out useless commandConn.CloseWrite warning message that can occur - // when using a remote context that is unreachable: "commandConn.CloseWrite: commandconn: failed to wait: signal: killed" - // https://github.com/docker/cli/blob/e1f24d3c93df6752d3c27c8d61d18260f141310c/cli/connhelper/commandconn/commandconn.go#L203-L215 - logrus.AddHook(logutil.NewFilter([]logrus.Level{ - logrus.WarnLevel, - }, - "commandConn.CloseWrite:", - "commandConn.CloseRead:", - )) - - opts := ProjectOptions{} - var ( - ansi string - noAnsi bool - verbose bool - version bool - parallel int - dryRun bool - ) - c := &cobra.Command{ - Short: "Docker Compose", - Long: "Define and run multi-container applications with Docker", - Use: PluginName, - TraverseChildren: true, - // By default (no Run/RunE in parent c) for typos in subcommands, cobra displays the help of parent c but exit(0) ! - RunE: func(cmd *cobra.Command, args []string) error { - if len(args) == 0 { - return cmd.Help() - } - if version { - return versionCommand(dockerCli).Execute() - } - _ = cmd.Help() - return dockercli.StatusError{ - StatusCode: 1, - Status: fmt.Sprintf("unknown docker command: %q", "compose "+args[0]), - } - }, - PersistentPreRunE: func(cmd *cobra.Command, args []string) error { - parent := cmd.Root() - if parent != nil { - parentPrerun := parent.PersistentPreRunE - if parentPrerun != nil { - err := parentPrerun(cmd, args) - if err != nil { - return err - } - } - } - - if verbose { - logrus.SetLevel(logrus.TraceLevel) - } - - err := setEnvWithDotEnv(opts, dockerCli) - if err != nil { - return err - } - if noAnsi { - if ansi != "auto" { - return errors.New(`cannot specify DEPRECATED "--no-ansi" and "--ansi". Please use only "--ansi"`) - } - ansi = "never" - fmt.Fprint(os.Stderr, "option '--no-ansi' is DEPRECATED ! Please use '--ansi' instead.\n") - } - if v, ok := os.LookupEnv("COMPOSE_ANSI"); ok && !cmd.Flags().Changed("ansi") { - ansi = v - } - formatter.SetANSIMode(dockerCli, ansi) - - if noColor, ok := os.LookupEnv("NO_COLOR"); ok && noColor != "" { - display.NoColor() - formatter.SetANSIMode(dockerCli, formatter.Never) - } - - switch ansi { - case "never": - display.Mode = display.ModePlain - case "always": - display.Mode = display.ModeTTY - } - - detached, _ := cmd.Flags().GetBool("detach") - var ep api.EventProcessor - switch opts.Progress { - case "", display.ModeAuto: - switch { - case ansi == "never": - display.Mode = display.ModePlain - ep = display.Plain(dockerCli.Err()) - case dockerCli.Out().IsTerminal(): - ep = display.Full(dockerCli.Err(), stdinfo(dockerCli), detached) - default: - ep = display.Plain(dockerCli.Err()) - } - case display.ModeTTY: - if ansi == "never" { - return fmt.Errorf("can't use --progress tty while ANSI support is disabled") - } - display.Mode = display.ModeTTY - ep = display.Full(dockerCli.Err(), stdinfo(dockerCli), detached) - - case display.ModePlain: - if ansi == "always" { - return fmt.Errorf("can't use --progress plain while ANSI support is forced") - } - display.Mode = display.ModePlain - ep = display.Plain(dockerCli.Err()) - case display.ModeQuiet, "none": - display.Mode = display.ModeQuiet - ep = display.Quiet() - case display.ModeJSON: - display.Mode = display.ModeJSON - logrus.SetFormatter(&logrus.JSONFormatter{}) - ep = display.JSON(dockerCli.Err()) - default: - return fmt.Errorf("unsupported --progress value %q", opts.Progress) - } - backendOptions.Add(compose.WithEventProcessor(ep)) - - // (4) options validation / normalization - if opts.WorkDir != "" { - if opts.ProjectDir != "" { - return errors.New(`cannot specify DEPRECATED "--workdir" and "--project-directory". Please use only "--project-directory" instead`) - } - opts.ProjectDir = opts.WorkDir - fmt.Fprint(os.Stderr, aec.Apply("option '--workdir' is DEPRECATED at root level! Please use '--project-directory' instead.\n", aec.RedF)) - } - for i, file := range opts.EnvFiles { - file = composepaths.ExpandUser(file) - if !filepath.IsAbs(file) { - file, err := filepath.Abs(file) - if err != nil { - return err - } - opts.EnvFiles[i] = file - } else { - opts.EnvFiles[i] = file - } - } - - composeCmd := cmd - for composeCmd.Name() != PluginName { - if !composeCmd.HasParent() { - return fmt.Errorf("error parsing command line, expected %q", PluginName) - } - composeCmd = composeCmd.Parent() - } - - if v, ok := os.LookupEnv(ComposeParallelLimit); ok && !composeCmd.Flags().Changed("parallel") { - i, err := strconv.Atoi(v) - if err != nil { - return fmt.Errorf("%s must be an integer (found: %q)", ComposeParallelLimit, v) - } - parallel = i - } - if parallel > 0 { - logrus.Debugf("Limiting max concurrency to %d jobs", parallel) - backendOptions.Add(compose.WithMaxConcurrency(parallel)) - } - - // dry run detection - if dryRun { - backendOptions.Add(compose.WithDryRun) - } - return nil - }, - } - - c.AddCommand( - upCommand(&opts, dockerCli, backendOptions), - downCommand(&opts, dockerCli, backendOptions), - startCommand(&opts, dockerCli, backendOptions), - restartCommand(&opts, dockerCli, backendOptions), - stopCommand(&opts, dockerCli, backendOptions), - psCommand(&opts, dockerCli, backendOptions), - listCommand(dockerCli, backendOptions), - logsCommand(&opts, dockerCli, backendOptions), - configCommand(&opts, dockerCli), - killCommand(&opts, dockerCli, backendOptions), - runCommand(&opts, dockerCli, backendOptions), - removeCommand(&opts, dockerCli, backendOptions), - execCommand(&opts, dockerCli, backendOptions), - attachCommand(&opts, dockerCli, backendOptions), - exportCommand(&opts, dockerCli, backendOptions), - commitCommand(&opts, dockerCli, backendOptions), - pauseCommand(&opts, dockerCli, backendOptions), - unpauseCommand(&opts, dockerCli, backendOptions), - topCommand(&opts, dockerCli, backendOptions), - eventsCommand(&opts, dockerCli, backendOptions), - portCommand(&opts, dockerCli, backendOptions), - imagesCommand(&opts, dockerCli, backendOptions), - versionCommand(dockerCli), - buildCommand(&opts, dockerCli, backendOptions), - pushCommand(&opts, dockerCli, backendOptions), - pullCommand(&opts, dockerCli, backendOptions), - createCommand(&opts, dockerCli, backendOptions), - copyCommand(&opts, dockerCli, backendOptions), - waitCommand(&opts, dockerCli, backendOptions), - scaleCommand(&opts, dockerCli, backendOptions), - statsCommand(&opts, dockerCli), - watchCommand(&opts, dockerCli, backendOptions), - publishCommand(&opts, dockerCli, backendOptions), - alphaCommand(&opts, dockerCli, backendOptions), - bridgeCommand(&opts, dockerCli), - volumesCommand(&opts, dockerCli, backendOptions), - ) - - c.Flags().SetInterspersed(false) - opts.addProjectFlags(c.Flags()) - c.RegisterFlagCompletionFunc( //nolint:errcheck - "project-name", - completeProjectNames(dockerCli, backendOptions), - ) - c.RegisterFlagCompletionFunc( //nolint:errcheck - "project-directory", - func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - return []string{}, cobra.ShellCompDirectiveFilterDirs - }, - ) - c.RegisterFlagCompletionFunc( //nolint:errcheck - "file", - func(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) { - return []string{"yaml", "yml"}, cobra.ShellCompDirectiveFilterFileExt - }, - ) - c.RegisterFlagCompletionFunc( //nolint:errcheck - "profile", - completeProfileNames(dockerCli, &opts), - ) - c.RegisterFlagCompletionFunc( //nolint:errcheck - "progress", - cobra.FixedCompletions(printerModes, cobra.ShellCompDirectiveNoFileComp), - ) - - c.Flags().StringVar(&ansi, "ansi", "auto", `Control when to print ANSI control characters ("never"|"always"|"auto")`) - c.Flags().IntVar(¶llel, "parallel", -1, `Control max parallelism, -1 for unlimited`) - c.Flags().BoolVarP(&version, "version", "v", false, "Show the Docker Compose version information") - c.PersistentFlags().BoolVar(&dryRun, "dry-run", false, "Execute command in dry run mode") - c.Flags().MarkHidden("version") //nolint:errcheck - c.Flags().BoolVar(&noAnsi, "no-ansi", false, `Do not print ANSI control characters (DEPRECATED)`) - c.Flags().MarkHidden("no-ansi") //nolint:errcheck - c.Flags().BoolVar(&verbose, "verbose", false, "Show more output") - c.Flags().MarkHidden("verbose") //nolint:errcheck - return c -} - -func stdinfo(dockerCli command.Cli) io.Writer { - if stdioToStdout { - return dockerCli.Out() - } - return dockerCli.Err() -} - -func setEnvWithDotEnv(opts ProjectOptions, dockerCli command.Cli) error { - // Check if we're using a remote config (OCI or Git) - // If so, skip env loading as remote loaders haven't been initialized yet - // and trying to process the path would fail - remoteLoaders := opts.remoteLoaders(dockerCli) - for _, path := range opts.ConfigPaths { - for _, loader := range remoteLoaders { - if loader.Accept(path) { - // Remote config - skip env loading for now - // It will be loaded later when the project is fully initialized - return nil - } - } - } - - options, err := cli.NewProjectOptions(opts.ConfigPaths, - cli.WithWorkingDirectory(opts.ProjectDir), - cli.WithOsEnv, - cli.WithEnvFiles(opts.EnvFiles...), - cli.WithDotEnv, - ) - if err != nil { - return err - } - envFromFile, err := dotenv.GetEnvFromFile(composegoutils.GetAsEqualsMap(os.Environ()), options.EnvFiles) - if err != nil { - return err - } - for k, v := range envFromFile { - if _, ok := os.LookupEnv(k); !ok && strings.HasPrefix(k, "COMPOSE_") { - if err := os.Setenv(k, v); err != nil { - return err - } - } - } - return nil -} - -var printerModes = []string{ - display.ModeAuto, - display.ModeTTY, - display.ModePlain, - display.ModeJSON, - display.ModeQuiet, -} diff --git a/cmd/compose/compose_oci_test.go b/cmd/compose/compose_oci_test.go deleted file mode 100644 index 7450f7633cf..00000000000 --- a/cmd/compose/compose_oci_test.go +++ /dev/null @@ -1,76 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/mocks" -) - -func TestSetEnvWithDotEnv_WithOCIArtifact(t *testing.T) { - // Test that setEnvWithDotEnv doesn't fail when using OCI artifacts - ctrl := gomock.NewController(t) - defer ctrl.Finish() - cli := mocks.NewMockCli(ctrl) - - opts := ProjectOptions{ - ConfigPaths: []string{"oci://docker.io/dockersamples/welcome-to-docker"}, - ProjectDir: "", - EnvFiles: []string{}, - } - - err := setEnvWithDotEnv(opts, cli) - assert.NilError(t, err, "setEnvWithDotEnv should not fail with OCI artifact path") -} - -func TestSetEnvWithDotEnv_WithGitRemote(t *testing.T) { - // Test that setEnvWithDotEnv doesn't fail when using Git remotes - ctrl := gomock.NewController(t) - defer ctrl.Finish() - cli := mocks.NewMockCli(ctrl) - - opts := ProjectOptions{ - ConfigPaths: []string{"https://github.com/docker/compose.git"}, - ProjectDir: "", - EnvFiles: []string{}, - } - - err := setEnvWithDotEnv(opts, cli) - assert.NilError(t, err, "setEnvWithDotEnv should not fail with Git remote path") -} - -func TestSetEnvWithDotEnv_WithLocalPath(t *testing.T) { - // Test that setEnvWithDotEnv still works with local paths - // This will fail if the file doesn't exist, but it should not panic - // or produce invalid paths - ctrl := gomock.NewController(t) - defer ctrl.Finish() - cli := mocks.NewMockCli(ctrl) - - opts := ProjectOptions{ - ConfigPaths: []string{"compose.yaml"}, - ProjectDir: "", - EnvFiles: []string{}, - } - - // This may error if files don't exist, but should not panic - _ = setEnvWithDotEnv(opts, cli) -} diff --git a/cmd/compose/compose_test.go b/cmd/compose/compose_test.go deleted file mode 100644 index 708929ff8cd..00000000000 --- a/cmd/compose/compose_test.go +++ /dev/null @@ -1,55 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "gotest.tools/v3/assert" -) - -func TestFilterServices(t *testing.T) { - p := &types.Project{ - Services: types.Services{ - "foo": { - Name: "foo", - Links: []string{"bar"}, - }, - "bar": { - Name: "bar", - DependsOn: map[string]types.ServiceDependency{ - "zot": {}, - }, - }, - "zot": { - Name: "zot", - }, - "qix": { - Name: "qix", - }, - }, - } - p, err := p.WithSelectedServices([]string{"bar"}) - assert.NilError(t, err) - - assert.Equal(t, len(p.Services), 2) - _, err = p.GetService("bar") - assert.NilError(t, err) - _, err = p.GetService("zot") - assert.NilError(t, err) -} diff --git a/cmd/compose/config.go b/cmd/compose/config.go deleted file mode 100644 index 4b5f1892730..00000000000 --- a/cmd/compose/config.go +++ /dev/null @@ -1,567 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "context" - "encoding/json" - "fmt" - "io" - "os" - "sort" - "strings" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/template" - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - "go.yaml.in/yaml/v4" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type configOptions struct { - *ProjectOptions - Format string - Output string - quiet bool - resolveImageDigests bool - noInterpolate bool - noNormalize bool - noResolvePath bool - noResolveEnv bool - services bool - volumes bool - networks bool - models bool - profiles bool - images bool - hash string - noConsistency bool - variables bool - environment bool - lockImageDigests bool -} - -func (o *configOptions) ToProject(ctx context.Context, dockerCli command.Cli, backend api.Compose, services []string) (*types.Project, error) { - project, _, err := o.ProjectOptions.ToProject(ctx, dockerCli, backend, services, o.toProjectOptionsFns()...) - return project, err -} - -func (o *configOptions) ToModel(ctx context.Context, dockerCli command.Cli, services []string, po ...cli.ProjectOptionsFn) (map[string]any, error) { - po = append(po, o.toProjectOptionsFns()...) - return o.ProjectOptions.ToModel(ctx, dockerCli, services, po...) -} - -// toProjectOptionsFns converts config options to cli.ProjectOptionsFn -func (o *configOptions) toProjectOptionsFns() []cli.ProjectOptionsFn { - fns := []cli.ProjectOptionsFn{ - cli.WithInterpolation(!o.noInterpolate), - cli.WithResolvedPaths(!o.noResolvePath), - cli.WithNormalization(!o.noNormalize), - cli.WithConsistency(!o.noConsistency), - cli.WithDefaultProfiles(o.Profiles...), - cli.WithDiscardEnvFile, - } - if o.noResolveEnv { - fns = append(fns, cli.WithoutEnvironmentResolution) - } - return fns -} - -func configCommand(p *ProjectOptions, dockerCli command.Cli) *cobra.Command { - opts := configOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "config [OPTIONS] [SERVICE...]", - Short: "Parse, resolve and render compose file in canonical format", - PreRunE: Adapt(func(ctx context.Context, args []string) error { - if opts.quiet { - devnull, err := os.Open(os.DevNull) - if err != nil { - return err - } - os.Stdout = devnull - } - if p.Compatibility { - opts.noNormalize = true - } - if opts.lockImageDigests { - opts.resolveImageDigests = true - } - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - if opts.services { - return runServices(ctx, dockerCli, opts) - } - if opts.volumes { - return runVolumes(ctx, dockerCli, opts) - } - if opts.networks { - return runNetworks(ctx, dockerCli, opts) - } - if opts.models { - return runModels(ctx, dockerCli, opts) - } - if opts.hash != "" { - return runHash(ctx, dockerCli, opts) - } - if opts.profiles { - return runProfiles(ctx, dockerCli, opts, args) - } - if opts.images { - return runConfigImages(ctx, dockerCli, opts, args) - } - if opts.variables { - return runVariables(ctx, dockerCli, opts, args) - } - if opts.environment { - return runEnvironment(ctx, dockerCli, opts, args) - } - - if opts.Format == "" { - opts.Format = "yaml" - } - return runConfig(ctx, dockerCli, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.StringVar(&opts.Format, "format", "", "Format the output. Values: [yaml | json]") - flags.BoolVar(&opts.resolveImageDigests, "resolve-image-digests", false, "Pin image tags to digests") - flags.BoolVar(&opts.lockImageDigests, "lock-image-digests", false, "Produces an override file with image digests") - flags.BoolVarP(&opts.quiet, "quiet", "q", false, "Only validate the configuration, don't print anything") - flags.BoolVar(&opts.noInterpolate, "no-interpolate", false, "Don't interpolate environment variables") - flags.BoolVar(&opts.noNormalize, "no-normalize", false, "Don't normalize compose model") - flags.BoolVar(&opts.noResolvePath, "no-path-resolution", false, "Don't resolve file paths") - flags.BoolVar(&opts.noConsistency, "no-consistency", false, "Don't check model consistency - warning: may produce invalid Compose output") - flags.BoolVar(&opts.noResolveEnv, "no-env-resolution", false, "Don't resolve service env files") - - flags.BoolVar(&opts.services, "services", false, "Print the service names, one per line.") - flags.BoolVar(&opts.volumes, "volumes", false, "Print the volume names, one per line.") - flags.BoolVar(&opts.networks, "networks", false, "Print the network names, one per line.") - flags.BoolVar(&opts.models, "models", false, "Print the model names, one per line.") - flags.BoolVar(&opts.profiles, "profiles", false, "Print the profile names, one per line.") - flags.BoolVar(&opts.images, "images", false, "Print the image names, one per line.") - flags.StringVar(&opts.hash, "hash", "", "Print the service config hash, one per line.") - flags.BoolVar(&opts.variables, "variables", false, "Print model variables and default values.") - flags.BoolVar(&opts.environment, "environment", false, "Print environment used for interpolation.") - flags.StringVarP(&opts.Output, "output", "o", "", "Save to file (default to stdout)") - - return cmd -} - -func runConfig(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) (err error) { - var content []byte - if opts.noInterpolate { - content, err = runConfigNoInterpolate(ctx, dockerCli, opts, services) - if err != nil { - return err - } - } else { - content, err = runConfigInterpolate(ctx, dockerCli, opts, services) - if err != nil { - return err - } - } - - if !opts.noInterpolate { - content = escapeDollarSign(content) - } - - if opts.quiet { - return nil - } - - if opts.Output != "" && len(content) > 0 { - return os.WriteFile(opts.Output, content, 0o666) - } - _, err = fmt.Fprint(dockerCli.Out(), string(content)) - return err -} - -func runConfigInterpolate(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) ([]byte, error) { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return nil, err - } - - project, err := opts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return nil, err - } - - if opts.resolveImageDigests { - project, err = project.WithImagesResolved(compose.ImageDigestResolver(ctx, dockerCli.ConfigFile(), dockerCli.Client())) - if err != nil { - return nil, err - } - } - - if !opts.noResolveEnv { - project, err = project.WithServicesEnvironmentResolved(true) - if err != nil { - return nil, err - } - } - - if !opts.noConsistency { - err := project.CheckContainerNameUnicity() - if err != nil { - return nil, err - } - } - - if opts.lockImageDigests { - project = imagesOnly(project) - } - - var content []byte - switch opts.Format { - case "json": - content, err = project.MarshalJSON() - case "yaml": - content, err = project.MarshalYAML() - default: - return nil, fmt.Errorf("unsupported format %q", opts.Format) - } - if err != nil { - return nil, err - } - return content, nil -} - -// imagesOnly return project with all attributes removed but service.images -func imagesOnly(project *types.Project) *types.Project { - digests := types.Services{} - for name, config := range project.Services { - digests[name] = types.ServiceConfig{ - Image: config.Image, - } - } - project = &types.Project{Services: digests} - return project -} - -func runConfigNoInterpolate(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) ([]byte, error) { - // we can't use ToProject, so the model we render here is only partially resolved - model, err := opts.ToModel(ctx, dockerCli, services) - if err != nil { - return nil, err - } - - if opts.resolveImageDigests { - err = resolveImageDigests(ctx, dockerCli, model) - if err != nil { - return nil, err - } - } - - if opts.lockImageDigests { - for key, e := range model { - if key != "services" { - delete(model, key) - } else { - for _, s := range e.(map[string]any) { - service := s.(map[string]any) - for key := range service { - if key != "image" { - delete(service, key) - } - } - } - } - } - } - - return formatModel(model, opts.Format) -} - -func resolveImageDigests(ctx context.Context, dockerCli command.Cli, model map[string]any) (err error) { - // create a pseudo-project so we can rely on WithImagesResolved to resolve images - p := &types.Project{ - Services: types.Services{}, - } - services := model["services"].(map[string]any) - for name, s := range services { - service := s.(map[string]any) - if image, ok := service["image"]; ok { - p.Services[name] = types.ServiceConfig{ - Image: image.(string), - } - } - } - - p, err = p.WithImagesResolved(compose.ImageDigestResolver(ctx, dockerCli.ConfigFile(), dockerCli.Client())) - if err != nil { - return err - } - - // Collect image resolved with digest and update model accordingly - for name, s := range services { - service := s.(map[string]any) - config := p.Services[name] - if config.Image != "" { - service["image"] = config.Image - } - services[name] = service - } - model["services"] = services - return nil -} - -func formatModel(model map[string]any, format string) (content []byte, err error) { - switch format { - case "json": - return json.MarshalIndent(model, "", " ") - case "yaml": - buf := bytes.NewBuffer([]byte{}) - encoder := yaml.NewEncoder(buf) - encoder.SetIndent(2) - err = encoder.Encode(model) - return buf.Bytes(), err - default: - return nil, fmt.Errorf("unsupported format %q", format) - } -} - -func runServices(ctx context.Context, dockerCli command.Cli, opts configOptions) error { - if opts.noInterpolate { - // we can't use ToProject, so the model we render here is only partially resolved - data, err := opts.ToModel(ctx, dockerCli, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - if _, ok := data["services"]; ok { - for serviceName := range data["services"].(map[string]any) { - _, _ = fmt.Fprintln(dockerCli.Out(), serviceName) - } - } - - return nil - } - - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, _, err := opts.ProjectOptions.ToProject(ctx, dockerCli, backend, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - err = project.ForEachService(project.ServiceNames(), func(serviceName string, _ *types.ServiceConfig) error { - _, _ = fmt.Fprintln(dockerCli.Out(), serviceName) - return nil - }) - - return err -} - -func runVolumes(ctx context.Context, dockerCli command.Cli, opts configOptions) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, _, err := opts.ProjectOptions.ToProject(ctx, dockerCli, backend, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - for n := range project.Volumes { - _, _ = fmt.Fprintln(dockerCli.Out(), n) - } - return nil -} - -func runNetworks(ctx context.Context, dockerCli command.Cli, opts configOptions) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, _, err := opts.ProjectOptions.ToProject(ctx, dockerCli, backend, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - for n := range project.Networks { - _, _ = fmt.Fprintln(dockerCli.Out(), n) - } - return nil -} - -func runModels(ctx context.Context, dockerCli command.Cli, opts configOptions) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, _, err := opts.ProjectOptions.ToProject(ctx, dockerCli, backend, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - for _, model := range project.Models { - if model.Model != "" { - _, _ = fmt.Fprintln(dockerCli.Out(), model.Model) - } - } - return nil -} - -func runHash(ctx context.Context, dockerCli command.Cli, opts configOptions) error { - var services []string - if opts.hash != "*" { - services = append(services, strings.Split(opts.hash, ",")...) - } - - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, _, err := opts.ProjectOptions.ToProject(ctx, dockerCli, backend, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - if err := applyPlatforms(project, true); err != nil { - return err - } - - if len(services) == 0 { - services = project.ServiceNames() - } - - sorted := services - sort.Slice(sorted, func(i, j int) bool { - return sorted[i] < sorted[j] - }) - - for _, name := range sorted { - s, err := project.GetService(name) - if err != nil { - return err - } - - hash, err := compose.ServiceHash(s) - if err != nil { - return err - } - _, _ = fmt.Fprintf(dockerCli.Out(), "%s %s\n", name, hash) - } - return nil -} - -func runProfiles(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) error { - set := map[string]struct{}{} - - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, err := opts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return err - } - for _, s := range project.AllServices() { - for _, p := range s.Profiles { - set[p] = struct{}{} - } - } - profiles := make([]string, 0, len(set)) - for p := range set { - profiles = append(profiles, p) - } - sort.Strings(profiles) - for _, p := range profiles { - _, _ = fmt.Fprintln(dockerCli.Out(), p) - } - return nil -} - -func runConfigImages(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, err := opts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return err - } - - for _, s := range project.Services { - _, _ = fmt.Fprintln(dockerCli.Out(), api.GetImageNameOrDefault(s, project.Name)) - } - return nil -} - -func runVariables(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) error { - opts.noInterpolate = true - model, err := opts.ToModel(ctx, dockerCli, services, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - variables := template.ExtractVariables(model, template.DefaultPattern) - - if opts.Format == "yaml" { - result, err := yaml.Marshal(variables) - if err != nil { - return err - } - fmt.Print(string(result)) - return nil - } - - return formatter.Print(variables, opts.Format, dockerCli.Out(), func(w io.Writer) { - for name, variable := range variables { - _, _ = fmt.Fprintf(w, "%s\t%t\t%s\t%s\n", name, variable.Required, variable.DefaultValue, variable.PresenceValue) - } - }, "NAME", "REQUIRED", "DEFAULT VALUE", "ALTERNATE VALUE") -} - -func runEnvironment(ctx context.Context, dockerCli command.Cli, opts configOptions, services []string) error { - backend, err := compose.NewComposeService(dockerCli) - if err != nil { - return err - } - - project, err := opts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return err - } - - for _, v := range project.Environment.Values() { - fmt.Println(v) - } - return nil -} - -func escapeDollarSign(marshal []byte) []byte { - dollar := []byte{'$'} - escDollar := []byte{'$', '$'} - return bytes.ReplaceAll(marshal, dollar, escDollar) -} diff --git a/cmd/compose/cp.go b/cmd/compose/cp.go deleted file mode 100644 index 17f7411309d..00000000000 --- a/cmd/compose/cp.go +++ /dev/null @@ -1,95 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type copyOptions struct { - *ProjectOptions - - source string - destination string - index int - all bool - followLink bool - copyUIDGID bool -} - -func copyCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := copyOptions{ - ProjectOptions: p, - } - copyCmd := &cobra.Command{ - Use: `cp [OPTIONS] SERVICE:SRC_PATH DEST_PATH|- - docker compose cp [OPTIONS] SRC_PATH|- SERVICE:DEST_PATH`, - Short: "Copy files/folders between a service container and the local filesystem", - Args: cli.ExactArgs(2), - PreRunE: Adapt(func(ctx context.Context, args []string) error { - if args[0] == "" { - return errors.New("source can not be empty") - } - if args[1] == "" { - return errors.New("destination can not be empty") - } - return nil - }), - RunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - opts.source = args[0] - opts.destination = args[1] - return runCopy(ctx, dockerCli, backendOptions, opts) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - flags := copyCmd.Flags() - flags.IntVar(&opts.index, "index", 0, "Index of the container if service has multiple replicas") - flags.BoolVar(&opts.all, "all", false, "Include containers created by the run command") - flags.BoolVarP(&opts.followLink, "follow-link", "L", false, "Always follow symbol link in SRC_PATH") - flags.BoolVarP(&opts.copyUIDGID, "archive", "a", false, "Archive mode (copy all uid/gid information)") - - return copyCmd -} - -func runCopy(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts copyOptions) error { - name, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Copy(ctx, name, api.CopyOptions{ - Source: opts.source, - Destination: opts.destination, - All: opts.all, - Index: opts.index, - FollowLink: opts.followLink, - CopyUIDGID: opts.copyUIDGID, - }) -} diff --git a/cmd/compose/create.go b/cmd/compose/create.go deleted file mode 100644 index 5f9f7908315..00000000000 --- a/cmd/compose/create.go +++ /dev/null @@ -1,223 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "slices" - "strconv" - "strings" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type createOptions struct { - Build bool - noBuild bool - Pull string - pullChanged bool - removeOrphans bool - ignoreOrphans bool - forceRecreate bool - noRecreate bool - recreateDeps bool - noInherit bool - timeChanged bool - timeout int - quietPull bool - scale []string - AssumeYes bool -} - -func createCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := createOptions{} - buildOpts := buildOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "create [OPTIONS] [SERVICE...]", - Short: "Creates containers for a service", - PreRunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - opts.pullChanged = cmd.Flags().Changed("pull") - if opts.Build && opts.noBuild { - return fmt.Errorf("--build and --no-build are incompatible") - } - if opts.forceRecreate && opts.noRecreate { - return fmt.Errorf("--force-recreate and --no-recreate are incompatible") - } - return nil - }), - RunE: p.WithServices(dockerCli, func(ctx context.Context, project *types.Project, services []string) error { - return runCreate(ctx, dockerCli, backendOptions, opts, buildOpts, project, services) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.BoolVar(&opts.Build, "build", false, "Build images before starting containers") - flags.BoolVar(&opts.noBuild, "no-build", false, "Don't build an image, even if it's policy") - flags.StringVar(&opts.Pull, "pull", "policy", `Pull image before running ("always"|"missing"|"never"|"build")`) - flags.BoolVar(&opts.quietPull, "quiet-pull", false, "Pull without printing progress information") - flags.BoolVar(&opts.forceRecreate, "force-recreate", false, "Recreate containers even if their configuration and image haven't changed") - flags.BoolVar(&opts.noRecreate, "no-recreate", false, "If containers already exist, don't recreate them. Incompatible with --force-recreate.") - flags.BoolVar(&opts.removeOrphans, "remove-orphans", false, "Remove containers for services not defined in the Compose file") - flags.StringArrayVar(&opts.scale, "scale", []string{}, "Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present.") - flags.BoolVarP(&opts.AssumeYes, "yes", "y", false, `Assume "yes" as answer to all prompts and run non-interactively`) - flags.SetNormalizeFunc(func(f *pflag.FlagSet, name string) pflag.NormalizedName { - // assumeYes was introduced by mistake as `--y` - if name == "y" { - logrus.Warn("--y is deprecated, please use --yes instead") - name = "yes" - } - return pflag.NormalizedName(name) - }) - return cmd -} - -func runCreate(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, createOpts createOptions, buildOpts buildOptions, project *types.Project, services []string) error { - if err := createOpts.Apply(project); err != nil { - return err - } - - var build *api.BuildOptions - if !createOpts.noBuild { - bo, err := buildOpts.toAPIBuildOptions(services) - if err != nil { - return err - } - build = &bo - } - - if createOpts.AssumeYes { - backendOptions.Options = append(backendOptions.Options, compose.WithPrompt(compose.AlwaysOkPrompt())) - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Create(ctx, project, api.CreateOptions{ - Build: build, - Services: services, - RemoveOrphans: createOpts.removeOrphans, - IgnoreOrphans: createOpts.ignoreOrphans, - Recreate: createOpts.recreateStrategy(), - RecreateDependencies: createOpts.dependenciesRecreateStrategy(), - Inherit: !createOpts.noInherit, - Timeout: createOpts.GetTimeout(), - QuietPull: createOpts.quietPull, - }) -} - -func (opts createOptions) recreateStrategy() string { - if opts.noRecreate { - return api.RecreateNever - } - if opts.forceRecreate { - return api.RecreateForce - } - if opts.noInherit { - return api.RecreateForce - } - return api.RecreateDiverged -} - -func (opts createOptions) dependenciesRecreateStrategy() string { - if opts.noRecreate { - return api.RecreateNever - } - if opts.recreateDeps { - return api.RecreateForce - } - return api.RecreateDiverged -} - -func (opts createOptions) GetTimeout() *time.Duration { - if opts.timeChanged { - t := time.Duration(opts.timeout) * time.Second - return &t - } - return nil -} - -func (opts createOptions) Apply(project *types.Project) error { - if opts.pullChanged { - if !opts.isPullPolicyValid() { - return fmt.Errorf("invalid --pull option %q", opts.Pull) - } - for i, service := range project.Services { - service.PullPolicy = opts.Pull - project.Services[i] = service - } - } - // N.B. opts.Build means "force build all", but images can still be built - // when this is false - // e.g. if a service has pull_policy: build or its local image is policy - if opts.Build { - for i, service := range project.Services { - if service.Build == nil { - continue - } - service.PullPolicy = types.PullPolicyBuild - project.Services[i] = service - } - } - - if err := applyPlatforms(project, true); err != nil { - return err - } - - err := applyScaleOpts(project, opts.scale) - if err != nil { - return err - } - return nil -} - -func applyScaleOpts(project *types.Project, opts []string) error { - for _, scale := range opts { - name, val, ok := strings.Cut(scale, "=") - if !ok || val == "" { - return fmt.Errorf("invalid --scale option %q. Should be SERVICE=NUM", scale) - } - replicas, err := strconv.Atoi(val) - if err != nil { - return err - } - err = setServiceScale(project, name, replicas) - if err != nil { - return err - } - } - return nil -} - -func (opts createOptions) isPullPolicyValid() bool { - pullPolicies := []string{ - types.PullPolicyAlways, types.PullPolicyNever, types.PullPolicyBuild, - types.PullPolicyMissing, types.PullPolicyIfNotPresent, - } - return slices.Contains(pullPolicies, opts.Pull) -} diff --git a/cmd/compose/down.go b/cmd/compose/down.go deleted file mode 100644 index d74c8175292..00000000000 --- a/cmd/compose/down.go +++ /dev/null @@ -1,104 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - "time" - - "github.com/docker/cli/cli/command" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" - "github.com/docker/compose/v5/pkg/utils" -) - -type downOptions struct { - *ProjectOptions - removeOrphans bool - timeChanged bool - timeout int - volumes bool - images string -} - -func downCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := downOptions{ - ProjectOptions: p, - } - downCmd := &cobra.Command{ - Use: "down [OPTIONS] [SERVICES]", - Short: "Stop and remove containers, networks", - PreRunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - opts.timeChanged = cmd.Flags().Changed("timeout") - if opts.images != "" { - if opts.images != "all" && opts.images != "local" { - return fmt.Errorf("invalid value for --rmi: %q", opts.images) - } - } - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runDown(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := downCmd.Flags() - removeOrphans := utils.StringToBool(os.Getenv(ComposeRemoveOrphans)) - flags.BoolVar(&opts.removeOrphans, "remove-orphans", removeOrphans, "Remove containers for services not defined in the Compose file") - flags.IntVarP(&opts.timeout, "timeout", "t", 0, "Specify a shutdown timeout in seconds") - flags.BoolVarP(&opts.volumes, "volumes", "v", false, `Remove named volumes declared in the "volumes" section of the Compose file and anonymous volumes attached to containers`) - flags.StringVar(&opts.images, "rmi", "", `Remove images used by services. "local" remove only images that don't have a custom tag ("local"|"all")`) - flags.SetNormalizeFunc(func(f *pflag.FlagSet, name string) pflag.NormalizedName { - if name == "volume" { - name = "volumes" - logrus.Warn("--volume is deprecated, please use --volumes") - } - return pflag.NormalizedName(name) - }) - return downCmd -} - -func runDown(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts downOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - var timeout *time.Duration - if opts.timeChanged { - timeoutValue := time.Duration(opts.timeout) * time.Second - timeout = &timeoutValue - } - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Down(ctx, name, api.DownOptions{ - RemoveOrphans: opts.removeOrphans, - Project: project, - Timeout: timeout, - Images: opts.images, - Volumes: opts.volumes, - Services: services, - }) -} diff --git a/cmd/compose/events.go b/cmd/compose/events.go deleted file mode 100644 index 7f8a4a77df4..00000000000 --- a/cmd/compose/events.go +++ /dev/null @@ -1,93 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "encoding/json" - "fmt" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type eventsOpts struct { - *composeOptions - json bool - since string - until string -} - -func eventsCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := eventsOpts{ - composeOptions: &composeOptions{ - ProjectOptions: p, - }, - } - cmd := &cobra.Command{ - Use: "events [OPTIONS] [SERVICE...]", - Short: "Receive real time events from containers", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runEvents(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - cmd.Flags().BoolVar(&opts.json, "json", false, "Output events as a stream of json objects") - cmd.Flags().StringVar(&opts.since, "since", "", "Show all events created since timestamp") - cmd.Flags().StringVar(&opts.until, "until", "", "Stream events until this timestamp") - return cmd -} - -func runEvents(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts eventsOpts, services []string) error { - name, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Events(ctx, name, api.EventsOptions{ - Services: services, - Since: opts.since, - Until: opts.until, - Consumer: func(event api.Event) error { - if opts.json { - marshal, err := json.Marshal(map[string]any{ - "time": event.Timestamp, - "type": "container", - "service": event.Service, - "id": event.Container, - "action": event.Status, - "attributes": event.Attributes, - }) - if err != nil { - return err - } - _, _ = fmt.Fprintln(dockerCli.Out(), string(marshal)) - } else { - _, _ = fmt.Fprintln(dockerCli.Out(), event) - } - return nil - }, - }) -} diff --git a/cmd/compose/exec.go b/cmd/compose/exec.go deleted file mode 100644 index f548730dc06..00000000000 --- a/cmd/compose/exec.go +++ /dev/null @@ -1,143 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "os" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type execOpts struct { - *composeOptions - - service string - command []string - environment []string - workingDir string - - noTty bool - user string - detach bool - index int - privileged bool - interactive bool -} - -func execCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := execOpts{ - composeOptions: &composeOptions{ - ProjectOptions: p, - }, - } - runCmd := &cobra.Command{ - Use: "exec [OPTIONS] SERVICE COMMAND [ARGS...]", - Short: "Execute a command in a running container", - Args: cobra.MinimumNArgs(2), - PreRunE: Adapt(func(ctx context.Context, args []string) error { - opts.service = args[0] - opts.command = args[1:] - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - err := runExec(ctx, dockerCli, backendOptions, opts) - if err != nil { - logrus.Debugf("%v", err) - var cliError cli.StatusError - if ok := errors.As(err, &cliError); ok { - os.Exit(err.(cli.StatusError).StatusCode) //nolint: errorlint - } - } - return err - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - runCmd.Flags().BoolVarP(&opts.detach, "detach", "d", false, "Detached mode: Run command in the background") - runCmd.Flags().StringArrayVarP(&opts.environment, "env", "e", []string{}, "Set environment variables") - runCmd.Flags().IntVar(&opts.index, "index", 0, "Index of the container if service has multiple replicas") - runCmd.Flags().BoolVarP(&opts.privileged, "privileged", "", false, "Give extended privileges to the process") - runCmd.Flags().StringVarP(&opts.user, "user", "u", "", "Run the command as this user") - runCmd.Flags().BoolVarP(&opts.noTty, "no-tty", "T", !dockerCli.Out().IsTerminal(), "Disable pseudo-TTY allocation. By default 'docker compose exec' allocates a TTY.") - runCmd.Flags().StringVarP(&opts.workingDir, "workdir", "w", "", "Path to workdir directory for this command") - - runCmd.Flags().BoolVarP(&opts.interactive, "interactive", "i", true, "Keep STDIN open even if not attached") - runCmd.Flags().MarkHidden("interactive") //nolint:errcheck - runCmd.Flags().BoolP("tty", "t", true, "Allocate a pseudo-TTY") - runCmd.Flags().MarkHidden("tty") //nolint:errcheck - - runCmd.Flags().SetInterspersed(false) - runCmd.Flags().SetNormalizeFunc(func(f *pflag.FlagSet, name string) pflag.NormalizedName { - if name == "no-TTY" { // legacy - name = "no-tty" - } - return pflag.NormalizedName(name) - }) - return runCmd -} - -func runExec(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts execOpts) error { - projectName, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - projectOptions, err := opts.composeOptions.toProjectOptions() //nolint:staticcheck - if err != nil { - return err - } - lookupFn := func(k string) (string, bool) { - v, ok := projectOptions.Environment[k] - return v, ok - } - execOpts := api.RunOptions{ - Service: opts.service, - Command: opts.command, - Environment: compose.ToMobyEnv(types.NewMappingWithEquals(opts.environment).Resolve(lookupFn)), - Tty: !opts.noTty, - User: opts.user, - Privileged: opts.privileged, - Index: opts.index, - Detach: opts.detach, - WorkingDir: opts.workingDir, - Interactive: opts.interactive, - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - exitCode, err := backend.Exec(ctx, projectName, execOpts) - if exitCode != 0 { - errMsg := fmt.Sprintf("exit status %d", exitCode) - if err != nil && err.Error() != "" { - errMsg = err.Error() - } - return cli.StatusError{StatusCode: exitCode, Status: errMsg} - } - return err -} diff --git a/cmd/compose/export.go b/cmd/compose/export.go deleted file mode 100644 index 4c7eaf7ef5d..00000000000 --- a/cmd/compose/export.go +++ /dev/null @@ -1,79 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type exportOptions struct { - *ProjectOptions - - service string - output string - index int -} - -func exportCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - options := exportOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "export [OPTIONS] SERVICE", - Short: "Export a service container's filesystem as a tar archive", - Args: cobra.MinimumNArgs(1), - PreRunE: Adapt(func(ctx context.Context, args []string) error { - options.service = args[0] - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runExport(ctx, dockerCli, backendOptions, options) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - flags := cmd.Flags() - flags.IntVar(&options.index, "index", 0, "index of the container if service has multiple replicas.") - flags.StringVarP(&options.output, "output", "o", "", "Write to a file, instead of STDOUT") - - return cmd -} - -func runExport(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, options exportOptions) error { - projectName, err := options.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - exportOptions := api.ExportOptions{ - Service: options.service, - Index: options.index, - Output: options.output, - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Export(ctx, projectName, exportOptions) -} diff --git a/cmd/compose/generate.go b/cmd/compose/generate.go deleted file mode 100644 index 3fe5b2389e5..00000000000 --- a/cmd/compose/generate.go +++ /dev/null @@ -1,91 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type generateOptions struct { - *ProjectOptions - Format string -} - -func generateCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := generateOptions{ - ProjectOptions: p, - } - - cmd := &cobra.Command{ - Use: "generate [OPTIONS] [CONTAINERS...]", - Short: "EXPERIMENTAL - Generate a Compose file from existing containers", - PreRunE: Adapt(func(ctx context.Context, args []string) error { - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runGenerate(ctx, dockerCli, backendOptions, opts, args) - }), - } - - cmd.Flags().StringVar(&opts.ProjectName, "name", "", "Project name to set in the Compose file") - cmd.Flags().StringVar(&opts.ProjectDir, "project-dir", "", "Directory to use for the project") - cmd.Flags().StringVar(&opts.Format, "format", "yaml", "Format the output. Values: [yaml | json]") - return cmd -} - -func runGenerate(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts generateOptions, containers []string) error { - _, _ = fmt.Fprintln(os.Stderr, "generate command is EXPERIMENTAL") - if len(containers) == 0 { - return fmt.Errorf("at least one container must be specified") - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - project, err := backend.Generate(ctx, api.GenerateOptions{ - Containers: containers, - ProjectName: opts.ProjectName, - }) - if err != nil { - return err - } - - var content []byte - switch opts.Format { - case "json": - content, err = project.MarshalJSON() - case "yaml": - content, err = project.MarshalYAML() - default: - return fmt.Errorf("unsupported format %q", opts.Format) - } - if err != nil { - return err - } - fmt.Println(string(content)) - - return nil -} diff --git a/cmd/compose/images.go b/cmd/compose/images.go deleted file mode 100644 index dbb7a56471d..00000000000 --- a/cmd/compose/images.go +++ /dev/null @@ -1,153 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - "maps" - "slices" - "strings" - "time" - - "github.com/containerd/platforms" - "github.com/docker/cli/cli/command" - "github.com/docker/docker/pkg/stringid" - "github.com/docker/go-units" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type imageOptions struct { - *ProjectOptions - Quiet bool - Format string -} - -func imagesCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := imageOptions{ - ProjectOptions: p, - } - imgCmd := &cobra.Command{ - Use: "images [OPTIONS] [SERVICE...]", - Short: "List images used by the created containers", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runImages(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - imgCmd.Flags().StringVar(&opts.Format, "format", "table", "Format the output. Values: [table | json]") - imgCmd.Flags().BoolVarP(&opts.Quiet, "quiet", "q", false, "Only display IDs") - return imgCmd -} - -func runImages(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts imageOptions, services []string) error { - projectName, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - images, err := backend.Images(ctx, projectName, api.ImagesOptions{ - Services: services, - }) - if err != nil { - return err - } - - if opts.Quiet { - ids := []string{} - for _, img := range images { - id := img.ID - if i := strings.IndexRune(img.ID, ':'); i >= 0 { - id = id[i+1:] - } - if !slices.Contains(ids, id) { - ids = append(ids, id) - } - } - for _, img := range ids { - _, _ = fmt.Fprintln(dockerCli.Out(), img) - } - return nil - } - if opts.Format == "json" { - - type img struct { - ID string `json:"ID"` - ContainerName string `json:"ContainerName"` - Repository string `json:"Repository"` - Tag string `json:"Tag"` - Platform string `json:"Platform"` - Size int64 `json:"Size"` - Created *time.Time `json:"Created,omitempty"` - LastTagTime time.Time `json:"LastTagTime,omitzero"` - } - // Convert map to slice - var imageList []img - for ctr, i := range images { - lastTagTime := i.LastTagTime - imageList = append(imageList, img{ - ContainerName: ctr, - ID: i.ID, - Repository: i.Repository, - Tag: i.Tag, - Platform: platforms.Format(i.Platform), - Size: i.Size, - Created: i.Created, - LastTagTime: lastTagTime, - }) - } - json, err := formatter.ToJSON(imageList, "", "") - if err != nil { - return err - } - _, err = fmt.Fprintln(dockerCli.Out(), json) - return err - } - - return formatter.Print(images, opts.Format, dockerCli.Out(), - func(w io.Writer) { - for _, container := range slices.Sorted(maps.Keys(images)) { - img := images[container] - id := stringid.TruncateID(img.ID) - size := units.HumanSizeWithPrecision(float64(img.Size), 3) - repo := img.Repository - if repo == "" { - repo = "<none>" - } - tag := img.Tag - if tag == "" { - tag = "<none>" - } - created := "N/A" - if img.Created != nil { - created = units.HumanDuration(time.Now().UTC().Sub(*img.Created)) + " ago" - } - _, _ = fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\t%s\n", - container, repo, tag, platforms.Format(img.Platform), id, size, created) - } - }, - "CONTAINER", "REPOSITORY", "TAG", "PLATFORM", "IMAGE ID", "SIZE", "CREATED") -} diff --git a/cmd/compose/kill.go b/cmd/compose/kill.go deleted file mode 100644 index ee488d2ec6f..00000000000 --- a/cmd/compose/kill.go +++ /dev/null @@ -1,81 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "os" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" - "github.com/docker/compose/v5/pkg/utils" -) - -type killOptions struct { - *ProjectOptions - removeOrphans bool - signal string -} - -func killCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := killOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "kill [OPTIONS] [SERVICE...]", - Short: "Force stop service containers", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runKill(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - flags := cmd.Flags() - removeOrphans := utils.StringToBool(os.Getenv(ComposeRemoveOrphans)) - flags.BoolVar(&opts.removeOrphans, "remove-orphans", removeOrphans, "Remove containers for services not defined in the Compose file") - flags.StringVarP(&opts.signal, "signal", "s", "SIGKILL", "SIGNAL to send to the container") - - return cmd -} - -func runKill(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts killOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - err = backend.Kill(ctx, name, api.KillOptions{ - RemoveOrphans: opts.removeOrphans, - Project: project, - Services: services, - Signal: opts.signal, - }) - if errors.Is(err, api.ErrNoResources) { - _, _ = fmt.Fprintln(stdinfo(dockerCli), "No container to kill") - return nil - } - return err -} diff --git a/cmd/compose/list.go b/cmd/compose/list.go deleted file mode 100644 index eaa843ac372..00000000000 --- a/cmd/compose/list.go +++ /dev/null @@ -1,122 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/opts" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type lsOptions struct { - Format string - Quiet bool - All bool - Filter opts.FilterOpt -} - -func listCommand(dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - lsOpts := lsOptions{Filter: opts.NewFilterOpt()} - lsCmd := &cobra.Command{ - Use: "ls [OPTIONS]", - Short: "List running compose projects", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runList(ctx, dockerCli, backendOptions, lsOpts) - }), - Args: cobra.NoArgs, - ValidArgsFunction: noCompletion(), - } - lsCmd.Flags().StringVar(&lsOpts.Format, "format", "table", "Format the output. Values: [table | json]") - lsCmd.Flags().BoolVarP(&lsOpts.Quiet, "quiet", "q", false, "Only display project names") - lsCmd.Flags().Var(&lsOpts.Filter, "filter", "Filter output based on conditions provided") - lsCmd.Flags().BoolVarP(&lsOpts.All, "all", "a", false, "Show all stopped Compose projects") - - return lsCmd -} - -var acceptedListFilters = map[string]bool{ - "name": true, -} - -func runList(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, lsOpts lsOptions) error { - filters := lsOpts.Filter.Value() - err := filters.Validate(acceptedListFilters) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - stackList, err := backend.List(ctx, api.ListOptions{All: lsOpts.All}) - if err != nil { - return err - } - - if filters.Len() > 0 { - var filtered []api.Stack - for _, s := range stackList { - if filters.Contains("name") && !filters.Match("name", s.Name) { - continue - } - filtered = append(filtered, s) - } - stackList = filtered - } - - if lsOpts.Quiet { - for _, s := range stackList { - _, _ = fmt.Fprintln(dockerCli.Out(), s.Name) - } - return nil - } - - view := viewFromStackList(stackList) - return formatter.Print(view, lsOpts.Format, dockerCli.Out(), func(w io.Writer) { - for _, stack := range view { - _, _ = fmt.Fprintf(w, "%s\t%s\t%s\n", stack.Name, stack.Status, stack.ConfigFiles) - } - }, "NAME", "STATUS", "CONFIG FILES") -} - -type stackView struct { - Name string - Status string - ConfigFiles string -} - -func viewFromStackList(stackList []api.Stack) []stackView { - retList := make([]stackView, len(stackList)) - for i, s := range stackList { - retList[i] = stackView{ - Name: s.Name, - Status: strings.TrimSpace(fmt.Sprintf("%s %s", s.Status, s.Reason)), - ConfigFiles: s.ConfigFiles, - } - } - return retList -} diff --git a/cmd/compose/logs.go b/cmd/compose/logs.go deleted file mode 100644 index de1452220b3..00000000000 --- a/cmd/compose/logs.go +++ /dev/null @@ -1,133 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type logsOptions struct { - *ProjectOptions - composeOptions - follow bool - index int - tail string - since string - until string - noColor bool - noPrefix bool - timestamps bool -} - -func logsCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := logsOptions{ - ProjectOptions: p, - } - logsCmd := &cobra.Command{ - Use: "logs [OPTIONS] [SERVICE...]", - Short: "View output from containers", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runLogs(ctx, dockerCli, backendOptions, opts, args) - }), - PreRunE: func(cmd *cobra.Command, args []string) error { - if opts.index > 0 && len(args) != 1 { - return errors.New("--index requires one service to be selected") - } - return nil - }, - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := logsCmd.Flags() - flags.BoolVarP(&opts.follow, "follow", "f", false, "Follow log output") - flags.IntVar(&opts.index, "index", 0, "index of the container if service has multiple replicas") - flags.StringVar(&opts.since, "since", "", "Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)") - flags.StringVar(&opts.until, "until", "", "Show logs before a timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)") - flags.BoolVar(&opts.noColor, "no-color", false, "Produce monochrome output") - flags.BoolVar(&opts.noPrefix, "no-log-prefix", false, "Don't print prefix in logs") - flags.BoolVarP(&opts.timestamps, "timestamps", "t", false, "Show timestamps") - flags.StringVarP(&opts.tail, "tail", "n", "all", "Number of lines to show from the end of the logs for each container") - return logsCmd -} - -func runLogs(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts logsOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - // exclude services configured to ignore output (attach: false), until explicitly selected - if project != nil && len(services) == 0 { - for n, service := range project.Services { - if service.Attach == nil || *service.Attach { - services = append(services, n) - } - } - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - consumer := formatter.NewLogConsumer(ctx, dockerCli.Out(), dockerCli.Err(), !opts.noColor, !opts.noPrefix, false) - return backend.Logs(ctx, name, consumer, api.LogOptions{ - Project: project, - Services: services, - Follow: opts.follow, - Index: opts.index, - Tail: opts.tail, - Since: opts.since, - Until: opts.until, - Timestamps: opts.timestamps, - }) -} - -var _ api.LogConsumer = &logConsumer{} - -type logConsumer struct { - events api.EventProcessor -} - -func (l logConsumer) Log(containerName, message string) { - l.events.On(api.Resource{ - ID: containerName, - Text: message, - }) -} - -func (l logConsumer) Err(containerName, message string) { - l.events.On(api.Resource{ - ID: containerName, - Status: api.Error, - Text: message, - }) -} - -func (l logConsumer) Status(containerName, message string) { - l.events.On(api.Resource{ - ID: containerName, - Status: api.Error, - Text: message, - }) -} diff --git a/cmd/compose/options.go b/cmd/compose/options.go deleted file mode 100644 index 6454f6aed9b..00000000000 --- a/cmd/compose/options.go +++ /dev/null @@ -1,292 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - "os" - "slices" - "sort" - "strings" - "text/tabwriter" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/template" - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - - "github.com/docker/compose/v5/cmd/display" - "github.com/docker/compose/v5/cmd/prompt" - "github.com/docker/compose/v5/internal/tracing" -) - -func applyPlatforms(project *types.Project, buildForSinglePlatform bool) error { - defaultPlatform := project.Environment["DOCKER_DEFAULT_PLATFORM"] - for name, service := range project.Services { - if service.Build == nil { - continue - } - - // default platform only applies if the service doesn't specify - if defaultPlatform != "" && service.Platform == "" { - if len(service.Build.Platforms) > 0 && !slices.Contains(service.Build.Platforms, defaultPlatform) { - return fmt.Errorf("service %q build.platforms does not support value set by DOCKER_DEFAULT_PLATFORM: %s", name, defaultPlatform) - } - service.Platform = defaultPlatform - } - - if service.Platform != "" { - if len(service.Build.Platforms) > 0 { - if !slices.Contains(service.Build.Platforms, service.Platform) { - return fmt.Errorf("service %q build configuration does not support platform: %s", name, service.Platform) - } - } - - if buildForSinglePlatform || len(service.Build.Platforms) == 0 { - // if we're building for a single platform, we want to build for the platform we'll use to run the image - // similarly, if no build platforms were explicitly specified, it makes sense to build for the platform - // the image is designed for rather than allowing the builder to infer the platform - service.Build.Platforms = []string{service.Platform} - } - } - - // services can specify that they should be built for multiple platforms, which can be used - // with `docker compose build` to produce a multi-arch image - // other cases, such as `up` and `run`, need a single architecture to actually run - // if there is only a single platform present (which might have been inferred - // from service.Platform above), it will be used, even if it requires emulation. - // if there's more than one platform, then the list is cleared so that the builder - // can decide. - // TODO(milas): there's no validation that the platform the builder will pick is actually one - // of the supported platforms from the build definition - // e.g. `build.platforms: [linux/arm64, linux/amd64]` on a `linux/ppc64` machine would build - // for `linux/ppc64` instead of returning an error that it's not a valid platform for the service. - if buildForSinglePlatform && len(service.Build.Platforms) > 1 { - // empty indicates that the builder gets to decide - service.Build.Platforms = nil - } - project.Services[name] = service - } - return nil -} - -// isRemoteConfig checks if the main compose file is from a remote source (OCI or Git) -func isRemoteConfig(dockerCli command.Cli, options buildOptions) bool { - if len(options.ConfigPaths) == 0 { - return false - } - remoteLoaders := options.remoteLoaders(dockerCli) - for _, loader := range remoteLoaders { - if loader.Accept(options.ConfigPaths[0]) { - return true - } - } - return false -} - -// checksForRemoteStack handles environment variable prompts for remote configurations -func checksForRemoteStack(ctx context.Context, dockerCli command.Cli, project *types.Project, options buildOptions, assumeYes bool, cmdEnvs []string) error { - if !isRemoteConfig(dockerCli, options) { - return nil - } - if metrics, ok := ctx.Value(tracing.MetricsKey{}).(tracing.Metrics); ok && metrics.CountIncludesRemote > 0 { - if err := confirmRemoteIncludes(dockerCli, options, assumeYes); err != nil { - return err - } - } - displayLocationRemoteStack(dockerCli, project, options) - return promptForInterpolatedVariables(ctx, dockerCli, options.ProjectOptions, assumeYes, cmdEnvs) -} - -// Prepare the values map and collect all variables info -type varInfo struct { - name string - value string - source string - required bool - defaultValue string -} - -// promptForInterpolatedVariables displays all variables and their values at once, -// then prompts for confirmation -func promptForInterpolatedVariables(ctx context.Context, dockerCli command.Cli, projectOptions *ProjectOptions, assumeYes bool, cmdEnvs []string) error { - if assumeYes { - return nil - } - - varsInfo, noVariables, err := extractInterpolationVariablesFromModel(ctx, dockerCli, projectOptions, cmdEnvs) - if err != nil { - return err - } - - if noVariables { - return nil - } - - displayInterpolationVariables(dockerCli.Out(), varsInfo) - - // Prompt for confirmation - userInput := prompt.NewPrompt(dockerCli.In(), dockerCli.Out()) - msg := "\nDo you want to proceed with these variables? [Y/n]: " - confirmed, err := userInput.Confirm(msg, true) - if err != nil { - return err - } - - if !confirmed { - return fmt.Errorf("operation cancelled by user") - } - - return nil -} - -func extractInterpolationVariablesFromModel(ctx context.Context, dockerCli command.Cli, projectOptions *ProjectOptions, cmdEnvs []string) ([]varInfo, bool, error) { - cmdEnvMap := extractEnvCLIDefined(cmdEnvs) - - // Create a model without interpolation to extract variables - opts := configOptions{ - noInterpolate: true, - ProjectOptions: projectOptions, - } - - model, err := opts.ToModel(ctx, dockerCli, nil, cli.WithoutEnvironmentResolution) - if err != nil { - return nil, false, err - } - - // Extract variables that need interpolation - variables := template.ExtractVariables(model, template.DefaultPattern) - if len(variables) == 0 { - return nil, true, nil - } - - var varsInfo []varInfo - proposedValues := make(map[string]string) - - for name, variable := range variables { - info := varInfo{ - name: name, - required: variable.Required, - defaultValue: variable.DefaultValue, - } - - // Determine value and source based on priority - if value, exists := cmdEnvMap[name]; exists { - info.value = value - info.source = "command-line" - proposedValues[name] = value - } else if value, exists := os.LookupEnv(name); exists { - info.value = value - info.source = "environment" - proposedValues[name] = value - } else if variable.DefaultValue != "" { - info.value = variable.DefaultValue - info.source = "compose file" - proposedValues[name] = variable.DefaultValue - } else { - info.value = "<unset>" - info.source = "none" - } - - varsInfo = append(varsInfo, info) - } - return varsInfo, false, nil -} - -func extractEnvCLIDefined(cmdEnvs []string) map[string]string { - // Parse command-line environment variables - cmdEnvMap := make(map[string]string) - for _, env := range cmdEnvs { - key, val, ok := strings.Cut(env, "=") - if ok { - cmdEnvMap[key] = val - } - } - return cmdEnvMap -} - -func displayInterpolationVariables(writer io.Writer, varsInfo []varInfo) { - // Display all variables in a table format - _, _ = fmt.Fprintln(writer, "\nFound the following variables in configuration:") - - w := tabwriter.NewWriter(writer, 0, 0, 3, ' ', 0) - _, _ = fmt.Fprintln(w, "VARIABLE\tVALUE\tSOURCE\tREQUIRED\tDEFAULT") - sort.Slice(varsInfo, func(a, b int) bool { - return varsInfo[a].name < varsInfo[b].name - }) - for _, info := range varsInfo { - required := "no" - if info.required { - required = "yes" - } - _, _ = fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", - info.name, - info.value, - info.source, - required, - info.defaultValue, - ) - } - _ = w.Flush() -} - -func displayLocationRemoteStack(dockerCli command.Cli, project *types.Project, options buildOptions) { - mainComposeFile := options.ProjectOptions.ConfigPaths[0] //nolint:staticcheck - if display.Mode != display.ModeQuiet && display.Mode != display.ModeJSON { - _, _ = fmt.Fprintf(dockerCli.Out(), "Your compose stack %q is stored in %q\n", mainComposeFile, project.WorkingDir) - } -} - -func confirmRemoteIncludes(dockerCli command.Cli, options buildOptions, assumeYes bool) error { - if assumeYes { - return nil - } - - var remoteIncludes []string - remoteLoaders := options.ProjectOptions.remoteLoaders(dockerCli) //nolint:staticcheck - for _, cf := range options.ProjectOptions.ConfigPaths { //nolint:staticcheck - for _, loader := range remoteLoaders { - if loader.Accept(cf) { - remoteIncludes = append(remoteIncludes, cf) - break - } - } - } - - if len(remoteIncludes) == 0 { - return nil - } - - _, _ = fmt.Fprintln(dockerCli.Out(), "\nWarning: This Compose project includes files from remote sources:") - for _, include := range remoteIncludes { - _, _ = fmt.Fprintf(dockerCli.Out(), " - %s\n", include) - } - _, _ = fmt.Fprintln(dockerCli.Out(), "\nRemote includes could potentially be malicious. Make sure you trust the source.") - - msg := "Do you want to continue? [y/N]: " - confirmed, err := prompt.NewPrompt(dockerCli.In(), dockerCli.Out()).Confirm(msg, false) - if err != nil { - return err - } - if !confirmed { - return fmt.Errorf("operation cancelled by user") - } - - return nil -} diff --git a/cmd/compose/options_test.go b/cmd/compose/options_test.go deleted file mode 100644 index 50f82752130..00000000000 --- a/cmd/compose/options_test.go +++ /dev/null @@ -1,385 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "fmt" - "io" - "os" - "path/filepath" - "strings" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/streams" - "github.com/stretchr/testify/require" - "go.uber.org/mock/gomock" - - "github.com/docker/compose/v5/pkg/mocks" -) - -func TestApplyPlatforms_InferFromRuntime(t *testing.T) { - makeProject := func() *types.Project { - return &types.Project{ - Services: types.Services{ - "test": { - Name: "test", - Image: "foo", - Build: &types.BuildConfig{ - Context: ".", - Platforms: []string{ - "linux/amd64", - "linux/arm64", - "alice/32", - }, - }, - Platform: "alice/32", - }, - }, - } - } - - t.Run("SinglePlatform", func(t *testing.T) { - project := makeProject() - require.NoError(t, applyPlatforms(project, true)) - require.EqualValues(t, []string{"alice/32"}, project.Services["test"].Build.Platforms) - }) - - t.Run("MultiPlatform", func(t *testing.T) { - project := makeProject() - require.NoError(t, applyPlatforms(project, false)) - require.EqualValues(t, []string{"linux/amd64", "linux/arm64", "alice/32"}, - project.Services["test"].Build.Platforms) - }) -} - -func TestApplyPlatforms_DockerDefaultPlatform(t *testing.T) { - makeProject := func() *types.Project { - return &types.Project{ - Environment: map[string]string{ - "DOCKER_DEFAULT_PLATFORM": "linux/amd64", - }, - Services: types.Services{ - "test": { - Name: "test", - Image: "foo", - Build: &types.BuildConfig{ - Context: ".", - Platforms: []string{ - "linux/amd64", - "linux/arm64", - }, - }, - }, - }, - } - } - - t.Run("SinglePlatform", func(t *testing.T) { - project := makeProject() - require.NoError(t, applyPlatforms(project, true)) - require.EqualValues(t, []string{"linux/amd64"}, project.Services["test"].Build.Platforms) - }) - - t.Run("MultiPlatform", func(t *testing.T) { - project := makeProject() - require.NoError(t, applyPlatforms(project, false)) - require.EqualValues(t, []string{"linux/amd64", "linux/arm64"}, - project.Services["test"].Build.Platforms) - }) -} - -func TestApplyPlatforms_UnsupportedPlatform(t *testing.T) { - makeProject := func() *types.Project { - return &types.Project{ - Environment: map[string]string{ - "DOCKER_DEFAULT_PLATFORM": "commodore/64", - }, - Services: types.Services{ - "test": { - Name: "test", - Image: "foo", - Build: &types.BuildConfig{ - Context: ".", - Platforms: []string{ - "linux/amd64", - "linux/arm64", - }, - }, - }, - }, - } - } - - t.Run("SinglePlatform", func(t *testing.T) { - project := makeProject() - require.EqualError(t, applyPlatforms(project, true), - `service "test" build.platforms does not support value set by DOCKER_DEFAULT_PLATFORM: commodore/64`) - }) - - t.Run("MultiPlatform", func(t *testing.T) { - project := makeProject() - require.EqualError(t, applyPlatforms(project, false), - `service "test" build.platforms does not support value set by DOCKER_DEFAULT_PLATFORM: commodore/64`) - }) -} - -func TestIsRemoteConfig(t *testing.T) { - ctrl := gomock.NewController(t) - defer ctrl.Finish() - cli := mocks.NewMockCli(ctrl) - - tests := []struct { - name string - configPaths []string - want bool - }{ - { - name: "empty config paths", - configPaths: []string{}, - want: false, - }, - { - name: "local file", - configPaths: []string{"docker-compose.yaml"}, - want: false, - }, - { - name: "OCI reference", - configPaths: []string{"oci://registry.example.com/stack:latest"}, - want: true, - }, - { - name: "GIT reference", - configPaths: []string{"git://github.com/user/repo.git"}, - want: true, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - opts := buildOptions{ - ProjectOptions: &ProjectOptions{ - ConfigPaths: tt.configPaths, - }, - } - got := isRemoteConfig(cli, opts) - require.Equal(t, tt.want, got) - }) - } -} - -func TestDisplayLocationRemoteStack(t *testing.T) { - ctrl := gomock.NewController(t) - defer ctrl.Finish() - cli := mocks.NewMockCli(ctrl) - - buf := new(bytes.Buffer) - cli.EXPECT().Out().Return(streams.NewOut(buf)).AnyTimes() - - project := &types.Project{ - Name: "test-project", - WorkingDir: "/tmp/test", - } - - options := buildOptions{ - ProjectOptions: &ProjectOptions{ - ConfigPaths: []string{"oci://registry.example.com/stack:latest"}, - }, - } - - displayLocationRemoteStack(cli, project, options) - - output := buf.String() - require.Equal(t, output, fmt.Sprintf("Your compose stack %q is stored in %q\n", "oci://registry.example.com/stack:latest", "/tmp/test")) -} - -func TestDisplayInterpolationVariables(t *testing.T) { - ctrl := gomock.NewController(t) - defer ctrl.Finish() - - tmpDir := t.TempDir() - - // Create a temporary compose file - composeContent := ` -services: - app: - image: nginx - environment: - - TEST_VAR=${TEST_VAR:?required} # required with default - - API_KEY=${API_KEY:?} # required without default - - DEBUG=${DEBUG:-true} # optional with default - - UNSET_VAR # optional without default -` - composePath := filepath.Join(tmpDir, "docker-compose.yml") - require.NoError(t, os.WriteFile(composePath, []byte(composeContent), 0o644)) - - buf := new(bytes.Buffer) - cli := mocks.NewMockCli(ctrl) - cli.EXPECT().Out().Return(streams.NewOut(buf)).AnyTimes() - - // Create ProjectOptions with the temporary compose file - projectOptions := &ProjectOptions{ - ConfigPaths: []string{composePath}, - } - - // Set up the context with necessary environment variables - t.Setenv("TEST_VAR", "test-value") - t.Setenv("API_KEY", "123456") - - // Extract variables from the model - info, noVariables, err := extractInterpolationVariablesFromModel(t.Context(), cli, projectOptions, []string{}) - require.NoError(t, err) - require.False(t, noVariables) - - // Display the variables - displayInterpolationVariables(cli.Out(), info) - - // Expected output format with proper spacing - expected := "\nFound the following variables in configuration:\n" + - "VARIABLE VALUE SOURCE REQUIRED DEFAULT\n" + - "API_KEY 123456 environment yes \n" + - "DEBUG true compose file no true\n" + - "TEST_VAR test-value environment yes \n" - - // Normalize spaces and newlines for comparison - normalizeSpaces := func(s string) string { - // Replace multiple spaces with a single space - s = strings.Join(strings.Fields(strings.TrimSpace(s)), " ") - return s - } - - actualOutput := buf.String() - - // Compare normalized strings - require.Equal(t, - normalizeSpaces(expected), - normalizeSpaces(actualOutput), - "\nExpected:\n%s\nGot:\n%s", expected, actualOutput) -} - -func TestConfirmRemoteIncludes(t *testing.T) { - ctrl := gomock.NewController(t) - defer ctrl.Finish() - cli := mocks.NewMockCli(ctrl) - - tests := []struct { - name string - opts buildOptions - assumeYes bool - userInput string - wantErr bool - errMessage string - wantPrompt bool - wantOutput string - }{ - { - name: "no remote includes", - opts: buildOptions{ - ProjectOptions: &ProjectOptions{ - ConfigPaths: []string{ - "docker-compose.yaml", - "./local/path/compose.yaml", - }, - }, - }, - assumeYes: false, - wantErr: false, - wantPrompt: false, - }, - { - name: "assume yes with remote includes", - opts: buildOptions{ - ProjectOptions: &ProjectOptions{ - ConfigPaths: []string{ - "oci://registry.example.com/stack:latest", - "git://github.com/user/repo.git", - }, - }, - }, - assumeYes: true, - wantErr: false, - wantPrompt: false, - }, - { - name: "user confirms remote includes", - opts: buildOptions{ - ProjectOptions: &ProjectOptions{ - ConfigPaths: []string{ - "oci://registry.example.com/stack:latest", - "git://github.com/user/repo.git", - }, - }, - }, - assumeYes: false, - userInput: "y\n", - wantErr: false, - wantPrompt: true, - wantOutput: "\nWarning: This Compose project includes files from remote sources:\n" + - " - oci://registry.example.com/stack:latest\n" + - " - git://github.com/user/repo.git\n" + - "\nRemote includes could potentially be malicious. Make sure you trust the source.\n" + - "Do you want to continue? [y/N]: ", - }, - { - name: "user rejects remote includes", - opts: buildOptions{ - ProjectOptions: &ProjectOptions{ - ConfigPaths: []string{ - "oci://registry.example.com/stack:latest", - }, - }, - }, - assumeYes: false, - userInput: "n\n", - wantErr: true, - errMessage: "operation cancelled by user", - wantPrompt: true, - wantOutput: "\nWarning: This Compose project includes files from remote sources:\n" + - " - oci://registry.example.com/stack:latest\n" + - "\nRemote includes could potentially be malicious. Make sure you trust the source.\n" + - "Do you want to continue? [y/N]: ", - }, - } - - buf := new(bytes.Buffer) - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - cli.EXPECT().Out().Return(streams.NewOut(buf)).AnyTimes() - - if tt.wantPrompt { - inbuf := io.NopCloser(bytes.NewBufferString(tt.userInput)) - cli.EXPECT().In().Return(streams.NewIn(inbuf)).AnyTimes() - } - - err := confirmRemoteIncludes(cli, tt.opts, tt.assumeYes) - - if tt.wantErr { - require.Error(t, err) - require.Equal(t, tt.errMessage, err.Error()) - } else { - require.NoError(t, err) - } - - if tt.wantOutput != "" { - require.Equal(t, tt.wantOutput, buf.String()) - } - buf.Reset() - }) - } -} diff --git a/cmd/compose/pause.go b/cmd/compose/pause.go deleted file mode 100644 index bb4cedba2ad..00000000000 --- a/cmd/compose/pause.go +++ /dev/null @@ -1,97 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type pauseOptions struct { - *ProjectOptions -} - -func pauseCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := pauseOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "pause [SERVICE...]", - Short: "Pause services", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runPause(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - return cmd -} - -func runPause(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts pauseOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Pause(ctx, name, api.PauseOptions{ - Services: services, - Project: project, - }) -} - -type unpauseOptions struct { - *ProjectOptions -} - -func unpauseCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := unpauseOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "unpause [SERVICE...]", - Short: "Unpause services", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runUnPause(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - return cmd -} - -func runUnPause(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts unpauseOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.UnPause(ctx, name, api.PauseOptions{ - Services: services, - Project: project, - }) -} diff --git a/cmd/compose/port.go b/cmd/compose/port.go deleted file mode 100644 index 862e3b5d68d..00000000000 --- a/cmd/compose/port.go +++ /dev/null @@ -1,86 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "strconv" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type portOptions struct { - *ProjectOptions - port uint16 - protocol string - index int -} - -func portCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := portOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "port [OPTIONS] SERVICE PRIVATE_PORT", - Short: "Print the public port for a port binding", - Args: cobra.MinimumNArgs(2), - PreRunE: Adapt(func(ctx context.Context, args []string) error { - port, err := strconv.ParseUint(args[1], 10, 16) - if err != nil { - return err - } - opts.port = uint16(port) - opts.protocol = strings.ToLower(opts.protocol) - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runPort(ctx, dockerCli, backendOptions, opts, args[0]) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - cmd.Flags().StringVar(&opts.protocol, "protocol", "tcp", "tcp or udp") - cmd.Flags().IntVar(&opts.index, "index", 0, "Index of the container if service has multiple replicas") - return cmd -} - -func runPort(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts portOptions, service string) error { - projectName, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - ip, port, err := backend.Port(ctx, projectName, service, opts.port, api.PortOptions{ - Protocol: opts.protocol, - Index: opts.index, - }) - if err != nil { - return err - } - - _, _ = fmt.Fprintf(dockerCli.Out(), "%s:%d\n", ip, port) - return nil -} diff --git a/cmd/compose/ps.go b/cmd/compose/ps.go deleted file mode 100644 index 878bb3186e1..00000000000 --- a/cmd/compose/ps.go +++ /dev/null @@ -1,184 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "slices" - "sort" - "strings" - - "github.com/docker/cli/cli/command" - cliformatter "github.com/docker/cli/cli/command/formatter" - cliflags "github.com/docker/cli/cli/flags" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type psOptions struct { - *ProjectOptions - Format string - All bool - Quiet bool - Services bool - Filter string - Status []string - noTrunc bool - Orphans bool -} - -func (p *psOptions) parseFilter() error { - if p.Filter == "" { - return nil - } - key, val, ok := strings.Cut(p.Filter, "=") - if !ok { - return errors.New("arguments to --filter should be in form KEY=VAL") - } - switch key { - case "status": - p.Status = append(p.Status, val) - return nil - case "source": - return api.ErrNotImplemented - default: - return fmt.Errorf("unknown filter %s", key) - } -} - -func psCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := psOptions{ - ProjectOptions: p, - } - psCmd := &cobra.Command{ - Use: "ps [OPTIONS] [SERVICE...]", - Short: "List containers", - PreRunE: func(cmd *cobra.Command, args []string) error { - return opts.parseFilter() - }, - RunE: Adapt(func(ctx context.Context, args []string) error { - return runPs(ctx, dockerCli, backendOptions, args, opts) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := psCmd.Flags() - flags.StringVar(&opts.Format, "format", "table", cliflags.FormatHelp) - flags.StringVar(&opts.Filter, "filter", "", "Filter services by a property (supported filters: status)") - flags.StringArrayVar(&opts.Status, "status", []string{}, "Filter services by status. Values: [paused | restarting | removing | running | dead | created | exited]") - flags.BoolVarP(&opts.Quiet, "quiet", "q", false, "Only display IDs") - flags.BoolVar(&opts.Services, "services", false, "Display services") - flags.BoolVar(&opts.Orphans, "orphans", true, "Include orphaned services (not declared by project)") - flags.BoolVarP(&opts.All, "all", "a", false, "Show all stopped containers (including those created by the run command)") - flags.BoolVar(&opts.noTrunc, "no-trunc", false, "Don't truncate output") - return psCmd -} - -func runPs(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, services []string, opts psOptions) error { //nolint:gocyclo - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - if project != nil { - names := project.ServiceNames() - if len(services) > 0 { - for _, service := range services { - if !slices.Contains(names, service) { - return fmt.Errorf("no such service: %s", service) - } - } - } else if !opts.Orphans { - // until user asks to list orphaned services, we only include those declared in project - services = names - } - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - containers, err := backend.Ps(ctx, name, api.PsOptions{ - Project: project, - All: opts.All || len(opts.Status) != 0, - Services: services, - }) - if err != nil { - return err - } - - if len(opts.Status) != 0 { - containers = filterByStatus(containers, opts.Status) - } - - sort.Slice(containers, func(i, j int) bool { - return containers[i].Name < containers[j].Name - }) - - if opts.Quiet { - for _, c := range containers { - _, _ = fmt.Fprintln(dockerCli.Out(), c.ID) - } - return nil - } - - if opts.Services { - services := []string{} - for _, c := range containers { - s := c.Service - if !slices.Contains(services, s) { - services = append(services, s) - } - } - _, _ = fmt.Fprintln(dockerCli.Out(), strings.Join(services, "\n")) - return nil - } - - if opts.Format == "" { - opts.Format = dockerCli.ConfigFile().PsFormat - } - - containerCtx := cliformatter.Context{ - Output: dockerCli.Out(), - Format: formatter.NewContainerFormat(opts.Format, opts.Quiet, false), - Trunc: !opts.noTrunc, - } - return formatter.ContainerWrite(containerCtx, containers) -} - -func filterByStatus(containers []api.ContainerSummary, statuses []string) []api.ContainerSummary { - var filtered []api.ContainerSummary - for _, c := range containers { - if hasStatus(c, statuses) { - filtered = append(filtered, c) - } - } - return filtered -} - -func hasStatus(c api.ContainerSummary, statuses []string) bool { - for _, status := range statuses { - if c.State == status { - return true - } - } - return false -} diff --git a/cmd/compose/publish.go b/cmd/compose/publish.go deleted file mode 100644 index 6dba282ebc4..00000000000 --- a/cmd/compose/publish.go +++ /dev/null @@ -1,102 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type publishOptions struct { - *ProjectOptions - resolveImageDigests bool - ociVersion string - withEnvironment bool - assumeYes bool - app bool - insecureRegistry bool -} - -func publishCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := publishOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "publish [OPTIONS] REPOSITORY[:TAG]", - Short: "Publish compose application", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runPublish(ctx, dockerCli, backendOptions, opts, args[0]) - }), - Args: cli.ExactArgs(1), - } - flags := cmd.Flags() - flags.BoolVar(&opts.resolveImageDigests, "resolve-image-digests", false, "Pin image tags to digests") - flags.StringVar(&opts.ociVersion, "oci-version", "", "OCI image/artifact specification version (automatically determined by default)") - flags.BoolVar(&opts.withEnvironment, "with-env", false, "Include environment variables in the published OCI artifact") - flags.BoolVarP(&opts.assumeYes, "yes", "y", false, `Assume "yes" as answer to all prompts`) - flags.BoolVar(&opts.app, "app", false, "Published compose application (includes referenced images)") - flags.BoolVar(&opts.insecureRegistry, "insecure-registry", false, "Use insecure registry") - flags.SetNormalizeFunc(func(f *pflag.FlagSet, name string) pflag.NormalizedName { - // assumeYes was introduced by mistake as `--y` - if name == "y" { - logrus.Warn("--y is deprecated, please use --yes instead") - name = "yes" - } - return pflag.NormalizedName(name) - }) - // Should **only** be used for testing purpose, we don't want to promote use of insecure registries - _ = flags.MarkHidden("insecure-registry") - - return cmd -} - -func runPublish(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts publishOptions, repository string) error { - if opts.assumeYes { - backendOptions.Options = append(backendOptions.Options, compose.WithPrompt(compose.AlwaysOkPrompt())) - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - project, metrics, err := opts.ToProject(ctx, dockerCli, backend, nil) - if err != nil { - return err - } - - if metrics.CountIncludesLocal > 0 { - return errors.New("cannot publish compose file with local includes") - } - - return backend.Publish(ctx, project, repository, api.PublishOptions{ - ResolveImageDigests: opts.resolveImageDigests || opts.app, - Application: opts.app, - OCIVersion: api.OCIVersion(opts.ociVersion), - WithEnvironment: opts.withEnvironment, - InsecureRegistry: opts.insecureRegistry, - }) -} diff --git a/cmd/compose/pull.go b/cmd/compose/pull.go deleted file mode 100644 index 694731155f6..00000000000 --- a/cmd/compose/pull.go +++ /dev/null @@ -1,122 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - "github.com/morikuni/aec" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type pullOptions struct { - *ProjectOptions - composeOptions - quiet bool - parallel bool - noParallel bool - includeDeps bool - ignorePullFailures bool - noBuildable bool - policy string -} - -func pullCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := pullOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "pull [OPTIONS] [SERVICE...]", - Short: "Pull service images", - PreRunE: func(cmd *cobra.Command, args []string) error { - if cmd.Flags().Changed("no-parallel") { - fmt.Fprint(os.Stderr, aec.Apply("option '--no-parallel' is DEPRECATED and will be ignored.\n", aec.RedF)) - } - if cmd.Flags().Changed("parallel") { - fmt.Fprint(os.Stderr, aec.Apply("option '--parallel' is DEPRECATED and will be ignored.\n", aec.RedF)) - } - return nil - }, - RunE: Adapt(func(ctx context.Context, args []string) error { - return runPull(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.BoolVarP(&opts.quiet, "quiet", "q", false, "Pull without printing progress information") - cmd.Flags().BoolVar(&opts.includeDeps, "include-deps", false, "Also pull services declared as dependencies") - cmd.Flags().BoolVar(&opts.parallel, "parallel", true, "DEPRECATED pull multiple images in parallel") - flags.MarkHidden("parallel") //nolint:errcheck - cmd.Flags().BoolVar(&opts.noParallel, "no-parallel", true, "DEPRECATED disable parallel pulling") - flags.MarkHidden("no-parallel") //nolint:errcheck - cmd.Flags().BoolVar(&opts.ignorePullFailures, "ignore-pull-failures", false, "Pull what it can and ignores images with pull failures") - cmd.Flags().BoolVar(&opts.noBuildable, "ignore-buildable", false, "Ignore images that can be built") - cmd.Flags().StringVar(&opts.policy, "policy", "", `Apply pull policy ("missing"|"always")`) - return cmd -} - -func (opts pullOptions) apply(project *types.Project, services []string) (*types.Project, error) { - if !opts.includeDeps { - var err error - project, err = project.WithSelectedServices(services, types.IgnoreDependencies) - if err != nil { - return nil, err - } - } - - if opts.policy != "" { - for i, service := range project.Services { - if service.Image == "" { - continue - } - service.PullPolicy = opts.policy - project.Services[i] = service - } - } - return project, nil -} - -func runPull(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts pullOptions, services []string) error { - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - project, _, err := opts.ToProject(ctx, dockerCli, backend, services, cli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - project, err = opts.apply(project, services) - if err != nil { - return err - } - - return backend.Pull(ctx, project, api.PullOptions{ - Quiet: opts.quiet, - IgnoreFailures: opts.ignorePullFailures, - IgnoreBuildable: opts.noBuildable, - }) -} diff --git a/cmd/compose/pullOptions_test.go b/cmd/compose/pullOptions_test.go deleted file mode 100644 index 05dd868edf7..00000000000 --- a/cmd/compose/pullOptions_test.go +++ /dev/null @@ -1,57 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "gotest.tools/v3/assert" -) - -func TestApplyPullOptions(t *testing.T) { - project := &types.Project{ - Services: types.Services{ - "must-build": { - Name: "must-build", - // No image, local build only - Build: &types.BuildConfig{ - Context: ".", - }, - }, - "has-build": { - Name: "has-build", - Image: "registry.example.com/myservice", - Build: &types.BuildConfig{ - Context: ".", - }, - }, - "must-pull": { - Name: "must-pull", - Image: "registry.example.com/another-service", - }, - }, - } - project, err := pullOptions{ - policy: types.PullPolicyMissing, - }.apply(project, nil) - assert.NilError(t, err) - - assert.Equal(t, project.Services["must-build"].PullPolicy, "") // still default - assert.Equal(t, project.Services["has-build"].PullPolicy, types.PullPolicyMissing) - assert.Equal(t, project.Services["must-pull"].PullPolicy, types.PullPolicyMissing) -} diff --git a/cmd/compose/push.go b/cmd/compose/push.go deleted file mode 100644 index 4dd23aedb6f..00000000000 --- a/cmd/compose/push.go +++ /dev/null @@ -1,79 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type pushOptions struct { - *ProjectOptions - composeOptions - IncludeDeps bool - Ignorefailures bool - Quiet bool -} - -func pushCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := pushOptions{ - ProjectOptions: p, - } - pushCmd := &cobra.Command{ - Use: "push [OPTIONS] [SERVICE...]", - Short: "Push service images", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runPush(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - pushCmd.Flags().BoolVar(&opts.Ignorefailures, "ignore-push-failures", false, "Push what it can and ignores images with push failures") - pushCmd.Flags().BoolVar(&opts.IncludeDeps, "include-deps", false, "Also push images of services declared as dependencies") - pushCmd.Flags().BoolVarP(&opts.Quiet, "quiet", "q", false, "Push without printing progress information") - - return pushCmd -} - -func runPush(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts pushOptions, services []string) error { - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - project, _, err := opts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return err - } - - if !opts.IncludeDeps { - project, err = project.WithSelectedServices(services, types.IgnoreDependencies) - if err != nil { - return err - } - } - - return backend.Push(ctx, project, api.PushOptions{ - IgnoreFailures: opts.Ignorefailures, - Quiet: opts.Quiet, - }) -} diff --git a/cmd/compose/remove.go b/cmd/compose/remove.go deleted file mode 100644 index d0765b20a53..00000000000 --- a/cmd/compose/remove.go +++ /dev/null @@ -1,88 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type removeOptions struct { - *ProjectOptions - force bool - stop bool - volumes bool -} - -func removeCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := removeOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "rm [OPTIONS] [SERVICE...]", - Short: "Removes stopped service containers", - Long: `Removes stopped service containers - -By default, anonymous volumes attached to containers will not be removed. You -can override this with -v. To list all volumes, use "docker volume ls". - -Any data which is not in a volume will be lost.`, - RunE: Adapt(func(ctx context.Context, args []string) error { - return runRemove(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - f := cmd.Flags() - f.BoolVarP(&opts.force, "force", "f", false, "Don't ask to confirm removal") - f.BoolVarP(&opts.stop, "stop", "s", false, "Stop the containers, if required, before removing") - f.BoolVarP(&opts.volumes, "volumes", "v", false, "Remove any anonymous volumes attached to containers") - f.BoolP("all", "a", false, "Deprecated - no effect") - f.MarkHidden("all") //nolint:errcheck - - return cmd -} - -func runRemove(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts removeOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - err = backend.Remove(ctx, name, api.RemoveOptions{ - Services: services, - Force: opts.force, - Volumes: opts.volumes, - Project: project, - Stop: opts.stop, - }) - if errors.Is(err, api.ErrNoResources) { - _, _ = fmt.Fprintln(stdinfo(dockerCli), "No stopped containers") - return nil - } - return err -} diff --git a/cmd/compose/restart.go b/cmd/compose/restart.go deleted file mode 100644 index e014b2a8efe..00000000000 --- a/cmd/compose/restart.go +++ /dev/null @@ -1,88 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "time" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type restartOptions struct { - *ProjectOptions - timeChanged bool - timeout int - noDeps bool -} - -func restartCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := restartOptions{ - ProjectOptions: p, - } - restartCmd := &cobra.Command{ - Use: "restart [OPTIONS] [SERVICE...]", - Short: "Restart service containers", - PreRun: func(cmd *cobra.Command, args []string) { - opts.timeChanged = cmd.Flags().Changed("timeout") - }, - RunE: Adapt(func(ctx context.Context, args []string) error { - return runRestart(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := restartCmd.Flags() - flags.IntVarP(&opts.timeout, "timeout", "t", 0, "Specify a shutdown timeout in seconds") - flags.BoolVar(&opts.noDeps, "no-deps", false, "Don't restart dependent services") - - return restartCmd -} - -func runRestart(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts restartOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli) - if err != nil { - return err - } - - if project != nil && len(services) > 0 { - project, err = project.WithServicesEnabled(services...) - if err != nil { - return err - } - } - - var timeout *time.Duration - if opts.timeChanged { - timeoutValue := time.Duration(opts.timeout) * time.Second - timeout = &timeoutValue - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Restart(ctx, name, api.RestartOptions{ - Timeout: timeout, - Services: services, - Project: project, - NoDeps: opts.noDeps, - }) -} diff --git a/cmd/compose/run.go b/cmd/compose/run.go deleted file mode 100644 index 573c0e14d4f..00000000000 --- a/cmd/compose/run.go +++ /dev/null @@ -1,351 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - "strings" - - composecli "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/dotenv" - "github.com/compose-spec/compose-go/v2/format" - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command" - "github.com/docker/cli/opts" - "github.com/mattn/go-shellwords" - xprogress "github.com/moby/buildkit/util/progress/progressui" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/cmd/display" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" - "github.com/docker/compose/v5/pkg/utils" -) - -type runOptions struct { - *composeOptions - Service string - Command []string - environment []string - envFiles []string - Detach bool - Remove bool - noTty bool - interactive bool - user string - workdir string - entrypoint string - entrypointCmd []string - capAdd opts.ListOpts - capDrop opts.ListOpts - labels []string - volumes []string - publish []string - useAliases bool - servicePorts bool - name string - noDeps bool - ignoreOrphans bool - removeOrphans bool - quiet bool - quietPull bool -} - -func (options runOptions) apply(project *types.Project) (*types.Project, error) { - if options.noDeps { - var err error - project, err = project.WithSelectedServices([]string{options.Service}, types.IgnoreDependencies) - if err != nil { - return nil, err - } - } - - target, err := project.GetService(options.Service) - if err != nil { - return nil, err - } - - target.Tty = !options.noTty - target.StdinOpen = options.interactive - - // --service-ports and --publish are incompatible - if !options.servicePorts { - if len(target.Ports) > 0 { - logrus.Debug("Running service without ports exposed as --service-ports=false") - } - target.Ports = []types.ServicePortConfig{} - for _, p := range options.publish { - config, err := types.ParsePortConfig(p) - if err != nil { - return nil, err - } - target.Ports = append(target.Ports, config...) - } - } - - for _, v := range options.volumes { - volume, err := format.ParseVolume(v) - if err != nil { - return nil, err - } - target.Volumes = append(target.Volumes, volume) - } - - for name := range project.Services { - if name == options.Service { - project.Services[name] = target - break - } - } - return project, nil -} - -func (options runOptions) getEnvironment(resolve func(string) (string, bool)) (types.Mapping, error) { - environment := types.NewMappingWithEquals(options.environment).Resolve(resolve).ToMapping() - for _, file := range options.envFiles { - f, err := os.Open(file) - if err != nil { - return nil, err - } - vars, err := dotenv.ParseWithLookup(f, func(k string) (string, bool) { - value, ok := environment[k] - return value, ok - }) - if err != nil { - return nil, nil - } - for k, v := range vars { - if _, ok := environment[k]; !ok { - environment[k] = v - } - } - } - return environment, nil -} - -func runCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - options := runOptions{ - composeOptions: &composeOptions{ - ProjectOptions: p, - }, - capAdd: opts.NewListOpts(nil), - capDrop: opts.NewListOpts(nil), - } - createOpts := createOptions{} - buildOpts := buildOptions{ - ProjectOptions: p, - } - // We remove the attribute from the option struct and use a dedicated var, to limit confusion and avoid anyone to use options.tty. - // The tty flag is here for convenience and let user do "docker compose run -it" the same way as they use the "docker run" command. - var ttyFlag bool - - cmd := &cobra.Command{ - Use: "run [OPTIONS] SERVICE [COMMAND] [ARGS...]", - Short: "Run a one-off command on a service", - Args: cobra.MinimumNArgs(1), - PreRunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - options.Service = args[0] - if len(args) > 1 { - options.Command = args[1:] - } - if len(options.publish) > 0 && options.servicePorts { - return fmt.Errorf("--service-ports and --publish are incompatible") - } - if cmd.Flags().Changed("entrypoint") { - command, err := shellwords.Parse(options.entrypoint) - if err != nil { - return err - } - options.entrypointCmd = command - } - if cmd.Flags().Changed("tty") { - if cmd.Flags().Changed("no-TTY") { - return fmt.Errorf("--tty and --no-TTY can't be used together") - } else { - options.noTty = !ttyFlag - } - } else if !cmd.Flags().Changed("no-TTY") && !cmd.Flags().Changed("interactive") && !dockerCli.In().IsTerminal() { - // while `docker run` requires explicit `-it` flags, Compose enables interactive mode and TTY by default - // but when compose is used from a script that has stdin piped from another command, we just can't - // Here, we detect we run "by default" (user didn't passed explicit flags) and disable TTY allocation if - // we don't have an actual terminal to attach to for interactive mode - options.noTty = true - } - - if options.quiet { - display.Mode = display.ModeQuiet - backendOptions.Add(compose.WithEventProcessor(display.Quiet())) - } - createOpts.pullChanged = cmd.Flags().Changed("pull") - return nil - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - project, _, err := p.ToProject(ctx, dockerCli, backend, []string{options.Service}, composecli.WithoutEnvironmentResolution) - if err != nil { - return err - } - - project, err = project.WithServicesEnvironmentResolved(true) - if err != nil { - return err - } - - if createOpts.quietPull { - buildOpts.Progress = string(xprogress.QuietMode) - } - - options.ignoreOrphans = utils.StringToBool(project.Environment[ComposeIgnoreOrphans]) - return runRun(ctx, backend, project, options, createOpts, buildOpts, dockerCli) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.BoolVarP(&options.Detach, "detach", "d", false, "Run container in background and print container ID") - flags.StringArrayVarP(&options.environment, "env", "e", []string{}, "Set environment variables") - flags.StringArrayVar(&options.envFiles, "env-from-file", []string{}, "Set environment variables from file") - flags.StringArrayVarP(&options.labels, "label", "l", []string{}, "Add or override a label") - flags.BoolVar(&options.Remove, "rm", false, "Automatically remove the container when it exits") - flags.BoolVarP(&options.noTty, "no-TTY", "T", !dockerCli.Out().IsTerminal(), "Disable pseudo-TTY allocation (default: auto-detected)") - flags.StringVar(&options.name, "name", "", "Assign a name to the container") - flags.StringVarP(&options.user, "user", "u", "", "Run as specified username or uid") - flags.StringVarP(&options.workdir, "workdir", "w", "", "Working directory inside the container") - flags.StringVar(&options.entrypoint, "entrypoint", "", "Override the entrypoint of the image") - flags.Var(&options.capAdd, "cap-add", "Add Linux capabilities") - flags.Var(&options.capDrop, "cap-drop", "Drop Linux capabilities") - flags.BoolVar(&options.noDeps, "no-deps", false, "Don't start linked services") - flags.StringArrayVarP(&options.volumes, "volume", "v", []string{}, "Bind mount a volume") - flags.StringArrayVarP(&options.publish, "publish", "p", []string{}, "Publish a container's port(s) to the host") - flags.BoolVar(&options.useAliases, "use-aliases", false, "Use the service's network useAliases in the network(s) the container connects to") - flags.BoolVarP(&options.servicePorts, "service-ports", "P", false, "Run command with all service's ports enabled and mapped to the host") - flags.StringVar(&createOpts.Pull, "pull", "policy", `Pull image before running ("always"|"missing"|"never")`) - flags.BoolVarP(&options.quiet, "quiet", "q", false, "Don't print anything to STDOUT") - flags.BoolVar(&buildOpts.quiet, "quiet-build", false, "Suppress progress output from the build process") - flags.BoolVar(&options.quietPull, "quiet-pull", false, "Pull without printing progress information") - flags.BoolVar(&createOpts.Build, "build", false, "Build image before starting container") - flags.BoolVar(&options.removeOrphans, "remove-orphans", false, "Remove containers for services not defined in the Compose file") - - cmd.Flags().BoolVarP(&options.interactive, "interactive", "i", true, "Keep STDIN open even if not attached") - cmd.Flags().BoolVarP(&ttyFlag, "tty", "t", true, "Allocate a pseudo-TTY") - cmd.Flags().MarkHidden("tty") //nolint:errcheck - - flags.SetNormalizeFunc(normalizeRunFlags) - flags.SetInterspersed(false) - return cmd -} - -func normalizeRunFlags(f *pflag.FlagSet, name string) pflag.NormalizedName { - switch name { - case "volumes": - name = "volume" - case "labels": - name = "label" - } - return pflag.NormalizedName(name) -} - -func runRun(ctx context.Context, backend api.Compose, project *types.Project, options runOptions, createOpts createOptions, buildOpts buildOptions, dockerCli command.Cli) error { - project, err := options.apply(project) - if err != nil { - return err - } - - err = createOpts.Apply(project) - if err != nil { - return err - } - - if err := checksForRemoteStack(ctx, dockerCli, project, buildOpts, createOpts.AssumeYes, []string{}); err != nil { - return err - } - - labels := types.Labels{} - for _, s := range options.labels { - key, val, ok := strings.Cut(s, "=") - if !ok { - return fmt.Errorf("label must be set as KEY=VALUE") - } - labels[key] = val - } - - var buildForRun *api.BuildOptions - if !createOpts.noBuild { - bo, err := buildOpts.toAPIBuildOptions(nil) - if err != nil { - return err - } - buildForRun = &bo - } - - environment, err := options.getEnvironment(project.Environment.Resolve) - if err != nil { - return err - } - - // start container and attach to container streams - runOpts := api.RunOptions{ - CreateOptions: api.CreateOptions{ - Build: buildForRun, - RemoveOrphans: options.removeOrphans, - IgnoreOrphans: options.ignoreOrphans, - QuietPull: options.quietPull, - }, - Name: options.name, - Service: options.Service, - Command: options.Command, - Detach: options.Detach, - AutoRemove: options.Remove, - Tty: !options.noTty, - Interactive: options.interactive, - WorkingDir: options.workdir, - User: options.user, - CapAdd: options.capAdd.GetSlice(), - CapDrop: options.capDrop.GetSlice(), - Environment: environment.Values(), - Entrypoint: options.entrypointCmd, - Labels: labels, - UseNetworkAliases: options.useAliases, - NoDeps: options.noDeps, - Index: 0, - } - - for name, service := range project.Services { - if name == options.Service { - service.StdinOpen = options.interactive - project.Services[name] = service - } - } - - exitCode, err := backend.RunOneOffContainer(ctx, project, runOpts) - if exitCode != 0 { - errMsg := "" - if err != nil { - errMsg = err.Error() - } - return cli.StatusError{StatusCode: exitCode, Status: errMsg} - } - return err -} diff --git a/cmd/compose/scale.go b/cmd/compose/scale.go deleted file mode 100644 index 8070858e9a1..00000000000 --- a/cmd/compose/scale.go +++ /dev/null @@ -1,107 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "maps" - "slices" - "strconv" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type scaleOptions struct { - *ProjectOptions - noDeps bool -} - -func scaleCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := scaleOptions{ - ProjectOptions: p, - } - scaleCmd := &cobra.Command{ - Use: "scale [SERVICE=REPLICAS...]", - Short: "Scale services ", - Args: cobra.MinimumNArgs(1), - RunE: Adapt(func(ctx context.Context, args []string) error { - serviceTuples, err := parseServicesReplicasArgs(args) - if err != nil { - return err - } - return runScale(ctx, dockerCli, backendOptions, opts, serviceTuples) - }), - ValidArgsFunction: completeScaleArgs(dockerCli, p), - } - flags := scaleCmd.Flags() - flags.BoolVar(&opts.noDeps, "no-deps", false, "Don't start linked services") - - return scaleCmd -} - -func runScale(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts scaleOptions, serviceReplicaTuples map[string]int) error { - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - services := slices.Sorted(maps.Keys(serviceReplicaTuples)) - project, _, err := opts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return err - } - - if opts.noDeps { - if project, err = project.WithSelectedServices(services, types.IgnoreDependencies); err != nil { - return err - } - } - - for key, value := range serviceReplicaTuples { - service, err := project.GetService(key) - if err != nil { - return err - } - service.SetScale(value) - project.Services[key] = service - } - - return backend.Scale(ctx, project, api.ScaleOptions{Services: services}) -} - -func parseServicesReplicasArgs(args []string) (map[string]int, error) { - serviceReplicaTuples := map[string]int{} - for _, arg := range args { - key, val, ok := strings.Cut(arg, "=") - if !ok || key == "" || val == "" { - return nil, fmt.Errorf("invalid scale specifier: %s", arg) - } - intValue, err := strconv.Atoi(val) - if err != nil { - return nil, fmt.Errorf("invalid scale specifier: can't parse replica value as int: %v", arg) - } - serviceReplicaTuples[key] = intValue - } - return serviceReplicaTuples, nil -} diff --git a/cmd/compose/start.go b/cmd/compose/start.go deleted file mode 100644 index bd5f10c4634..00000000000 --- a/cmd/compose/start.go +++ /dev/null @@ -1,77 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "time" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type startOptions struct { - *ProjectOptions - wait bool - waitTimeout int -} - -func startCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := startOptions{ - ProjectOptions: p, - } - startCmd := &cobra.Command{ - Use: "start [SERVICE...]", - Short: "Start services", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runStart(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := startCmd.Flags() - flags.BoolVar(&opts.wait, "wait", false, "Wait for services to be running|healthy. Implies detached mode.") - flags.IntVar(&opts.waitTimeout, "wait-timeout", 0, "Maximum duration in seconds to wait for the project to be running|healthy") - - return startCmd -} - -func runStart(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts startOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - var timeout time.Duration - if opts.waitTimeout > 0 { - timeout = time.Duration(opts.waitTimeout) * time.Second - } - return backend.Start(ctx, name, api.StartOptions{ - AttachTo: services, - Project: project, - Services: services, - Wait: opts.wait, - WaitTimeout: timeout, - }) -} diff --git a/cmd/compose/stats.go b/cmd/compose/stats.go deleted file mode 100644 index cef2daf275d..00000000000 --- a/cmd/compose/stats.go +++ /dev/null @@ -1,84 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/command/container" - "github.com/docker/docker/api/types/filters" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" -) - -type statsOptions struct { - ProjectOptions *ProjectOptions - all bool - format string - noStream bool - noTrunc bool -} - -func statsCommand(p *ProjectOptions, dockerCli command.Cli) *cobra.Command { - opts := statsOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "stats [OPTIONS] [SERVICE]", - Short: "Display a live stream of container(s) resource usage statistics", - Args: cobra.MaximumNArgs(1), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runStats(ctx, dockerCli, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.BoolVarP(&opts.all, "all", "a", false, "Show all containers (default shows just running)") - flags.StringVar(&opts.format, "format", "", `Format output using a custom template: -'table': Print output in table format with column headers (default) -'table TEMPLATE': Print output in table format using the given Go template -'json': Print in JSON format -'TEMPLATE': Print output using the given Go template. -Refer to https://docs.docker.com/engine/cli/formatting/ for more information about formatting output with templates`) - flags.BoolVar(&opts.noStream, "no-stream", false, "Disable streaming stats and only pull the first result") - flags.BoolVar(&opts.noTrunc, "no-trunc", false, "Do not truncate output") - return cmd -} - -func runStats(ctx context.Context, dockerCli command.Cli, opts statsOptions, service []string) error { - name, err := opts.ProjectOptions.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - filter := []filters.KeyValuePair{ - filters.Arg("label", fmt.Sprintf("%s=%s", api.ProjectLabel, name)), - } - if len(service) > 0 { - filter = append(filter, filters.Arg("label", fmt.Sprintf("%s=%s", api.ServiceLabel, service[0]))) - } - args := filters.NewArgs(filter...) - return container.RunStats(ctx, dockerCli, &container.StatsOptions{ - All: opts.all, - NoStream: opts.noStream, - NoTrunc: opts.noTrunc, - Format: opts.format, - Filters: &args, - }) -} diff --git a/cmd/compose/stop.go b/cmd/compose/stop.go deleted file mode 100644 index 6bc3faaa96b..00000000000 --- a/cmd/compose/stop.go +++ /dev/null @@ -1,77 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "time" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type stopOptions struct { - *ProjectOptions - timeChanged bool - timeout int -} - -func stopCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := stopOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "stop [OPTIONS] [SERVICE...]", - Short: "Stop services", - PreRun: func(cmd *cobra.Command, args []string) { - opts.timeChanged = cmd.Flags().Changed("timeout") - }, - RunE: Adapt(func(ctx context.Context, args []string) error { - return runStop(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := cmd.Flags() - flags.IntVarP(&opts.timeout, "timeout", "t", 0, "Specify a shutdown timeout in seconds") - - return cmd -} - -func runStop(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts stopOptions, services []string) error { - project, name, err := opts.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - var timeout *time.Duration - if opts.timeChanged { - timeoutValue := time.Duration(opts.timeout) * time.Second - timeout = &timeoutValue - } - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - return backend.Stop(ctx, name, api.StopOptions{ - Timeout: timeout, - Services: services, - Project: project, - }) -} diff --git a/cmd/compose/top.go b/cmd/compose/top.go deleted file mode 100644 index 0d7f969c841..00000000000 --- a/cmd/compose/top.go +++ /dev/null @@ -1,149 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - "sort" - "strings" - "text/tabwriter" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type topOptions struct { - *ProjectOptions -} - -func topCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := topOptions{ - ProjectOptions: p, - } - topCmd := &cobra.Command{ - Use: "top [SERVICES...]", - Short: "Display the running processes", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runTop(ctx, dockerCli, backendOptions, opts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - return topCmd -} - -type ( - topHeader map[string]int // maps a proc title to its output index - topEntries map[string]string -) - -func runTop(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts topOptions, services []string) error { - projectName, err := opts.toProjectName(ctx, dockerCli) - if err != nil { - return err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - containers, err := backend.Top(ctx, projectName, services) - if err != nil { - return err - } - - sort.Slice(containers, func(i, j int) bool { - return containers[i].Name < containers[j].Name - }) - - header, entries := collectTop(containers) - return topPrint(dockerCli.Out(), header, entries) -} - -func collectTop(containers []api.ContainerProcSummary) (topHeader, []topEntries) { - // map column name to its header (should keep working if backend.Top returns - // varying columns for different containers) - header := topHeader{"SERVICE": 0, "#": 1} - - // assume one process per container and grow if needed - entries := make([]topEntries, 0, len(containers)) - - for _, container := range containers { - for _, proc := range container.Processes { - entry := topEntries{ - "SERVICE": container.Service, - "#": container.Replica, - } - for i, title := range container.Titles { - if _, exists := header[title]; !exists { - header[title] = len(header) - } - entry[title] = proc[i] - } - entries = append(entries, entry) - } - } - - // ensure CMD is the right-most column - if pos, ok := header["CMD"]; ok { - maxPos := pos - for h, i := range header { - if i > maxPos { - maxPos = i - } - if i > pos { - header[h] = i - 1 - } - } - header["CMD"] = maxPos - } - - return header, entries -} - -func topPrint(out io.Writer, headers topHeader, rows []topEntries) error { - if len(rows) == 0 { - return nil - } - - w := tabwriter.NewWriter(out, 4, 1, 2, ' ', 0) - - // write headers in the order we've encountered them - h := make([]string, len(headers)) - for title, index := range headers { - h[index] = title - } - _, _ = fmt.Fprintln(w, strings.Join(h, "\t")) - - for _, row := range rows { - // write proc data in header order - r := make([]string, len(headers)) - for title, index := range headers { - if v, ok := row[title]; ok { - r[index] = v - } else { - r[index] = "-" - } - } - _, _ = fmt.Fprintln(w, strings.Join(r, "\t")) - } - return w.Flush() -} diff --git a/cmd/compose/top_test.go b/cmd/compose/top_test.go deleted file mode 100644 index 3e212ed85b0..00000000000 --- a/cmd/compose/top_test.go +++ /dev/null @@ -1,330 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "strings" - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - - "github.com/docker/compose/v5/pkg/api" -) - -var topTestCases = []struct { - name string - titles []string - procs [][]string - - header topHeader - entries []topEntries - output string -}{ - { - name: "noprocs", - titles: []string{"UID", "PID", "PPID", "C", "STIME", "TTY", "TIME", "CMD"}, - procs: [][]string{}, - header: topHeader{"SERVICE": 0, "#": 1}, - entries: []topEntries{}, - output: "", - }, - { - name: "simple", - titles: []string{"UID", "PID", "PPID", "C", "STIME", "TTY", "TIME", "CMD"}, - procs: [][]string{{"root", "1", "1", "0", "12:00", "?", "00:00:01", "/entrypoint"}}, - header: topHeader{ - "SERVICE": 0, - "#": 1, - "UID": 2, - "PID": 3, - "PPID": 4, - "C": 5, - "STIME": 6, - "TTY": 7, - "TIME": 8, - "CMD": 9, - }, - entries: []topEntries{ - { - "SERVICE": "simple", - "#": "1", - "UID": "root", - "PID": "1", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:01", - "CMD": "/entrypoint", - }, - }, - output: trim(` - SERVICE # UID PID PPID C STIME TTY TIME CMD - simple 1 root 1 1 0 12:00 ? 00:00:01 /entrypoint - `), - }, - { - name: "noppid", - titles: []string{"UID", "PID", "C", "STIME", "TTY", "TIME", "CMD"}, - procs: [][]string{{"root", "1", "0", "12:00", "?", "00:00:02", "/entrypoint"}}, - header: topHeader{ - "SERVICE": 0, - "#": 1, - "UID": 2, - "PID": 3, - "C": 4, - "STIME": 5, - "TTY": 6, - "TIME": 7, - "CMD": 8, - }, - entries: []topEntries{ - { - "SERVICE": "noppid", - "#": "1", - "UID": "root", - "PID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:02", - "CMD": "/entrypoint", - }, - }, - output: trim(` - SERVICE # UID PID C STIME TTY TIME CMD - noppid 1 root 1 0 12:00 ? 00:00:02 /entrypoint - `), - }, - { - name: "extra-hdr", - titles: []string{"UID", "GID", "PID", "PPID", "C", "STIME", "TTY", "TIME", "CMD"}, - procs: [][]string{{"root", "1", "1", "1", "0", "12:00", "?", "00:00:03", "/entrypoint"}}, - header: topHeader{ - "SERVICE": 0, - "#": 1, - "UID": 2, - "GID": 3, - "PID": 4, - "PPID": 5, - "C": 6, - "STIME": 7, - "TTY": 8, - "TIME": 9, - "CMD": 10, - }, - entries: []topEntries{ - { - "SERVICE": "extra-hdr", - "#": "1", - "UID": "root", - "GID": "1", - "PID": "1", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:03", - "CMD": "/entrypoint", - }, - }, - output: trim(` - SERVICE # UID GID PID PPID C STIME TTY TIME CMD - extra-hdr 1 root 1 1 1 0 12:00 ? 00:00:03 /entrypoint - `), - }, - { - name: "multiple", - titles: []string{"UID", "PID", "PPID", "C", "STIME", "TTY", "TIME", "CMD"}, - procs: [][]string{ - {"root", "1", "1", "0", "12:00", "?", "00:00:04", "/entrypoint"}, - {"root", "123", "1", "0", "12:00", "?", "00:00:42", "sleep infinity"}, - }, - header: topHeader{ - "SERVICE": 0, - "#": 1, - "UID": 2, - "PID": 3, - "PPID": 4, - "C": 5, - "STIME": 6, - "TTY": 7, - "TIME": 8, - "CMD": 9, - }, - entries: []topEntries{ - { - "SERVICE": "multiple", - "#": "1", - "UID": "root", - "PID": "1", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:04", - "CMD": "/entrypoint", - }, - { - "SERVICE": "multiple", - "#": "1", - "UID": "root", - "PID": "123", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:42", - "CMD": "sleep infinity", - }, - }, - output: trim(` - SERVICE # UID PID PPID C STIME TTY TIME CMD - multiple 1 root 1 1 0 12:00 ? 00:00:04 /entrypoint - multiple 1 root 123 1 0 12:00 ? 00:00:42 sleep infinity - `), - }, -} - -// TestRunTopCore only tests the core functionality of runTop: formatting -// and printing of the output of (api.Compose).Top(). -func TestRunTopCore(t *testing.T) { - t.Parallel() - - all := []api.ContainerProcSummary{} - - for _, tc := range topTestCases { - summary := api.ContainerProcSummary{ - Name: "not used", - Titles: tc.titles, - Processes: tc.procs, - Service: tc.name, - Replica: "1", - } - all = append(all, summary) - - t.Run(tc.name, func(t *testing.T) { - header, entries := collectTop([]api.ContainerProcSummary{summary}) - assert.Equal(t, tc.header, header) - assert.Equal(t, tc.entries, entries) - - var buf bytes.Buffer - err := topPrint(&buf, header, entries) - - require.NoError(t, err) - assert.Equal(t, tc.output, buf.String()) - }) - } - - t.Run("all", func(t *testing.T) { - header, entries := collectTop(all) - assert.Equal(t, topHeader{ - "SERVICE": 0, - "#": 1, - "UID": 2, - "PID": 3, - "PPID": 4, - "C": 5, - "STIME": 6, - "TTY": 7, - "TIME": 8, - "GID": 9, - "CMD": 10, - }, header) - assert.Equal(t, []topEntries{ - { - "SERVICE": "simple", - "#": "1", - "UID": "root", - "PID": "1", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:01", - "CMD": "/entrypoint", - }, { - "SERVICE": "noppid", - "#": "1", - "UID": "root", - "PID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:02", - "CMD": "/entrypoint", - }, { - "SERVICE": "extra-hdr", - "#": "1", - "UID": "root", - "GID": "1", - "PID": "1", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:03", - "CMD": "/entrypoint", - }, { - "SERVICE": "multiple", - "#": "1", - "UID": "root", - "PID": "1", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:04", - "CMD": "/entrypoint", - }, { - "SERVICE": "multiple", - "#": "1", - "UID": "root", - "PID": "123", - "PPID": "1", - "C": "0", - "STIME": "12:00", - "TTY": "?", - "TIME": "00:00:42", - "CMD": "sleep infinity", - }, - }, entries) - - var buf bytes.Buffer - err := topPrint(&buf, header, entries) - require.NoError(t, err) - assert.Equal(t, trim(` - SERVICE # UID PID PPID C STIME TTY TIME GID CMD - simple 1 root 1 1 0 12:00 ? 00:00:01 - /entrypoint - noppid 1 root 1 - 0 12:00 ? 00:00:02 - /entrypoint - extra-hdr 1 root 1 1 0 12:00 ? 00:00:03 1 /entrypoint - multiple 1 root 1 1 0 12:00 ? 00:00:04 - /entrypoint - multiple 1 root 123 1 0 12:00 ? 00:00:42 - sleep infinity - `), buf.String()) - }) -} - -func trim(s string) string { - var out bytes.Buffer - for line := range strings.SplitSeq(strings.TrimSpace(s), "\n") { - out.WriteString(strings.TrimSpace(line)) - out.WriteRune('\n') - } - return out.String() -} diff --git a/cmd/compose/up.go b/cmd/compose/up.go deleted file mode 100644 index cda2678bbb2..00000000000 --- a/cmd/compose/up.go +++ /dev/null @@ -1,363 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "os" - "strings" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - xprogress "github.com/moby/buildkit/util/progress/progressui" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "github.com/spf13/pflag" - - "github.com/docker/compose/v5/cmd/display" - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" - "github.com/docker/compose/v5/pkg/utils" -) - -// composeOptions hold options common to `up` and `run` to run compose project -type composeOptions struct { - *ProjectOptions -} - -type upOptions struct { - *composeOptions - Detach bool - noStart bool - noDeps bool - cascadeStop bool - cascadeFail bool - exitCodeFrom string - noColor bool - noPrefix bool - attachDependencies bool - attach []string - noAttach []string - timestamp bool - wait bool - waitTimeout int - watch bool - navigationMenu bool - navigationMenuChanged bool -} - -func (opts upOptions) apply(project *types.Project, services []string) (*types.Project, error) { - if opts.noDeps { - var err error - project, err = project.WithSelectedServices(services, types.IgnoreDependencies) - if err != nil { - return nil, err - } - } - - if opts.exitCodeFrom != "" { - _, err := project.GetService(opts.exitCodeFrom) - if err != nil { - return nil, err - } - } - - return project, nil -} - -func (opts *upOptions) validateNavigationMenu(dockerCli command.Cli) { - if !dockerCli.Out().IsTerminal() { - opts.navigationMenu = false - return - } - // If --menu flag was not set - if !opts.navigationMenuChanged { - if envVar, ok := os.LookupEnv(ComposeMenu); ok { - opts.navigationMenu = utils.StringToBool(envVar) - return - } - // ...and COMPOSE_MENU env var is not defined we want the default value to be true - opts.navigationMenu = true - } -} - -func (opts upOptions) OnExit() api.Cascade { - switch { - case opts.cascadeStop: - return api.CascadeStop - case opts.cascadeFail: - return api.CascadeFail - default: - return api.CascadeIgnore - } -} - -func upCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - up := upOptions{} - create := createOptions{} - build := buildOptions{ProjectOptions: p} - upCmd := &cobra.Command{ - Use: "up [OPTIONS] [SERVICE...]", - Short: "Create and start containers", - PreRunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - create.pullChanged = cmd.Flags().Changed("pull") - create.timeChanged = cmd.Flags().Changed("timeout") - up.navigationMenuChanged = cmd.Flags().Changed("menu") - if !cmd.Flags().Changed("remove-orphans") { - create.removeOrphans = utils.StringToBool(os.Getenv(ComposeRemoveOrphans)) - } - return validateFlags(&up, &create) - }), - RunE: p.WithServices(dockerCli, func(ctx context.Context, project *types.Project, services []string) error { - create.ignoreOrphans = utils.StringToBool(project.Environment[ComposeIgnoreOrphans]) - if create.ignoreOrphans && create.removeOrphans { - return fmt.Errorf("cannot combine %s and --remove-orphans", ComposeIgnoreOrphans) - } - if len(up.attach) != 0 && up.attachDependencies { - return errors.New("cannot combine --attach and --attach-dependencies") - } - - up.validateNavigationMenu(dockerCli) - - if !p.All && len(project.Services) == 0 { - return fmt.Errorf("no service selected") - } - - return runUp(ctx, dockerCli, backendOptions, create, up, build, project, services) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - flags := upCmd.Flags() - flags.BoolVarP(&up.Detach, "detach", "d", false, "Detached mode: Run containers in the background") - flags.BoolVar(&create.Build, "build", false, "Build images before starting containers") - flags.BoolVar(&create.noBuild, "no-build", false, "Don't build an image, even if it's policy") - flags.StringVar(&create.Pull, "pull", "policy", `Pull image before running ("always"|"missing"|"never")`) - flags.BoolVar(&create.removeOrphans, "remove-orphans", false, "Remove containers for services not defined in the Compose file") - flags.StringArrayVar(&create.scale, "scale", []string{}, "Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present.") - flags.BoolVar(&up.noColor, "no-color", false, "Produce monochrome output") - flags.BoolVar(&up.noPrefix, "no-log-prefix", false, "Don't print prefix in logs") - flags.BoolVar(&create.forceRecreate, "force-recreate", false, "Recreate containers even if their configuration and image haven't changed") - flags.BoolVar(&create.noRecreate, "no-recreate", false, "If containers already exist, don't recreate them. Incompatible with --force-recreate.") - flags.BoolVar(&up.noStart, "no-start", false, "Don't start the services after creating them") - flags.BoolVar(&up.cascadeStop, "abort-on-container-exit", false, "Stops all containers if any container was stopped. Incompatible with -d") - flags.BoolVar(&up.cascadeFail, "abort-on-container-failure", false, "Stops all containers if any container exited with failure. Incompatible with -d") - flags.StringVar(&up.exitCodeFrom, "exit-code-from", "", "Return the exit code of the selected service container. Implies --abort-on-container-exit") - flags.IntVarP(&create.timeout, "timeout", "t", 0, "Use this timeout in seconds for container shutdown when attached or when containers are already running") - flags.BoolVar(&up.timestamp, "timestamps", false, "Show timestamps") - flags.BoolVar(&up.noDeps, "no-deps", false, "Don't start linked services") - flags.BoolVar(&create.recreateDeps, "always-recreate-deps", false, "Recreate dependent containers. Incompatible with --no-recreate.") - flags.BoolVarP(&create.noInherit, "renew-anon-volumes", "V", false, "Recreate anonymous volumes instead of retrieving data from the previous containers") - flags.BoolVar(&create.quietPull, "quiet-pull", false, "Pull without printing progress information") - flags.BoolVar(&build.quiet, "quiet-build", false, "Suppress the build output") - flags.StringArrayVar(&up.attach, "attach", []string{}, "Restrict attaching to the specified services. Incompatible with --attach-dependencies.") - flags.StringArrayVar(&up.noAttach, "no-attach", []string{}, "Do not attach (stream logs) to the specified services") - flags.BoolVar(&up.attachDependencies, "attach-dependencies", false, "Automatically attach to log output of dependent services") - flags.BoolVar(&up.wait, "wait", false, "Wait for services to be running|healthy. Implies detached mode.") - flags.IntVar(&up.waitTimeout, "wait-timeout", 0, "Maximum duration in seconds to wait for the project to be running|healthy") - flags.BoolVarP(&up.watch, "watch", "w", false, "Watch source code and rebuild/refresh containers when files are updated.") - flags.BoolVar(&up.navigationMenu, "menu", false, "Enable interactive shortcuts when running attached. Incompatible with --detach. Can also be enable/disable by setting COMPOSE_MENU environment var.") - flags.BoolVarP(&create.AssumeYes, "yes", "y", false, `Assume "yes" as answer to all prompts and run non-interactively`) - flags.SetNormalizeFunc(func(f *pflag.FlagSet, name string) pflag.NormalizedName { - // assumeYes was introduced by mistake as `--y` - if name == "y" { - logrus.Warn("--y is deprecated, please use --yes instead") - name = "yes" - } - return pflag.NormalizedName(name) - }) - return upCmd -} - -//nolint:gocyclo -func validateFlags(up *upOptions, create *createOptions) error { - if up.waitTimeout < 0 { - return fmt.Errorf("--wait-timeout must be a non-negative integer") - } - if up.exitCodeFrom != "" && !up.cascadeFail { - up.cascadeStop = true - } - if up.cascadeStop && up.cascadeFail { - return fmt.Errorf("--abort-on-container-failure cannot be combined with --abort-on-container-exit") - } - if up.wait { - if up.attachDependencies || up.cascadeStop || len(up.attach) > 0 { - return fmt.Errorf("--wait cannot be combined with --abort-on-container-exit, --attach or --attach-dependencies") - } - up.Detach = true - } - if create.Build && create.noBuild { - return fmt.Errorf("--build and --no-build are incompatible") - } - if up.Detach && (up.attachDependencies || up.cascadeStop || up.cascadeFail || len(up.attach) > 0 || up.watch) { - if up.wait { - return fmt.Errorf("--wait cannot be combined with --abort-on-container-exit, --abort-on-container-failure, --attach, --attach-dependencies or --watch") - } else { - return fmt.Errorf("--detach cannot be combined with --abort-on-container-exit, --abort-on-container-failure, --attach, --attach-dependencies or --watch") - } - } - if create.noInherit && create.noRecreate { - return fmt.Errorf("--no-recreate and --renew-anon-volumes are incompatible") - } - if create.forceRecreate && create.noRecreate { - return fmt.Errorf("--force-recreate and --no-recreate are incompatible") - } - if create.recreateDeps && create.noRecreate { - return fmt.Errorf("--always-recreate-deps and --no-recreate are incompatible") - } - if create.noBuild && up.watch { - return fmt.Errorf("--no-build and --watch are incompatible") - } - return nil -} - -//nolint:gocyclo -func runUp( - ctx context.Context, - dockerCli command.Cli, - backendOptions *BackendOptions, - createOptions createOptions, - upOptions upOptions, - buildOptions buildOptions, - project *types.Project, - services []string, -) error { - if err := checksForRemoteStack(ctx, dockerCli, project, buildOptions, createOptions.AssumeYes, []string{}); err != nil { - return err - } - - err := createOptions.Apply(project) - if err != nil { - return err - } - - project, err = upOptions.apply(project, services) - if err != nil { - return err - } - - var build *api.BuildOptions - if !createOptions.noBuild { - if createOptions.quietPull { - buildOptions.Progress = string(xprogress.QuietMode) - } - // BuildOptions here is nested inside CreateOptions, so - // no service list is passed, it will implicitly pick all - // services being created, which includes any explicitly - // specified via "services" arg here as well as deps - bo, err := buildOptions.toAPIBuildOptions(nil) - if err != nil { - return err - } - bo.Services = project.ServiceNames() - bo.Deps = !upOptions.noDeps - build = &bo - } - - create := api.CreateOptions{ - Build: build, - Services: services, - RemoveOrphans: createOptions.removeOrphans, - IgnoreOrphans: createOptions.ignoreOrphans, - Recreate: createOptions.recreateStrategy(), - RecreateDependencies: createOptions.dependenciesRecreateStrategy(), - Inherit: !createOptions.noInherit, - Timeout: createOptions.GetTimeout(), - QuietPull: createOptions.quietPull, - } - - if createOptions.AssumeYes { - backendOptions.Options = append(backendOptions.Options, compose.WithPrompt(compose.AlwaysOkPrompt())) - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - if upOptions.noStart { - return backend.Create(ctx, project, create) - } - - var consumer api.LogConsumer - var attach []string - if !upOptions.Detach { - consumer = formatter.NewLogConsumer(ctx, dockerCli.Out(), dockerCli.Err(), !upOptions.noColor, !upOptions.noPrefix, upOptions.timestamp) - - var attachSet utils.Set[string] - if len(upOptions.attach) != 0 { - // services are passed explicitly with --attach, verify they're valid and then use them as-is - attachSet = utils.NewSet(upOptions.attach...) - unexpectedSvcs := attachSet.Diff(utils.NewSet(project.ServiceNames()...)) - if len(unexpectedSvcs) != 0 { - return fmt.Errorf("cannot attach to services not included in up: %s", strings.Join(unexpectedSvcs.Elements(), ", ")) - } - } else { - // mark services being launched (and potentially their deps) for attach - // if they didn't opt-out via Compose YAML - attachSet = utils.NewSet[string]() - var dependencyOpt types.DependencyOption = types.IgnoreDependencies - if upOptions.attachDependencies { - dependencyOpt = types.IncludeDependencies - } - if err := project.ForEachService(services, func(serviceName string, s *types.ServiceConfig) error { - if s.Attach == nil || *s.Attach { - attachSet.Add(serviceName) - } - return nil - }, dependencyOpt); err != nil { - return err - } - } - // filter out any services that have been explicitly marked for ignore with `--no-attach` - attachSet.RemoveAll(upOptions.noAttach...) - attach = attachSet.Elements() - } - - var timeout time.Duration - if upOptions.waitTimeout > 0 { - timeout = time.Duration(upOptions.waitTimeout) * time.Second - } - return backend.Up(ctx, project, api.UpOptions{ - Create: create, - Start: api.StartOptions{ - Project: project, - Attach: consumer, - AttachTo: attach, - ExitCodeFrom: upOptions.exitCodeFrom, - OnExit: upOptions.OnExit(), - Wait: upOptions.wait, - WaitTimeout: timeout, - Watch: upOptions.watch, - Services: services, - NavigationMenu: upOptions.navigationMenu && display.Mode != "plain" && dockerCli.In().IsTerminal(), - }, - }) -} - -func setServiceScale(project *types.Project, name string, replicas int) error { - service, err := project.GetService(name) - if err != nil { - return err - } - service.SetScale(replicas) - project.Services[name] = service - return nil -} diff --git a/cmd/compose/up_test.go b/cmd/compose/up_test.go deleted file mode 100644 index f567e97ad44..00000000000 --- a/cmd/compose/up_test.go +++ /dev/null @@ -1,91 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestApplyScaleOpt(t *testing.T) { - p := types.Project{ - Services: types.Services{ - "foo": { - Name: "foo", - }, - "bar": { - Name: "bar", - Deploy: &types.DeployConfig{ - Mode: "test", - }, - }, - }, - } - err := applyScaleOpts(&p, []string{"foo=2", "bar=3"}) - assert.NilError(t, err) - foo, err := p.GetService("foo") - assert.NilError(t, err) - assert.Equal(t, *foo.Scale, 2) - - bar, err := p.GetService("bar") - assert.NilError(t, err) - assert.Equal(t, *bar.Scale, 3) - assert.Equal(t, *bar.Deploy.Replicas, 3) -} - -func TestUpOptions_OnExit(t *testing.T) { - tests := []struct { - name string - args upOptions - want api.Cascade - }{ - { - name: "no cascade", - args: upOptions{}, - want: api.CascadeIgnore, - }, - { - name: "cascade stop", - args: upOptions{cascadeStop: true}, - want: api.CascadeStop, - }, - { - name: "cascade fail", - args: upOptions{cascadeFail: true}, - want: api.CascadeFail, - }, - { - name: "both set - stop takes precedence", - args: upOptions{ - cascadeStop: true, - cascadeFail: true, - }, - want: api.CascadeStop, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - got := tt.args.OnExit() - assert.Equal(t, got, tt.want) - }) - } -} diff --git a/cmd/compose/version.go b/cmd/compose/version.go deleted file mode 100644 index 4302c15697b..00000000000 --- a/cmd/compose/version.go +++ /dev/null @@ -1,69 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "fmt" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/internal" -) - -type versionOptions struct { - format string - short bool -} - -func versionCommand(dockerCli command.Cli) *cobra.Command { - opts := versionOptions{} - cmd := &cobra.Command{ - Use: "version [OPTIONS]", - Short: "Show the Docker Compose version information", - Args: cobra.NoArgs, - RunE: func(cmd *cobra.Command, _ []string) error { - runVersion(opts, dockerCli) - return nil - }, - PersistentPreRunE: func(cmd *cobra.Command, args []string) error { - // overwrite parent PersistentPreRunE to avoid trying to load - // compose file on version command if COMPOSE_FILE is set - return nil - }, - } - // define flags for backward compatibility with com.docker.cli - flags := cmd.Flags() - flags.StringVarP(&opts.format, "format", "f", "", "Format the output. Values: [pretty | json]. (Default: pretty)") - flags.BoolVar(&opts.short, "short", false, "Shows only Compose's version number") - - return cmd -} - -func runVersion(opts versionOptions, dockerCli command.Cli) { - if opts.short { - _, _ = fmt.Fprintln(dockerCli.Out(), strings.TrimPrefix(internal.Version, "v")) - return - } - if opts.format == formatter.JSON { - _, _ = fmt.Fprintf(dockerCli.Out(), "{\"version\":%q}\n", internal.Version) - return - } - _, _ = fmt.Fprintln(dockerCli.Out(), "Docker Compose version", internal.Version) -} diff --git a/cmd/compose/version_test.go b/cmd/compose/version_test.go deleted file mode 100644 index c9bf9b74eae..00000000000 --- a/cmd/compose/version_test.go +++ /dev/null @@ -1,77 +0,0 @@ -/* - Copyright 2025 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "testing" - - "github.com/docker/cli/cli/streams" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/internal" - "github.com/docker/compose/v5/pkg/mocks" -) - -func TestVersionCommand(t *testing.T) { - originalVersion := internal.Version - defer func() { - internal.Version = originalVersion - }() - internal.Version = "v9.9.9-test" - - tests := []struct { - name string - args []string - want string - }{ - { - name: "default", - args: []string{}, - want: "Docker Compose version v9.9.9-test\n", - }, - { - name: "short flag", - args: []string{"--short"}, - want: "9.9.9-test\n", - }, - { - name: "json flag", - args: []string{"--format", "json"}, - want: `{"version":"v9.9.9-test"}` + "\n", - }, - } - - for _, test := range tests { - t.Run(test.name, func(t *testing.T) { - ctrl := gomock.NewController(t) - defer ctrl.Finish() - - buf := new(bytes.Buffer) - cli := mocks.NewMockCli(ctrl) - cli.EXPECT().Out().Return(streams.NewOut(buf)).AnyTimes() - - cmd := versionCommand(cli) - cmd.SetArgs(test.args) - err := cmd.Execute() - assert.NilError(t, err) - - assert.Equal(t, test.want, buf.String()) - }) - } -} diff --git a/cmd/compose/viz.go b/cmd/compose/viz.go deleted file mode 100644 index 443d78c6261..00000000000 --- a/cmd/compose/viz.go +++ /dev/null @@ -1,105 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type vizOptions struct { - *ProjectOptions - includeNetworks bool - includePorts bool - includeImageName bool - indentationStr string -} - -func vizCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := vizOptions{ - ProjectOptions: p, - } - var indentationSize int - var useSpaces bool - - cmd := &cobra.Command{ - Use: "viz [OPTIONS]", - Short: "EXPERIMENTAL - Generate a graphviz graph from your compose file", - PreRunE: Adapt(func(ctx context.Context, args []string) error { - var err error - opts.indentationStr, err = preferredIndentationStr(indentationSize, useSpaces) - return err - }), - RunE: Adapt(func(ctx context.Context, args []string) error { - return runViz(ctx, dockerCli, backendOptions, &opts) - }), - } - - cmd.Flags().BoolVar(&opts.includePorts, "ports", false, "Include service's exposed ports in output graph") - cmd.Flags().BoolVar(&opts.includeNetworks, "networks", false, "Include service's attached networks in output graph") - cmd.Flags().BoolVar(&opts.includeImageName, "image", false, "Include service's image name in output graph") - cmd.Flags().IntVar(&indentationSize, "indentation-size", 1, "Number of tabs or spaces to use for indentation") - cmd.Flags().BoolVar(&useSpaces, "spaces", false, "If given, space character ' ' will be used to indent,\notherwise tab character '\\t' will be used") - return cmd -} - -func runViz(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts *vizOptions) error { - _, _ = fmt.Fprintln(os.Stderr, "viz command is EXPERIMENTAL") - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - project, _, err := opts.ToProject(ctx, dockerCli, backend, nil) - if err != nil { - return err - } - - // build graph - graphStr, _ := backend.Viz(ctx, project, api.VizOptions{ - IncludeNetworks: opts.includeNetworks, - IncludePorts: opts.includePorts, - IncludeImageName: opts.includeImageName, - Indentation: opts.indentationStr, - }) - - fmt.Println(graphStr) - - return nil -} - -// preferredIndentationStr returns a single string given the indentation preference -func preferredIndentationStr(size int, useSpace bool) (string, error) { - if size < 0 { - return "", fmt.Errorf("invalid indentation size: %d", size) - } - - indentationStr := "\t" - if useSpace { - indentationStr = " " - } - return strings.Repeat(indentationStr, size), nil -} diff --git a/cmd/compose/viz_test.go b/cmd/compose/viz_test.go deleted file mode 100644 index f4a90501e33..00000000000 --- a/cmd/compose/viz_test.go +++ /dev/null @@ -1,94 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -func TestPreferredIndentationStr(t *testing.T) { - type args struct { - size int - useSpace bool - } - tests := []struct { - name string - args args - want string - wantErr bool - }{ - { - name: "should return '\\t\\t'", - args: args{ - size: 2, - useSpace: false, - }, - want: "\t\t", - wantErr: false, - }, - { - name: "should return ' '", - args: args{ - size: 4, - useSpace: true, - }, - want: " ", - wantErr: false, - }, - { - name: "should return ''", - args: args{ - size: 0, - useSpace: false, - }, - want: "", - wantErr: false, - }, - { - name: "should return ''", - args: args{ - size: 0, - useSpace: true, - }, - want: "", - wantErr: false, - }, - { - name: "should throw error because indentation size < 0", - args: args{ - size: -1, - useSpace: false, - }, - want: "", - wantErr: true, - }, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - got, err := preferredIndentationStr(tt.args.size, tt.args.useSpace) - if tt.wantErr { - require.Errorf(t, err, "preferredIndentationStr(%v, %v)", tt.args.size, tt.args.useSpace) - } else { - require.NoError(t, err) - assert.Equalf(t, tt.want, got, "preferredIndentationStr(%v, %v)", tt.args.size, tt.args.useSpace) - } - }) - } -} diff --git a/cmd/compose/volumes.go b/cmd/compose/volumes.go deleted file mode 100644 index e0da4f82e3b..00000000000 --- a/cmd/compose/volumes.go +++ /dev/null @@ -1,98 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "slices" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/command/formatter" - "github.com/docker/cli/cli/flags" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type volumesOptions struct { - *ProjectOptions - Quiet bool - Format string -} - -func volumesCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - options := volumesOptions{ - ProjectOptions: p, - } - - cmd := &cobra.Command{ - Use: "volumes [OPTIONS] [SERVICE...]", - Short: "List volumes", - RunE: Adapt(func(ctx context.Context, args []string) error { - return runVol(ctx, dockerCli, backendOptions, args, options) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - cmd.Flags().BoolVarP(&options.Quiet, "quiet", "q", false, "Only display volume names") - cmd.Flags().StringVar(&options.Format, "format", "table", flags.FormatHelp) - - return cmd -} - -func runVol(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, services []string, options volumesOptions) error { - project, name, err := options.projectOrName(ctx, dockerCli, services...) - if err != nil { - return err - } - - if project != nil { - names := project.ServiceNames() - for _, service := range services { - if !slices.Contains(names, service) { - return fmt.Errorf("no such service: %s", service) - } - } - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - volumes, err := backend.Volumes(ctx, name, api.VolumesOptions{ - Services: services, - }) - if err != nil { - return err - } - - if options.Quiet { - for _, v := range volumes { - _, _ = fmt.Fprintln(dockerCli.Out(), v.Name) - } - return nil - } - - volumeCtx := formatter.Context{ - Output: dockerCli.Out(), - Format: formatter.NewVolumeFormat(options.Format, options.Quiet), - } - - return formatter.VolumeWrite(volumeCtx, volumes) -} diff --git a/cmd/compose/wait.go b/cmd/compose/wait.go deleted file mode 100644 index 9d86fd314cf..00000000000 --- a/cmd/compose/wait.go +++ /dev/null @@ -1,79 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "os" - - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type waitOptions struct { - *ProjectOptions - - services []string - - downProject bool -} - -func waitCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - opts := waitOptions{ - ProjectOptions: p, - } - - var statusCode int64 - var err error - cmd := &cobra.Command{ - Use: "wait SERVICE [SERVICE...] [OPTIONS]", - Short: "Block until containers of all (or specified) services stop.", - Args: cli.RequiresMinArgs(1), - RunE: Adapt(func(ctx context.Context, services []string) error { - opts.services = services - statusCode, err = runWait(ctx, dockerCli, backendOptions, &opts) - return err - }), - PostRun: func(cmd *cobra.Command, args []string) { - os.Exit(int(statusCode)) - }, - } - - cmd.Flags().BoolVar(&opts.downProject, "down-project", false, "Drops project when the first container stops") - - return cmd -} - -func runWait(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, opts *waitOptions) (int64, error) { - _, name, err := opts.projectOrName(ctx, dockerCli) - if err != nil { - return 0, err - } - - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return 0, err - } - return backend.Wait(ctx, name, api.WaitOptions{ - Services: opts.services, - DownProjectOnContainerExit: opts.downProject, - }) -} diff --git a/cmd/compose/watch.go b/cmd/compose/watch.go deleted file mode 100644 index c60b243860e..00000000000 --- a/cmd/compose/watch.go +++ /dev/null @@ -1,132 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/internal/locker" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -type watchOptions struct { - *ProjectOptions - prune bool - noUp bool -} - -func watchCommand(p *ProjectOptions, dockerCli command.Cli, backendOptions *BackendOptions) *cobra.Command { - watchOpts := watchOptions{ - ProjectOptions: p, - } - buildOpts := buildOptions{ - ProjectOptions: p, - } - cmd := &cobra.Command{ - Use: "watch [SERVICE...]", - Short: "Watch build context for service and rebuild/refresh containers when files are updated", - PreRunE: Adapt(func(ctx context.Context, args []string) error { - return nil - }), - RunE: AdaptCmd(func(ctx context.Context, cmd *cobra.Command, args []string) error { - if cmd.Parent().Name() == "alpha" { - logrus.Warn("watch command is now available as a top level command") - } - return runWatch(ctx, dockerCli, backendOptions, watchOpts, buildOpts, args) - }), - ValidArgsFunction: completeServiceNames(dockerCli, p), - } - - cmd.Flags().BoolVar(&buildOpts.quiet, "quiet", false, "hide build output") - cmd.Flags().BoolVar(&watchOpts.prune, "prune", true, "Prune dangling images on rebuild") - cmd.Flags().BoolVar(&watchOpts.noUp, "no-up", false, "Do not build & start services before watching") - return cmd -} - -func runWatch(ctx context.Context, dockerCli command.Cli, backendOptions *BackendOptions, watchOpts watchOptions, buildOpts buildOptions, services []string) error { - backend, err := compose.NewComposeService(dockerCli, backendOptions.Options...) - if err != nil { - return err - } - - project, _, err := watchOpts.ToProject(ctx, dockerCli, backend, services) - if err != nil { - return err - } - - if err := applyPlatforms(project, true); err != nil { - return err - } - - build, err := buildOpts.toAPIBuildOptions(nil) - if err != nil { - return err - } - - // validation done -- ensure we have the lockfile for this project before doing work - l, err := locker.NewPidfile(project.Name) - if err != nil { - return fmt.Errorf("cannot take exclusive lock for project %q: %w", project.Name, err) - } - if err := l.Lock(); err != nil { - return fmt.Errorf("cannot take exclusive lock for project %q: %w", project.Name, err) - } - - if !watchOpts.noUp { - for index, service := range project.Services { - if service.Build != nil && service.Develop != nil { - service.PullPolicy = types.PullPolicyBuild - } - project.Services[index] = service - } - upOpts := api.UpOptions{ - Create: api.CreateOptions{ - Build: &build, - Services: services, - RemoveOrphans: false, - Recreate: api.RecreateDiverged, - RecreateDependencies: api.RecreateNever, - Inherit: true, - QuietPull: buildOpts.quiet, - }, - Start: api.StartOptions{ - Project: project, - Attach: nil, - Services: services, - }, - } - if err := backend.Up(ctx, project, upOpts); err != nil { - return err - } - } - - consumer := formatter.NewLogConsumer(ctx, dockerCli.Out(), dockerCli.Err(), false, false, false) - return backend.Watch(ctx, project, api.WatchOptions{ - Build: &build, - LogTo: consumer, - Prune: watchOpts.prune, - Services: services, - }) -} diff --git a/cmd/display/colors.go b/cmd/display/colors.go deleted file mode 100644 index a00b4ed6ad5..00000000000 --- a/cmd/display/colors.go +++ /dev/null @@ -1,47 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "github.com/morikuni/aec" -) - -type colorFunc func(string) string - -var ( - nocolor colorFunc = func(s string) string { - return s - } - - DoneColor colorFunc = aec.BlueF.Apply - TimerColor colorFunc = aec.BlueF.Apply - CountColor colorFunc = aec.YellowF.Apply - WarningColor colorFunc = aec.YellowF.With(aec.Bold).Apply - SuccessColor colorFunc = aec.GreenF.Apply - ErrorColor colorFunc = aec.RedF.With(aec.Bold).Apply - PrefixColor colorFunc = aec.CyanF.Apply -) - -func NoColor() { - DoneColor = nocolor - TimerColor = nocolor - CountColor = nocolor - WarningColor = nocolor - SuccessColor = nocolor - ErrorColor = nocolor - PrefixColor = nocolor -} diff --git a/cmd/display/dryrun.go b/cmd/display/dryrun.go deleted file mode 100644 index 2ab542e5b05..00000000000 --- a/cmd/display/dryrun.go +++ /dev/null @@ -1,21 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -const ( - DRYRUN_PREFIX = " DRY-RUN MODE - " -) diff --git a/cmd/display/json.go b/cmd/display/json.go deleted file mode 100644 index b8873596374..00000000000 --- a/cmd/display/json.go +++ /dev/null @@ -1,81 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "context" - "encoding/json" - "fmt" - "io" - - "github.com/docker/compose/v5/pkg/api" -) - -func JSON(out io.Writer) api.EventProcessor { - return &jsonWriter{ - out: out, - } -} - -type jsonWriter struct { - out io.Writer - dryRun bool -} - -type jsonMessage struct { - DryRun bool `json:"dry-run,omitempty"` - Tail bool `json:"tail,omitempty"` - ID string `json:"id,omitempty"` - ParentID string `json:"parent_id,omitempty"` - Status string `json:"status,omitempty"` - Text string `json:"text,omitempty"` - Details string `json:"details,omitempty"` - Current int64 `json:"current,omitempty"` - Total int64 `json:"total,omitempty"` - Percent int `json:"percent,omitempty"` -} - -func (p *jsonWriter) Start(ctx context.Context, operation string) { -} - -func (p *jsonWriter) Event(e api.Resource) { - message := &jsonMessage{ - DryRun: p.dryRun, - Tail: false, - ID: e.ID, - Status: e.StatusText(), - Text: e.Text, - Details: e.Details, - ParentID: e.ParentID, - Current: e.Current, - Total: e.Total, - Percent: e.Percent, - } - marshal, err := json.Marshal(message) - if err == nil { - _, _ = fmt.Fprintln(p.out, string(marshal)) - } -} - -func (p *jsonWriter) On(events ...api.Resource) { - for _, e := range events { - p.Event(e) - } -} - -func (p *jsonWriter) Done(_ string, _ bool) { -} diff --git a/cmd/display/json_test.go b/cmd/display/json_test.go deleted file mode 100644 index 0f0dff23a61..00000000000 --- a/cmd/display/json_test.go +++ /dev/null @@ -1,62 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "bytes" - "encoding/json" - "testing" - - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestJsonWriter_Event(t *testing.T) { - var out bytes.Buffer - w := &jsonWriter{ - out: &out, - dryRun: true, - } - - event := api.Resource{ - ID: "service1", - ParentID: "project", - Status: api.Working, - Text: api.StatusCreating, - Current: 50, - Total: 100, - Percent: 50, - } - w.Event(event) - - var actual jsonMessage - err := json.Unmarshal(out.Bytes(), &actual) - assert.NilError(t, err) - - expected := jsonMessage{ - DryRun: true, - ID: event.ID, - ParentID: event.ParentID, - Text: api.StatusCreating, - Status: "Working", - Current: event.Current, - Total: event.Total, - Percent: event.Percent, - } - assert.DeepEqual(t, expected, actual) -} diff --git a/cmd/display/mode.go b/cmd/display/mode.go deleted file mode 100644 index d66777b472c..00000000000 --- a/cmd/display/mode.go +++ /dev/null @@ -1,33 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -// Mode define how progress should be rendered, either as ModePlain or ModeTTY -var Mode = ModeAuto - -const ( - // ModeAuto detect console capabilities - ModeAuto = "auto" - // ModeTTY use terminal capability for advanced rendering - ModeTTY = "tty" - // ModePlain dump raw events to output - ModePlain = "plain" - // ModeQuiet don't display events - ModeQuiet = "quiet" - // ModeJSON outputs a machine-readable JSON stream - ModeJSON = "json" -) diff --git a/cmd/display/plain.go b/cmd/display/plain.go deleted file mode 100644 index 16f2816c011..00000000000 --- a/cmd/display/plain.go +++ /dev/null @@ -1,56 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "context" - "fmt" - "io" - - "github.com/docker/compose/v5/pkg/api" -) - -func Plain(out io.Writer) api.EventProcessor { - return &plainWriter{ - out: out, - } -} - -type plainWriter struct { - out io.Writer - dryRun bool -} - -func (p *plainWriter) Start(ctx context.Context, operation string) { -} - -func (p *plainWriter) Event(e api.Resource) { - prefix := "" - if p.dryRun { - prefix = DRYRUN_PREFIX - } - _, _ = fmt.Fprintln(p.out, prefix, e.ID, e.Text, e.Details) -} - -func (p *plainWriter) On(events ...api.Resource) { - for _, e := range events { - p.Event(e) - } -} - -func (p *plainWriter) Done(_ string, _ bool) { -} diff --git a/cmd/display/quiet.go b/cmd/display/quiet.go deleted file mode 100644 index 8e1537d8061..00000000000 --- a/cmd/display/quiet.go +++ /dev/null @@ -1,38 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "context" - - "github.com/docker/compose/v5/pkg/api" -) - -func Quiet() api.EventProcessor { - return &quiet{} -} - -type quiet struct{} - -func (q *quiet) Start(_ context.Context, _ string) { -} - -func (q *quiet) Done(_ string, _ bool) { -} - -func (q *quiet) On(_ ...api.Resource) { -} diff --git a/cmd/display/spinner.go b/cmd/display/spinner.go deleted file mode 100644 index e476deae80f..00000000000 --- a/cmd/display/spinner.go +++ /dev/null @@ -1,70 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "runtime" - "time" -) - -type Spinner struct { - time time.Time - index int - chars []string - stop bool - done string -} - -func NewSpinner() *Spinner { - chars := []string{ - "⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏", - } - done := "⠿" - - if runtime.GOOS == "windows" { - chars = []string{"-"} - done = "-" - } - - return &Spinner{ - index: 0, - time: time.Now(), - chars: chars, - done: done, - } -} - -func (s *Spinner) String() string { - if s.stop { - return s.done - } - - d := time.Since(s.time) - if d.Milliseconds() > 100 { - s.index = (s.index + 1) % len(s.chars) - } - - return s.chars[s.index] -} - -func (s *Spinner) Stop() { - s.stop = true -} - -func (s *Spinner) Restart() { - s.stop = false -} diff --git a/cmd/display/tty.go b/cmd/display/tty.go deleted file mode 100644 index 3a69ebaadc7..00000000000 --- a/cmd/display/tty.go +++ /dev/null @@ -1,664 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "context" - "fmt" - "io" - "iter" - "slices" - "strings" - "sync" - "time" - "unicode/utf8" - - "github.com/buger/goterm" - "github.com/docker/go-units" - "github.com/morikuni/aec" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -// Full creates an EventProcessor that render advanced UI within a terminal. -// On Start, TUI lists task with a progress timer -func Full(out io.Writer, info io.Writer, detached bool) api.EventProcessor { - return &ttyWriter{ - out: out, - info: info, - tasks: map[string]*task{}, - done: make(chan bool), - mtx: &sync.Mutex{}, - detached: detached, - } -} - -type ttyWriter struct { - out io.Writer - ids []string // tasks ids ordered as first event appeared - tasks map[string]*task - repeated bool - numLines int - done chan bool - mtx *sync.Mutex - dryRun bool // FIXME(ndeloof) (re)implement support for dry-run - operation string - ticker *time.Ticker - suspended bool - info io.Writer - detached bool -} - -type task struct { - ID string - parent string // the resource this task receives updates from - other parents will be ignored - parents utils.Set[string] // all resources to depend on this task - startTime time.Time - endTime time.Time - text string - details string - status api.EventStatus - current int64 - percent int - total int64 - spinner *Spinner -} - -func newTask(e api.Resource) task { - t := task{ - ID: e.ID, - parents: utils.NewSet[string](), - startTime: time.Now(), - text: e.Text, - details: e.Details, - status: e.Status, - current: e.Current, - percent: e.Percent, - total: e.Total, - spinner: NewSpinner(), - } - if e.ParentID != "" { - t.parent = e.ParentID - t.parents.Add(e.ParentID) - } - if e.Status == api.Done || e.Status == api.Error { - t.stop() - } - return t -} - -// update adjusts task state based on last received event -func (t *task) update(e api.Resource) { - if e.ParentID != "" { - t.parents.Add(e.ParentID) - // we may receive same event from distinct parents (typically: images sharing layers) - // to avoid status to flicker, only accept updates from our first declared parent - if t.parent != e.ParentID { - return - } - } - - // update task based on received event - switch e.Status { - case api.Done, api.Error, api.Warning: - if t.status != e.Status { - t.stop() - } - case api.Working: - t.hasMore() - } - t.status = e.Status - t.text = e.Text - t.details = e.Details - // progress can only go up - if e.Total > t.total { - t.total = e.Total - } - if e.Current > t.current { - t.current = e.Current - } - if e.Percent > t.percent { - t.percent = e.Percent - } -} - -func (t *task) stop() { - t.endTime = time.Now() - t.spinner.Stop() -} - -func (t *task) hasMore() { - t.spinner.Restart() -} - -func (t *task) Completed() bool { - switch t.status { - case api.Done, api.Error, api.Warning: - return true - default: - return false - } -} - -func (w *ttyWriter) Start(ctx context.Context, operation string) { - w.ticker = time.NewTicker(100 * time.Millisecond) - w.operation = operation - go func() { - for { - select { - case <-ctx.Done(): - // interrupted - w.ticker.Stop() - return - case <-w.done: - return - case <-w.ticker.C: - w.print() - } - } - }() -} - -func (w *ttyWriter) Done(operation string, success bool) { - w.print() - w.mtx.Lock() - defer w.mtx.Unlock() - w.ticker.Stop() - w.operation = "" - w.done <- true -} - -func (w *ttyWriter) On(events ...api.Resource) { - w.mtx.Lock() - defer w.mtx.Unlock() - for _, e := range events { - if e.ID == "Compose" { - _, _ = fmt.Fprintln(w.info, ErrorColor(e.Details)) - continue - } - - if w.operation != "start" && (e.Text == api.StatusStarted || e.Text == api.StatusStarting) && !w.detached { - // skip those events to avoid mix with container logs - continue - } - w.event(e) - } -} - -func (w *ttyWriter) event(e api.Resource) { - // Suspend print while a build is in progress, to avoid collision with buildkit Display - if e.Text == api.StatusBuilding { - w.ticker.Stop() - w.suspended = true - } else if w.suspended { - w.ticker.Reset(100 * time.Millisecond) - w.suspended = false - } - - if last, ok := w.tasks[e.ID]; ok { - last.update(e) - } else { - t := newTask(e) - w.tasks[e.ID] = &t - w.ids = append(w.ids, e.ID) - } - w.printEvent(e) -} - -func (w *ttyWriter) printEvent(e api.Resource) { - if w.operation != "" { - // event will be displayed by progress UI on ticker's ticks - return - } - - var color colorFunc - switch e.Status { - case api.Working: - color = SuccessColor - case api.Done: - color = SuccessColor - case api.Warning: - color = WarningColor - case api.Error: - color = ErrorColor - } - _, _ = fmt.Fprintf(w.out, "%s %s %s\n", e.ID, color(e.Text), e.Details) -} - -func (w *ttyWriter) parentTasks() iter.Seq[*task] { - return func(yield func(*task) bool) { - for _, id := range w.ids { // iterate on ids to enforce a consistent order - t := w.tasks[id] - if len(t.parents) == 0 { - yield(t) - } - } - } -} - -func (w *ttyWriter) childrenTasks(parent string) iter.Seq[*task] { - return func(yield func(*task) bool) { - for _, id := range w.ids { // iterate on ids to enforce a consistent order - t := w.tasks[id] - if t.parents.Has(parent) { - yield(t) - } - } - } -} - -// lineData holds pre-computed formatting for a task line -type lineData struct { - spinner string // rendered spinner with color - prefix string // dry-run prefix if any - taskID string // possibly abbreviated - progress string // progress bar and size info - status string // rendered status with color - details string // possibly abbreviated - timer string // rendered timer with color - statusPad int // padding before status to align - timerPad int // padding before timer to align - statusColor colorFunc -} - -func (w *ttyWriter) print() { - terminalWidth := goterm.Width() - terminalHeight := goterm.Height() - if terminalWidth <= 0 { - terminalWidth = 80 - } - if terminalHeight <= 0 { - terminalHeight = 24 - } - w.printWithDimensions(terminalWidth, terminalHeight) -} - -func (w *ttyWriter) printWithDimensions(terminalWidth, terminalHeight int) { - w.mtx.Lock() - defer w.mtx.Unlock() - if len(w.tasks) == 0 { - return - } - - up := w.numLines + 1 - if !w.repeated { - up-- - w.repeated = true - } - b := aec.NewBuilder( - aec.Hide, // Hide the cursor while we are printing - aec.Up(uint(up)), - aec.Column(0), - ) - _, _ = fmt.Fprint(w.out, b.ANSI) - defer func() { - _, _ = fmt.Fprint(w.out, aec.Show) - }() - - firstLine := fmt.Sprintf("[+] %s %d/%d", w.operation, numDone(w.tasks), len(w.tasks)) - _, _ = fmt.Fprintln(w.out, firstLine) - - // Collect parent tasks in original order - allTasks := slices.Collect(w.parentTasks()) - - // Available lines: terminal height - 2 (header line + potential "more" line) - maxLines := terminalHeight - 2 - if maxLines < 1 { - maxLines = 1 - } - - showMore := len(allTasks) > maxLines - tasksToShow := allTasks - if showMore { - tasksToShow = allTasks[:maxLines-1] // Reserve one line for "more" message - } - - // collect line data and compute timerLen - lines := make([]lineData, len(tasksToShow)) - var timerLen int - for i, t := range tasksToShow { - lines[i] = w.prepareLineData(t) - if len(lines[i].timer) > timerLen { - timerLen = len(lines[i].timer) - } - } - - // shorten details/taskID to fit terminal width - w.adjustLineWidth(lines, timerLen, terminalWidth) - - // compute padding - w.applyPadding(lines, terminalWidth, timerLen) - - // Render lines - numLines := 0 - for _, l := range lines { - _, _ = fmt.Fprint(w.out, lineText(l)) - numLines++ - } - - if showMore { - moreCount := len(allTasks) - len(tasksToShow) - moreText := fmt.Sprintf(" ... %d more", moreCount) - pad := terminalWidth - len(moreText) - if pad < 0 { - pad = 0 - } - _, _ = fmt.Fprintf(w.out, "%s%s\n", moreText, strings.Repeat(" ", pad)) - numLines++ - } - - // Clear any remaining lines from previous render - for i := numLines; i < w.numLines; i++ { - _, _ = fmt.Fprintln(w.out, strings.Repeat(" ", terminalWidth)) - numLines++ - } - w.numLines = numLines -} - -func (w *ttyWriter) applyPadding(lines []lineData, terminalWidth int, timerLen int) { - var maxBeforeStatus int - for i := range lines { - l := &lines[i] - // Width before statusPad: space(1) + spinner(1) + prefix + space(1) + taskID + progress - beforeStatus := 3 + lenAnsi(l.prefix) + utf8.RuneCountInString(l.taskID) + lenAnsi(l.progress) - if beforeStatus > maxBeforeStatus { - maxBeforeStatus = beforeStatus - } - } - - for i, l := range lines { - // Position before statusPad: space(1) + spinner(1) + prefix + space(1) + taskID + progress - beforeStatus := 3 + lenAnsi(l.prefix) + utf8.RuneCountInString(l.taskID) + lenAnsi(l.progress) - // statusPad aligns status; lineText adds 1 more space after statusPad - l.statusPad = maxBeforeStatus - beforeStatus - - // Format: beforeStatus + statusPad + space(1) + status - lineLen := beforeStatus + l.statusPad + 1 + utf8.RuneCountInString(l.status) - if l.details != "" { - lineLen += 1 + utf8.RuneCountInString(l.details) - } - l.timerPad = terminalWidth - lineLen - timerLen - if l.timerPad < 1 { - l.timerPad = 1 - } - lines[i] = l - - } -} - -func (w *ttyWriter) adjustLineWidth(lines []lineData, timerLen int, terminalWidth int) { - const minIDLen = 10 - maxStatusLen := maxStatusLength(lines) - - // Iteratively truncate until all lines fit - for range 100 { // safety limit - maxBeforeStatus := maxBeforeStatusWidth(lines) - overflow := computeOverflow(lines, maxBeforeStatus, maxStatusLen, timerLen, terminalWidth) - - if overflow <= 0 { - break - } - - // First try to truncate details, then taskID - if !truncateDetails(lines, overflow) && !truncateLongestTaskID(lines, overflow, minIDLen) { - break // Can't truncate further - } - } -} - -// maxStatusLength returns the maximum status text length across all lines. -func maxStatusLength(lines []lineData) int { - var maxLen int - for i := range lines { - if len(lines[i].status) > maxLen { - maxLen = len(lines[i].status) - } - } - return maxLen -} - -// maxBeforeStatusWidth computes the maximum width before statusPad across all lines. -// This is: space(1) + spinner(1) + prefix + space(1) + taskID + progress -func maxBeforeStatusWidth(lines []lineData) int { - var maxWidth int - for i := range lines { - l := &lines[i] - width := 3 + lenAnsi(l.prefix) + len(l.taskID) + lenAnsi(l.progress) - if width > maxWidth { - maxWidth = width - } - } - return maxWidth -} - -// computeOverflow calculates how many characters the widest line exceeds the terminal width. -// Returns 0 or negative if all lines fit. -func computeOverflow(lines []lineData, maxBeforeStatus, maxStatusLen, timerLen, terminalWidth int) int { - var maxOverflow int - for i := range lines { - l := &lines[i] - detailsLen := len(l.details) - if detailsLen > 0 { - detailsLen++ // space before details - } - // Line width: maxBeforeStatus + space(1) + status + details + minTimerPad(1) + timer - lineWidth := maxBeforeStatus + 1 + maxStatusLen + detailsLen + 1 + timerLen - overflow := lineWidth - terminalWidth - if overflow > maxOverflow { - maxOverflow = overflow - } - } - return maxOverflow -} - -// truncateDetails tries to truncate the first line's details to reduce overflow. -// Returns true if any truncation was performed. -func truncateDetails(lines []lineData, overflow int) bool { - for i := range lines { - l := &lines[i] - if len(l.details) > 3 { - reduction := overflow - if reduction > len(l.details)-3 { - reduction = len(l.details) - 3 - } - l.details = l.details[:len(l.details)-reduction-3] + "..." - return true - } else if l.details != "" { - l.details = "" - return true - } - } - return false -} - -// truncateLongestTaskID truncates the longest taskID to reduce overflow. -// Returns true if truncation was performed. -func truncateLongestTaskID(lines []lineData, overflow, minIDLen int) bool { - longestIdx := -1 - longestLen := minIDLen - for i := range lines { - if len(lines[i].taskID) > longestLen { - longestLen = len(lines[i].taskID) - longestIdx = i - } - } - - if longestIdx < 0 { - return false - } - - l := &lines[longestIdx] - reduction := overflow + 3 // account for "..." - newLen := len(l.taskID) - reduction - if newLen < minIDLen-3 { - newLen = minIDLen - 3 - } - if newLen > 0 { - l.taskID = l.taskID[:newLen] + "..." - } - return true -} - -func (w *ttyWriter) prepareLineData(t *task) lineData { - endTime := time.Now() - if t.status != api.Working { - endTime = t.startTime - if (t.endTime != time.Time{}) { - endTime = t.endTime - } - } - - prefix := "" - if w.dryRun { - prefix = PrefixColor(DRYRUN_PREFIX) - } - - elapsed := endTime.Sub(t.startTime).Seconds() - - var ( - hideDetails bool - total int64 - current int64 - completion []string - ) - - // only show the aggregated progress while the root operation is in-progress - if t.status == api.Working { - for child := range w.childrenTasks(t.ID) { - if child.status == api.Working && child.total == 0 { - hideDetails = true - } - total += child.total - current += child.current - r := len(percentChars) - 1 - p := child.percent - if p > 100 { - p = 100 - } - completion = append(completion, percentChars[r*p/100]) - } - } - - if total == 0 { - hideDetails = true - } - - var progress string - if len(completion) > 0 { - progress = " [" + SuccessColor(strings.Join(completion, "")) + "]" - if !hideDetails { - progress += fmt.Sprintf(" %7s / %-7s", units.HumanSize(float64(current)), units.HumanSize(float64(total))) - } - } - - return lineData{ - spinner: spinner(t), - prefix: prefix, - taskID: t.ID, - progress: progress, - status: t.text, - statusColor: colorFn(t.status), - details: t.details, - timer: fmt.Sprintf("%.1fs", elapsed), - } -} - -func lineText(l lineData) string { - var sb strings.Builder - sb.WriteString(" ") - sb.WriteString(l.spinner) - sb.WriteString(l.prefix) - sb.WriteString(" ") - sb.WriteString(l.taskID) - sb.WriteString(l.progress) - sb.WriteString(strings.Repeat(" ", l.statusPad)) - sb.WriteString(" ") - sb.WriteString(l.statusColor(l.status)) - if l.details != "" { - sb.WriteString(" ") - sb.WriteString(l.details) - } - sb.WriteString(strings.Repeat(" ", l.timerPad)) - sb.WriteString(TimerColor(l.timer)) - sb.WriteString("\n") - return sb.String() -} - -var ( - spinnerDone = "✔" - spinnerWarning = "!" - spinnerError = "✘" -) - -func spinner(t *task) string { - switch t.status { - case api.Done: - return SuccessColor(spinnerDone) - case api.Warning: - return WarningColor(spinnerWarning) - case api.Error: - return ErrorColor(spinnerError) - default: - return CountColor(t.spinner.String()) - } -} - -func colorFn(s api.EventStatus) colorFunc { - switch s { - case api.Done: - return SuccessColor - case api.Warning: - return WarningColor - case api.Error: - return ErrorColor - default: - return nocolor - } -} - -func numDone(tasks map[string]*task) int { - i := 0 - for _, t := range tasks { - if t.status != api.Working { - i++ - } - } - return i -} - -// lenAnsi count of user-perceived characters in ANSI string. -func lenAnsi(s string) int { - length := 0 - ansiCode := false - for _, r := range s { - if r == '\x1b' { - ansiCode = true - continue - } - if ansiCode && r == 'm' { - ansiCode = false - continue - } - if !ansiCode { - length++ - } - } - return length -} - -var percentChars = strings.Split("⠀⡀⣀⣄⣤⣦⣶⣷⣿", "") diff --git a/cmd/display/tty_test.go b/cmd/display/tty_test.go deleted file mode 100644 index 875f5f029f7..00000000000 --- a/cmd/display/tty_test.go +++ /dev/null @@ -1,424 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package display - -import ( - "bytes" - "strings" - "sync" - "testing" - "time" - "unicode/utf8" - - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/api" -) - -func newTestWriter() (*ttyWriter, *bytes.Buffer) { - var buf bytes.Buffer - w := &ttyWriter{ - out: &buf, - info: &buf, - tasks: map[string]*task{}, - done: make(chan bool), - mtx: &sync.Mutex{}, - operation: "pull", - } - return w, &buf -} - -func addTask(w *ttyWriter, id, text, details string, status api.EventStatus) { - t := &task{ - ID: id, - parents: make(map[string]struct{}), - startTime: time.Now(), - text: text, - details: details, - status: status, - spinner: NewSpinner(), - } - w.tasks[id] = t - w.ids = append(w.ids, id) -} - -// extractLines parses the output buffer and returns lines without ANSI control sequences -func extractLines(buf *bytes.Buffer) []string { - content := buf.String() - // Split by newline - rawLines := strings.Split(content, "\n") - var lines []string - for _, line := range rawLines { - // Skip empty lines and lines that are just ANSI codes - if lenAnsi(line) > 0 { - lines = append(lines, line) - } - } - return lines -} - -func TestPrintWithDimensions_LinesFitTerminalWidth(t *testing.T) { - testCases := []struct { - name string - taskID string - status string - details string - terminalWidth int - }{ - { - name: "short task fits wide terminal", - taskID: "Image foo", - status: "Pulling", - details: "layer abc123", - terminalWidth: 100, - }, - { - name: "long details truncated to fit", - taskID: "Image foo", - status: "Pulling", - details: "downloading layer sha256:abc123def456789xyz0123456789abcdef", - terminalWidth: 50, - }, - { - name: "long taskID truncated to fit", - taskID: "very-long-image-name-that-exceeds-terminal-width", - status: "Pulling", - details: "", - terminalWidth: 40, - }, - { - name: "both long taskID and details", - taskID: "my-very-long-service-name-here", - status: "Downloading", - details: "layer sha256:abc123def456789xyz0123456789", - terminalWidth: 50, - }, - { - name: "narrow terminal", - taskID: "service-name", - status: "Pulling", - details: "some details", - terminalWidth: 35, - }, - } - - for _, tc := range testCases { - t.Run(tc.name, func(t *testing.T) { - w, buf := newTestWriter() - addTask(w, tc.taskID, tc.status, tc.details, api.Working) - - w.printWithDimensions(tc.terminalWidth, 24) - - lines := extractLines(buf) - for i, line := range lines { - lineLen := lenAnsi(line) - assert.Assert(t, lineLen <= tc.terminalWidth, - "line %d has length %d which exceeds terminal width %d: %q", - i, lineLen, tc.terminalWidth, line) - } - }) - } -} - -func TestPrintWithDimensions_MultipleTasksFitTerminalWidth(t *testing.T) { - w, buf := newTestWriter() - - // Add multiple tasks with varying lengths - addTask(w, "Image nginx", "Pulling", "layer sha256:abc123", api.Working) - addTask(w, "Image postgres-database", "Pulling", "downloading", api.Working) - addTask(w, "Image redis", "Pulled", "", api.Done) - - terminalWidth := 60 - w.printWithDimensions(terminalWidth, 24) - - lines := extractLines(buf) - for i, line := range lines { - lineLen := lenAnsi(line) - assert.Assert(t, lineLen <= terminalWidth, - "line %d has length %d which exceeds terminal width %d: %q", - i, lineLen, terminalWidth, line) - } -} - -func TestPrintWithDimensions_VeryNarrowTerminal(t *testing.T) { - w, buf := newTestWriter() - addTask(w, "Image nginx", "Pulling", "details", api.Working) - - terminalWidth := 30 - w.printWithDimensions(terminalWidth, 24) - - lines := extractLines(buf) - for i, line := range lines { - lineLen := lenAnsi(line) - assert.Assert(t, lineLen <= terminalWidth, - "line %d has length %d which exceeds terminal width %d: %q", - i, lineLen, terminalWidth, line) - } -} - -func TestPrintWithDimensions_TaskWithProgress(t *testing.T) { - w, buf := newTestWriter() - - // Create parent task - parent := &task{ - ID: "Image nginx", - parents: make(map[string]struct{}), - startTime: time.Now(), - text: "Pulling", - status: api.Working, - spinner: NewSpinner(), - } - w.tasks["Image nginx"] = parent - w.ids = append(w.ids, "Image nginx") - - // Create child tasks to trigger progress display - for i := 0; i < 3; i++ { - child := &task{ - ID: "layer" + string(rune('a'+i)), - parents: map[string]struct{}{"Image nginx": {}}, - startTime: time.Now(), - text: "Downloading", - status: api.Working, - total: 1000, - current: 500, - percent: 50, - spinner: NewSpinner(), - } - w.tasks[child.ID] = child - w.ids = append(w.ids, child.ID) - } - - terminalWidth := 80 - w.printWithDimensions(terminalWidth, 24) - - lines := extractLines(buf) - for i, line := range lines { - lineLen := lenAnsi(line) - assert.Assert(t, lineLen <= terminalWidth, - "line %d has length %d which exceeds terminal width %d: %q", - i, lineLen, terminalWidth, line) - } -} - -func TestAdjustLineWidth_DetailsCorrectlyTruncated(t *testing.T) { - w := &ttyWriter{} - lines := []lineData{ - { - taskID: "Image foo", - status: "Pulling", - details: "downloading layer sha256:abc123def456789xyz", - }, - } - - terminalWidth := 50 - timerLen := 5 - w.adjustLineWidth(lines, timerLen, terminalWidth) - - // Verify the line fits - detailsLen := len(lines[0].details) - if detailsLen > 0 { - detailsLen++ // space before details - } - // widthWithoutDetails = 5 + prefix(0) + taskID(9) + progress(0) + status(7) + timer(5) = 26 - lineWidth := 5 + len(lines[0].taskID) + len(lines[0].status) + detailsLen + timerLen - - assert.Assert(t, lineWidth <= terminalWidth, - "line width %d should not exceed terminal width %d (taskID=%q, details=%q)", - lineWidth, terminalWidth, lines[0].taskID, lines[0].details) - - // Verify details were truncated (not removed entirely) - assert.Assert(t, lines[0].details != "", "details should be truncated, not removed") - assert.Assert(t, strings.HasSuffix(lines[0].details, "..."), "truncated details should end with ...") -} - -func TestAdjustLineWidth_TaskIDCorrectlyTruncated(t *testing.T) { - w := &ttyWriter{} - lines := []lineData{ - { - taskID: "very-long-image-name-that-exceeds-minimum-length", - status: "Pulling", - details: "", - }, - } - - terminalWidth := 40 - timerLen := 5 - w.adjustLineWidth(lines, timerLen, terminalWidth) - - lineWidth := 5 + len(lines[0].taskID) + 7 + timerLen - - assert.Assert(t, lineWidth <= terminalWidth, - "line width %d should not exceed terminal width %d (taskID=%q)", - lineWidth, terminalWidth, lines[0].taskID) - - assert.Assert(t, strings.HasSuffix(lines[0].taskID, "..."), "truncated taskID should end with ...") -} - -func TestAdjustLineWidth_NoTruncationNeeded(t *testing.T) { - w := &ttyWriter{} - originalDetails := "short" - originalTaskID := "Image foo" - lines := []lineData{ - { - taskID: originalTaskID, - status: "Pulling", - details: originalDetails, - }, - } - - // Wide terminal, nothing should be truncated - w.adjustLineWidth(lines, 5, 100) - - assert.Equal(t, originalTaskID, lines[0].taskID, "taskID should not be modified") - assert.Equal(t, originalDetails, lines[0].details, "details should not be modified") -} - -func TestAdjustLineWidth_DetailsRemovedWhenTooShort(t *testing.T) { - w := &ttyWriter{} - lines := []lineData{ - { - taskID: "Image foo", - status: "Pulling", - details: "abc", // Very short, can't be meaningfully truncated - }, - } - - // Terminal so narrow that even minimal details + "..." wouldn't help - w.adjustLineWidth(lines, 5, 28) - - assert.Equal(t, "", lines[0].details, "details should be removed entirely when too short to truncate") -} - -// stripAnsi removes ANSI escape codes from a string -func stripAnsi(s string) string { - var result strings.Builder - inAnsi := false - for _, r := range s { - if r == '\x1b' { - inAnsi = true - continue - } - if inAnsi { - // ANSI sequences end with a letter (m, h, l, G, etc.) - if (r >= 'A' && r <= 'Z') || (r >= 'a' && r <= 'z') { - inAnsi = false - } - continue - } - result.WriteRune(r) - } - return result.String() -} - -func TestPrintWithDimensions_PulledAndPullingWithLongIDs(t *testing.T) { - w, buf := newTestWriter() - - // Add a completed task with long ID - completedTask := &task{ - ID: "Image docker.io/library/nginx-long-name", - parents: make(map[string]struct{}), - startTime: time.Now().Add(-2 * time.Second), - endTime: time.Now(), - text: "Pulled", - status: api.Done, - spinner: NewSpinner(), - } - completedTask.spinner.Stop() - w.tasks[completedTask.ID] = completedTask - w.ids = append(w.ids, completedTask.ID) - - // Add a pending task with long ID - pendingTask := &task{ - ID: "Image docker.io/library/postgres-database", - parents: make(map[string]struct{}), - startTime: time.Now(), - text: "Pulling", - status: api.Working, - spinner: NewSpinner(), - } - w.tasks[pendingTask.ID] = pendingTask - w.ids = append(w.ids, pendingTask.ID) - - terminalWidth := 50 - w.printWithDimensions(terminalWidth, 24) - - // Strip all ANSI codes from output and split by newline - stripped := stripAnsi(buf.String()) - lines := strings.Split(stripped, "\n") - - // Filter non-empty lines - var nonEmptyLines []string - for _, line := range lines { - if strings.TrimSpace(line) != "" { - nonEmptyLines = append(nonEmptyLines, line) - } - } - - // Expected output format (50 runes per task line) - expected := `[+] pull 1/2 - ✔ Image docker.io/library/nginx-l... Pulled 2.0s - ⠋ Image docker.io/library/postgre... Pulling 0.0s` - - expectedLines := strings.Split(expected, "\n") - - // Debug output - t.Logf("Actual output:\n") - for i, line := range nonEmptyLines { - t.Logf(" line %d (%2d runes): %q", i, utf8.RuneCountInString(line), line) - } - - // Verify number of lines - assert.Equal(t, len(expectedLines), len(nonEmptyLines), "number of lines should match") - - // Verify each line matches expected - for i, line := range nonEmptyLines { - if i < len(expectedLines) { - assert.Equal(t, expectedLines[i], line, - "line %d should match expected", i) - } - } - - // Verify task lines fit within terminal width (strict - no tolerance) - for i, line := range nonEmptyLines { - if i > 0 { // Skip header line - runeCount := utf8.RuneCountInString(line) - assert.Assert(t, runeCount <= terminalWidth, - "line %d has %d runes which exceeds terminal width %d: %q", - i, runeCount, terminalWidth, line) - } - } -} - -func TestLenAnsi(t *testing.T) { - testCases := []struct { - input string - expected int - }{ - {"hello", 5}, - {"\x1b[32mhello\x1b[0m", 5}, - {"\x1b[1;32mgreen\x1b[0m text", 10}, - {"", 0}, - {"\x1b[0m", 0}, - } - - for _, tc := range testCases { - t.Run(tc.input, func(t *testing.T) { - result := lenAnsi(tc.input) - assert.Equal(t, tc.expected, result) - }) - } -} diff --git a/cmd/formatter/ansi.go b/cmd/formatter/ansi.go deleted file mode 100644 index ad1031946ef..00000000000 --- a/cmd/formatter/ansi.go +++ /dev/null @@ -1,99 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "fmt" - - "github.com/acarl005/stripansi" - "github.com/morikuni/aec" -) - -var disableAnsi bool - -func saveCursor() { - if disableAnsi { - return - } - // see https://github.com/morikuni/aec/pull/5 - fmt.Print(aec.Save) -} - -func restoreCursor() { - if disableAnsi { - return - } - // see https://github.com/morikuni/aec/pull/5 - fmt.Print(aec.Restore) -} - -func showCursor() { - if disableAnsi { - return - } - fmt.Print(aec.Show) -} - -func moveCursor(y, x int) { - if disableAnsi { - return - } - fmt.Print(aec.Position(uint(y), uint(x))) -} - -func carriageReturn() { - if disableAnsi { - return - } - fmt.Print(aec.Column(0)) -} - -func clearLine() { - if disableAnsi { - return - } - // Does not move cursor from its current position - fmt.Print(aec.EraseLine(aec.EraseModes.Tail)) -} - -func moveCursorUp(lines int) { - if disableAnsi { - return - } - // Does not add new lines - fmt.Print(aec.Up(uint(lines))) -} - -func moveCursorDown(lines int) { - if disableAnsi { - return - } - // Does not add new lines - fmt.Print(aec.Down(uint(lines))) -} - -func newLine() { - // Like \n - fmt.Print("\012") -} - -func lenAnsi(s string) int { - // len has into consideration ansi codes, if we want - // the len of the actual len(string) we need to strip - // all ansi codes - return len(stripansi.Strip(s)) -} diff --git a/cmd/formatter/colors.go b/cmd/formatter/colors.go deleted file mode 100644 index ea0e1a26362..00000000000 --- a/cmd/formatter/colors.go +++ /dev/null @@ -1,148 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "fmt" - "strconv" - "strings" - "sync" - - "github.com/docker/cli/cli/command" -) - -var names = []string{ - "grey", - "red", - "green", - "yellow", - "blue", - "magenta", - "cyan", - "white", -} - -const ( - BOLD = "1" - FAINT = "2" - ITALIC = "3" - UNDERLINE = "4" -) - -const ( - RESET = "0" - CYAN = "36" -) - -const ( - // Never use ANSI codes - Never = "never" - - // Always use ANSI codes - Always = "always" - - // Auto detect terminal is a tty and can use ANSI codes - Auto = "auto" -) - -// ansiColorOffset is the offset for basic foreground colors in ANSI escape codes. -const ansiColorOffset = 30 - -// SetANSIMode configure formatter for colored output on ANSI-compliant console -func SetANSIMode(streams command.Streams, ansi string) { - if !useAnsi(streams, ansi) { - nextColor = func() colorFunc { - return monochrome - } - disableAnsi = true - } -} - -func useAnsi(streams command.Streams, ansi string) bool { - switch ansi { - case Always: - return true - case Auto: - return streams.Out().IsTerminal() - } - return false -} - -// colorFunc use ANSI codes to render colored text on console -type colorFunc func(s string) string - -var monochrome = func(s string) string { - return s -} - -func ansiColor(code, s string, formatOpts ...string) string { - return fmt.Sprintf("%s%s%s", ansiColorCode(code, formatOpts...), s, ansiColorCode("0")) -} - -// Everything about ansiColorCode color https://hyperskill.org/learn/step/18193 -func ansiColorCode(code string, formatOpts ...string) string { - var sb strings.Builder - sb.WriteString("\033[") - for _, c := range formatOpts { - sb.WriteString(c) - sb.WriteString(";") - } - sb.WriteString(code) - sb.WriteString("m") - return sb.String() -} - -func makeColorFunc(code string) colorFunc { - return func(s string) string { - return ansiColor(code, s) - } -} - -var ( - nextColor = rainbowColor - rainbow []colorFunc - currentIndex = 0 - mutex sync.Mutex -) - -func rainbowColor() colorFunc { - mutex.Lock() - defer mutex.Unlock() - result := rainbow[currentIndex] - currentIndex = (currentIndex + 1) % len(rainbow) - return result -} - -func init() { - colors := map[string]colorFunc{} - for i, name := range names { - colors[name] = makeColorFunc(strconv.Itoa(ansiColorOffset + i)) - colors["intense_"+name] = makeColorFunc(strconv.Itoa(ansiColorOffset+i) + ";1") - } - rainbow = []colorFunc{ - colors["cyan"], - colors["yellow"], - colors["green"], - colors["magenta"], - colors["blue"], - colors["intense_cyan"], - colors["intense_yellow"], - colors["intense_green"], - colors["intense_magenta"], - colors["intense_blue"], - } -} diff --git a/cmd/formatter/consts.go b/cmd/formatter/consts.go deleted file mode 100644 index 0ae447c432f..00000000000 --- a/cmd/formatter/consts.go +++ /dev/null @@ -1,30 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -const ( - // JSON Print in JSON format - JSON = "json" - // TemplateLegacyJSON the legacy json formatting value using go template - TemplateLegacyJSON = "{{json.}}" - // PRETTY is the constant for default formats on list commands - // - // Deprecated: use TABLE - PRETTY = "pretty" - // TABLE Print output in table format with column headers (default) - TABLE = "table" -) diff --git a/cmd/formatter/container.go b/cmd/formatter/container.go deleted file mode 100644 index 488caf4afad..00000000000 --- a/cmd/formatter/container.go +++ /dev/null @@ -1,288 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "fmt" - "strconv" - "strings" - "time" - - "github.com/docker/cli/cli/command/formatter" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/pkg/stringid" - "github.com/docker/go-units" - - "github.com/docker/compose/v5/pkg/api" -) - -const ( - defaultContainerTableFormat = "table {{.Name}}\t{{.Image}}\t{{.Command}}\t{{.Service}}\t{{.RunningFor}}\t{{.Status}}\t{{.Ports}}" - - nameHeader = "NAME" - projectHeader = "PROJECT" - serviceHeader = "SERVICE" - commandHeader = "COMMAND" - runningForHeader = "CREATED" - mountsHeader = "MOUNTS" - localVolumes = "LOCAL VOLUMES" - networksHeader = "NETWORKS" -) - -// NewContainerFormat returns a Format for rendering using a Context -func NewContainerFormat(source string, quiet bool, size bool) formatter.Format { - switch source { - case formatter.TableFormatKey, "": // table formatting is the default if none is set. - if quiet { - return formatter.DefaultQuietFormat - } - format := defaultContainerTableFormat - if size { - format += `\t{{.Size}}` - } - return formatter.Format(format) - case formatter.RawFormatKey: - if quiet { - return `container_id: {{.ID}}` - } - format := `container_id: {{.ID}} -image: {{.Image}} -command: {{.Command}} -created_at: {{.CreatedAt}} -state: {{- pad .State 1 0}} -status: {{- pad .Status 1 0}} -names: {{.Names}} -labels: {{- pad .Labels 1 0}} -ports: {{- pad .Ports 1 0}} -` - if size { - format += `size: {{.Size}}\n` - } - return formatter.Format(format) - default: // custom format - if quiet { - return formatter.DefaultQuietFormat - } - return formatter.Format(source) - } -} - -// ContainerWrite renders the context for a list of containers -func ContainerWrite(ctx formatter.Context, containers []api.ContainerSummary) error { - render := func(format func(subContext formatter.SubContext) error) error { - for _, container := range containers { - err := format(&ContainerContext{trunc: ctx.Trunc, c: container}) - if err != nil { - return err - } - } - return nil - } - return ctx.Write(NewContainerContext(), render) -} - -// ContainerContext is a struct used for rendering a list of containers in a Go template. -type ContainerContext struct { - formatter.HeaderContext - trunc bool - c api.ContainerSummary - - // FieldsUsed is used in the pre-processing step to detect which fields are - // used in the template. It's currently only used to detect use of the .Size - // field which (if used) automatically sets the '--size' option when making - // the API call. - FieldsUsed map[string]any -} - -// NewContainerContext creates a new context for rendering containers -func NewContainerContext() *ContainerContext { - containerCtx := ContainerContext{} - containerCtx.Header = formatter.SubHeaderContext{ - "ID": formatter.ContainerIDHeader, - "Name": nameHeader, - "Project": projectHeader, - "Service": serviceHeader, - "Image": formatter.ImageHeader, - "Command": commandHeader, - "CreatedAt": formatter.CreatedAtHeader, - "RunningFor": runningForHeader, - "Ports": formatter.PortsHeader, - "State": formatter.StateHeader, - "Status": formatter.StatusHeader, - "Size": formatter.SizeHeader, - "Labels": formatter.LabelsHeader, - } - return &containerCtx -} - -// MarshalJSON makes ContainerContext implement json.Marshaler -func (c *ContainerContext) MarshalJSON() ([]byte, error) { - return formatter.MarshalJSON(c) -} - -// ID returns the container's ID as a string. Depending on the `--no-trunc` -// option being set, the full or truncated ID is returned. -func (c *ContainerContext) ID() string { - if c.trunc { - return stringid.TruncateID(c.c.ID) - } - return c.c.ID -} - -func (c *ContainerContext) Name() string { - return c.c.Name -} - -// Names returns a comma-separated string of the container's names, with their -// slash (/) prefix stripped. Additional names for the container (related to the -// legacy `--link` feature) are omitted. -func (c *ContainerContext) Names() string { - names := formatter.StripNamePrefix(c.c.Names) - if c.trunc { - for _, name := range names { - if len(strings.Split(name, "/")) == 1 { - names = []string{name} - break - } - } - } - return strings.Join(names, ",") -} - -func (c *ContainerContext) Service() string { - return c.c.Service -} - -func (c *ContainerContext) Project() string { - return c.c.Project -} - -func (c *ContainerContext) Image() string { - return c.c.Image -} - -func (c *ContainerContext) Command() string { - command := c.c.Command - if c.trunc { - command = formatter.Ellipsis(command, 20) - } - return strconv.Quote(command) -} - -func (c *ContainerContext) CreatedAt() string { - return time.Unix(c.c.Created, 0).String() -} - -func (c *ContainerContext) RunningFor() string { - createdAt := time.Unix(c.c.Created, 0) - return units.HumanDuration(time.Now().UTC().Sub(createdAt)) + " ago" -} - -func (c *ContainerContext) ExitCode() int { - return c.c.ExitCode -} - -func (c *ContainerContext) State() string { - return c.c.State -} - -func (c *ContainerContext) Status() string { - return c.c.Status -} - -func (c *ContainerContext) Health() string { - return c.c.Health -} - -func (c *ContainerContext) Publishers() api.PortPublishers { - return c.c.Publishers -} - -func (c *ContainerContext) Ports() string { - var ports []container.Port - for _, publisher := range c.c.Publishers { - ports = append(ports, container.Port{ - IP: publisher.URL, - PrivatePort: uint16(publisher.TargetPort), - PublicPort: uint16(publisher.PublishedPort), - Type: publisher.Protocol, - }) - } - return formatter.DisplayablePorts(ports) -} - -// Labels returns a comma-separated string of labels present on the container. -func (c *ContainerContext) Labels() string { - if c.c.Labels == nil { - return "" - } - - var joinLabels []string - for k, v := range c.c.Labels { - joinLabels = append(joinLabels, fmt.Sprintf("%s=%s", k, v)) - } - return strings.Join(joinLabels, ",") -} - -// Label returns the value of the label with the given name or an empty string -// if the given label does not exist. -func (c *ContainerContext) Label(name string) string { - if c.c.Labels == nil { - return "" - } - return c.c.Labels[name] -} - -// Mounts returns a comma-separated string of mount names present on the container. -// If the trunc option is set, names can be truncated (ellipsized). -func (c *ContainerContext) Mounts() string { - var mounts []string - for _, name := range c.c.Mounts { - if c.trunc { - name = formatter.Ellipsis(name, 15) - } - mounts = append(mounts, name) - } - return strings.Join(mounts, ",") -} - -// LocalVolumes returns the number of volumes using the "local" volume driver. -func (c *ContainerContext) LocalVolumes() string { - return fmt.Sprintf("%d", c.c.LocalVolumes) -} - -// Networks returns a comma-separated string of networks that the container is -// attached to. -func (c *ContainerContext) Networks() string { - return strings.Join(c.c.Networks, ",") -} - -// Size returns the container's size and virtual size (e.g. "2B (virtual 21.5MB)") -func (c *ContainerContext) Size() string { - if c.FieldsUsed == nil { - c.FieldsUsed = map[string]any{} - } - c.FieldsUsed["Size"] = struct{}{} - srw := units.HumanSizeWithPrecision(float64(c.c.SizeRw), 3) - sv := units.HumanSizeWithPrecision(float64(c.c.SizeRootFs), 3) - - sf := srw - if c.c.SizeRootFs > 0 { - sf = fmt.Sprintf("%s (virtual %s)", srw, sv) - } - return sf -} diff --git a/cmd/formatter/formatter.go b/cmd/formatter/formatter.go deleted file mode 100644 index 6af92c478db..00000000000 --- a/cmd/formatter/formatter.go +++ /dev/null @@ -1,71 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "fmt" - "io" - "reflect" - "strings" - - "github.com/docker/compose/v5/pkg/api" -) - -// Print prints formatted lists in different formats -func Print(toJSON any, format string, outWriter io.Writer, writerFn func(w io.Writer), headers ...string) error { - switch strings.ToLower(format) { - case TABLE, PRETTY, "": - return PrintPrettySection(outWriter, writerFn, headers...) - case TemplateLegacyJSON: - switch reflect.TypeOf(toJSON).Kind() { - case reflect.Slice: - s := reflect.ValueOf(toJSON) - for i := 0; i < s.Len(); i++ { - obj := s.Index(i).Interface() - outJSON, err := ToJSON(obj, "", "") - if err != nil { - return err - } - _, _ = fmt.Fprint(outWriter, outJSON) - } - default: - outJSON, err := ToStandardJSON(toJSON) - if err != nil { - return err - } - _, _ = fmt.Fprintln(outWriter, outJSON) - } - case JSON: - switch reflect.TypeOf(toJSON).Kind() { - case reflect.Slice: - outJSON, err := ToJSON(toJSON, "", "") - if err != nil { - return err - } - _, _ = fmt.Fprint(outWriter, outJSON) - default: - outJSON, err := ToStandardJSON(toJSON) - if err != nil { - return err - } - _, _ = fmt.Fprintln(outWriter, outJSON) - } - default: - return fmt.Errorf("format value %q could not be parsed: %w", format, api.ErrParsingFailed) - } - return nil -} diff --git a/cmd/formatter/formatter_test.go b/cmd/formatter/formatter_test.go deleted file mode 100644 index 07b152559c1..00000000000 --- a/cmd/formatter/formatter_test.go +++ /dev/null @@ -1,78 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "bytes" - "fmt" - "io" - "testing" - - "go.uber.org/goleak" - "gotest.tools/v3/assert" -) - -type testStruct struct { - Name string - Status string -} - -// Print prints formatted lists in different formats -func TestPrint(t *testing.T) { - testList := []testStruct{ - { - Name: "myName1", - Status: "myStatus1", - }, - { - Name: "myName2", - Status: "myStatus2", - }, - } - - b := &bytes.Buffer{} - assert.NilError(t, Print(testList, TABLE, b, func(w io.Writer) { - for _, t := range testList { - _, _ = fmt.Fprintf(w, "%s\t%s\n", t.Name, t.Status) - } - }, "NAME", "STATUS")) - assert.Equal(t, b.String(), "NAME STATUS\nmyName1 myStatus1\nmyName2 myStatus2\n") - - b.Reset() - assert.NilError(t, Print(testList, JSON, b, func(w io.Writer) { - for _, t := range testList { - _, _ = fmt.Fprintf(w, "%s\t%s\n", t.Name, t.Status) - } - }, "NAME", "STATUS")) - assert.Equal(t, b.String(), `[{"Name":"myName1","Status":"myStatus1"},{"Name":"myName2","Status":"myStatus2"}] -`) - - b.Reset() - assert.NilError(t, Print(testList, TemplateLegacyJSON, b, func(w io.Writer) { - for _, t := range testList { - _, _ = fmt.Fprintf(w, "%s\t%s\n", t.Name, t.Status) - } - }, "NAME", "STATUS")) - json := b.String() - assert.Equal(t, json, `{"Name":"myName1","Status":"myStatus1"} -{"Name":"myName2","Status":"myStatus2"} -`) -} - -func TestColorsGoroutinesLeak(t *testing.T) { - goleak.VerifyNone(t) -} diff --git a/cmd/formatter/json.go b/cmd/formatter/json.go deleted file mode 100644 index b09e721aa27..00000000000 --- a/cmd/formatter/json.go +++ /dev/null @@ -1,39 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "bytes" - "encoding/json" -) - -const standardIndentation = " " - -// ToStandardJSON return a string with the JSON representation of the interface{} -func ToStandardJSON(i any) (string, error) { - return ToJSON(i, "", standardIndentation) -} - -// ToJSON return a string with the JSON representation of the interface{} -func ToJSON(i any, prefix string, indentation string) (string, error) { - buffer := &bytes.Buffer{} - encoder := json.NewEncoder(buffer) - encoder.SetEscapeHTML(false) - encoder.SetIndent(prefix, indentation) - err := encoder.Encode(i) - return buffer.String(), err -} diff --git a/cmd/formatter/logs.go b/cmd/formatter/logs.go deleted file mode 100644 index 1ee35e96b32..00000000000 --- a/cmd/formatter/logs.go +++ /dev/null @@ -1,186 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "context" - "fmt" - "io" - "strconv" - "strings" - "sync" - "time" - - "github.com/buger/goterm" - "github.com/docker/docker/pkg/jsonmessage" - - "github.com/docker/compose/v5/pkg/api" -) - -// LogConsumer consume logs from services and format them -type logConsumer struct { - ctx context.Context - presenters sync.Map // map[string]*presenter - width int - stdout io.Writer - stderr io.Writer - color bool - prefix bool - timestamp bool -} - -// NewLogConsumer creates a new LogConsumer -func NewLogConsumer(ctx context.Context, stdout, stderr io.Writer, color, prefix, timestamp bool) api.LogConsumer { - return &logConsumer{ - ctx: ctx, - presenters: sync.Map{}, - width: 0, - stdout: stdout, - stderr: stderr, - color: color, - prefix: prefix, - timestamp: timestamp, - } -} - -func (l *logConsumer) register(name string) *presenter { - var p *presenter - root, _, found := strings.Cut(name, " ") - if found { - parent := l.getPresenter(root) - p = &presenter{ - colors: parent.colors, - name: name, - prefix: parent.prefix, - } - } else { - cf := monochrome - if l.color { - switch name { - case "": - cf = monochrome - case api.WatchLogger: - cf = makeColorFunc("92") - default: - cf = nextColor() - } - } - p = &presenter{ - colors: cf, - name: name, - } - } - l.presenters.Store(name, p) - l.computeWidth() - if l.prefix { - l.presenters.Range(func(key, value any) bool { - p := value.(*presenter) - p.setPrefix(l.width) - return true - }) - } - return p -} - -func (l *logConsumer) getPresenter(container string) *presenter { - p, ok := l.presenters.Load(container) - if !ok { // should have been registered, but ¯\_(ツ)_/¯ - return l.register(container) - } - return p.(*presenter) -} - -// Log formats a log message as received from name/container -func (l *logConsumer) Log(container, message string) { - l.write(l.stdout, container, message) -} - -// Err formats a log message as received from name/container -func (l *logConsumer) Err(container, message string) { - l.write(l.stderr, container, message) -} - -func (l *logConsumer) write(w io.Writer, container, message string) { - if l.ctx.Err() != nil { - return - } - p := l.getPresenter(container) - timestamp := time.Now().Format(jsonmessage.RFC3339NanoFixed) - for line := range strings.SplitSeq(message, "\n") { - if l.timestamp { - _, _ = fmt.Fprintf(w, "%s%s %s\n", p.prefix, timestamp, line) - } else { - _, _ = fmt.Fprintf(w, "%s%s\n", p.prefix, line) - } - } -} - -func (l *logConsumer) Status(container, msg string) { - p := l.getPresenter(container) - s := p.colors(fmt.Sprintf("%s%s %s\n", goterm.RESET_LINE, container, msg)) - l.stdout.Write([]byte(s)) //nolint:errcheck -} - -func (l *logConsumer) computeWidth() { - width := 0 - l.presenters.Range(func(key, value any) bool { - p := value.(*presenter) - if len(p.name) > width { - width = len(p.name) - } - return true - }) - l.width = width + 1 -} - -type presenter struct { - colors colorFunc - name string - prefix string -} - -func (p *presenter) setPrefix(width int) { - if p.name == api.WatchLogger { - p.prefix = p.colors(strings.Repeat(" ", width) + " ⦿ ") - return - } - p.prefix = p.colors(fmt.Sprintf("%-"+strconv.Itoa(width)+"s | ", p.name)) -} - -type logDecorator struct { - decorated api.LogConsumer - Before func() - After func() -} - -func (l logDecorator) Log(containerName, message string) { - l.Before() - l.decorated.Log(containerName, message) - l.After() -} - -func (l logDecorator) Err(containerName, message string) { - l.Before() - l.decorated.Err(containerName, message) - l.After() -} - -func (l logDecorator) Status(container, msg string) { - l.Before() - l.decorated.Status(container, msg) - l.After() -} diff --git a/cmd/formatter/pretty.go b/cmd/formatter/pretty.go deleted file mode 100644 index bb85dedef61..00000000000 --- a/cmd/formatter/pretty.go +++ /dev/null @@ -1,32 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "fmt" - "io" - "strings" - "text/tabwriter" -) - -// PrintPrettySection prints a tabbed section on the writer parameter -func PrintPrettySection(out io.Writer, printer func(writer io.Writer), headers ...string) error { - w := tabwriter.NewWriter(out, 20, 1, 3, ' ', 0) - _, _ = fmt.Fprintln(w, strings.Join(headers, "\t")) - printer(w) - return w.Flush() -} diff --git a/cmd/formatter/shortcut.go b/cmd/formatter/shortcut.go deleted file mode 100644 index 16e8fc2e7af..00000000000 --- a/cmd/formatter/shortcut.go +++ /dev/null @@ -1,365 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import ( - "context" - "errors" - "fmt" - "math" - "os" - "strings" - "syscall" - "time" - - "github.com/buger/goterm" - "github.com/compose-spec/compose-go/v2/types" - "github.com/eiannone/keyboard" - "github.com/skratchdot/open-golang/open" - - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" -) - -const DISPLAY_ERROR_TIME = 10 - -type KeyboardError struct { - err error - timeStart time.Time -} - -func (ke *KeyboardError) shouldDisplay() bool { - return ke.err != nil && int(time.Since(ke.timeStart).Seconds()) < DISPLAY_ERROR_TIME -} - -func (ke *KeyboardError) printError(height int, info string) { - if ke.shouldDisplay() { - errMessage := ke.err.Error() - - moveCursor(height-1-extraLines(info)-extraLines(errMessage), 0) - clearLine() - - fmt.Print(errMessage) - } -} - -func (ke *KeyboardError) addError(prefix string, err error) { - ke.timeStart = time.Now() - - prefix = ansiColor(CYAN, fmt.Sprintf("%s →", prefix), BOLD) - errorString := fmt.Sprintf("%s %s", prefix, err.Error()) - - ke.err = errors.New(errorString) -} - -func (ke *KeyboardError) error() string { - return ke.err.Error() -} - -type KeyboardWatch struct { - Watching bool - Watcher Feature -} - -// Feature is an compose feature that can be started/stopped by a menu command -type Feature interface { - Start(context.Context) error - Stop() error -} - -type KEYBOARD_LOG_LEVEL int - -const ( - NONE KEYBOARD_LOG_LEVEL = 0 - INFO KEYBOARD_LOG_LEVEL = 1 - DEBUG KEYBOARD_LOG_LEVEL = 2 -) - -type LogKeyboard struct { - kError KeyboardError - Watch *KeyboardWatch - Detach func() - IsDockerDesktopActive bool - logLevel KEYBOARD_LOG_LEVEL - signalChannel chan<- os.Signal -} - -func NewKeyboardManager(isDockerDesktopActive bool, sc chan<- os.Signal) *LogKeyboard { - return &LogKeyboard{ - IsDockerDesktopActive: isDockerDesktopActive, - logLevel: INFO, - signalChannel: sc, - } -} - -func (lk *LogKeyboard) Decorate(l api.LogConsumer) api.LogConsumer { - return logDecorator{ - decorated: l, - Before: lk.clearNavigationMenu, - After: lk.PrintKeyboardInfo, - } -} - -func (lk *LogKeyboard) PrintKeyboardInfo() { - if lk.logLevel == INFO { - lk.printNavigationMenu() - } -} - -// Creates space to print error and menu string -func (lk *LogKeyboard) createBuffer(lines int) { - if lk.kError.shouldDisplay() { - extraLines := extraLines(lk.kError.error()) + 1 - lines += extraLines - } - - // get the string - infoMessage := lk.navigationMenu() - // calculate how many lines we need to display the menu info - // might be needed a line break - extraLines := extraLines(infoMessage) + 1 - lines += extraLines - - if lines > 0 { - allocateSpace(lines) - moveCursorUp(lines) - } -} - -func (lk *LogKeyboard) printNavigationMenu() { - offset := 1 - lk.clearNavigationMenu() - lk.createBuffer(offset) - - if lk.logLevel == INFO { - height := goterm.Height() - menu := lk.navigationMenu() - - carriageReturn() - saveCursor() - - lk.kError.printError(height, menu) - - moveCursor(height-extraLines(menu), 0) - clearLine() - fmt.Print(menu) - - carriageReturn() - restoreCursor() - } -} - -func (lk *LogKeyboard) navigationMenu() string { - var items []string - if lk.IsDockerDesktopActive { - items = append(items, shortcutKeyColor("v")+navColor(" View in Docker Desktop")) - } - - if lk.IsDockerDesktopActive { - items = append(items, shortcutKeyColor("o")+navColor(" View Config")) - } - - isEnabled := " Enable" - if lk.Watch != nil && lk.Watch.Watching { - isEnabled = " Disable" - } - items = append(items, shortcutKeyColor("w")+navColor(isEnabled+" Watch")) - items = append(items, shortcutKeyColor("d")+navColor(" Detach")) - - return strings.Join(items, " ") -} - -func (lk *LogKeyboard) clearNavigationMenu() { - height := goterm.Height() - carriageReturn() - saveCursor() - - // clearLine() - for i := 0; i < height; i++ { - moveCursorDown(1) - clearLine() - } - restoreCursor() -} - -func (lk *LogKeyboard) openDockerDesktop(ctx context.Context, project *types.Project) { - if !lk.IsDockerDesktopActive { - return - } - go func() { - _ = tracing.EventWrapFuncForErrGroup(ctx, "menu/gui", tracing.SpanOptions{}, - func(ctx context.Context) error { - link := fmt.Sprintf("docker-desktop://dashboard/apps/%s", project.Name) - err := open.Run(link) - if err != nil { - err = fmt.Errorf("could not open Docker Desktop") - lk.keyboardError("View", err) - } - return err - })() - }() -} - -func (lk *LogKeyboard) openDDComposeUI(ctx context.Context, project *types.Project) { - if !lk.IsDockerDesktopActive { - return - } - go func() { - _ = tracing.EventWrapFuncForErrGroup(ctx, "menu/gui/composeview", tracing.SpanOptions{}, - func(ctx context.Context) error { - link := fmt.Sprintf("docker-desktop://dashboard/docker-compose/%s", project.Name) - err := open.Run(link) - if err != nil { - err = fmt.Errorf("could not open Docker Desktop Compose UI") - lk.keyboardError("View Config", err) - } - return err - })() - }() -} - -func (lk *LogKeyboard) openDDWatchDocs(ctx context.Context, project *types.Project) { - go func() { - _ = tracing.EventWrapFuncForErrGroup(ctx, "menu/gui/watch", tracing.SpanOptions{}, - func(ctx context.Context) error { - link := fmt.Sprintf("docker-desktop://dashboard/docker-compose/%s/watch", project.Name) - err := open.Run(link) - if err != nil { - err = fmt.Errorf("could not open Docker Desktop Compose UI") - lk.keyboardError("Watch Docs", err) - } - return err - })() - }() -} - -func (lk *LogKeyboard) keyboardError(prefix string, err error) { - lk.kError.addError(prefix, err) - - lk.printNavigationMenu() - timer1 := time.NewTimer((DISPLAY_ERROR_TIME + 1) * time.Second) - go func() { - <-timer1.C - lk.printNavigationMenu() - }() -} - -func (lk *LogKeyboard) ToggleWatch(ctx context.Context, options api.UpOptions) { - if lk.Watch == nil { - return - } - if lk.Watch.Watching { - err := lk.Watch.Watcher.Stop() - if err != nil { - options.Start.Attach.Err(api.WatchLogger, err.Error()) - } else { - lk.Watch.Watching = false - } - } else { - go func() { - _ = tracing.EventWrapFuncForErrGroup(ctx, "menu/watch", tracing.SpanOptions{}, - func(ctx context.Context) error { - err := lk.Watch.Watcher.Start(ctx) - if err != nil { - options.Start.Attach.Err(api.WatchLogger, err.Error()) - } else { - lk.Watch.Watching = true - } - return err - })() - }() - } -} - -func (lk *LogKeyboard) HandleKeyEvents(ctx context.Context, event keyboard.KeyEvent, project *types.Project, options api.UpOptions) { - switch kRune := event.Rune; kRune { - case 'd': - lk.clearNavigationMenu() - lk.Detach() - case 'v': - lk.openDockerDesktop(ctx, project) - case 'w': - if lk.Watch == nil { - // we try to open watch docs if DD is installed - if lk.IsDockerDesktopActive { - lk.openDDWatchDocs(ctx, project) - } - // either way we mark menu/watch as an error - go func() { - _ = tracing.EventWrapFuncForErrGroup(ctx, "menu/watch", tracing.SpanOptions{}, - func(ctx context.Context) error { - err := fmt.Errorf("watch is not yet configured. Learn more: %s", ansiColor(CYAN, "https://docs.docker.com/compose/file-watch/")) - lk.keyboardError("Watch", err) - return err - })() - }() - } - lk.ToggleWatch(ctx, options) - case 'o': - lk.openDDComposeUI(ctx, project) - } - switch key := event.Key; key { - case keyboard.KeyCtrlC: - _ = keyboard.Close() - lk.clearNavigationMenu() - showCursor() - - lk.logLevel = NONE - // will notify main thread to kill and will handle gracefully - lk.signalChannel <- syscall.SIGINT - case keyboard.KeyCtrlZ: - handleCtrlZ() - case keyboard.KeyEnter: - newLine() - lk.printNavigationMenu() - } -} - -func (lk *LogKeyboard) EnableWatch(enabled bool, watcher Feature) { - lk.Watch = &KeyboardWatch{ - Watching: enabled, - Watcher: watcher, - } -} - -func (lk *LogKeyboard) EnableDetach(detach func()) { - lk.Detach = detach -} - -func allocateSpace(lines int) { - for i := 0; i < lines; i++ { - clearLine() - newLine() - carriageReturn() - } -} - -func extraLines(s string) int { - return int(math.Floor(float64(lenAnsi(s)) / float64(goterm.Width()))) -} - -func shortcutKeyColor(key string) string { - foreground := "38;2" - black := "0;0;0" - background := "48;2" - white := "255;255;255" - return ansiColor(foreground+";"+black+";"+background+";"+white, key, BOLD) -} - -func navColor(key string) string { - return ansiColor(FAINT, key) -} diff --git a/cmd/formatter/shortcut_unix.go b/cmd/formatter/shortcut_unix.go deleted file mode 100644 index 0baa3a949cc..00000000000 --- a/cmd/formatter/shortcut_unix.go +++ /dev/null @@ -1,25 +0,0 @@ -//go:build !windows - -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -import "syscall" - -func handleCtrlZ() { - _ = syscall.Kill(0, syscall.SIGSTOP) -} diff --git a/cmd/formatter/shortcut_windows.go b/cmd/formatter/shortcut_windows.go deleted file mode 100644 index 1efa14cc2dd..00000000000 --- a/cmd/formatter/shortcut_windows.go +++ /dev/null @@ -1,25 +0,0 @@ -//go:build windows - -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package formatter - -// handleCtrlZ is a no-op on Windows as SIGSTOP is not supported -func handleCtrlZ() { - // Windows doesn't support SIGSTOP/SIGCONT signals - // Ctrl+Z behavior is handled differently by the Windows terminal -} diff --git a/cmd/main.go b/cmd/main.go deleted file mode 100644 index 7a5ec58d60b..00000000000 --- a/cmd/main.go +++ /dev/null @@ -1,85 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package main - -import ( - "os" - - dockercli "github.com/docker/cli/cli" - "github.com/docker/cli/cli-plugins/metadata" - "github.com/docker/cli/cli-plugins/plugin" - "github.com/docker/cli/cli/command" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/cmdtrace" - "github.com/docker/compose/v5/cmd/compatibility" - commands "github.com/docker/compose/v5/cmd/compose" - "github.com/docker/compose/v5/cmd/prompt" - "github.com/docker/compose/v5/internal" - "github.com/docker/compose/v5/pkg/compose" -) - -func pluginMain() { - plugin.Run( - func(cli command.Cli) *cobra.Command { - backendOptions := &commands.BackendOptions{ - Options: []compose.Option{ - compose.WithPrompt(prompt.NewPrompt(cli.In(), cli.Out()).Confirm), - }, - } - - cmd := commands.RootCommand(cli, backendOptions) - originalPreRunE := cmd.PersistentPreRunE - cmd.PersistentPreRunE = func(cmd *cobra.Command, args []string) error { - // initialize the cli instance - if err := plugin.PersistentPreRunE(cmd, args); err != nil { - return err - } - if err := cmdtrace.Setup(cmd, cli, os.Args[1:]); err != nil { - logrus.Debugf("failed to enable tracing: %v", err) - } - - if originalPreRunE != nil { - return originalPreRunE(cmd, args) - } - return nil - } - - cmd.SetFlagErrorFunc(func(c *cobra.Command, err error) error { - return dockercli.StatusError{ - StatusCode: 1, - Status: err.Error(), - } - }) - return cmd - }, - metadata.Metadata{ - SchemaVersion: "0.1.0", - Vendor: "Docker Inc.", - Version: internal.Version, - }, - command.WithUserAgent("compose/"+internal.Version), - ) -} - -func main() { - if plugin.RunningStandalone() { - os.Args = append([]string{"docker"}, compatibility.Convert(os.Args[1:])...) - } - pluginMain() -} diff --git a/cmd/prompt/prompt.go b/cmd/prompt/prompt.go deleted file mode 100644 index 87379f5eb61..00000000000 --- a/cmd/prompt/prompt.go +++ /dev/null @@ -1,102 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package prompt - -import ( - "fmt" - "io" - - "github.com/AlecAivazis/survey/v2" - "github.com/docker/cli/cli/streams" - - "github.com/docker/compose/v5/pkg/utils" -) - -//go:generate mockgen -destination=./prompt_mock.go -self_package "github.com/docker/compose/v5/pkg/prompt" -package=prompt . UI - -// UI - prompt user input -type UI interface { - Confirm(message string, defaultValue bool) (bool, error) -} - -func NewPrompt(stdin *streams.In, stdout *streams.Out) UI { - if stdin.IsTerminal() { - return User{stdin: streamsFileReader{stdin}, stdout: streamsFileWriter{stdout}} - } - return Pipe{stdin: stdin, stdout: stdout} -} - -// User - in a terminal -type User struct { - stdout streamsFileWriter - stdin streamsFileReader -} - -// adapt streams.Out to terminal.FileWriter -type streamsFileWriter struct { - stream *streams.Out -} - -func (s streamsFileWriter) Write(p []byte) (n int, err error) { - return s.stream.Write(p) -} - -func (s streamsFileWriter) Fd() uintptr { - return s.stream.FD() -} - -// adapt streams.In to terminal.FileReader -type streamsFileReader struct { - stream *streams.In -} - -func (s streamsFileReader) Read(p []byte) (n int, err error) { - return s.stream.Read(p) -} - -func (s streamsFileReader) Fd() uintptr { - return s.stream.FD() -} - -// Confirm asks for yes or no input -func (u User) Confirm(message string, defaultValue bool) (bool, error) { - qs := &survey.Confirm{ - Message: message, - Default: defaultValue, - } - var b bool - err := survey.AskOne(qs, &b, func(options *survey.AskOptions) error { - options.Stdio.In = u.stdin - options.Stdio.Out = u.stdout - return nil - }) - return b, err -} - -// Pipe - aggregates prompt methods -type Pipe struct { - stdout io.Writer - stdin io.Reader -} - -// Confirm asks for yes or no input -func (u Pipe) Confirm(message string, defaultValue bool) (bool, error) { - _, _ = fmt.Fprint(u.stdout, message) - var answer string - _, _ = fmt.Fscanln(u.stdin, &answer) - return utils.StringToBool(answer), nil -} diff --git a/cmd/prompt/prompt_mock.go b/cmd/prompt/prompt_mock.go deleted file mode 100644 index 83b0ff1189b..00000000000 --- a/cmd/prompt/prompt_mock.go +++ /dev/null @@ -1,94 +0,0 @@ -// Code generated by MockGen. DO NOT EDIT. -// Source: github.com/docker/compose-cli/pkg/prompt (interfaces: UI) - -// Package prompt is a generated GoMock package. -package prompt - -import ( - reflect "reflect" - - gomock "go.uber.org/mock/gomock" -) - -// MockUI is a mock of UI interface -type MockUI struct { - ctrl *gomock.Controller - recorder *MockUIMockRecorder -} - -// MockUIMockRecorder is the mock recorder for MockUI -type MockUIMockRecorder struct { - mock *MockUI -} - -// NewMockUI creates a new mock instance -func NewMockUI(ctrl *gomock.Controller) *MockUI { - mock := &MockUI{ctrl: ctrl} - mock.recorder = &MockUIMockRecorder{mock} - return mock -} - -// EXPECT returns an object that allows the caller to indicate expected use -func (m *MockUI) EXPECT() *MockUIMockRecorder { - return m.recorder -} - -// Confirm mocks base method -func (m *MockUI) Confirm(arg0 string, arg1 bool) (bool, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Confirm", arg0, arg1) - ret0, _ := ret[0].(bool) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Confirm indicates an expected call of Confirm -func (mr *MockUIMockRecorder) Confirm(arg0, arg1 interface{}) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Confirm", reflect.TypeOf((*MockUI)(nil).Confirm), arg0, arg1) -} - -// Input mocks base method -func (m *MockUI) Input(arg0, arg1 string) (string, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Input", arg0, arg1) - ret0, _ := ret[0].(string) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Input indicates an expected call of Input -func (mr *MockUIMockRecorder) Input(arg0, arg1 interface{}) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Input", reflect.TypeOf((*MockUI)(nil).Input), arg0, arg1) -} - -// Password mocks base method -func (m *MockUI) Password(arg0 string) (string, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Password", arg0) - ret0, _ := ret[0].(string) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Password indicates an expected call of Password -func (mr *MockUIMockRecorder) Password(arg0 interface{}) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Password", reflect.TypeOf((*MockUI)(nil).Password), arg0) -} - -// Select mocks base method -func (m *MockUI) Select(arg0 string, arg1 []string) (int, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Select", arg0, arg1) - ret0, _ := ret[0].(int) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Select indicates an expected call of Select -func (mr *MockUIMockRecorder) Select(arg0, arg1 interface{}) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Select", reflect.TypeOf((*MockUI)(nil).Select), arg0, arg1) -} diff --git a/codecov.yml b/codecov.yml deleted file mode 100644 index a66912f32e0..00000000000 --- a/codecov.yml +++ /dev/null @@ -1,21 +0,0 @@ -coverage: - status: - project: - default: - informational: true - target: auto - threshold: 2% - patch: - default: - informational: true - -comment: - require_changes: true - -ignore: - - "packaging" - - "docs" - - "bin" - - "e2e" - - "pkg/e2e" - - "**/*_test.go" diff --git a/compose/__init__.py b/compose/__init__.py new file mode 100644 index 00000000000..444d27edb84 --- /dev/null +++ b/compose/__init__.py @@ -0,0 +1 @@ +__version__ = '1.28.0dev' diff --git a/compose/__main__.py b/compose/__main__.py new file mode 100644 index 00000000000..199ba2ae9b4 --- /dev/null +++ b/compose/__main__.py @@ -0,0 +1,3 @@ +from compose.cli.main import main + +main() diff --git a/pkg/e2e/fixtures/environment/env-priority/.env.empty b/compose/cli/__init__.py similarity index 100% rename from pkg/e2e/fixtures/environment/env-priority/.env.empty rename to compose/cli/__init__.py diff --git a/compose/cli/colors.py b/compose/cli/colors.py new file mode 100644 index 00000000000..a4983a9f51a --- /dev/null +++ b/compose/cli/colors.py @@ -0,0 +1,46 @@ +from ..const import IS_WINDOWS_PLATFORM + +NAMES = [ + 'grey', + 'red', + 'green', + 'yellow', + 'blue', + 'magenta', + 'cyan', + 'white' +] + + +def get_pairs(): + for i, name in enumerate(NAMES): + yield (name, str(30 + i)) + yield ('intense_' + name, str(30 + i) + ';1') + + +def ansi(code): + return '\033[{}m'.format(code) + + +def ansi_color(code, s): + return '{}{}{}'.format(ansi(code), s, ansi(0)) + + +def make_color_fn(code): + return lambda s: ansi_color(code, s) + + +if IS_WINDOWS_PLATFORM: + import colorama + colorama.init(strip=False) +for (name, code) in get_pairs(): + globals()[name] = make_color_fn(code) + + +def rainbow(): + cs = ['cyan', 'yellow', 'green', 'magenta', 'blue', + 'intense_cyan', 'intense_yellow', 'intense_green', + 'intense_magenta', 'intense_blue'] + + for c in cs: + yield globals()[c] diff --git a/compose/cli/command.py b/compose/cli/command.py new file mode 100644 index 00000000000..599df9969a7 --- /dev/null +++ b/compose/cli/command.py @@ -0,0 +1,210 @@ +import logging +import os +import re + +from . import errors +from .. import config +from .. import parallel +from ..config.environment import Environment +from ..const import LABEL_CONFIG_FILES +from ..const import LABEL_ENVIRONMENT_FILE +from ..const import LABEL_WORKING_DIR +from ..project import Project +from .docker_client import get_client +from .docker_client import load_context +from .docker_client import make_context +from .errors import UserError + +log = logging.getLogger(__name__) + +SILENT_COMMANDS = { + 'events', + 'exec', + 'kill', + 'logs', + 'pause', + 'ps', + 'restart', + 'rm', + 'start', + 'stop', + 'top', + 'unpause', +} + + +def project_from_options(project_dir, options, additional_options=None): + additional_options = additional_options or {} + override_dir = get_project_dir(options) + environment_file = options.get('--env-file') + environment = Environment.from_env_file(override_dir or project_dir, environment_file) + environment.silent = options.get('COMMAND', None) in SILENT_COMMANDS + set_parallel_limit(environment) + + # get the context for the run + context = None + context_name = options.get('--context', None) + if context_name: + context = load_context(context_name) + if not context: + raise UserError("Context '{}' not found".format(context_name)) + + host = options.get('--host', None) + if host is not None: + if context: + raise UserError( + "-H, --host and -c, --context are mutually exclusive. Only one should be set.") + host = host.lstrip('=') + context = make_context(host, options, environment) + + return get_project( + project_dir, + get_config_path_from_options(options, environment), + project_name=options.get('--project-name'), + verbose=options.get('--verbose'), + context=context, + environment=environment, + override_dir=override_dir, + interpolate=(not additional_options.get('--no-interpolate')), + environment_file=environment_file, + enabled_profiles=get_profiles_from_options(options, environment) + ) + + +def set_parallel_limit(environment): + parallel_limit = environment.get('COMPOSE_PARALLEL_LIMIT') + if parallel_limit: + try: + parallel_limit = int(parallel_limit) + except ValueError: + raise errors.UserError( + 'COMPOSE_PARALLEL_LIMIT must be an integer (found: "{}")'.format( + environment.get('COMPOSE_PARALLEL_LIMIT') + ) + ) + if parallel_limit <= 1: + raise errors.UserError('COMPOSE_PARALLEL_LIMIT can not be less than 2') + parallel.GlobalLimit.set_global_limit(parallel_limit) + + +def get_project_dir(options): + override_dir = None + files = get_config_path_from_options(options, os.environ) + if files: + if files[0] == '-': + return '.' + override_dir = os.path.dirname(files[0]) + return options.get('--project-directory') or override_dir + + +def get_config_from_options(base_dir, options, additional_options=None): + additional_options = additional_options or {} + override_dir = get_project_dir(options) + environment_file = options.get('--env-file') + environment = Environment.from_env_file(override_dir or base_dir, environment_file) + config_path = get_config_path_from_options(options, environment) + return config.load( + config.find(base_dir, config_path, environment, override_dir), + not additional_options.get('--no-interpolate') + ) + + +def get_config_path_from_options(options, environment): + def unicode_paths(paths): + return [p.decode('utf-8') if isinstance(p, bytes) else p for p in paths] + + file_option = options.get('--file') + if file_option: + return unicode_paths(file_option) + + config_files = environment.get('COMPOSE_FILE') + if config_files: + pathsep = environment.get('COMPOSE_PATH_SEPARATOR', os.pathsep) + return unicode_paths(config_files.split(pathsep)) + return None + + +def get_profiles_from_options(options, environment): + profile_option = options.get('--profile') + if profile_option: + return profile_option + + profiles = environment.get('COMPOSE_PROFILE') + if profiles: + return profiles.split(',') + + return [] + + +def get_project(project_dir, config_path=None, project_name=None, verbose=False, + context=None, environment=None, override_dir=None, + interpolate=True, environment_file=None, enabled_profiles=None): + if not environment: + environment = Environment.from_env_file(project_dir) + config_details = config.find(project_dir, config_path, environment, override_dir) + project_name = get_project_name( + config_details.working_dir, project_name, environment + ) + config_data = config.load(config_details, interpolate) + + api_version = environment.get('COMPOSE_API_VERSION') + + client = get_client( + verbose=verbose, version=api_version, context=context, environment=environment + ) + + with errors.handle_connection_errors(client): + return Project.from_config( + project_name, + config_data, + client, + environment.get('DOCKER_DEFAULT_PLATFORM'), + execution_context_labels(config_details, environment_file), + enabled_profiles, + ) + + +def execution_context_labels(config_details, environment_file): + extra_labels = [ + '{}={}'.format(LABEL_WORKING_DIR, os.path.abspath(config_details.working_dir)) + ] + + if not use_config_from_stdin(config_details): + extra_labels.append('{}={}'.format(LABEL_CONFIG_FILES, config_files_label(config_details))) + + if environment_file is not None: + extra_labels.append('{}={}'.format( + LABEL_ENVIRONMENT_FILE, + os.path.normpath(environment_file)) + ) + return extra_labels + + +def use_config_from_stdin(config_details): + for c in config_details.config_files: + if not c.filename: + return True + return False + + +def config_files_label(config_details): + return ",".join( + os.path.normpath(c.filename) for c in config_details.config_files + ) + + +def get_project_name(working_dir, project_name=None, environment=None): + def normalize_name(name): + return re.sub(r'[^-_a-z0-9]', '', name.lower()) + + if not environment: + environment = Environment.from_env_file(working_dir) + project_name = project_name or environment.get('COMPOSE_PROJECT_NAME') + if project_name: + return normalize_name(project_name) + + project = os.path.basename(os.path.abspath(working_dir)) + if project: + return normalize_name(project) + + return 'default' diff --git a/compose/cli/docker_client.py b/compose/cli/docker_client.py new file mode 100644 index 00000000000..e4a0fea61b4 --- /dev/null +++ b/compose/cli/docker_client.py @@ -0,0 +1,173 @@ +import logging +import os.path +import ssl + +from docker import APIClient +from docker import Context +from docker import ContextAPI +from docker import TLSConfig +from docker.errors import TLSParameterError +from docker.utils import kwargs_from_env +from docker.utils.config import home_dir + +from . import verbose_proxy +from ..config.environment import Environment +from ..const import HTTP_TIMEOUT +from ..utils import unquote_path +from .errors import UserError +from .utils import generate_user_agent +from .utils import get_version_info + +log = logging.getLogger(__name__) + + +def default_cert_path(): + return os.path.join(home_dir(), '.docker') + + +def make_context(host, options, environment): + tls = tls_config_from_options(options, environment) + ctx = Context("compose", host=host, tls=tls.verify if tls else False) + if tls: + ctx.set_endpoint("docker", host, tls, skip_tls_verify=not tls.verify) + return ctx + + +def load_context(name=None): + return ContextAPI.get_context(name) + + +def get_client(environment, verbose=False, version=None, context=None): + client = docker_client( + version=version, context=context, + environment=environment, tls_version=get_tls_version(environment) + ) + if verbose: + version_info = client.version().items() + log.info(get_version_info('full')) + log.info("Docker base_url: %s", client.base_url) + log.info("Docker version: %s", + ", ".join("%s=%s" % item for item in version_info)) + return verbose_proxy.VerboseProxy('docker', client) + return client + + +def get_tls_version(environment): + compose_tls_version = environment.get('COMPOSE_TLS_VERSION', None) + if not compose_tls_version: + return None + + tls_attr_name = "PROTOCOL_{}".format(compose_tls_version) + if not hasattr(ssl, tls_attr_name): + log.warning( + 'The "{}" protocol is unavailable. You may need to update your ' + 'version of Python or OpenSSL. Falling back to TLSv1 (default).' + .format(compose_tls_version) + ) + return None + + return getattr(ssl, tls_attr_name) + + +def tls_config_from_options(options, environment=None): + environment = environment or Environment() + cert_path = environment.get('DOCKER_CERT_PATH') or None + + tls = options.get('--tls', False) + ca_cert = unquote_path(options.get('--tlscacert')) + cert = unquote_path(options.get('--tlscert')) + key = unquote_path(options.get('--tlskey')) + # verify is a special case - with docopt `--tlsverify` = False means it + # wasn't used, so we set it if either the environment or the flag is True + # see https://github.com/docker/compose/issues/5632 + verify = options.get('--tlsverify') or environment.get_boolean('DOCKER_TLS_VERIFY') + + skip_hostname_check = options.get('--skip-hostname-check', False) + if cert_path is not None and not any((ca_cert, cert, key)): + # FIXME: Modify TLSConfig to take a cert_path argument and do this internally + cert = os.path.join(cert_path, 'cert.pem') + key = os.path.join(cert_path, 'key.pem') + ca_cert = os.path.join(cert_path, 'ca.pem') + + if verify and not any((ca_cert, cert, key)): + # Default location for cert files is ~/.docker + ca_cert = os.path.join(default_cert_path(), 'ca.pem') + cert = os.path.join(default_cert_path(), 'cert.pem') + key = os.path.join(default_cert_path(), 'key.pem') + + tls_version = get_tls_version(environment) + + advanced_opts = any([ca_cert, cert, key, verify, tls_version]) + + if tls is True and not advanced_opts: + return True + elif advanced_opts: # --tls is a noop + client_cert = None + if cert or key: + client_cert = (cert, key) + + return TLSConfig( + client_cert=client_cert, verify=verify, ca_cert=ca_cert, + assert_hostname=False if skip_hostname_check else None, + ssl_version=tls_version + ) + + return None + + +def docker_client(environment, version=None, context=None, tls_version=None): + """ + Returns a docker-py client configured using environment variables + according to the same logic as the official Docker client. + """ + try: + kwargs = kwargs_from_env(environment=environment, ssl_version=tls_version) + except TLSParameterError: + raise UserError( + "TLS configuration is invalid - make sure your DOCKER_TLS_VERIFY " + "and DOCKER_CERT_PATH are set correctly.\n" + "You might need to run `eval \"$(docker-machine env default)\"`") + + if not context: + # check env for DOCKER_HOST and certs path + host = kwargs.get("base_url", None) + tls = kwargs.get("tls", None) + verify = False if not tls else tls.verify + if host: + context = Context("compose", host=host, tls=verify) + else: + context = ContextAPI.get_current_context() + if tls: + context.set_endpoint("docker", host=host, tls_cfg=tls, skip_tls_verify=not verify) + + if not context.is_docker_host(): + raise UserError( + "The platform targeted with the current context is not supported.\n" + "Make sure the context in use targets a Docker Engine.\n") + + kwargs['base_url'] = context.Host + if context.TLSConfig: + kwargs['tls'] = context.TLSConfig + + if version: + kwargs['version'] = version + + timeout = environment.get('COMPOSE_HTTP_TIMEOUT') + if timeout: + kwargs['timeout'] = int(timeout) + else: + kwargs['timeout'] = HTTP_TIMEOUT + + kwargs['user_agent'] = generate_user_agent() + + # Workaround for + # https://pyinstaller.readthedocs.io/en/v3.3.1/runtime-information.html#ld-library-path-libpath-considerations + if 'LD_LIBRARY_PATH_ORIG' in environment: + kwargs['credstore_env'] = { + 'LD_LIBRARY_PATH': environment.get('LD_LIBRARY_PATH_ORIG'), + } + use_paramiko_ssh = int(environment.get('COMPOSE_PARAMIKO_SSH', 0)) + client = APIClient(use_ssh_client=not use_paramiko_ssh, **kwargs) + client._original_base_url = kwargs.get('base_url') + + return client diff --git a/compose/cli/docopt_command.py b/compose/cli/docopt_command.py new file mode 100644 index 00000000000..e56b37835bc --- /dev/null +++ b/compose/cli/docopt_command.py @@ -0,0 +1,62 @@ +from inspect import getdoc + +from docopt import docopt +from docopt import DocoptExit + + +def docopt_full_help(docstring, *args, **kwargs): + try: + return docopt(docstring, *args, **kwargs) + except DocoptExit: + raise SystemExit(docstring) + + +class DocoptDispatcher: + + def __init__(self, command_class, options): + self.command_class = command_class + self.options = options + + @classmethod + def get_command_and_options(cls, doc_entity, argv, options): + command_help = getdoc(doc_entity) + opt = docopt_full_help(command_help, argv, **options) + command = opt['COMMAND'] + return command_help, opt, command + + def parse(self, argv): + command_help, options, command = DocoptDispatcher.get_command_and_options( + self.command_class, argv, self.options) + + if command is None: + raise SystemExit(command_help) + + handler = get_handler(self.command_class, command) + docstring = getdoc(handler) + + if docstring is None: + raise NoSuchCommand(command, self) + + command_options = docopt_full_help(docstring, options['ARGS'], options_first=True) + return options, handler, command_options + + +def get_handler(command_class, command): + command = command.replace('-', '_') + # we certainly want to have "exec" command, since that's what docker client has + # but in python exec is a keyword + if command == "exec": + command = "exec_command" + + if not hasattr(command_class, command): + raise NoSuchCommand(command, command_class) + + return getattr(command_class, command) + + +class NoSuchCommand(Exception): + def __init__(self, command, supercommand): + super().__init__("No such command: %s" % command) + + self.command = command + self.supercommand = supercommand diff --git a/compose/cli/errors.py b/compose/cli/errors.py new file mode 100644 index 00000000000..a807c7d1c78 --- /dev/null +++ b/compose/cli/errors.py @@ -0,0 +1,165 @@ +import contextlib +import logging +import socket +from distutils.spawn import find_executable +from textwrap import dedent + +from docker.errors import APIError +from requests.exceptions import ConnectionError as RequestsConnectionError +from requests.exceptions import ReadTimeout +from requests.exceptions import SSLError +from requests.packages.urllib3.exceptions import ReadTimeoutError + +from ..const import API_VERSION_TO_ENGINE_VERSION +from .utils import binarystr_to_unicode +from .utils import is_docker_for_mac_installed +from .utils import is_mac +from .utils import is_ubuntu +from .utils import is_windows + + +log = logging.getLogger(__name__) + + +class UserError(Exception): + + def __init__(self, msg): + self.msg = dedent(msg).strip() + + def __str__(self): + return self.msg + + +class ConnectionError(Exception): + pass + + +@contextlib.contextmanager +def handle_connection_errors(client): + try: + yield + except SSLError as e: + log.error('SSL error: %s' % e) + raise ConnectionError() + except RequestsConnectionError as e: + if e.args and isinstance(e.args[0], ReadTimeoutError): + log_timeout_error(client.timeout) + raise ConnectionError() + exit_with_error(get_conn_error_message(client.base_url)) + except APIError as e: + log_api_error(e, client.api_version) + raise ConnectionError() + except (ReadTimeout, socket.timeout): + log_timeout_error(client.timeout) + raise ConnectionError() + except Exception as e: + if is_windows(): + import pywintypes + if isinstance(e, pywintypes.error): + log_windows_pipe_error(e) + raise ConnectionError() + raise + + +def log_windows_pipe_error(exc): + if exc.winerror == 2: + log.error("Couldn't connect to Docker daemon. You might need to start Docker for Windows.") + elif exc.winerror == 232: # https://github.com/docker/compose/issues/5005 + log.error( + "The current Compose file version is not compatible with your engine version. " + "Please upgrade your Compose file to a more recent version, or set " + "a COMPOSE_API_VERSION in your environment." + ) + else: + log.error( + "Windows named pipe error: {} (code: {})".format( + binarystr_to_unicode(exc.strerror), exc.winerror + ) + ) + + +def log_timeout_error(timeout): + log.error( + "An HTTP request took too long to complete. Retry with --verbose to " + "obtain debug information.\n" + "If you encounter this issue regularly because of slow network " + "conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher " + "value (current value: %s)." % timeout) + + +def log_api_error(e, client_version): + explanation = binarystr_to_unicode(e.explanation) + + if 'client is newer than server' not in explanation: + log.error(explanation) + return + + version = API_VERSION_TO_ENGINE_VERSION.get(client_version) + if not version: + # They've set a custom API version + log.error(explanation) + return + + log.error( + "The Docker Engine version is less than the minimum required by " + "Compose. Your current project requires a Docker Engine of " + "version {version} or greater.".format(version=version) + ) + + +def exit_with_error(msg): + log.error(dedent(msg).strip()) + raise ConnectionError() + + +def get_conn_error_message(url): + try: + if find_executable('docker') is None: + return docker_not_found_msg("Couldn't connect to Docker daemon.") + if is_docker_for_mac_installed(): + return conn_error_docker_for_mac + if find_executable('docker-machine') is not None: + return conn_error_docker_machine + except UnicodeDecodeError: + # https://github.com/docker/compose/issues/5442 + # Ignore the error and print the generic message instead. + pass + return conn_error_generic.format(url=url) + + +def docker_not_found_msg(problem): + return "{} You might need to install Docker:\n\n{}".format( + problem, docker_install_url()) + + +def docker_install_url(): + if is_mac(): + return docker_install_url_mac + elif is_ubuntu(): + return docker_install_url_ubuntu + elif is_windows(): + return docker_install_url_windows + else: + return docker_install_url_generic + + +docker_install_url_mac = "https://docs.docker.com/engine/installation/mac/" +docker_install_url_ubuntu = "https://docs.docker.com/engine/installation/ubuntulinux/" +docker_install_url_windows = "https://docs.docker.com/engine/installation/windows/" +docker_install_url_generic = "https://docs.docker.com/engine/installation/" + + +conn_error_docker_machine = """ + Couldn't connect to Docker daemon - you might need to run `docker-machine start default`. +""" + +conn_error_docker_for_mac = """ + Couldn't connect to Docker daemon. You might need to start Docker for Mac. +""" + + +conn_error_generic = """ + Couldn't connect to Docker daemon at {url} - is it running? + + If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable. +""" diff --git a/compose/cli/formatter.py b/compose/cli/formatter.py new file mode 100644 index 00000000000..ff81ee65163 --- /dev/null +++ b/compose/cli/formatter.py @@ -0,0 +1,54 @@ +import logging +from shutil import get_terminal_size + +import texttable + +from compose.cli import colors + + +def get_tty_width(): + try: + # get_terminal_size can't determine the size if compose is piped + # to another command. But in such case it doesn't make sense to + # try format the output by terminal size as this output is consumed + # by another command. So let's pretend we have a huge terminal so + # output is single-lined + width, _ = get_terminal_size(fallback=(999, 0)) + return int(width) + except OSError: + return 0 + + +class Formatter: + """Format tabular data for printing.""" + + @staticmethod + def table(headers, rows): + table = texttable.Texttable(max_width=get_tty_width()) + table.set_cols_dtype(['t' for h in headers]) + table.add_rows([headers] + rows) + table.set_deco(table.HEADER) + table.set_chars(['-', '|', '+', '-']) + + return table.draw() + + +class ConsoleWarningFormatter(logging.Formatter): + """A logging.Formatter which prints WARNING and ERROR messages with + a prefix of the log level colored appropriate for the log level. + """ + + def get_level_message(self, record): + separator = ': ' + if record.levelno >= logging.ERROR: + return colors.red(record.levelname) + separator + if record.levelno >= logging.WARNING: + return colors.yellow(record.levelname) + separator + + return '' + + def format(self, record): + if isinstance(record.msg, bytes): + record.msg = record.msg.decode('utf-8') + message = super().format(record) + return '{}{}'.format(self.get_level_message(record), message) diff --git a/compose/cli/log_printer.py b/compose/cli/log_printer.py new file mode 100644 index 00000000000..c49b817de7e --- /dev/null +++ b/compose/cli/log_printer.py @@ -0,0 +1,271 @@ +import _thread as thread +import sys +from collections import namedtuple +from itertools import cycle +from operator import attrgetter +from queue import Empty +from queue import Queue +from threading import Thread + +from docker.errors import APIError + +from . import colors +from compose.cli.signals import ShutdownException +from compose.utils import split_buffer + + +class LogPresenter: + + def __init__(self, prefix_width, color_func, keep_prefix=True): + self.prefix_width = prefix_width + self.color_func = color_func + self.keep_prefix = keep_prefix + + def present(self, container, line): + to_log = '{line}'.format(line=line) + + if self.keep_prefix: + prefix = container.name_without_project.ljust(self.prefix_width) + to_log = '{prefix} '.format(prefix=self.color_func(prefix + ' |')) + to_log + + return to_log + + +def build_log_presenters(service_names, monochrome, keep_prefix=True): + """Return an iterable of functions. + + Each function can be used to format the logs output of a container. + """ + prefix_width = max_name_width(service_names) + + def no_color(text): + return text + + for color_func in cycle([no_color] if monochrome else colors.rainbow()): + yield LogPresenter(prefix_width, color_func, keep_prefix) + + +def max_name_width(service_names, max_index_width=3): + """Calculate the maximum width of container names so we can make the log + prefixes line up like so: + + db_1 | Listening + web_1 | Listening + """ + return max(len(name) for name in service_names) + max_index_width + + +class LogPrinter: + """Print logs from many containers to a single output stream.""" + + def __init__(self, + containers, + presenters, + event_stream, + output=sys.stdout, + cascade_stop=False, + log_args=None): + self.containers = containers + self.presenters = presenters + self.event_stream = event_stream + self.output = output + self.cascade_stop = cascade_stop + self.log_args = log_args or {} + + def run(self): + if not self.containers: + return + + queue = Queue() + thread_args = queue, self.log_args + thread_map = build_thread_map(self.containers, self.presenters, thread_args) + start_producer_thread(( + thread_map, + self.event_stream, + self.presenters, + thread_args)) + + for line in consume_queue(queue, self.cascade_stop): + remove_stopped_threads(thread_map) + + if self.cascade_stop: + matching_container = [cont.name for cont in self.containers if cont.name == line] + if line in matching_container: + # Returning the name of the container that started the + # the cascade_stop so we can return the correct exit code + return line + + if not line: + if not thread_map: + # There are no running containers left to tail, so exit + return + # We got an empty line because of a timeout, but there are still + # active containers to tail, so continue + continue + + self.write(line) + + def write(self, line): + try: + self.output.write(line) + except UnicodeEncodeError: + # This may happen if the user's locale settings don't support UTF-8 + # and UTF-8 characters are present in the log line. The following + # will output a "degraded" log with unsupported characters + # replaced by `?` + self.output.write(line.encode('ascii', 'replace').decode()) + self.output.flush() + + +def remove_stopped_threads(thread_map): + for container_id, tailer_thread in list(thread_map.items()): + if not tailer_thread.is_alive(): + thread_map.pop(container_id, None) + + +def build_thread(container, presenter, queue, log_args): + tailer = Thread( + target=tail_container_logs, + args=(container, presenter, queue, log_args)) + tailer.daemon = True + tailer.start() + return tailer + + +def build_thread_map(initial_containers, presenters, thread_args): + return { + container.id: build_thread(container, next(presenters), *thread_args) + # Container order is unspecified, so they are sorted by name in order to make + # container:presenter (log color) assignment deterministic when given a list of containers + # with the same names. + for container in sorted(initial_containers, key=attrgetter('name')) + } + + +class QueueItem(namedtuple('_QueueItem', 'item is_stop exc')): + + @classmethod + def new(cls, item): + return cls(item, None, None) + + @classmethod + def exception(cls, exc): + return cls(None, None, exc) + + @classmethod + def stop(cls, item=None): + return cls(item, True, None) + + +def tail_container_logs(container, presenter, queue, log_args): + generator = get_log_generator(container) + + try: + for item in generator(container, log_args): + queue.put(QueueItem.new(presenter.present(container, item))) + except Exception as e: + queue.put(QueueItem.exception(e)) + return + if log_args.get('follow'): + queue.put(QueueItem.new(presenter.color_func(wait_on_exit(container)))) + queue.put(QueueItem.stop(container.name)) + + +def get_log_generator(container): + if container.has_api_logs: + return build_log_generator + return build_no_log_generator + + +def build_no_log_generator(container, log_args): + """Return a generator that prints a warning about logs and waits for + container to exit. + """ + yield "WARNING: no logs are available with the '{}' log driver\n".format( + container.log_driver) + + +def build_log_generator(container, log_args): + # if the container doesn't have a log_stream we need to attach to container + # before log printer starts running + if container.log_stream is None: + stream = container.logs(stdout=True, stderr=True, stream=True, **log_args) + else: + stream = container.log_stream + + return split_buffer(stream) + + +def wait_on_exit(container): + try: + exit_code = container.wait() + return "{} exited with code {}\n".format(container.name, exit_code) + except APIError as e: + return "Unexpected API error for {} (HTTP code {})\nResponse body:\n{}\n".format( + container.name, e.response.status_code, + e.response.text or '[empty]' + ) + + +def start_producer_thread(thread_args): + producer = Thread(target=watch_events, args=thread_args) + producer.daemon = True + producer.start() + + +def watch_events(thread_map, event_stream, presenters, thread_args): + crashed_containers = set() + for event in event_stream: + if event['action'] == 'stop': + thread_map.pop(event['id'], None) + + if event['action'] == 'die': + thread_map.pop(event['id'], None) + crashed_containers.add(event['id']) + + if event['action'] != 'start': + continue + + if event['id'] in thread_map: + if thread_map[event['id']].is_alive(): + continue + # Container was stopped and started, we need a new thread + thread_map.pop(event['id'], None) + + # Container crashed so we should reattach to it + if event['id'] in crashed_containers: + container = event['container'] + if not container.is_restarting: + try: + container.attach_log_stream() + except APIError: + # Just ignore errors when reattaching to already crashed containers + pass + crashed_containers.remove(event['id']) + + thread_map[event['id']] = build_thread( + event['container'], + next(presenters), + *thread_args + ) + + +def consume_queue(queue, cascade_stop): + """Consume the queue by reading lines off of it and yielding them.""" + while True: + try: + item = queue.get(timeout=0.1) + except Empty: + yield None + continue + # See https://github.com/docker/compose/issues/189 + except thread.error: + raise ShutdownException() + + if item.exc: + raise item.exc + + if item.is_stop and not cascade_stop: + continue + + yield item.item diff --git a/compose/cli/main.py b/compose/cli/main.py new file mode 100644 index 00000000000..37521cc7b3d --- /dev/null +++ b/compose/cli/main.py @@ -0,0 +1,1657 @@ +import contextlib +import functools +import json +import logging +import os +import pipes +import re +import subprocess +import sys +from distutils.spawn import find_executable +from inspect import getdoc +from operator import attrgetter + +import docker.errors +import docker.utils + +from . import errors +from . import signals +from .. import __version__ +from ..config import ConfigurationError +from ..config import parse_environment +from ..config import parse_labels +from ..config import resolve_build_args +from ..config.environment import Environment +from ..config.serialize import serialize_config +from ..config.types import VolumeSpec +from ..const import IS_WINDOWS_PLATFORM +from ..errors import StreamParseError +from ..metrics.decorator import metrics +from ..progress_stream import StreamOutputError +from ..project import get_image_digests +from ..project import MissingDigests +from ..project import NoSuchService +from ..project import OneOffFilter +from ..project import ProjectError +from ..service import BuildAction +from ..service import BuildError +from ..service import ConvergenceStrategy +from ..service import ImageType +from ..service import NeedsBuildError +from ..service import OperationFailedError +from ..utils import filter_attached_for_up +from .command import get_config_from_options +from .command import get_project_dir +from .command import project_from_options +from .docopt_command import DocoptDispatcher +from .docopt_command import get_handler +from .docopt_command import NoSuchCommand +from .errors import UserError +from .formatter import ConsoleWarningFormatter +from .formatter import Formatter +from .log_printer import build_log_presenters +from .log_printer import LogPrinter +from .utils import get_version_info +from .utils import human_readable_file_size +from .utils import yesno +from compose.metrics.client import MetricsCommand +from compose.metrics.client import Status + + +if not IS_WINDOWS_PLATFORM: + from dockerpty.pty import PseudoTerminal, RunOperation, ExecOperation + +log = logging.getLogger(__name__) +console_handler = logging.StreamHandler(sys.stderr) + + +def main(): # noqa: C901 + signals.ignore_sigpipe() + command = None + try: + _, opts, command = DocoptDispatcher.get_command_and_options( + TopLevelCommand, + get_filtered_args(sys.argv[1:]), + {'options_first': True, 'version': get_version_info('compose')}) + except Exception: + pass + try: + command_func = dispatch() + command_func() + except (KeyboardInterrupt, signals.ShutdownException): + exit_with_metrics(command, "Aborting.", status=Status.FAILURE) + except (UserError, NoSuchService, ConfigurationError, + ProjectError, OperationFailedError) as e: + exit_with_metrics(command, e.msg, status=Status.FAILURE) + except BuildError as e: + reason = "" + if e.reason: + reason = " : " + e.reason + exit_with_metrics(command, + "Service '{}' failed to build{}".format(e.service.name, reason), + status=Status.FAILURE) + except StreamOutputError as e: + exit_with_metrics(command, e, status=Status.FAILURE) + except NeedsBuildError as e: + exit_with_metrics(command, + "Service '{}' needs to be built, but --no-build was passed.".format( + e.service.name), status=Status.FAILURE) + except NoSuchCommand as e: + commands = "\n".join(parse_doc_section("commands:", getdoc(e.supercommand))) + exit_with_metrics(e.command, "No such command: {}\n\n{}".format(e.command, commands)) + except (errors.ConnectionError, StreamParseError): + exit_with_metrics(command, status=Status.FAILURE) + except SystemExit as e: + status = Status.SUCCESS + if len(sys.argv) > 1 and '--help' not in sys.argv: + status = Status.FAILURE + + if command and len(sys.argv) >= 3 and sys.argv[2] == '--help': + command = '--help ' + command + + if not command and len(sys.argv) >= 2 and sys.argv[1] == '--help': + command = '--help' + + msg = e.args[0] if len(e.args) else "" + code = 0 + if isinstance(e.code, int): + code = e.code + exit_with_metrics(command, log_msg=msg, status=status, + exit_code=code) + + +def get_filtered_args(args): + if args[0] in ('-h', '--help'): + return [] + if args[0] == '--version': + return ['version'] + + +def exit_with_metrics(command, log_msg=None, status=Status.SUCCESS, exit_code=1): + if log_msg: + if not exit_code: + log.info(log_msg) + else: + log.error(log_msg) + + MetricsCommand(command, status=status).send_metrics() + sys.exit(exit_code) + + +def dispatch(): + setup_logging() + dispatcher = DocoptDispatcher( + TopLevelCommand, + {'options_first': True, 'version': get_version_info('compose')}) + + options, handler, command_options = dispatcher.parse(sys.argv[1:]) + setup_console_handler(console_handler, + options.get('--verbose'), + set_no_color_if_clicolor(options.get('--no-ansi')), + options.get("--log-level")) + setup_parallel_logger(set_no_color_if_clicolor(options.get('--no-ansi'))) + if options.get('--no-ansi'): + command_options['--no-color'] = True + return functools.partial(perform_command, options, handler, command_options) + + +def perform_command(options, handler, command_options): + if options['COMMAND'] in ('help', 'version'): + # Skip looking up the compose file. + handler(command_options) + return + + if options['COMMAND'] == 'config': + command = TopLevelCommand(None, options=options) + handler(command, command_options) + return + + project = project_from_options('.', options) + command = TopLevelCommand(project, options=options) + with errors.handle_connection_errors(project.client): + handler(command, command_options) + + +def setup_logging(): + root_logger = logging.getLogger() + root_logger.addHandler(console_handler) + root_logger.setLevel(logging.DEBUG) + + # Disable requests and docker-py logging + logging.getLogger("urllib3").propagate = False + logging.getLogger("requests").propagate = False + logging.getLogger("docker").propagate = False + + +def setup_parallel_logger(noansi): + if noansi: + import compose.parallel + compose.parallel.ParallelStreamWriter.set_noansi() + + +def setup_console_handler(handler, verbose, noansi=False, level=None): + if handler.stream.isatty() and noansi is False: + format_class = ConsoleWarningFormatter + else: + format_class = logging.Formatter + + if verbose: + handler.setFormatter(format_class('%(name)s.%(funcName)s: %(message)s')) + loglevel = logging.DEBUG + else: + handler.setFormatter(format_class()) + loglevel = logging.INFO + + if level is not None: + levels = { + 'DEBUG': logging.DEBUG, + 'INFO': logging.INFO, + 'WARNING': logging.WARNING, + 'ERROR': logging.ERROR, + 'CRITICAL': logging.CRITICAL, + } + loglevel = levels.get(level.upper()) + if loglevel is None: + raise UserError( + 'Invalid value for --log-level. Expected one of DEBUG, INFO, WARNING, ERROR, CRITICAL.' + ) + + handler.setLevel(loglevel) + + +# stolen from docopt master +def parse_doc_section(name, source): + pattern = re.compile('^([^\n]*' + name + '[^\n]*\n?(?:[ \t].*?(?:\n|$))*)', + re.IGNORECASE | re.MULTILINE) + return [s.strip() for s in pattern.findall(source)] + + +class TopLevelCommand: + """Define and run multi-container applications with Docker. + + Usage: + docker-compose [-f <arg>...] [--profile <name>...] [options] [--] [COMMAND] [ARGS...] + docker-compose -h|--help + + Options: + -f, --file FILE Specify an alternate compose file + (default: docker-compose.yml) + -p, --project-name NAME Specify an alternate project name + (default: directory name) + --profile NAME Specify a profile to enable + -c, --context NAME Specify a context name + --verbose Show more output + --log-level LEVEL Set log level (DEBUG, INFO, WARNING, ERROR, CRITICAL) + --no-ansi Do not print ANSI control characters + -v, --version Print version and exit + -H, --host HOST Daemon socket to connect to + + --tls Use TLS; implied by --tlsverify + --tlscacert CA_PATH Trust certs signed only by this CA + --tlscert CLIENT_CERT_PATH Path to TLS certificate file + --tlskey TLS_KEY_PATH Path to TLS key file + --tlsverify Use TLS and verify the remote + --skip-hostname-check Don't check the daemon's hostname against the + name specified in the client certificate + --project-directory PATH Specify an alternate working directory + (default: the path of the Compose file) + --compatibility If set, Compose will attempt to convert keys + in v3 files to their non-Swarm equivalent (DEPRECATED) + --env-file PATH Specify an alternate environment file + + Commands: + build Build or rebuild services + config Validate and view the Compose file + create Create services + down Stop and remove resources + events Receive real time events from containers + exec Execute a command in a running container + help Get help on a command + images List images + kill Kill containers + logs View output from containers + pause Pause services + port Print the public port for a port binding + ps List containers + pull Pull service images + push Push service images + restart Restart services + rm Remove stopped containers + run Run a one-off command + scale Set number of containers for a service + start Start services + stop Stop services + top Display the running processes + unpause Unpause services + up Create and start containers + version Show version information and quit + """ + + def __init__(self, project, options=None): + self.project = project + self.toplevel_options = options or {} + + @property + def project_dir(self): + return get_project_dir(self.toplevel_options) + + @property + def toplevel_environment(self): + environment_file = self.toplevel_options.get('--env-file') + return Environment.from_env_file(self.project_dir, environment_file) + + @metrics() + def build(self, options): + """ + Build or rebuild services. + + Services are built once and then tagged as `project_service`, + e.g. `composetest_db`. If you change a service's `Dockerfile` or the + contents of its build directory, you can run `docker-compose build` to rebuild it. + + Usage: build [options] [--build-arg key=val...] [--] [SERVICE...] + + Options: + --build-arg key=val Set build-time variables for services. + --compress Compress the build context using gzip. + --force-rm Always remove intermediate containers. + -m, --memory MEM Set memory limit for the build container. + --no-cache Do not use cache when building the image. + --no-rm Do not remove intermediate containers after a successful build. + --parallel Build images in parallel. + --progress string Set type of progress output (auto, plain, tty). + --pull Always attempt to pull a newer version of the image. + -q, --quiet Don't print anything to STDOUT + """ + service_names = options['SERVICE'] + build_args = options.get('--build-arg', None) + if build_args: + if not service_names and docker.utils.version_lt(self.project.client.api_version, '1.25'): + raise UserError( + '--build-arg is only supported when services are specified for API version < 1.25.' + ' Please use a Compose file version > 2.2 or specify which services to build.' + ) + build_args = resolve_build_args(build_args, self.toplevel_environment) + + native_builder = self.toplevel_environment.get_boolean('COMPOSE_DOCKER_CLI_BUILD', True) + + self.project.build( + service_names=options['SERVICE'], + no_cache=bool(options.get('--no-cache', False)), + pull=bool(options.get('--pull', False)), + force_rm=bool(options.get('--force-rm', False)), + memory=options.get('--memory'), + rm=not bool(options.get('--no-rm', False)), + build_args=build_args, + gzip=options.get('--compress', False), + parallel_build=options.get('--parallel', False), + silent=options.get('--quiet', False), + cli=native_builder, + progress=options.get('--progress'), + ) + + @metrics() + def config(self, options): + """ + Validate and view the Compose file. + + Usage: config [options] + + Options: + --resolve-image-digests Pin image tags to digests. + --no-interpolate Don't interpolate environment variables. + -q, --quiet Only validate the configuration, don't print + anything. + --services Print the service names, one per line. + --volumes Print the volume names, one per line. + --hash="*" Print the service config hash, one per line. + Set "service1,service2" for a list of specified services + or use the wildcard symbol to display all services. + """ + + additional_options = {'--no-interpolate': options.get('--no-interpolate')} + compose_config = get_config_from_options('.', self.toplevel_options, additional_options) + image_digests = None + + if options['--resolve-image-digests']: + self.project = project_from_options('.', self.toplevel_options, additional_options) + with errors.handle_connection_errors(self.project.client): + image_digests = image_digests_for_project(self.project) + + if options['--quiet']: + return + + if options['--services']: + print('\n'.join(service['name'] for service in compose_config.services)) + return + + if options['--volumes']: + print('\n'.join(volume for volume in compose_config.volumes)) + return + + if options['--hash'] is not None: + h = options['--hash'] + self.project = project_from_options('.', self.toplevel_options, additional_options) + services = [svc for svc in options['--hash'].split(',')] if h != '*' else None + with errors.handle_connection_errors(self.project.client): + for service in self.project.get_services(services): + print('{} {}'.format(service.name, service.config_hash)) + return + + print(serialize_config(compose_config, image_digests, not options['--no-interpolate'])) + + @metrics() + def create(self, options): + """ + Creates containers for a service. + This command is deprecated. Use the `up` command with `--no-start` instead. + + Usage: create [options] [SERVICE...] + + Options: + --force-recreate Recreate containers even if their configuration and + image haven't changed. Incompatible with --no-recreate. + --no-recreate If containers already exist, don't recreate them. + Incompatible with --force-recreate. + --no-build Don't build an image, even if it's missing. + --build Build images before creating containers. + """ + service_names = options['SERVICE'] + + log.warning( + 'The create command is deprecated. ' + 'Use the up command with the --no-start flag instead.' + ) + + self.project.create( + service_names=service_names, + strategy=convergence_strategy_from_opts(options), + do_build=build_action_from_opts(options), + ) + + @metrics() + def down(self, options): + """ + Stops containers and removes containers, networks, volumes, and images + created by `up`. + + By default, the only things removed are: + + - Containers for services defined in the Compose file + - Networks defined in the `networks` section of the Compose file + - The default network, if one is used + + Networks and volumes defined as `external` are never removed. + + Usage: down [options] + + Options: + --rmi type Remove images. Type must be one of: + 'all': Remove all images used by any service. + 'local': Remove only images that don't have a + custom tag set by the `image` field. + -v, --volumes Remove named volumes declared in the `volumes` + section of the Compose file and anonymous volumes + attached to containers. + --remove-orphans Remove containers for services not defined in the + Compose file + -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. + (default: 10) + """ + ignore_orphans = self.toplevel_environment.get_boolean('COMPOSE_IGNORE_ORPHANS') + + if ignore_orphans and options['--remove-orphans']: + raise UserError("COMPOSE_IGNORE_ORPHANS and --remove-orphans cannot be combined.") + + image_type = image_type_from_opt('--rmi', options['--rmi']) + timeout = timeout_from_opts(options) + self.project.down( + image_type, + options['--volumes'], + options['--remove-orphans'], + timeout=timeout, + ignore_orphans=ignore_orphans) + + def events(self, options): + """ + Receive real time events from containers. + + Usage: events [options] [--] [SERVICE...] + + Options: + --json Output events as a stream of json objects + """ + + def format_event(event): + attributes = ["%s=%s" % item for item in event['attributes'].items()] + return ("{time} {type} {action} {id} ({attrs})").format( + attrs=", ".join(sorted(attributes)), + **event) + + def json_format_event(event): + event['time'] = event['time'].isoformat() + event.pop('container') + return json.dumps(event) + + for event in self.project.events(): + formatter = json_format_event if options['--json'] else format_event + print(formatter(event)) + sys.stdout.flush() + + @metrics("exec") + def exec_command(self, options): + """ + Execute a command in a running container + + Usage: exec [options] [-e KEY=VAL...] [--] SERVICE COMMAND [ARGS...] + + Options: + -d, --detach Detached mode: Run command in the background. + --privileged Give extended privileges to the process. + -u, --user USER Run the command as this user. + -T Disable pseudo-tty allocation. By default `docker-compose exec` + allocates a TTY. + --index=index index of the container if there are multiple + instances of a service [default: 1] + -e, --env KEY=VAL Set environment variables (can be used multiple times, + not supported in API < 1.25) + -w, --workdir DIR Path to workdir directory for this command. + """ + use_cli = not self.toplevel_environment.get_boolean('COMPOSE_INTERACTIVE_NO_CLI') + index = int(options.get('--index')) + service = self.project.get_service(options['SERVICE']) + detach = options.get('--detach') + + if options['--env'] and docker.utils.version_lt(self.project.client.api_version, '1.25'): + raise UserError("Setting environment for exec is not supported in API < 1.25 (%s)" + % self.project.client.api_version) + + if options['--workdir'] and docker.utils.version_lt(self.project.client.api_version, '1.35'): + raise UserError("Setting workdir for exec is not supported in API < 1.35 (%s)" + % self.project.client.api_version) + + try: + container = service.get_container(number=index) + except ValueError as e: + raise UserError(str(e)) + command = [options['COMMAND']] + options['ARGS'] + tty = not options["-T"] + + if IS_WINDOWS_PLATFORM or use_cli and not detach: + sys.exit(call_docker( + build_exec_command(options, container.id, command), + self.toplevel_options, self.toplevel_environment) + ) + + create_exec_options = { + "privileged": options["--privileged"], + "user": options["--user"], + "tty": tty, + "stdin": True, + "workdir": options["--workdir"], + } + + if docker.utils.version_gte(self.project.client.api_version, '1.25'): + create_exec_options["environment"] = options["--env"] + + exec_id = container.create_exec(command, **create_exec_options) + + if detach: + container.start_exec(exec_id, tty=tty, stream=True) + return + + signals.set_signal_handler_to_shutdown() + try: + operation = ExecOperation( + self.project.client, + exec_id, + interactive=tty, + ) + pty = PseudoTerminal(self.project.client, operation) + pty.start() + except signals.ShutdownException: + log.info("received shutdown exception: closing") + exit_code = self.project.client.exec_inspect(exec_id).get("ExitCode") + sys.exit(exit_code) + + @classmethod + @metrics() + def help(cls, options): + """ + Get help on a command. + + Usage: help [COMMAND] + """ + if options['COMMAND']: + subject = get_handler(cls, options['COMMAND']) + else: + subject = cls + + print(getdoc(subject)) + + @metrics() + def images(self, options): + """ + List images used by the created containers. + Usage: images [options] [--] [SERVICE...] + + Options: + -q, --quiet Only display IDs + """ + containers = sorted( + self.project.containers(service_names=options['SERVICE'], stopped=True) + + self.project.containers(service_names=options['SERVICE'], one_off=OneOffFilter.only), + key=attrgetter('name')) + + if options['--quiet']: + for image in {c.image for c in containers}: + print(image.split(':')[1]) + return + + def add_default_tag(img_name): + if ':' not in img_name.split('/')[-1]: + return '{}:latest'.format(img_name) + return img_name + + headers = [ + 'Container', + 'Repository', + 'Tag', + 'Image Id', + 'Size' + ] + rows = [] + for container in containers: + image_config = container.image_config + service = self.project.get_service(container.service) + index = 0 + img_name = add_default_tag(service.image_name) + if img_name in image_config['RepoTags']: + index = image_config['RepoTags'].index(img_name) + repo_tags = ( + image_config['RepoTags'][index].rsplit(':', 1) if image_config['RepoTags'] + else ('<none>', '<none>') + ) + + image_id = image_config['Id'].split(':')[1][:12] + size = human_readable_file_size(image_config['Size']) + rows.append([ + container.name, + repo_tags[0], + repo_tags[1], + image_id, + size + ]) + print(Formatter.table(headers, rows)) + + @metrics() + def kill(self, options): + """ + Force stop service containers. + + Usage: kill [options] [--] [SERVICE...] + + Options: + -s SIGNAL SIGNAL to send to the container. + Default signal is SIGKILL. + """ + signal = options.get('-s', 'SIGKILL') + + self.project.kill(service_names=options['SERVICE'], signal=signal) + + @metrics() + def logs(self, options): + """ + View output from containers. + + Usage: logs [options] [--] [SERVICE...] + + Options: + --no-color Produce monochrome output. + -f, --follow Follow log output. + -t, --timestamps Show timestamps. + --tail="all" Number of lines to show from the end of the logs + for each container. + --no-log-prefix Don't print prefix in logs. + """ + containers = self.project.containers(service_names=options['SERVICE'], stopped=True) + + tail = options['--tail'] + if tail is not None: + if tail.isdigit(): + tail = int(tail) + elif tail != 'all': + raise UserError("tail flag must be all or a number") + log_args = { + 'follow': options['--follow'], + 'tail': tail, + 'timestamps': options['--timestamps'] + } + print("Attaching to", list_containers(containers)) + log_printer_from_project( + self.project, + containers, + set_no_color_if_clicolor(options['--no-color']), + log_args, + event_stream=self.project.events(service_names=options['SERVICE']), + keep_prefix=not options['--no-log-prefix']).run() + + @metrics() + def pause(self, options): + """ + Pause services. + + Usage: pause [SERVICE...] + """ + containers = self.project.pause(service_names=options['SERVICE']) + exit_if(not containers, 'No containers to pause', 1) + + @metrics() + def port(self, options): + """ + Print the public port for a port binding. + + Usage: port [options] [--] SERVICE PRIVATE_PORT + + Options: + --protocol=proto tcp or udp [default: tcp] + --index=index index of the container if there are multiple + instances of a service [default: 1] + """ + index = int(options.get('--index')) + service = self.project.get_service(options['SERVICE']) + try: + container = service.get_container(number=index) + except ValueError as e: + raise UserError(str(e)) + print(container.get_local_port( + options['PRIVATE_PORT'], + protocol=options.get('--protocol') or 'tcp') or '') + + @metrics() + def ps(self, options): + """ + List containers. + + Usage: ps [options] [--] [SERVICE...] + + Options: + -q, --quiet Only display IDs + --services Display services + --filter KEY=VAL Filter services by a property + -a, --all Show all stopped containers (including those created by the run command) + """ + if options['--quiet'] and options['--services']: + raise UserError('--quiet and --services cannot be combined') + + if options['--services']: + filt = build_filter(options.get('--filter')) + services = self.project.services + if filt: + services = filter_services(filt, services, self.project) + print('\n'.join(service.name for service in services)) + return + + if options['--all']: + containers = sorted(self.project.containers(service_names=options['SERVICE'], + one_off=OneOffFilter.include, stopped=True), + key=attrgetter('name')) + else: + containers = sorted( + self.project.containers(service_names=options['SERVICE'], stopped=True) + + self.project.containers(service_names=options['SERVICE'], one_off=OneOffFilter.only), + key=attrgetter('name')) + + if options['--quiet']: + for container in containers: + print(container.id) + else: + headers = [ + 'Name', + 'Command', + 'State', + 'Ports', + ] + rows = [] + for container in containers: + command = container.human_readable_command + if len(command) > 30: + command = '%s ...' % command[:26] + rows.append([ + container.name, + command, + container.human_readable_state, + container.human_readable_ports, + ]) + print(Formatter.table(headers, rows)) + + @metrics() + def pull(self, options): + """ + Pulls images for services defined in a Compose file, but does not start the containers. + + Usage: pull [options] [--] [SERVICE...] + + Options: + --ignore-pull-failures Pull what it can and ignores images with pull failures. + --parallel Deprecated, pull multiple images in parallel (enabled by default). + --no-parallel Disable parallel pulling. + -q, --quiet Pull without printing progress information + --include-deps Also pull services declared as dependencies + """ + if options.get('--parallel'): + log.warning('--parallel option is deprecated and will be removed in future versions.') + self.project.pull( + service_names=options['SERVICE'], + ignore_pull_failures=options.get('--ignore-pull-failures'), + parallel_pull=not options.get('--no-parallel'), + silent=options.get('--quiet'), + include_deps=options.get('--include-deps'), + ) + + @metrics() + def push(self, options): + """ + Pushes images for services. + + Usage: push [options] [--] [SERVICE...] + + Options: + --ignore-push-failures Push what it can and ignores images with push failures. + """ + self.project.push( + service_names=options['SERVICE'], + ignore_push_failures=options.get('--ignore-push-failures') + ) + + @metrics() + def rm(self, options): + """ + Removes stopped service containers. + + By default, anonymous volumes attached to containers will not be removed. You + can override this with `-v`. To list all volumes, use `docker volume ls`. + + Any data which is not in a volume will be lost. + + Usage: rm [options] [--] [SERVICE...] + + Options: + -f, --force Don't ask to confirm removal + -s, --stop Stop the containers, if required, before removing + -v Remove any anonymous volumes attached to containers + -a, --all Deprecated - no effect. + """ + if options.get('--all'): + log.warning( + '--all flag is obsolete. This is now the default behavior ' + 'of `docker-compose rm`' + ) + one_off = OneOffFilter.include + + if options.get('--stop'): + self.project.stop(service_names=options['SERVICE'], one_off=one_off) + + all_containers = self.project.containers( + service_names=options['SERVICE'], stopped=True, one_off=one_off + ) + stopped_containers = [c for c in all_containers if not c.is_running] + + if len(stopped_containers) > 0: + print("Going to remove", list_containers(stopped_containers)) + if options.get('--force') \ + or yesno("Are you sure? [yN] ", default=False): + self.project.remove_stopped( + service_names=options['SERVICE'], + v=options.get('-v', False), + one_off=one_off + ) + else: + print("No stopped containers") + + @metrics() + def run(self, options): + """ + Run a one-off command on a service. + + For example: + + $ docker-compose run web python manage.py shell + + By default, linked services will be started, unless they are already + running. If you do not want to start linked services, use + `docker-compose run --no-deps SERVICE COMMAND [ARGS...]`. + + Usage: + run [options] [-v VOLUME...] [-p PORT...] [-e KEY=VAL...] [-l KEY=VALUE...] [--] + SERVICE [COMMAND] [ARGS...] + + Options: + -d, --detach Detached mode: Run container in the background, print + new container name. + --name NAME Assign a name to the container + --entrypoint CMD Override the entrypoint of the image. + -e KEY=VAL Set an environment variable (can be used multiple times) + -l, --label KEY=VAL Add or override a label (can be used multiple times) + -u, --user="" Run as specified username or uid + --no-deps Don't start linked services. + --rm Remove container after run. Ignored in detached mode. + -p, --publish=[] Publish a container's port(s) to the host + --service-ports Run command with the service's ports enabled and mapped + to the host. + --use-aliases Use the service's network aliases in the network(s) the + container connects to. + -v, --volume=[] Bind mount a volume (default []) + -T Disable pseudo-tty allocation. By default `docker-compose run` + allocates a TTY. + -w, --workdir="" Working directory inside the container + """ + service = self.project.get_service(options['SERVICE']) + detach = options.get('--detach') + + if options['--publish'] and options['--service-ports']: + raise UserError( + 'Service port mapping and manual port mapping ' + 'can not be used together' + ) + + if options['COMMAND'] is not None: + command = [options['COMMAND']] + options['ARGS'] + elif options['--entrypoint'] is not None: + command = [] + else: + command = service.options.get('command') + + options['stdin_open'] = service.options.get('stdin_open', True) + + container_options = build_one_off_container_options(options, detach, command) + run_one_off_container( + container_options, self.project, service, options, + self.toplevel_options, self.toplevel_environment + ) + + @metrics() + def scale(self, options): + """ + Set number of containers to run for a service. + + Numbers are specified in the form `service=num` as arguments. + For example: + + $ docker-compose scale web=2 worker=3 + + This command is deprecated. Use the up command with the `--scale` flag + instead. + + Usage: scale [options] [SERVICE=NUM...] + + Options: + -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. + (default: 10) + """ + timeout = timeout_from_opts(options) + + log.warning( + 'The scale command is deprecated. ' + 'Use the up command with the --scale flag instead.' + ) + + for service_name, num in parse_scale_args(options['SERVICE=NUM']).items(): + self.project.get_service(service_name).scale(num, timeout=timeout) + + @metrics() + def start(self, options): + """ + Start existing containers. + + Usage: start [SERVICE...] + """ + containers = self.project.start(service_names=options['SERVICE']) + exit_if(not containers, 'No containers to start', 1) + + @metrics() + def stop(self, options): + """ + Stop running containers without removing them. + + They can be started again with `docker-compose start`. + + Usage: stop [options] [--] [SERVICE...] + + Options: + -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. + (default: 10) + """ + timeout = timeout_from_opts(options) + self.project.stop(service_names=options['SERVICE'], timeout=timeout) + + @metrics() + def restart(self, options): + """ + Restart running containers. + + Usage: restart [options] [--] [SERVICE...] + + Options: + -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. + (default: 10) + """ + timeout = timeout_from_opts(options) + containers = self.project.restart(service_names=options['SERVICE'], timeout=timeout) + exit_if(not containers, 'No containers to restart', 1) + + @metrics() + def top(self, options): + """ + Display the running processes + + Usage: top [SERVICE...] + + """ + containers = sorted( + self.project.containers(service_names=options['SERVICE'], stopped=False) + + self.project.containers(service_names=options['SERVICE'], one_off=OneOffFilter.only), + key=attrgetter('name') + ) + + for idx, container in enumerate(containers): + if idx > 0: + print() + + top_data = self.project.client.top(container.name) + headers = top_data.get("Titles") + rows = [] + + for process in top_data.get("Processes", []): + rows.append(process) + + print(container.name) + print(Formatter.table(headers, rows)) + + @metrics() + def unpause(self, options): + """ + Unpause services. + + Usage: unpause [SERVICE...] + """ + containers = self.project.unpause(service_names=options['SERVICE']) + exit_if(not containers, 'No containers to unpause', 1) + + @metrics() + def up(self, options): + """ + Builds, (re)creates, starts, and attaches to containers for a service. + + Unless they are already running, this command also starts any linked services. + + The `docker-compose up` command aggregates the output of each container. When + the command exits, all containers are stopped. Running `docker-compose up -d` + starts the containers in the background and leaves them running. + + If there are existing containers for a service, and the service's configuration + or image was changed after the container's creation, `docker-compose up` picks + up the changes by stopping and recreating the containers (preserving mounted + volumes). To prevent Compose from picking up changes, use the `--no-recreate` + flag. + + If you want to force Compose to stop and recreate all containers, use the + `--force-recreate` flag. + + Usage: up [options] [--scale SERVICE=NUM...] [--] [SERVICE...] + + Options: + -d, --detach Detached mode: Run containers in the background, + print new container names. Incompatible with + --abort-on-container-exit. + --no-color Produce monochrome output. + --quiet-pull Pull without printing progress information + --no-deps Don't start linked services. + --force-recreate Recreate containers even if their configuration + and image haven't changed. + --always-recreate-deps Recreate dependent containers. + Incompatible with --no-recreate. + --no-recreate If containers already exist, don't recreate + them. Incompatible with --force-recreate and -V. + --no-build Don't build an image, even if it's missing. + --no-start Don't start the services after creating them. + --build Build images before starting containers. + --abort-on-container-exit Stops all containers if any container was + stopped. Incompatible with -d. + --attach-dependencies Attach to dependent containers. + -t, --timeout TIMEOUT Use this timeout in seconds for container + shutdown when attached or when containers are + already running. (default: 10) + -V, --renew-anon-volumes Recreate anonymous volumes instead of retrieving + data from the previous containers. + --remove-orphans Remove containers for services not defined + in the Compose file. + --exit-code-from SERVICE Return the exit code of the selected service + container. Implies --abort-on-container-exit. + --scale SERVICE=NUM Scale SERVICE to NUM instances. Overrides the + `scale` setting in the Compose file if present. + --no-log-prefix Don't print prefix in logs. + """ + start_deps = not options['--no-deps'] + always_recreate_deps = options['--always-recreate-deps'] + exit_value_from = exitval_from_opts(options, self.project) + cascade_stop = options['--abort-on-container-exit'] + service_names = options['SERVICE'] + timeout = timeout_from_opts(options) + remove_orphans = options['--remove-orphans'] + detached = options.get('--detach') + no_start = options.get('--no-start') + attach_dependencies = options.get('--attach-dependencies') + keep_prefix = not options['--no-log-prefix'] + + if detached and (cascade_stop or exit_value_from or attach_dependencies): + raise UserError( + "-d cannot be combined with --abort-on-container-exit or --attach-dependencies.") + + ignore_orphans = self.toplevel_environment.get_boolean('COMPOSE_IGNORE_ORPHANS') + + if ignore_orphans and remove_orphans: + raise UserError("COMPOSE_IGNORE_ORPHANS and --remove-orphans cannot be combined.") + + opts = ['--detach', '--abort-on-container-exit', '--exit-code-from', '--attach-dependencies'] + for excluded in [x for x in opts if options.get(x) and no_start]: + raise UserError('--no-start and {} cannot be combined.'.format(excluded)) + + native_builder = self.toplevel_environment.get_boolean('COMPOSE_DOCKER_CLI_BUILD', True) + + with up_shutdown_context(self.project, service_names, timeout, detached): + warn_for_swarm_mode(self.project.client) + + def up(rebuild): + return self.project.up( + service_names=service_names, + start_deps=start_deps, + strategy=convergence_strategy_from_opts(options), + do_build=build_action_from_opts(options), + timeout=timeout, + detached=detached, + remove_orphans=remove_orphans, + ignore_orphans=ignore_orphans, + scale_override=parse_scale_args(options['--scale']), + start=not no_start, + always_recreate_deps=always_recreate_deps, + reset_container_image=rebuild, + renew_anonymous_volumes=options.get('--renew-anon-volumes'), + silent=options.get('--quiet-pull'), + cli=native_builder, + attach_dependencies=attach_dependencies, + ) + + try: + to_attach = up(False) + except docker.errors.ImageNotFound as e: + log.error( + "The image for the service you're trying to recreate has been removed. " + "If you continue, volume data could be lost. Consider backing up your data " + "before continuing.\n" + ) + res = yesno("Continue with the new image? [yN]", False) + if res is None or not res: + raise e + + to_attach = up(True) + + if detached or no_start: + return + + attached_containers = filter_attached_containers( + to_attach, + service_names, + attach_dependencies) + + log_printer = log_printer_from_project( + self.project, + attached_containers, + set_no_color_if_clicolor(options['--no-color']), + {'follow': True}, + cascade_stop, + event_stream=self.project.events(service_names=service_names), + keep_prefix=keep_prefix) + print("Attaching to", list_containers(log_printer.containers)) + cascade_starter = log_printer.run() + + if cascade_stop: + print("Aborting on container exit...") + all_containers = self.project.containers(service_names=options['SERVICE'], stopped=True) + exit_code = compute_exit_code( + exit_value_from, attached_containers, cascade_starter, all_containers + ) + + self.project.stop(service_names=service_names, timeout=timeout) + if exit_value_from: + exit_code = compute_service_exit_code(exit_value_from, attached_containers) + + sys.exit(exit_code) + + @classmethod + @metrics() + def version(cls, options): + """ + Show version information and quit. + + Usage: version [--short] + + Options: + --short Shows only Compose's version number. + """ + if options['--short']: + print(__version__) + else: + print(get_version_info('full')) + + +def compute_service_exit_code(exit_value_from, attached_containers): + candidates = list(filter( + lambda c: c.service == exit_value_from, + attached_containers)) + if not candidates: + log.error( + 'No containers matching the spec "{}" ' + 'were run.'.format(exit_value_from) + ) + return 2 + if len(candidates) > 1: + exit_values = filter( + lambda e: e != 0, + [c.inspect()['State']['ExitCode'] for c in candidates] + ) + + return exit_values[0] + return candidates[0].inspect()['State']['ExitCode'] + + +def compute_exit_code(exit_value_from, attached_containers, cascade_starter, all_containers): + exit_code = 0 + for e in all_containers: + if (not e.is_running and cascade_starter == e.name): + if not e.exit_code == 0: + exit_code = e.exit_code + break + + return exit_code + + +def convergence_strategy_from_opts(options): + no_recreate = options['--no-recreate'] + force_recreate = options['--force-recreate'] + renew_anonymous_volumes = options.get('--renew-anon-volumes') + if force_recreate and no_recreate: + raise UserError("--force-recreate and --no-recreate cannot be combined.") + + if no_recreate and renew_anonymous_volumes: + raise UserError('--no-recreate and --renew-anon-volumes cannot be combined.') + + if force_recreate or renew_anonymous_volumes: + return ConvergenceStrategy.always + + if no_recreate: + return ConvergenceStrategy.never + + return ConvergenceStrategy.changed + + +def timeout_from_opts(options): + timeout = options.get('--timeout') + return None if timeout is None else int(timeout) + + +def image_digests_for_project(project): + try: + return get_image_digests(project) + + except MissingDigests as e: + def list_images(images): + return "\n".join(" {}".format(name) for name in sorted(images)) + + paras = ["Some images are missing digests."] + + if e.needs_push: + command_hint = ( + "Use `docker push {}` to push them. " + .format(" ".join(sorted(e.needs_push))) + ) + paras += [ + "The following images can be pushed:", + list_images(e.needs_push), + command_hint, + ] + + if e.needs_pull: + command_hint = ( + "Use `docker pull {}` to pull them. " + .format(" ".join(sorted(e.needs_pull))) + ) + + paras += [ + "The following images need to be pulled:", + list_images(e.needs_pull), + command_hint, + ] + + raise UserError("\n\n".join(paras)) + + +def exitval_from_opts(options, project): + exit_value_from = options.get('--exit-code-from') + if exit_value_from: + if not options.get('--abort-on-container-exit'): + log.warning('using --exit-code-from implies --abort-on-container-exit') + options['--abort-on-container-exit'] = True + if exit_value_from not in [s.name for s in project.get_services()]: + log.error('No service named "%s" was found in your compose file.', + exit_value_from) + sys.exit(2) + return exit_value_from + + +def image_type_from_opt(flag, value): + if not value: + return ImageType.none + try: + return ImageType[value] + except KeyError: + raise UserError("%s flag must be one of: all, local" % flag) + + +def build_action_from_opts(options): + if options['--build'] and options['--no-build']: + raise UserError("--build and --no-build can not be combined.") + + if options['--build']: + return BuildAction.force + + if options['--no-build']: + return BuildAction.skip + + return BuildAction.none + + +def build_one_off_container_options(options, detach, command): + container_options = { + 'command': command, + 'tty': not (detach or options['-T'] or not sys.stdin.isatty()), + 'stdin_open': options.get('stdin_open'), + 'detach': detach, + } + + if options['-e']: + container_options['environment'] = Environment.from_command_line( + parse_environment(options['-e']) + ) + + if options['--label']: + container_options['labels'] = parse_labels(options['--label']) + + if options.get('--entrypoint') is not None: + container_options['entrypoint'] = ( + [""] if options['--entrypoint'] == '' else options['--entrypoint'] + ) + + # Ensure that run command remains one-off (issue #6302) + container_options['restart'] = None + + if options['--user']: + container_options['user'] = options.get('--user') + + if not options['--service-ports']: + container_options['ports'] = [] + + if options['--publish']: + container_options['ports'] = options.get('--publish') + + if options['--name']: + container_options['name'] = options['--name'] + + if options['--workdir']: + container_options['working_dir'] = options['--workdir'] + + if options['--volume']: + volumes = [VolumeSpec.parse(i) for i in options['--volume']] + container_options['volumes'] = volumes + + return container_options + + +def run_one_off_container(container_options, project, service, options, toplevel_options, + toplevel_environment): + native_builder = toplevel_environment.get_boolean('COMPOSE_DOCKER_CLI_BUILD') + detach = options.get('--detach') + use_network_aliases = options.get('--use-aliases') + service.scale_num = 1 + containers = project.up( + service_names=[service.name], + start_deps=not options['--no-deps'], + strategy=ConvergenceStrategy.never, + detached=True, + rescale=False, + cli=native_builder, + one_off=True, + override_options=container_options, + ) + try: + container = next(c for c in containers if c.service == service.name) + except StopIteration: + raise OperationFailedError('Could not bring up the requested service') + + if detach: + service.start_container(container, use_network_aliases) + print(container.name) + return + + def remove_container(): + if options['--rm']: + project.client.remove_container(container.id, force=True, v=True) + + use_cli = not toplevel_environment.get_boolean('COMPOSE_INTERACTIVE_NO_CLI') + + signals.set_signal_handler_to_shutdown() + signals.set_signal_handler_to_hang_up() + try: + try: + if IS_WINDOWS_PLATFORM or use_cli: + service.connect_container_to_networks(container, use_network_aliases) + exit_code = call_docker( + get_docker_start_call(container_options, container.id), + toplevel_options, toplevel_environment + ) + else: + operation = RunOperation( + project.client, + container.id, + interactive=not options['-T'], + logs=False, + ) + pty = PseudoTerminal(project.client, operation) + sockets = pty.sockets() + service.start_container(container, use_network_aliases) + pty.start(sockets) + exit_code = container.wait() + except (signals.ShutdownException): + project.client.stop(container.id) + exit_code = 1 + except (signals.ShutdownException, signals.HangUpException): + project.client.kill(container.id) + remove_container() + sys.exit(2) + + remove_container() + sys.exit(exit_code) + + +def get_docker_start_call(container_options, container_id): + docker_call = ["start"] + if not container_options.get('detach'): + docker_call.append("--attach") + if container_options.get('stdin_open'): + docker_call.append("--interactive") + docker_call.append(container_id) + return docker_call + + +def log_printer_from_project( + project, + containers, + monochrome, + log_args, + cascade_stop=False, + event_stream=None, + keep_prefix=True, +): + return LogPrinter( + containers, + build_log_presenters(project.service_names, monochrome, keep_prefix), + event_stream or project.events(), + cascade_stop=cascade_stop, + log_args=log_args) + + +def filter_attached_containers(containers, service_names, attach_dependencies=False): + return filter_attached_for_up( + containers, + service_names, + attach_dependencies, + lambda container: container.service) + + +@contextlib.contextmanager +def up_shutdown_context(project, service_names, timeout, detached): + if detached: + yield + return + + signals.set_signal_handler_to_shutdown() + try: + try: + yield + except signals.ShutdownException: + print("Gracefully stopping... (press Ctrl+C again to force)") + project.stop(service_names=service_names, timeout=timeout) + except signals.ShutdownException: + project.kill(service_names=service_names) + sys.exit(2) + + +def list_containers(containers): + return ", ".join(c.name for c in containers) + + +def exit_if(condition, message, exit_code): + if condition: + log.error(message) + raise SystemExit(exit_code) + + +def call_docker(args, dockeropts, environment): + executable_path = find_executable('docker') + if not executable_path: + raise UserError(errors.docker_not_found_msg("Couldn't find `docker` binary.")) + + tls = dockeropts.get('--tls', False) + ca_cert = dockeropts.get('--tlscacert') + cert = dockeropts.get('--tlscert') + key = dockeropts.get('--tlskey') + verify = dockeropts.get('--tlsverify') + host = dockeropts.get('--host') + context = dockeropts.get('--context') + tls_options = [] + if tls: + tls_options.append('--tls') + if ca_cert: + tls_options.extend(['--tlscacert', ca_cert]) + if cert: + tls_options.extend(['--tlscert', cert]) + if key: + tls_options.extend(['--tlskey', key]) + if verify: + tls_options.append('--tlsverify') + if host: + tls_options.extend( + ['--host', re.sub(r'^https?://', 'tcp://', host.lstrip('='))] + ) + if context: + tls_options.extend( + ['--context', context] + ) + + args = [executable_path] + tls_options + args + log.debug(" ".join(map(pipes.quote, args))) + + filtered_env = {k: v for k, v in environment.items() if v is not None} + + return subprocess.call(args, env=filtered_env) + + +def parse_scale_args(options): + res = {} + for s in options: + if '=' not in s: + raise UserError('Arguments to scale should be in the form service=num') + service_name, num = s.split('=', 1) + try: + num = int(num) + except ValueError: + raise UserError( + 'Number of containers for service "%s" is not a number' % service_name + ) + res[service_name] = num + return res + + +def build_exec_command(options, container_id, command): + args = ["exec"] + + if options["--detach"]: + args += ["--detach"] + else: + args += ["--interactive"] + + if not options["-T"]: + args += ["--tty"] + + if options["--privileged"]: + args += ["--privileged"] + + if options["--user"]: + args += ["--user", options["--user"]] + + if options["--env"]: + for env_variable in options["--env"]: + args += ["--env", env_variable] + + if options["--workdir"]: + args += ["--workdir", options["--workdir"]] + + args += [container_id] + args += command + return args + + +def has_container_with_state(containers, state): + states = { + 'running': lambda c: c.is_running, + 'stopped': lambda c: not c.is_running, + 'paused': lambda c: c.is_paused, + 'restarting': lambda c: c.is_restarting, + } + for container in containers: + if state not in states: + raise UserError("Invalid state: %s" % state) + if states[state](container): + return True + + +def filter_services(filt, services, project): + def should_include(service): + for f in filt: + if f == 'status': + state = filt[f] + containers = project.containers([service.name], stopped=True) + if not has_container_with_state(containers, state): + return False + elif f == 'source': + source = filt[f] + if source == 'image' or source == 'build': + if source not in service.options: + return False + else: + raise UserError("Invalid value for source filter: %s" % source) + else: + raise UserError("Invalid filter: %s" % f) + return True + + return filter(should_include, services) + + +def build_filter(arg): + filt = {} + if arg is not None: + if '=' not in arg: + raise UserError("Arguments to --filter should be in form KEY=VAL") + key, val = arg.split('=', 1) + filt[key] = val + return filt + + +def warn_for_swarm_mode(client): + info = client.info() + if info.get('Swarm', {}).get('LocalNodeState') == 'active': + if info.get('ServerVersion', '').startswith('ucp'): + # UCP does multi-node scheduling with traditional Compose files. + return + + log.warning( + "The Docker Engine you're using is running in swarm mode.\n\n" + "Compose does not use swarm mode to deploy services to multiple nodes in a swarm. " + "All containers will be scheduled on the current node.\n\n" + "To deploy your application across the swarm, " + "use `docker stack deploy`.\n" + ) + + +def set_no_color_if_clicolor(no_color_flag): + return no_color_flag or os.environ.get('CLICOLOR') == "0" diff --git a/compose/cli/signals.py b/compose/cli/signals.py new file mode 100644 index 00000000000..0244e70189a --- /dev/null +++ b/compose/cli/signals.py @@ -0,0 +1,41 @@ +import signal + +from ..const import IS_WINDOWS_PLATFORM + + +class ShutdownException(Exception): + pass + + +class HangUpException(Exception): + pass + + +def shutdown(signal, frame): + raise ShutdownException() + + +def set_signal_handler(handler): + signal.signal(signal.SIGINT, handler) + signal.signal(signal.SIGTERM, handler) + + +def set_signal_handler_to_shutdown(): + set_signal_handler(shutdown) + + +def hang_up(signal, frame): + raise HangUpException() + + +def set_signal_handler_to_hang_up(): + # on Windows a ValueError will be raised if trying to set signal handler for SIGHUP + if not IS_WINDOWS_PLATFORM: + signal.signal(signal.SIGHUP, hang_up) + + +def ignore_sigpipe(): + # Restore default behavior for SIGPIPE instead of raising + # an exception when encountered. + if not IS_WINDOWS_PLATFORM: + signal.signal(signal.SIGPIPE, signal.SIG_DFL) diff --git a/compose/cli/utils.py b/compose/cli/utils.py new file mode 100644 index 00000000000..6a4615a9660 --- /dev/null +++ b/compose/cli/utils.py @@ -0,0 +1,144 @@ +import math +import os +import platform +import ssl +import subprocess +import sys + +import distro +import docker + +import compose +from ..const import IS_WINDOWS_PLATFORM + + +def yesno(prompt, default=None): + """ + Prompt the user for a yes or no. + + Can optionally specify a default value, which will only be + used if they enter a blank line. + + Unrecognised input (anything other than "y", "n", "yes", + "no" or "") will return None. + """ + answer = input(prompt).strip().lower() + + if answer == "y" or answer == "yes": + return True + elif answer == "n" or answer == "no": + return False + elif answer == "": + return default + else: + return None + + +def input(prompt): + """ + Version of input (raw_input in Python 2) which forces a flush of sys.stdout + to avoid problems where the prompt fails to appear due to line buffering + """ + sys.stdout.write(prompt) + sys.stdout.flush() + return sys.stdin.readline().rstrip('\n') + + +def call_silently(*args, **kwargs): + """ + Like subprocess.call(), but redirects stdout and stderr to /dev/null. + """ + with open(os.devnull, 'w') as shutup: + try: + return subprocess.call(*args, stdout=shutup, stderr=shutup, **kwargs) + except OSError: + # On Windows, subprocess.call() can still raise exceptions. Normalize + # to POSIXy behaviour by returning a nonzero exit code. + return 1 + + +def is_mac(): + return platform.system() == 'Darwin' + + +def is_ubuntu(): + return platform.system() == 'Linux' and distro.linux_distribution()[0] == 'Ubuntu' + + +def is_windows(): + return IS_WINDOWS_PLATFORM + + +def get_version_info(scope): + versioninfo = 'docker-compose version {}, build {}'.format( + compose.__version__, + get_build_version()) + + if scope == 'compose': + return versioninfo + if scope == 'full': + return ( + "{}\n" + "docker-py version: {}\n" + "{} version: {}\n" + "OpenSSL version: {}" + ).format( + versioninfo, + docker.version, + platform.python_implementation(), + platform.python_version(), + ssl.OPENSSL_VERSION) + + raise ValueError("{} is not a valid version scope".format(scope)) + + +def get_build_version(): + filename = os.path.join(os.path.dirname(compose.__file__), 'GITSHA') + if not os.path.exists(filename): + return 'unknown' + + with open(filename) as fh: + return fh.read().strip() + + +def is_docker_for_mac_installed(): + return is_mac() and os.path.isdir('/Applications/Docker.app') + + +def generate_user_agent(): + parts = [ + "docker-compose/{}".format(compose.__version__), + "docker-py/{}".format(docker.__version__), + ] + try: + p_system = platform.system() + p_release = platform.release() + except OSError: + pass + else: + parts.append("{}/{}".format(p_system, p_release)) + return " ".join(parts) + + +def human_readable_file_size(size): + suffixes = ['B', 'kB', 'MB', 'GB', 'TB', 'PB', 'EB', ] + order = int(math.log(size, 1000)) if size else 0 + if order >= len(suffixes): + order = len(suffixes) - 1 + + return '{:.4g} {}'.format( + size / pow(10, order * 3), + suffixes[order] + ) + + +def binarystr_to_unicode(s): + if not isinstance(s, bytes): + return s + + if IS_WINDOWS_PLATFORM: + try: + return s.decode('windows-1250') + except UnicodeDecodeError: + pass + return s.decode('utf-8', 'replace') diff --git a/compose/cli/verbose_proxy.py b/compose/cli/verbose_proxy.py new file mode 100644 index 00000000000..c9340c4e0d2 --- /dev/null +++ b/compose/cli/verbose_proxy.py @@ -0,0 +1,55 @@ +import functools +import logging +import pprint +from itertools import chain + + +def format_call(args, kwargs): + args = (repr(a) for a in args) + kwargs = ("{!s}={!r}".format(*item) for item in kwargs.items()) + return "({})".format(", ".join(chain(args, kwargs))) + + +def format_return(result, max_lines): + if isinstance(result, (list, tuple, set)): + return "({} with {} items)".format(type(result).__name__, len(result)) + + if result: + lines = pprint.pformat(result).split('\n') + extra = '\n...' if len(lines) > max_lines else '' + return '\n'.join(lines[:max_lines]) + extra + + return result + + +class VerboseProxy: + """Proxy all function calls to another class and log method name, arguments + and return values for each call. + """ + + def __init__(self, obj_name, obj, log_name=None, max_lines=10): + self.obj_name = obj_name + self.obj = obj + self.max_lines = max_lines + self.log = logging.getLogger(log_name or __name__) + + def __getattr__(self, name): + attr = getattr(self.obj, name) + + if not callable(attr): + return attr + + return functools.partial(self.proxy_callable, name) + + def proxy_callable(self, call_name, *args, **kwargs): + self.log.info("%s %s <- %s", + self.obj_name, + call_name, + format_call(args, kwargs)) + + result = getattr(self.obj, call_name)(*args, **kwargs) + self.log.info("%s %s -> %s", + self.obj_name, + call_name, + format_return(result, self.max_lines)) + return result diff --git a/compose/config/__init__.py b/compose/config/__init__.py new file mode 100644 index 00000000000..855b2401d94 --- /dev/null +++ b/compose/config/__init__.py @@ -0,0 +1,12 @@ +# flake8: noqa +from . import environment +from .config import ConfigurationError +from .config import DOCKER_CONFIG_KEYS +from .config import find +from .config import is_url +from .config import load +from .config import merge_environment +from .config import merge_labels +from .config import parse_environment +from .config import parse_labels +from .config import resolve_build_args diff --git a/compose/config/compose_spec.json b/compose/config/compose_spec.json new file mode 100644 index 00000000000..6d892376795 --- /dev/null +++ b/compose/config/compose_spec.json @@ -0,0 +1,812 @@ +{ + "$schema": "http://json-schema.org/draft/2019-09/schema#", + "id": "compose_spec.json", + "type": "object", + "title": "Compose Specification", + "description": "The Compose file is a YAML file defining a multi-containers based application.", + + "properties": { + "version": { + "type": "string", + "description": "Version of the Compose specification used. Tools not implementing required version MUST reject the configuration file." + }, + + "services": { + "id": "#/properties/services", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/service" + } + }, + "additionalProperties": false + }, + + "networks": { + "id": "#/properties/networks", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/network" + } + } + }, + + "volumes": { + "id": "#/properties/volumes", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/volume" + } + }, + "additionalProperties": false + }, + + "secrets": { + "id": "#/properties/secrets", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/secret" + } + }, + "additionalProperties": false + }, + + "configs": { + "id": "#/properties/configs", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/config" + } + }, + "additionalProperties": false + } + }, + + "patternProperties": {"^x-": {}}, + "additionalProperties": false, + + "definitions": { + + "service": { + "id": "#/definitions/service", + "type": "object", + + "properties": { + "deploy": {"$ref": "#/definitions/deployment"}, + "build": { + "oneOf": [ + {"type": "string"}, + { + "type": "object", + "properties": { + "context": {"type": "string"}, + "dockerfile": {"type": "string"}, + "args": {"$ref": "#/definitions/list_or_dict"}, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "cache_from": {"type": "array", "items": {"type": "string"}}, + "network": {"type": "string"}, + "target": {"type": "string"}, + "shm_size": {"type": ["integer", "string"]}, + "extra_hosts": {"$ref": "#/definitions/list_or_dict"}, + "isolation": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + ] + }, + "blkio_config": { + "type": "object", + "properties": { + "device_read_bps": { + "type": "array", + "items": {"$ref": "#/definitions/blkio_limit"} + }, + "device_read_iops": { + "type": "array", + "items": {"$ref": "#/definitions/blkio_limit"} + }, + "device_write_bps": { + "type": "array", + "items": {"$ref": "#/definitions/blkio_limit"} + }, + "device_write_iops": { + "type": "array", + "items": {"$ref": "#/definitions/blkio_limit"} + }, + "weight": {"type": "integer"}, + "weight_device": { + "type": "array", + "items": {"$ref": "#/definitions/blkio_weight"} + } + }, + "additionalProperties": false + }, + "cap_add": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "cap_drop": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "cgroup_parent": {"type": "string"}, + "command": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "configs": { + "type": "array", + "items": { + "oneOf": [ + {"type": "string"}, + { + "type": "object", + "properties": { + "source": {"type": "string"}, + "target": {"type": "string"}, + "uid": {"type": "string"}, + "gid": {"type": "string"}, + "mode": {"type": "number"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + ] + } + }, + "container_name": {"type": "string"}, + "cpu_count": {"type": "integer", "minimum": 0}, + "cpu_percent": {"type": "integer", "minimum": 0, "maximum": 100}, + "cpu_shares": {"type": ["number", "string"]}, + "cpu_quota": {"type": ["number", "string"]}, + "cpu_period": {"type": ["number", "string"]}, + "cpu_rt_period": {"type": ["number", "string"]}, + "cpu_rt_runtime": {"type": ["number", "string"]}, + "cpus": {"type": ["number", "string"]}, + "cpuset": {"type": "string"}, + "credential_spec": { + "type": "object", + "properties": { + "config": {"type": "string"}, + "file": {"type": "string"}, + "registry": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "depends_on": { + "oneOf": [ + {"$ref": "#/definitions/list_of_strings"}, + { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "type": "object", + "additionalProperties": false, + "properties": { + "condition": { + "type": "string", + "enum": ["service_started", "service_healthy"] + } + }, + "required": ["condition"] + } + } + } + ] + }, + "device_cgroup_rules": {"$ref": "#/definitions/list_of_strings"}, + "devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "dns": {"$ref": "#/definitions/string_or_list"}, + "dns_opt": {"type": "array","items": {"type": "string"}, "uniqueItems": true}, + "dns_search": {"$ref": "#/definitions/string_or_list"}, + "domainname": {"type": "string"}, + "entrypoint": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "env_file": {"$ref": "#/definitions/string_or_list"}, + "environment": {"$ref": "#/definitions/list_or_dict"}, + + "expose": { + "type": "array", + "items": { + "type": ["string", "number"], + "format": "expose" + }, + "uniqueItems": true + }, + "extends": { + "oneOf": [ + {"type": "string"}, + { + "type": "object", + + "properties": { + "service": {"type": "string"}, + "file": {"type": "string"} + }, + "required": ["service"], + "additionalProperties": false + } + ] + }, + "external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "extra_hosts": {"$ref": "#/definitions/list_or_dict"}, + "group_add": { + "type": "array", + "items": { + "type": ["string", "number"] + }, + "uniqueItems": true + }, + "healthcheck": {"$ref": "#/definitions/healthcheck"}, + "hostname": {"type": "string"}, + "image": {"type": "string"}, + "init": {"type": "boolean"}, + "ipc": {"type": "string"}, + "isolation": {"type": "string"}, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "logging": { + "type": "object", + + "properties": { + "driver": {"type": "string"}, + "options": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number", "null"]} + } + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "mac_address": {"type": "string"}, + "mem_limit": {"type": ["number", "string"]}, + "mem_reservation": {"type": ["string", "integer"]}, + "mem_swappiness": {"type": "integer"}, + "memswap_limit": {"type": ["number", "string"]}, + "network_mode": {"type": "string"}, + "networks": { + "oneOf": [ + {"$ref": "#/definitions/list_of_strings"}, + { + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "oneOf": [ + { + "type": "object", + "properties": { + "aliases": {"$ref": "#/definitions/list_of_strings"}, + "ipv4_address": {"type": "string"}, + "ipv6_address": {"type": "string"}, + "link_local_ips": {"$ref": "#/definitions/list_of_strings"}, + "priority": {"type": "number"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + {"type": "null"} + ] + } + }, + "additionalProperties": false + } + ] + }, + "oom_kill_disable": {"type": "boolean"}, + "oom_score_adj": {"type": "integer", "minimum": -1000, "maximum": 1000}, + "pid": {"type": ["string", "null"]}, + "pids_limit": {"type": ["number", "string"]}, + "platform": {"type": "string"}, + "ports": { + "type": "array", + "items": { + "oneOf": [ + {"type": "number", "format": "ports"}, + {"type": "string", "format": "ports"}, + { + "type": "object", + "properties": { + "mode": {"type": "string"}, + "target": {"type": "integer"}, + "published": {"type": "integer"}, + "protocol": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + ] + }, + "uniqueItems": true + }, + "privileged": {"type": "boolean"}, + "profiles": {"$ref": "#/definitions/list_of_strings"}, + "pull_policy": {"type": "string", "enum": [ + "always", "never", "if_not_present" + ]}, + "read_only": {"type": "boolean"}, + "restart": {"type": "string"}, + "runtime": { + "deprecated": true, + "type": "string" + }, + "scale": { + "type": "integer" + }, + "security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "shm_size": {"type": ["number", "string"]}, + "secrets": { + "type": "array", + "items": { + "oneOf": [ + {"type": "string"}, + { + "type": "object", + "properties": { + "source": {"type": "string"}, + "target": {"type": "string"}, + "uid": {"type": "string"}, + "gid": {"type": "string"}, + "mode": {"type": "number"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + ] + } + }, + "sysctls": {"$ref": "#/definitions/list_or_dict"}, + "stdin_open": {"type": "boolean"}, + "stop_grace_period": {"type": "string", "format": "duration"}, + "stop_signal": {"type": "string"}, + "tmpfs": {"$ref": "#/definitions/string_or_list"}, + "tty": {"type": "boolean"}, + "ulimits": { + "type": "object", + "patternProperties": { + "^[a-z]+$": { + "oneOf": [ + {"type": "integer"}, + { + "type": "object", + "properties": { + "hard": {"type": "integer"}, + "soft": {"type": "integer"} + }, + "required": ["soft", "hard"], + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + ] + } + } + }, + "user": {"type": "string"}, + "userns_mode": {"type": "string"}, + "volumes": { + "type": "array", + "items": { + "oneOf": [ + {"type": "string"}, + { + "type": "object", + "required": ["type"], + "properties": { + "type": {"type": "string"}, + "source": {"type": "string"}, + "target": {"type": "string"}, + "read_only": {"type": "boolean"}, + "consistency": {"type": "string"}, + "bind": { + "type": "object", + "properties": { + "propagation": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "volume": { + "type": "object", + "properties": { + "nocopy": {"type": "boolean"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "tmpfs": { + "type": "object", + "properties": { + "size": { + "type": "integer", + "minimum": 0 + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + ] + }, + "uniqueItems": true + }, + "volumes_from": { + "type": "array", + "items": {"type": "string"}, + "uniqueItems": true + }, + "working_dir": {"type": "string"} + }, + "patternProperties": {"^x-": {}}, + "additionalProperties": false + }, + + "healthcheck": { + "id": "#/definitions/healthcheck", + "type": "object", + "properties": { + "disable": {"type": "boolean"}, + "interval": {"type": "string", "format": "duration"}, + "retries": {"type": "number"}, + "test": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "timeout": {"type": "string", "format": "duration"}, + "start_period": {"type": "string", "format": "duration"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "deployment": { + "id": "#/definitions/deployment", + "type": ["object", "null"], + "properties": { + "mode": {"type": "string"}, + "endpoint_mode": {"type": "string"}, + "replicas": {"type": "integer"}, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "rollback_config": { + "type": "object", + "properties": { + "parallelism": {"type": "integer"}, + "delay": {"type": "string", "format": "duration"}, + "failure_action": {"type": "string"}, + "monitor": {"type": "string", "format": "duration"}, + "max_failure_ratio": {"type": "number"}, + "order": {"type": "string", "enum": [ + "start-first", "stop-first" + ]} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "update_config": { + "type": "object", + "properties": { + "parallelism": {"type": "integer"}, + "delay": {"type": "string", "format": "duration"}, + "failure_action": {"type": "string"}, + "monitor": {"type": "string", "format": "duration"}, + "max_failure_ratio": {"type": "number"}, + "order": {"type": "string", "enum": [ + "start-first", "stop-first" + ]} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "resources": { + "type": "object", + "properties": { + "limits": { + "type": "object", + "properties": { + "cpus": {"type": ["number", "string"]}, + "memory": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "reservations": { + "type": "object", + "properties": { + "cpus": {"type": ["number", "string"]}, + "memory": {"type": "string"}, + "generic_resources": {"$ref": "#/definitions/generic_resources"}, + "devices": {"$ref": "#/definitions/devices"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "restart_policy": { + "type": "object", + "properties": { + "condition": {"type": "string"}, + "delay": {"type": "string", "format": "duration"}, + "max_attempts": {"type": "integer"}, + "window": {"type": "string", "format": "duration"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "placement": { + "type": "object", + "properties": { + "constraints": {"type": "array", "items": {"type": "string"}}, + "preferences": { + "type": "array", + "items": { + "type": "object", + "properties": { + "spread": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + "max_replicas_per_node": {"type": "integer"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + + "generic_resources": { + "id": "#/definitions/generic_resources", + "type": "array", + "items": { + "type": "object", + "properties": { + "discrete_resource_spec": { + "type": "object", + "properties": { + "kind": {"type": "string"}, + "value": {"type": "number"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + + "devices": { + "id": "#/definitions/devices", + "type": "array", + "items": { + "type": "object", + "properties": { + "capabilities": {"$ref": "#/definitions/list_of_strings"}, + "count": {"type": ["string", "integer"]}, + "device_ids": {"$ref": "#/definitions/list_of_strings"}, + "driver":{"type": "string"}, + "options":{"$ref": "#/definitions/list_or_dict"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + + "network": { + "id": "#/definitions/network", + "type": ["object", "null"], + "properties": { + "name": {"type": "string"}, + "driver": {"type": "string"}, + "driver_opts": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number"]} + } + }, + "ipam": { + "type": "object", + "properties": { + "driver": {"type": "string"}, + "config": { + "type": "array", + "items": { + "type": "object", + "properties": { + "subnet": {"type": "string", "format": "subnet_ip_address"}, + "ip_range": {"type": "string"}, + "gateway": {"type": "string"}, + "aux_addresses": { + "type": "object", + "additionalProperties": false, + "patternProperties": {"^.+$": {"type": "string"}} + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + } + }, + "options": { + "type": "object", + "additionalProperties": false, + "patternProperties": {"^.+$": {"type": "string"}} + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "external": { + "type": ["boolean", "object"], + "properties": { + "name": { + "deprecated": true, + "type": "string" + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "internal": {"type": "boolean"}, + "enable_ipv6": {"type": "boolean"}, + "attachable": {"type": "boolean"}, + "labels": {"$ref": "#/definitions/list_or_dict"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + + "volume": { + "id": "#/definitions/volume", + "type": ["object", "null"], + "properties": { + "name": {"type": "string"}, + "driver": {"type": "string"}, + "driver_opts": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number"]} + } + }, + "external": { + "type": ["boolean", "object"], + "properties": { + "name": { + "deprecated": true, + "type": "string" + } + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + "labels": {"$ref": "#/definitions/list_or_dict"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + + "secret": { + "id": "#/definitions/secret", + "type": "object", + "properties": { + "name": {"type": "string"}, + "file": {"type": "string"}, + "external": { + "type": ["boolean", "object"], + "properties": { + "name": {"type": "string"} + } + }, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "driver": {"type": "string"}, + "driver_opts": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number"]} + } + }, + "template_driver": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + + "config": { + "id": "#/definitions/config", + "type": "object", + "properties": { + "name": {"type": "string"}, + "file": {"type": "string"}, + "external": { + "type": ["boolean", "object"], + "properties": { + "name": { + "deprecated": true, + "type": "string" + } + } + }, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "template_driver": {"type": "string"} + }, + "additionalProperties": false, + "patternProperties": {"^x-": {}} + }, + + "string_or_list": { + "oneOf": [ + {"type": "string"}, + {"$ref": "#/definitions/list_of_strings"} + ] + }, + + "list_of_strings": { + "type": "array", + "items": {"type": "string"}, + "uniqueItems": true + }, + + "list_or_dict": { + "oneOf": [ + { + "type": "object", + "patternProperties": { + ".+": { + "type": ["string", "number", "null"] + } + }, + "additionalProperties": false + }, + {"type": "array", "items": {"type": "string"}, "uniqueItems": true} + ] + }, + + "blkio_limit": { + "type": "object", + "properties": { + "path": {"type": "string"}, + "rate": {"type": ["integer", "string"]} + }, + "additionalProperties": false + }, + "blkio_weight": { + "type": "object", + "properties": { + "path": {"type": "string"}, + "weight": {"type": "integer"} + }, + "additionalProperties": false + }, + + "constraints": { + "service": { + "id": "#/definitions/constraints/service", + "anyOf": [ + {"required": ["build"]}, + {"required": ["image"]} + ], + "properties": { + "build": { + "required": ["context"] + } + } + } + } + } +} diff --git a/compose/config/config.py b/compose/config/config.py new file mode 100644 index 00000000000..39c815a9842 --- /dev/null +++ b/compose/config/config.py @@ -0,0 +1,1480 @@ +import functools +import logging +import os +import re +import string +import sys +from collections import namedtuple +from itertools import chain +from operator import attrgetter +from operator import itemgetter + +import yaml +from cached_property import cached_property + +from . import types +from ..const import COMPOSE_SPEC as VERSION +from ..const import COMPOSEFILE_V1 as V1 +from ..utils import build_string_dict +from ..utils import json_hash +from ..utils import parse_bytes +from ..utils import parse_nanoseconds_int +from ..utils import splitdrive +from ..version import ComposeVersion +from .environment import env_vars_from_file +from .environment import Environment +from .environment import split_env +from .errors import CircularReference +from .errors import ComposeFileNotFound +from .errors import ConfigurationError +from .errors import DuplicateOverrideFileFound +from .errors import VERSION_EXPLANATION +from .interpolation import interpolate_environment_variables +from .sort_services import get_container_name_from_network_mode +from .sort_services import get_service_name_from_network_mode +from .sort_services import sort_service_dicts +from .types import MountSpec +from .types import parse_extra_hosts +from .types import parse_restart_spec +from .types import SecurityOpt +from .types import ServiceLink +from .types import ServicePort +from .types import VolumeFromSpec +from .types import VolumeSpec +from .validation import match_named_volumes +from .validation import validate_against_config_schema +from .validation import validate_config_section +from .validation import validate_cpu +from .validation import validate_credential_spec +from .validation import validate_depends_on +from .validation import validate_extends_file_path +from .validation import validate_healthcheck +from .validation import validate_ipc_mode +from .validation import validate_links +from .validation import validate_network_mode +from .validation import validate_pid_mode +from .validation import validate_service_constraints +from .validation import validate_top_level_object +from .validation import validate_ulimits + + +DOCKER_CONFIG_KEYS = [ + 'cap_add', + 'cap_drop', + 'cgroup_parent', + 'command', + 'cpu_count', + 'cpu_percent', + 'cpu_period', + 'cpu_quota', + 'cpu_rt_period', + 'cpu_rt_runtime', + 'cpu_shares', + 'cpus', + 'cpuset', + 'detach', + 'device_cgroup_rules', + 'devices', + 'dns', + 'dns_search', + 'dns_opt', + 'domainname', + 'entrypoint', + 'env_file', + 'environment', + 'extra_hosts', + 'group_add', + 'hostname', + 'healthcheck', + 'image', + 'ipc', + 'isolation', + 'labels', + 'links', + 'mac_address', + 'mem_limit', + 'mem_reservation', + 'memswap_limit', + 'mem_swappiness', + 'net', + 'oom_score_adj', + 'oom_kill_disable', + 'pid', + 'ports', + 'privileged', + 'read_only', + 'restart', + 'runtime', + 'secrets', + 'security_opt', + 'shm_size', + 'pids_limit', + 'stdin_open', + 'stop_signal', + 'sysctls', + 'tty', + 'user', + 'userns_mode', + 'volume_driver', + 'volumes', + 'volumes_from', + 'working_dir', +] + +ALLOWED_KEYS = DOCKER_CONFIG_KEYS + [ + 'blkio_config', + 'build', + 'container_name', + 'credential_spec', + 'dockerfile', + 'init', + 'log_driver', + 'log_opt', + 'logging', + 'network_mode', + 'platform', + 'profiles', + 'scale', + 'stop_grace_period', +] + +DOCKER_VALID_URL_PREFIXES = ( + 'http://', + 'https://', + 'git://', + 'github.com/', + 'git@', +) + +SUPPORTED_FILENAMES = [ + 'docker-compose.yml', + 'docker-compose.yaml', +] + +DEFAULT_OVERRIDE_FILENAMES = ('docker-compose.override.yml', 'docker-compose.override.yaml') + + +log = logging.getLogger(__name__) + + +class ConfigDetails(namedtuple('_ConfigDetails', 'working_dir config_files environment')): + """ + :param working_dir: the directory to use for relative paths in the config + :type working_dir: string + :param config_files: list of configuration files to load + :type config_files: list of :class:`ConfigFile` + :param environment: computed environment values for this project + :type environment: :class:`environment.Environment` + """ + def __new__(cls, working_dir, config_files, environment=None): + if environment is None: + environment = Environment.from_env_file(working_dir) + return super().__new__( + cls, working_dir, config_files, environment + ) + + +class ConfigFile(namedtuple('_ConfigFile', 'filename config')): + """ + :param filename: filename of the config file + :type filename: string + :param config: contents of the config file + :type config: :class:`dict` + """ + + @classmethod + def from_filename(cls, filename): + return cls(filename, load_yaml(filename)) + + @cached_property + def config_version(self): + version = self.config.get('version', None) + if isinstance(version, dict): + return V1 + return ComposeVersion(version) if version else self.version + + @cached_property + def version(self): + version = self.config.get('version', None) + if not version: + # no version is specified in the config file + services = self.config.get('services', None) + networks = self.config.get('networks', None) + volumes = self.config.get('volumes', None) + if services or networks or volumes: + # validate V2/V3 structure + for section in ['services', 'networks', 'volumes']: + validate_config_section( + self.filename, self.config.get(section, {}), section) + return VERSION + + # validate V1 structure + validate_config_section( + self.filename, self.config, 'services') + return V1 + + if isinstance(version, dict): + log.warning('Unexpected type for "version" key in "{}". Assuming ' + '"version" is the name of a service, and defaulting to ' + 'Compose file version {}.'.format(self.filename, V1)) + return V1 + + if not isinstance(version, str): + raise ConfigurationError( + 'Version in "{}" is invalid - it should be a string.' + .format(self.filename)) + + if isinstance(version, str): + version_pattern = re.compile(r"^[1-3]+(\.\d+)?$") + if not version_pattern.match(version): + raise ConfigurationError( + 'Version "{}" in "{}" is invalid.' + .format(version, self.filename)) + + if version.startswith("1"): + raise ConfigurationError( + 'Version in "{}" is invalid. {}' + .format(self.filename, VERSION_EXPLANATION) + ) + + return VERSION + + def get_service(self, name): + return self.get_service_dicts()[name] + + def get_service_dicts(self): + if self.version == V1: + return self.config + return self.config.get('services', {}) + + def get_volumes(self): + return {} if self.version == V1 else self.config.get('volumes', {}) + + def get_networks(self): + return {} if self.version == V1 else self.config.get('networks', {}) + + def get_secrets(self): + return {} if self.version == V1 else self.config.get('secrets', {}) + + def get_configs(self): + return {} if self.version == V1 else self.config.get('configs', {}) + + +class Config(namedtuple('_Config', 'config_version version services volumes networks secrets configs')): + """ + :param config_version: configuration file version + :type config_version: int + :param version: configuration version + :type version: int + :param services: List of service description dictionaries + :type services: :class:`list` + :param volumes: Dictionary mapping volume names to description dictionaries + :type volumes: :class:`dict` + :param networks: Dictionary mapping network names to description dictionaries + :type networks: :class:`dict` + :param secrets: Dictionary mapping secret names to description dictionaries + :type secrets: :class:`dict` + :param configs: Dictionary mapping config names to description dictionaries + :type configs: :class:`dict` + """ + + +class ServiceConfig(namedtuple('_ServiceConfig', 'working_dir filename name config')): + + @classmethod + def with_abs_paths(cls, working_dir, filename, name, config): + if not working_dir: + raise ValueError("No working_dir for ServiceConfig.") + + return cls( + os.path.abspath(working_dir), + os.path.abspath(filename) if filename else filename, + name, + config) + + +def find(base_dir, filenames, environment, override_dir=None): + if filenames == ['-']: + return ConfigDetails( + os.path.abspath(override_dir) if override_dir else os.getcwd(), + [ConfigFile(None, yaml.safe_load(sys.stdin))], + environment + ) + + if filenames: + filenames = [os.path.join(base_dir, f) for f in filenames] + else: + filenames = get_default_config_files(base_dir) + + log.debug("Using configuration files: {}".format(",".join(filenames))) + return ConfigDetails( + override_dir if override_dir else os.path.dirname(filenames[0]), + [ConfigFile.from_filename(f) for f in filenames], + environment + ) + + +def validate_config_version(config_files): + main_file = config_files[0] + validate_top_level_object(main_file) + + for next_file in config_files[1:]: + validate_top_level_object(next_file) + + if main_file.version != next_file.version: + raise ConfigurationError( + "Version mismatch: file {} specifies version {} but " + "extension file {} uses version {}".format( + main_file.filename, + main_file.version, + next_file.filename, + next_file.version)) + + +def get_default_config_files(base_dir): + (candidates, path) = find_candidates_in_parent_dirs(SUPPORTED_FILENAMES, base_dir) + + if not candidates: + raise ComposeFileNotFound(SUPPORTED_FILENAMES) + + winner = candidates[0] + + if len(candidates) > 1: + log.warning("Found multiple config files with supported names: %s", ", ".join(candidates)) + log.warning("Using %s\n", winner) + + return [os.path.join(path, winner)] + get_default_override_file(path) + + +def get_default_override_file(path): + override_files_in_path = [os.path.join(path, override_filename) for override_filename + in DEFAULT_OVERRIDE_FILENAMES + if os.path.exists(os.path.join(path, override_filename))] + if len(override_files_in_path) > 1: + raise DuplicateOverrideFileFound(override_files_in_path) + return override_files_in_path + + +def find_candidates_in_parent_dirs(filenames, path): + """ + Given a directory path to start, looks for filenames in the + directory, and then each parent directory successively, + until found. + + Returns tuple (candidates, path). + """ + candidates = [filename for filename in filenames + if os.path.exists(os.path.join(path, filename))] + + if not candidates: + parent_dir = os.path.join(path, '..') + if os.path.abspath(parent_dir) != os.path.abspath(path): + return find_candidates_in_parent_dirs(filenames, parent_dir) + + return (candidates, path) + + +def check_swarm_only_config(service_dicts): + warning_template = ( + "Some services ({services}) use the '{key}' key, which will be ignored. " + "Compose does not support '{key}' configuration - use " + "`docker stack deploy` to deploy to a swarm." + ) + key = 'configs' + services = [s for s in service_dicts if s.get(key)] + if services: + log.warning( + warning_template.format( + services=", ".join(sorted(s['name'] for s in services)), + key=key + ) + ) + + +def load(config_details, interpolate=True): + """Load the configuration from a working directory and a list of + configuration files. Files are loaded in order, and merged on top + of each other to create the final configuration. + + Return a fully interpolated, extended and validated configuration. + """ + + # validate against latest version and if fails do it against v1 schema + validate_config_version(config_details.config_files) + + processed_files = [ + process_config_file(config_file, config_details.environment, interpolate=interpolate) + for config_file in config_details.config_files + ] + config_details = config_details._replace(config_files=processed_files) + + main_file = config_details.config_files[0] + volumes = load_mapping( + config_details.config_files, 'get_volumes', 'Volume' + ) + networks = load_mapping( + config_details.config_files, 'get_networks', 'Network' + ) + secrets = load_mapping( + config_details.config_files, 'get_secrets', 'Secret', config_details.working_dir + ) + configs = load_mapping( + config_details.config_files, 'get_configs', 'Config', config_details.working_dir + ) + service_dicts = load_services(config_details, main_file, interpolate=interpolate) + + if main_file.version != V1: + for service_dict in service_dicts: + match_named_volumes(service_dict, volumes) + + check_swarm_only_config(service_dicts) + + return Config(main_file.config_version, main_file.version, + service_dicts, volumes, networks, secrets, configs) + + +def load_mapping(config_files, get_func, entity_type, working_dir=None): + mapping = {} + + for config_file in config_files: + for name, config in getattr(config_file, get_func)().items(): + mapping[name] = config or {} + if not config: + continue + + external = config.get('external') + if external: + validate_external(entity_type, name, config, config_file.version) + if isinstance(external, dict): + config['name'] = external.get('name') + elif not config.get('name'): + config['name'] = name + + if 'labels' in config: + config['labels'] = parse_labels(config['labels']) + + if 'file' in config: + config['file'] = expand_path(working_dir, config['file']) + + if 'driver_opts' in config: + config['driver_opts'] = build_string_dict( + config['driver_opts'] + ) + device = format_device_option(entity_type, config) + if device: + config['driver_opts']['device'] = device + return mapping + + +def format_device_option(entity_type, config): + if entity_type != 'Volume': + return + # default driver is 'local' + driver = config.get('driver', 'local') + if driver != 'local': + return + o = config['driver_opts'].get('o') + device = config['driver_opts'].get('device') + if o and o == 'bind' and device: + fullpath = os.path.abspath(os.path.expanduser(device)) + return fullpath + + +def validate_external(entity_type, name, config, version): + for k in config.keys(): + if entity_type == 'Network' and k == 'driver': + continue + if k not in ['external', 'name']: + raise ConfigurationError( + "{} {} declared as external but specifies additional attributes " + "({}).".format( + entity_type, name, ', '.join(k for k in config if k != 'external'))) + + +def load_services(config_details, config_file, interpolate=True): + def build_service(service_name, service_dict, service_names): + service_config = ServiceConfig.with_abs_paths( + config_details.working_dir, + config_file.filename, + service_name, + service_dict) + resolver = ServiceExtendsResolver( + service_config, config_file, environment=config_details.environment + ) + service_dict = process_service(resolver.run()) + + service_config = service_config._replace(config=service_dict) + validate_service(service_config, service_names, config_file) + service_dict = finalize_service( + service_config, + service_names, + config_file.version, + config_details.environment, + interpolate + ) + return service_dict + + def build_services(service_config): + service_names = service_config.keys() + return sort_service_dicts([ + build_service(name, service_dict, service_names) + for name, service_dict in service_config.items() + ]) + + def merge_services(base, override): + all_service_names = set(base) | set(override) + return { + name: merge_service_dicts_from_files( + base.get(name, {}), + override.get(name, {}), + config_file.version) + for name in all_service_names + } + + service_configs = [ + file.get_service_dicts() for file in config_details.config_files + ] + + service_config = functools.reduce(merge_services, service_configs) + + return build_services(service_config) + + +def interpolate_config_section(config_file, config, section, environment): + return interpolate_environment_variables( + config_file.version, + config, + section, + environment + ) + + +def process_config_section(config_file, config, section, environment, interpolate): + validate_config_section(config_file.filename, config, section) + if interpolate: + return interpolate_environment_variables( + config_file.version, + config, + section, + environment + ) + else: + return config + + +def process_config_file(config_file, environment, service_name=None, interpolate=True): + services = process_config_section( + config_file, + config_file.get_service_dicts(), + 'service', + environment, + interpolate, + ) + + if config_file.version > V1: + processed_config = dict(config_file.config) + processed_config['services'] = services + processed_config['volumes'] = process_config_section( + config_file, + config_file.get_volumes(), + 'volume', + environment, + interpolate, + ) + processed_config['networks'] = process_config_section( + config_file, + config_file.get_networks(), + 'network', + environment, + interpolate, + ) + processed_config['secrets'] = process_config_section( + config_file, + config_file.get_secrets(), + 'secret', + environment, + interpolate, + ) + processed_config['configs'] = process_config_section( + config_file, + config_file.get_configs(), + 'config', + environment, + interpolate, + ) + else: + processed_config = services + + config_file = config_file._replace(config=processed_config) + validate_against_config_schema(config_file, config_file.version) + + if service_name and service_name not in services: + raise ConfigurationError( + "Cannot extend service '{}' in {}: Service not found".format( + service_name, config_file.filename)) + + return config_file + + +class ServiceExtendsResolver: + def __init__(self, service_config, config_file, environment, already_seen=None): + self.service_config = service_config + self.working_dir = service_config.working_dir + self.already_seen = already_seen or [] + self.config_file = config_file + self.environment = environment + + @property + def signature(self): + return self.service_config.filename, self.service_config.name + + def detect_cycle(self): + if self.signature in self.already_seen: + raise CircularReference(self.already_seen + [self.signature]) + + def run(self): + self.detect_cycle() + + if 'extends' in self.service_config.config: + service_dict = self.resolve_extends(*self.validate_and_construct_extends()) + return self.service_config._replace(config=service_dict) + + return self.service_config + + def validate_and_construct_extends(self): + extends = self.service_config.config['extends'] + if not isinstance(extends, dict): + extends = {'service': extends} + + config_path = self.get_extended_config_path(extends) + service_name = extends['service'] + + if config_path == os.path.abspath(self.config_file.filename): + try: + service_config = self.config_file.get_service(service_name) + except KeyError: + raise ConfigurationError( + "Cannot extend service '{}' in {}: Service not found".format( + service_name, config_path) + ) + else: + extends_file = ConfigFile.from_filename(config_path) + validate_config_version([self.config_file, extends_file]) + extended_file = process_config_file( + extends_file, self.environment, service_name=service_name + ) + service_config = extended_file.get_service(service_name) + + return config_path, service_config, service_name + + def resolve_extends(self, extended_config_path, service_dict, service_name): + resolver = ServiceExtendsResolver( + ServiceConfig.with_abs_paths( + os.path.dirname(extended_config_path), + extended_config_path, + service_name, + service_dict), + self.config_file, + already_seen=self.already_seen + [self.signature], + environment=self.environment + ) + + service_config = resolver.run() + other_service_dict = process_service(service_config) + validate_extended_service_dict( + other_service_dict, + extended_config_path, + service_name) + + return merge_service_dicts( + other_service_dict, + self.service_config.config, + self.config_file.version) + + def get_extended_config_path(self, extends_options): + """Service we are extending either has a value for 'file' set, which we + need to obtain a full path too or we are extending from a service + defined in our own file. + """ + filename = self.service_config.filename + validate_extends_file_path( + self.service_config.name, + extends_options, + filename) + if 'file' in extends_options: + return expand_path(self.working_dir, extends_options['file']) + return filename + + +def resolve_environment(service_dict, environment=None, interpolate=True): + """Unpack any environment variables from an env_file, if set. + Interpolate environment values if set. + """ + env = {} + for env_file in service_dict.get('env_file', []): + env.update(env_vars_from_file(env_file, interpolate)) + + env.update(parse_environment(service_dict.get('environment'))) + return dict(resolve_env_var(k, v, environment) for k, v in env.items()) + + +def resolve_build_args(buildargs, environment): + args = parse_build_arguments(buildargs) + return dict(resolve_env_var(k, v, environment) for k, v in args.items()) + + +def validate_extended_service_dict(service_dict, filename, service): + error_prefix = "Cannot extend service '{}' in {}:".format(service, filename) + + if 'links' in service_dict: + raise ConfigurationError( + "%s services with 'links' cannot be extended" % error_prefix) + + if 'volumes_from' in service_dict: + raise ConfigurationError( + "%s services with 'volumes_from' cannot be extended" % error_prefix) + + if 'net' in service_dict: + if get_container_name_from_network_mode(service_dict['net']): + raise ConfigurationError( + "%s services with 'net: container' cannot be extended" % error_prefix) + + if 'network_mode' in service_dict: + if get_service_name_from_network_mode(service_dict['network_mode']): + raise ConfigurationError( + "%s services with 'network_mode: service' cannot be extended" % error_prefix) + + if 'depends_on' in service_dict: + raise ConfigurationError( + "%s services with 'depends_on' cannot be extended" % error_prefix) + + +def validate_service(service_config, service_names, config_file): + def build_image(): + args = sys.argv[1:] + if 'pull' in args: + return False + + if '--no-build' in args: + return False + + return True + + service_dict, service_name = service_config.config, service_config.name + validate_service_constraints(service_dict, service_name, config_file) + + if build_image(): + # We only care about valid paths when actually building images + validate_paths(service_dict) + + validate_cpu(service_config) + validate_ulimits(service_config) + validate_ipc_mode(service_config, service_names) + validate_network_mode(service_config, service_names) + validate_pid_mode(service_config, service_names) + validate_depends_on(service_config, service_names) + validate_links(service_config, service_names) + validate_healthcheck(service_config) + validate_credential_spec(service_config) + + if not service_dict.get('image') and has_uppercase(service_name): + raise ConfigurationError( + "Service '{name}' contains uppercase characters which are not valid " + "as part of an image name. Either use a lowercase service name or " + "use the `image` field to set a custom name for the service image." + .format(name=service_name)) + + +def process_service(service_config): + working_dir = service_config.working_dir + service_dict = dict(service_config.config) + + if 'env_file' in service_dict: + service_dict['env_file'] = [ + expand_path(working_dir, path) + for path in to_list(service_dict['env_file']) + ] + + if 'build' in service_dict: + process_build_section(service_dict, working_dir) + + if 'volumes' in service_dict and service_dict.get('volume_driver') is None: + service_dict['volumes'] = resolve_volume_paths(working_dir, service_dict) + + if 'sysctls' in service_dict: + service_dict['sysctls'] = build_string_dict(parse_sysctls(service_dict['sysctls'])) + + if 'labels' in service_dict: + service_dict['labels'] = parse_labels(service_dict['labels']) + + service_dict = process_depends_on(service_dict) + + for field in ['dns', 'dns_search', 'tmpfs']: + if field in service_dict: + service_dict[field] = to_list(service_dict[field]) + + service_dict = process_security_opt(process_blkio_config(process_ports( + process_healthcheck(service_dict) + ))) + + return service_dict + + +def process_build_section(service_dict, working_dir): + if isinstance(service_dict['build'], str): + service_dict['build'] = resolve_build_path(working_dir, service_dict['build']) + elif isinstance(service_dict['build'], dict): + if 'context' in service_dict['build']: + path = service_dict['build']['context'] + service_dict['build']['context'] = resolve_build_path(working_dir, path) + if 'labels' in service_dict['build']: + service_dict['build']['labels'] = parse_labels(service_dict['build']['labels']) + + +def process_ports(service_dict): + if 'ports' not in service_dict: + return service_dict + + ports = [] + for port_definition in service_dict['ports']: + if isinstance(port_definition, ServicePort): + ports.append(port_definition) + else: + ports.extend(ServicePort.parse(port_definition)) + service_dict['ports'] = ports + return service_dict + + +def process_depends_on(service_dict): + if 'depends_on' in service_dict and not isinstance(service_dict['depends_on'], dict): + service_dict['depends_on'] = { + svc: {'condition': 'service_started'} for svc in service_dict['depends_on'] + } + return service_dict + + +def process_blkio_config(service_dict): + if not service_dict.get('blkio_config'): + return service_dict + + for field in ['device_read_bps', 'device_write_bps']: + if field in service_dict['blkio_config']: + for v in service_dict['blkio_config'].get(field, []): + rate = v.get('rate', 0) + v['rate'] = parse_bytes(rate) + if v['rate'] is None: + raise ConfigurationError('Invalid format for bytes value: "{}"'.format(rate)) + + for field in ['device_read_iops', 'device_write_iops']: + if field in service_dict['blkio_config']: + for v in service_dict['blkio_config'].get(field, []): + try: + v['rate'] = int(v.get('rate', 0)) + except ValueError: + raise ConfigurationError( + 'Invalid IOPS value: "{}". Must be a positive integer.'.format(v.get('rate')) + ) + + return service_dict + + +def process_healthcheck(service_dict): + if 'healthcheck' not in service_dict: + return service_dict + + hc = service_dict['healthcheck'] + + if 'disable' in hc: + del hc['disable'] + hc['test'] = ['NONE'] + + for field in ['interval', 'timeout', 'start_period']: + if field not in hc or isinstance(hc[field], int): + continue + hc[field] = parse_nanoseconds_int(hc[field]) + + return service_dict + + +def finalize_service_volumes(service_dict, environment): + if 'volumes' in service_dict: + finalized_volumes = [] + normalize = environment.get_boolean('COMPOSE_CONVERT_WINDOWS_PATHS') + win_host = environment.get_boolean('COMPOSE_FORCE_WINDOWS_HOST') + for v in service_dict['volumes']: + if isinstance(v, dict): + finalized_volumes.append(MountSpec.parse(v, normalize, win_host)) + else: + finalized_volumes.append(VolumeSpec.parse(v, normalize, win_host)) + + duplicate_mounts = [] + mounts = [v.as_volume_spec() if isinstance(v, MountSpec) else v for v in finalized_volumes] + for mount in mounts: + if list(map(attrgetter('internal'), mounts)).count(mount.internal) > 1: + duplicate_mounts.append(mount.repr()) + + if duplicate_mounts: + raise ConfigurationError("Duplicate mount points: [%s]" % ( + ', '.join(duplicate_mounts))) + + service_dict['volumes'] = finalized_volumes + + return service_dict + + +def finalize_service(service_config, service_names, version, environment, + interpolate=True): + service_dict = dict(service_config.config) + + if 'environment' in service_dict or 'env_file' in service_dict: + service_dict['environment'] = resolve_environment(service_dict, environment, interpolate) + service_dict.pop('env_file', None) + + if 'volumes_from' in service_dict: + service_dict['volumes_from'] = [ + VolumeFromSpec.parse(vf, service_names, version) + for vf in service_dict['volumes_from'] + ] + + service_dict = finalize_service_volumes(service_dict, environment) + + if 'net' in service_dict: + network_mode = service_dict.pop('net') + container_name = get_container_name_from_network_mode(network_mode) + if container_name and container_name in service_names: + service_dict['network_mode'] = 'service:{}'.format(container_name) + else: + service_dict['network_mode'] = network_mode + + if 'networks' in service_dict: + service_dict['networks'] = parse_networks(service_dict['networks']) + + if 'restart' in service_dict: + service_dict['restart'] = parse_restart_spec(service_dict['restart']) + + if 'secrets' in service_dict: + service_dict['secrets'] = [ + types.ServiceSecret.parse(s) for s in service_dict['secrets'] + ] + + if 'configs' in service_dict: + service_dict['configs'] = [ + types.ServiceConfig.parse(c) for c in service_dict['configs'] + ] + + normalize_build(service_dict, service_config.working_dir, environment) + + service_dict['name'] = service_config.name + return normalize_v1_service_format(service_dict) + + +def normalize_v1_service_format(service_dict): + if 'log_driver' in service_dict or 'log_opt' in service_dict: + if 'logging' not in service_dict: + service_dict['logging'] = {} + if 'log_driver' in service_dict: + service_dict['logging']['driver'] = service_dict['log_driver'] + del service_dict['log_driver'] + if 'log_opt' in service_dict: + service_dict['logging']['options'] = service_dict['log_opt'] + del service_dict['log_opt'] + + if 'dockerfile' in service_dict: + service_dict['build'] = service_dict.get('build', {}) + service_dict['build'].update({ + 'dockerfile': service_dict.pop('dockerfile') + }) + + return service_dict + + +def merge_service_dicts_from_files(base, override, version): + """When merging services from multiple files we need to merge the `extends` + field. This is not handled by `merge_service_dicts()` which is used to + perform the `extends`. + """ + new_service = merge_service_dicts(base, override, version) + if 'extends' in override: + new_service['extends'] = override['extends'] + elif 'extends' in base: + new_service['extends'] = base['extends'] + return new_service + + +class MergeDict(dict): + """A dict-like object responsible for merging two dicts into one.""" + + def __init__(self, base, override): + self.base = base + self.override = override + + def needs_merge(self, field): + return field in self.base or field in self.override + + def merge_field(self, field, merge_func, default=None): + if not self.needs_merge(field): + return + + self[field] = merge_func( + self.base.get(field, default), + self.override.get(field, default)) + + def merge_mapping(self, field, parse_func=None): + if not self.needs_merge(field): + return + + if parse_func is None: + def parse_func(m): + return m or {} + + self[field] = parse_func(self.base.get(field)) + self[field].update(parse_func(self.override.get(field))) + + def merge_sequence(self, field, parse_func): + def parse_sequence_func(seq): + return to_mapping((parse_func(item) for item in seq), 'merge_field') + + if not self.needs_merge(field): + return + + merged = parse_sequence_func(self.base.get(field, [])) + merged.update(parse_sequence_func(self.override.get(field, []))) + self[field] = [item.repr() for item in sorted(merged.values())] + + def merge_scalar(self, field): + if self.needs_merge(field): + self[field] = self.override.get(field, self.base.get(field)) + + +def merge_service_dicts(base, override, version): + md = MergeDict(base, override) + + md.merge_mapping('environment', parse_environment) + md.merge_mapping('labels', parse_labels) + md.merge_mapping('ulimits', parse_flat_dict) + md.merge_mapping('sysctls', parse_sysctls) + md.merge_mapping('depends_on', parse_depends_on) + md.merge_mapping('storage_opt', parse_flat_dict) + md.merge_sequence('links', ServiceLink.parse) + md.merge_sequence('secrets', types.ServiceSecret.parse) + md.merge_sequence('configs', types.ServiceConfig.parse) + md.merge_sequence('security_opt', types.SecurityOpt.parse) + md.merge_mapping('extra_hosts', parse_extra_hosts) + + md.merge_field('networks', merge_networks, default={}) + for field in ['volumes', 'devices']: + md.merge_field(field, merge_path_mappings) + + for field in [ + 'cap_add', 'cap_drop', 'expose', 'external_links', + 'volumes_from', 'device_cgroup_rules', 'profiles', + ]: + md.merge_field(field, merge_unique_items_lists, default=[]) + + for field in ['dns', 'dns_search', 'env_file', 'tmpfs']: + md.merge_field(field, merge_list_or_string) + + md.merge_field('logging', merge_logging, default={}) + merge_ports(md, base, override) + md.merge_field('blkio_config', merge_blkio_config, default={}) + md.merge_field('healthcheck', merge_healthchecks, default={}) + md.merge_field('deploy', merge_deploy, default={}) + + for field in set(ALLOWED_KEYS) - set(md): + md.merge_scalar(field) + + if version == V1: + legacy_v1_merge_image_or_build(md, base, override) + elif md.needs_merge('build'): + md['build'] = merge_build(md, base, override) + + return dict(md) + + +def merge_unique_items_lists(base, override): + override = (str(o) for o in override) + base = (str(b) for b in base) + return sorted(set(chain(base, override))) + + +def merge_healthchecks(base, override): + if override.get('disabled') is True: + return override + result = base.copy() + result.update(override) + return result + + +def merge_ports(md, base, override): + def parse_sequence_func(seq): + acc = [s for item in seq for s in ServicePort.parse(item)] + return to_mapping(acc, 'merge_field') + + field = 'ports' + + if not md.needs_merge(field): + return + + merged = parse_sequence_func(md.base.get(field, [])) + merged.update(parse_sequence_func(md.override.get(field, []))) + md[field] = [item for item in sorted(merged.values(), key=attrgetter("target"))] + + +def merge_build(output, base, override): + def to_dict(service): + build_config = service.get('build', {}) + if isinstance(build_config, str): + return {'context': build_config} + return build_config + + md = MergeDict(to_dict(base), to_dict(override)) + md.merge_scalar('context') + md.merge_scalar('dockerfile') + md.merge_scalar('network') + md.merge_scalar('target') + md.merge_scalar('shm_size') + md.merge_scalar('isolation') + md.merge_mapping('args', parse_build_arguments) + md.merge_field('cache_from', merge_unique_items_lists, default=[]) + md.merge_mapping('labels', parse_labels) + md.merge_mapping('extra_hosts', parse_extra_hosts) + return dict(md) + + +def merge_deploy(base, override): + md = MergeDict(base or {}, override or {}) + md.merge_scalar('mode') + md.merge_scalar('endpoint_mode') + md.merge_scalar('replicas') + md.merge_mapping('labels', parse_labels) + md.merge_mapping('update_config') + md.merge_mapping('rollback_config') + md.merge_mapping('restart_policy') + if md.needs_merge('resources'): + resources_md = MergeDict(md.base.get('resources') or {}, md.override.get('resources') or {}) + resources_md.merge_mapping('limits') + resources_md.merge_field('reservations', merge_reservations, default={}) + md['resources'] = dict(resources_md) + if md.needs_merge('placement'): + placement_md = MergeDict(md.base.get('placement') or {}, md.override.get('placement') or {}) + placement_md.merge_scalar('max_replicas_per_node') + placement_md.merge_field('constraints', merge_unique_items_lists, default=[]) + placement_md.merge_field('preferences', merge_unique_objects_lists, default=[]) + md['placement'] = dict(placement_md) + + return dict(md) + + +def merge_networks(base, override): + merged_networks = {} + all_network_names = set(base) | set(override) + base = {k: {} for k in base} if isinstance(base, list) else base + override = {k: {} for k in override} if isinstance(override, list) else override + for network_name in all_network_names: + md = MergeDict(base.get(network_name) or {}, override.get(network_name) or {}) + md.merge_field('aliases', merge_unique_items_lists, []) + md.merge_field('link_local_ips', merge_unique_items_lists, []) + md.merge_scalar('priority') + md.merge_scalar('ipv4_address') + md.merge_scalar('ipv6_address') + merged_networks[network_name] = dict(md) + return merged_networks + + +def merge_reservations(base, override): + md = MergeDict(base, override) + md.merge_scalar('cpus') + md.merge_scalar('memory') + md.merge_sequence('generic_resources', types.GenericResource.parse) + md.merge_field('devices', merge_unique_objects_lists, default=[]) + return dict(md) + + +def merge_unique_objects_lists(base, override): + result = {json_hash(i): i for i in base + override} + return [i[1] for i in sorted(((k, v) for k, v in result.items()), key=itemgetter(0))] + + +def merge_blkio_config(base, override): + md = MergeDict(base, override) + md.merge_scalar('weight') + + def merge_blkio_limits(base, override): + get_path = itemgetter('path') + index = {get_path(b): b for b in base} + index.update((get_path(o), o) for o in override) + + return sorted(index.values(), key=get_path) + + for field in [ + "device_read_bps", "device_read_iops", "device_write_bps", + "device_write_iops", "weight_device", + ]: + md.merge_field(field, merge_blkio_limits, default=[]) + + return dict(md) + + +def merge_logging(base, override): + md = MergeDict(base, override) + md.merge_scalar('driver') + if md.get('driver') == base.get('driver') or base.get('driver') is None: + md.merge_mapping('options', lambda m: m or {}) + elif override.get('options'): + md['options'] = override.get('options', {}) + return dict(md) + + +def legacy_v1_merge_image_or_build(output, base, override): + output.pop('image', None) + output.pop('build', None) + if 'image' in override: + output['image'] = override['image'] + elif 'build' in override: + output['build'] = override['build'] + elif 'image' in base: + output['image'] = base['image'] + elif 'build' in base: + output['build'] = base['build'] + + +def merge_environment(base, override): + env = parse_environment(base) + env.update(parse_environment(override)) + return env + + +def merge_labels(base, override): + labels = parse_labels(base) + labels.update(parse_labels(override)) + return labels + + +def split_kv(kvpair): + if '=' in kvpair: + return kvpair.split('=', 1) + else: + return kvpair, '' + + +def parse_dict_or_list(split_func, type_name, arguments): + if not arguments: + return {} + + if isinstance(arguments, list): + return dict(split_func(e) for e in arguments) + + if isinstance(arguments, dict): + return dict(arguments) + + raise ConfigurationError( + "%s \"%s\" must be a list or mapping," % + (type_name, arguments) + ) + + +parse_build_arguments = functools.partial(parse_dict_or_list, split_env, 'build arguments') +parse_environment = functools.partial(parse_dict_or_list, split_env, 'environment') +parse_labels = functools.partial(parse_dict_or_list, split_kv, 'labels') +parse_networks = functools.partial(parse_dict_or_list, lambda k: (k, None), 'networks') +parse_sysctls = functools.partial(parse_dict_or_list, split_kv, 'sysctls') +parse_depends_on = functools.partial( + parse_dict_or_list, lambda k: (k, {'condition': 'service_started'}), 'depends_on' +) + + +def parse_flat_dict(d): + if not d: + return {} + + if isinstance(d, dict): + return dict(d) + + raise ConfigurationError("Invalid type: expected mapping") + + +def resolve_env_var(key, val, environment): + if val is not None: + return key, val + elif environment and key in environment: + return key, environment[key] + else: + return key, None + + +def resolve_volume_paths(working_dir, service_dict): + return [ + resolve_volume_path(working_dir, volume) + for volume in service_dict['volumes'] + ] + + +def resolve_volume_path(working_dir, volume): + if isinstance(volume, dict): + if volume.get('source', '').startswith(('.', '~')) and volume['type'] == 'bind': + volume['source'] = expand_path(working_dir, volume['source']) + return volume + + mount_params = None + container_path, mount_params = split_path_mapping(volume) + + if mount_params is not None: + host_path, mode = mount_params + if host_path is None: + return container_path + if host_path.startswith('.'): + host_path = expand_path(working_dir, host_path) + host_path = os.path.expanduser(host_path) + return "{}:{}{}".format(host_path, container_path, (':' + mode if mode else '')) + + return container_path + + +def normalize_build(service_dict, working_dir, environment): + + if 'build' in service_dict: + build = {} + # Shortcut where specifying a string is treated as the build context + if isinstance(service_dict['build'], str): + build['context'] = service_dict.pop('build') + else: + build.update(service_dict['build']) + if 'args' in build: + build['args'] = build_string_dict( + resolve_build_args(build.get('args'), environment) + ) + + service_dict['build'] = build + + +def resolve_build_path(working_dir, build_path): + if is_url(build_path): + return build_path + return expand_path(working_dir, build_path) + + +def is_url(build_path): + return build_path.startswith(DOCKER_VALID_URL_PREFIXES) + + +def validate_paths(service_dict): + if 'build' in service_dict: + build = service_dict.get('build', {}) + + if isinstance(build, str): + build_path = build + elif isinstance(build, dict) and 'context' in build: + build_path = build['context'] + else: + # We have a build section but no context, so nothing to validate + return + + if ( + not is_url(build_path) and + (not os.path.exists(build_path) or not os.access(build_path, os.R_OK)) + ): + raise ConfigurationError( + "build path %s either does not exist, is not accessible, " + "or is not a valid URL." % build_path) + + +def merge_path_mappings(base, override): + d = dict_from_path_mappings(base) + d.update(dict_from_path_mappings(override)) + return path_mappings_from_dict(d) + + +def dict_from_path_mappings(path_mappings): + if path_mappings: + return dict(split_path_mapping(v) for v in path_mappings) + else: + return {} + + +def path_mappings_from_dict(d): + return [join_path_mapping(v) for v in sorted(d.items())] + + +def split_path_mapping(volume_path): + """ + Ascertain if the volume_path contains a host path as well as a container + path. Using splitdrive so windows absolute paths won't cause issues with + splitting on ':'. + """ + if isinstance(volume_path, dict): + return (volume_path.get('target'), volume_path) + drive, volume_config = splitdrive(volume_path) + + if ':' in volume_config: + (host, container) = volume_config.split(':', 1) + container_drive, container_path = splitdrive(container) + mode = None + if ':' in container_path: + container_path, mode = container_path.rsplit(':', 1) + + return (container_drive + container_path, (drive + host, mode)) + else: + return (volume_path, None) + + +def process_security_opt(service_dict): + security_opts = service_dict.get('security_opt', []) + result = [] + for value in security_opts: + result.append(SecurityOpt.parse(value)) + if result: + service_dict['security_opt'] = result + return service_dict + + +def join_path_mapping(pair): + (container, host) = pair + if isinstance(host, dict): + return host + elif host is None: + return container + else: + host, mode = host + result = ":".join((host, container)) + if mode: + result += ":" + mode + return result + + +def expand_path(working_dir, path): + return os.path.abspath(os.path.join(working_dir, os.path.expanduser(path))) + + +def merge_list_or_string(base, override): + return to_list(base) + to_list(override) + + +def to_list(value): + if value is None: + return [] + elif isinstance(value, str): + return [value] + else: + return value + + +def to_mapping(sequence, key_field): + return {getattr(item, key_field): item for item in sequence} + + +def has_uppercase(name): + return any(char in string.ascii_uppercase for char in name) + + +def load_yaml(filename, encoding=None, binary=True): + try: + with open(filename, 'rb' if binary else 'r', encoding=encoding) as fh: + return yaml.safe_load(fh) + except (OSError, yaml.YAMLError, UnicodeDecodeError) as e: + if encoding is None: + # Sometimes the user's locale sets an encoding that doesn't match + # the YAML files. Im such cases, retry once with the "default" + # UTF-8 encoding + return load_yaml(filename, encoding='utf-8-sig', binary=False) + error_name = getattr(e, '__module__', '') + '.' + e.__class__.__name__ + raise ConfigurationError("{}: {}".format(error_name, e)) diff --git a/compose/config/config_schema_v1.json b/compose/config/config_schema_v1.json new file mode 100644 index 00000000000..2771f9958fb --- /dev/null +++ b/compose/config/config_schema_v1.json @@ -0,0 +1,203 @@ +{ + "$schema": "http://json-schema.org/draft-04/schema#", + "id": "config_schema_v1.json", + + "type": "object", + + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/service" + } + }, + + "additionalProperties": false, + + "definitions": { + "service": { + "id": "#/definitions/service", + "type": "object", + + "properties": { + "build": {"type": "string"}, + "cap_add": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "cap_drop": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "cgroup_parent": {"type": "string"}, + "command": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "container_name": {"type": "string"}, + "cpu_shares": {"type": ["number", "string"]}, + "cpu_quota": {"type": ["number", "string"]}, + "cpuset": {"type": "string"}, + "devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "dns": {"$ref": "#/definitions/string_or_list"}, + "dns_search": {"$ref": "#/definitions/string_or_list"}, + "dockerfile": {"type": "string"}, + "domainname": {"type": "string"}, + "entrypoint": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "env_file": {"$ref": "#/definitions/string_or_list"}, + "environment": {"$ref": "#/definitions/list_or_dict"}, + + "expose": { + "type": "array", + "items": { + "type": ["string", "number"], + "format": "expose" + }, + "uniqueItems": true + }, + + "extends": { + "oneOf": [ + { + "type": "string" + }, + { + "type": "object", + + "properties": { + "service": {"type": "string"}, + "file": {"type": "string"} + }, + "required": ["service"], + "additionalProperties": false + } + ] + }, + + "extra_hosts": {"$ref": "#/definitions/list_or_dict"}, + "external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "hostname": {"type": "string"}, + "image": {"type": "string"}, + "ipc": {"type": "string"}, + "labels": {"$ref": "#/definitions/labels"}, + "links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "log_driver": {"type": "string"}, + "log_opt": {"type": "object"}, + "mac_address": {"type": "string"}, + "mem_limit": {"type": ["number", "string"]}, + "memswap_limit": {"type": ["number", "string"]}, + "mem_swappiness": {"type": "integer"}, + "net": {"type": "string"}, + "pid": {"type": ["string", "null"]}, + + "ports": { + "type": "array", + "items": { + "type": ["string", "number"], + "format": "ports" + }, + "uniqueItems": true + }, + + "privileged": {"type": "boolean"}, + "read_only": {"type": "boolean"}, + "restart": {"type": "string"}, + "security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "shm_size": {"type": ["number", "string"]}, + "stdin_open": {"type": "boolean"}, + "stop_signal": {"type": "string"}, + "tty": {"type": "boolean"}, + "ulimits": { + "type": "object", + "patternProperties": { + "^[a-z]+$": { + "oneOf": [ + {"type": "integer"}, + { + "type":"object", + "properties": { + "hard": {"type": "integer"}, + "soft": {"type": "integer"} + }, + "required": ["soft", "hard"], + "additionalProperties": false + } + ] + } + } + }, + "user": {"type": "string"}, + "volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "volume_driver": {"type": "string"}, + "volumes_from": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "working_dir": {"type": "string"} + }, + + "dependencies": { + "memswap_limit": ["mem_limit"] + }, + "additionalProperties": false + }, + + "string_or_list": { + "oneOf": [ + {"type": "string"}, + {"$ref": "#/definitions/list_of_strings"} + ] + }, + + "list_of_strings": { + "type": "array", + "items": {"type": "string"}, + "uniqueItems": true + }, + + "list_or_dict": { + "oneOf": [ + { + "type": "object", + "patternProperties": { + ".+": { + "type": ["string", "number", "null"] + } + }, + "additionalProperties": false + }, + {"type": "array", "items": {"type": "string"}, "uniqueItems": true} + ] + }, + + "labels": { + "oneOf": [ + { + "type": "object", + "patternProperties": { + ".+": { + "type": "string" + } + }, + "additionalProperties": false + }, + {"type": "array", "items": {"type": "string"}, "uniqueItems": true} + ] + }, + + "constraints": { + "service": { + "id": "#/definitions/constraints/service", + "anyOf": [ + { + "required": ["build"], + "not": {"required": ["image"]} + }, + { + "required": ["image"], + "not": {"anyOf": [ + {"required": ["build"]}, + {"required": ["dockerfile"]} + ]} + } + ] + } + } + } +} diff --git a/compose/config/environment.py b/compose/config/environment.py new file mode 100644 index 00000000000..8769df9f8f7 --- /dev/null +++ b/compose/config/environment.py @@ -0,0 +1,125 @@ +import logging +import os +import re + +import dotenv + +from ..const import IS_WINDOWS_PLATFORM +from .errors import ConfigurationError +from .errors import EnvFileNotFound + +log = logging.getLogger(__name__) + + +def split_env(env): + if isinstance(env, bytes): + env = env.decode('utf-8', 'replace') + key = value = None + if '=' in env: + key, value = env.split('=', 1) + else: + key = env + if re.search(r'\s', key): + raise ConfigurationError( + "environment variable name '{}' may not contain whitespace.".format(key) + ) + return key, value + + +def env_vars_from_file(filename, interpolate=True): + """ + Read in a line delimited file of environment variables. + """ + if not os.path.exists(filename): + raise EnvFileNotFound("Couldn't find env file: {}".format(filename)) + elif not os.path.isfile(filename): + raise EnvFileNotFound("{} is not a file.".format(filename)) + + env = dotenv.dotenv_values(dotenv_path=filename, encoding='utf-8-sig', interpolate=interpolate) + for k, v in env.items(): + env[k] = v if interpolate else v.replace('$', '$$') + return env + + +class Environment(dict): + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.missing_keys = [] + self.silent = False + + @classmethod + def from_env_file(cls, base_dir, env_file=None): + def _initialize(): + result = cls() + if base_dir is None: + return result + if env_file: + env_file_path = os.path.join(base_dir, env_file) + else: + env_file_path = os.path.join(base_dir, '.env') + try: + return cls(env_vars_from_file(env_file_path)) + except EnvFileNotFound: + pass + return result + + instance = _initialize() + instance.update(os.environ) + return instance + + @classmethod + def from_command_line(cls, parsed_env_opts): + result = cls() + for k, v in parsed_env_opts.items(): + # Values from the command line take priority, unless they're unset + # in which case they take the value from the system's environment + if v is None and k in os.environ: + result[k] = os.environ[k] + else: + result[k] = v + return result + + def __getitem__(self, key): + try: + return super().__getitem__(key) + except KeyError: + if IS_WINDOWS_PLATFORM: + try: + return super().__getitem__(key.upper()) + except KeyError: + pass + if not self.silent and key not in self.missing_keys: + log.warning( + "The {} variable is not set. Defaulting to a blank string." + .format(key) + ) + self.missing_keys.append(key) + + return "" + + def __contains__(self, key): + result = super().__contains__(key) + if IS_WINDOWS_PLATFORM: + return ( + result or super().__contains__(key.upper()) + ) + return result + + def get(self, key, *args, **kwargs): + if IS_WINDOWS_PLATFORM: + return super().get( + key, + super().get(key.upper(), *args, **kwargs) + ) + return super().get(key, *args, **kwargs) + + def get_boolean(self, key, default=False): + # Convert a value to a boolean using "common sense" rules. + # Unset, empty, "0" and "false" (i-case) yield False. + # All other values yield True. + value = self.get(key) + if not value: + return default + if value.lower() in ['0', 'false']: + return False + return True diff --git a/compose/config/errors.py b/compose/config/errors.py new file mode 100644 index 00000000000..b66433a7998 --- /dev/null +++ b/compose/config/errors.py @@ -0,0 +1,55 @@ +VERSION_EXPLANATION = ( + 'You might be seeing this error because you\'re using the wrong Compose file version. ' + 'Either specify a supported version (e.g "2.2" or "3.3") and place ' + 'your service definitions under the `services` key, or omit the `version` key ' + 'and place your service definitions at the root of the file to use ' + 'version 1.\nFor more on the Compose file format versions, see ' + 'https://docs.docker.com/compose/compose-file/') + + +class ConfigurationError(Exception): + def __init__(self, msg): + self.msg = msg + + def __str__(self): + return self.msg + + +class EnvFileNotFound(ConfigurationError): + pass + + +class DependencyError(ConfigurationError): + pass + + +class CircularReference(ConfigurationError): + def __init__(self, trail): + self.trail = trail + + @property + def msg(self): + lines = [ + "{} in {}".format(service_name, filename) + for (filename, service_name) in self.trail + ] + return "Circular reference:\n {}".format("\n extends ".join(lines)) + + +class ComposeFileNotFound(ConfigurationError): + def __init__(self, supported_filenames): + super().__init__(""" + Can't find a suitable configuration file in this directory or any + parent. Are you in the right directory? + + Supported filenames: %s + """ % ", ".join(supported_filenames)) + + +class DuplicateOverrideFileFound(ConfigurationError): + def __init__(self, override_filenames): + self.override_filenames = override_filenames + super().__init__( + "Multiple override files found: {}. You may only use a single " + "override file.".format(", ".join(override_filenames)) + ) diff --git a/compose/config/interpolation.py b/compose/config/interpolation.py new file mode 100644 index 00000000000..a464f946569 --- /dev/null +++ b/compose/config/interpolation.py @@ -0,0 +1,295 @@ +import logging +import re +from string import Template + +from .errors import ConfigurationError +from compose.const import COMPOSEFILE_V1 as V1 +from compose.utils import parse_bytes +from compose.utils import parse_nanoseconds_int + + +log = logging.getLogger(__name__) + + +class Interpolator: + + def __init__(self, templater, mapping): + self.templater = templater + self.mapping = mapping + + def interpolate(self, string): + try: + return self.templater(string).substitute(self.mapping) + except ValueError: + raise InvalidInterpolation(string) + + +def interpolate_environment_variables(version, config, section, environment): + if version == V1: + interpolator = Interpolator(Template, environment) + else: + interpolator = Interpolator(TemplateWithDefaults, environment) + + def process_item(name, config_dict): + return { + key: interpolate_value(name, key, val, section, interpolator) + for key, val in (config_dict or {}).items() + } + + return { + name: process_item(name, config_dict or {}) + for name, config_dict in config.items() + } + + +def get_config_path(config_key, section, name): + return '{}/{}/{}'.format(section, name, config_key) + + +def interpolate_value(name, config_key, value, section, interpolator): + try: + return recursive_interpolate(value, interpolator, get_config_path(config_key, section, name)) + except InvalidInterpolation as e: + raise ConfigurationError( + 'Invalid interpolation format for "{config_key}" option ' + 'in {section} "{name}": "{string}"'.format( + config_key=config_key, + name=name, + section=section, + string=e.string)) + except UnsetRequiredSubstitution as e: + raise ConfigurationError( + 'Missing mandatory value for "{config_key}" option interpolating {value} ' + 'in {section} "{name}": {err}'.format(config_key=config_key, + value=value, + name=name, + section=section, + err=e.err) + ) + + +def recursive_interpolate(obj, interpolator, config_path): + def append(config_path, key): + return '{}/{}'.format(config_path, key) + + if isinstance(obj, str): + return converter.convert(config_path, interpolator.interpolate(obj)) + if isinstance(obj, dict): + return { + key: recursive_interpolate(val, interpolator, append(config_path, key)) + for key, val in obj.items() + } + if isinstance(obj, list): + return [recursive_interpolate(val, interpolator, config_path) for val in obj] + return converter.convert(config_path, obj) + + +class TemplateWithDefaults(Template): + pattern = r""" + {delim}(?: + (?P<escaped>{delim}) | + (?P<named>{id}) | + {{(?P<braced>{bid})}} | + (?P<invalid>) + ) + """.format( + delim=re.escape('$'), + id=r'[_a-z][_a-z0-9]*', + bid=r'[_a-z][_a-z0-9]*(?:(?P<sep>:?[-?])[^}]*)?', + ) + + @staticmethod + def process_braced_group(braced, sep, mapping): + if ':-' == sep: + var, _, default = braced.partition(':-') + return mapping.get(var) or default + elif '-' == sep: + var, _, default = braced.partition('-') + return mapping.get(var, default) + + elif ':?' == sep: + var, _, err = braced.partition(':?') + result = mapping.get(var) + if not result: + err = err or var + raise UnsetRequiredSubstitution(err) + return result + elif '?' == sep: + var, _, err = braced.partition('?') + if var in mapping: + return mapping.get(var) + err = err or var + raise UnsetRequiredSubstitution(err) + + # Modified from python2.7/string.py + def substitute(self, mapping): + # Helper function for .sub() + + def convert(mo): + named = mo.group('named') or mo.group('braced') + braced = mo.group('braced') + if braced is not None: + sep = mo.group('sep') + if sep: + return self.process_braced_group(braced, sep, mapping) + + if named is not None: + val = mapping[named] + if isinstance(val, bytes): + val = val.decode('utf-8') + return '{}'.format(val) + if mo.group('escaped') is not None: + return self.delimiter + if mo.group('invalid') is not None: + self._invalid(mo) + raise ValueError('Unrecognized named group in pattern', + self.pattern) + return self.pattern.sub(convert, self.template) + + +class InvalidInterpolation(Exception): + def __init__(self, string): + self.string = string + + +class UnsetRequiredSubstitution(Exception): + def __init__(self, custom_err_msg): + self.err = custom_err_msg + + +PATH_JOKER = '[^/]+' +FULL_JOKER = '.+' + + +def re_path(*args): + return re.compile('^{}$'.format('/'.join(args))) + + +def re_path_basic(section, name): + return re_path(section, PATH_JOKER, name) + + +def service_path(*args): + return re_path('service', PATH_JOKER, *args) + + +def to_boolean(s): + if not isinstance(s, str): + return s + s = s.lower() + if s in ['y', 'yes', 'true', 'on']: + return True + elif s in ['n', 'no', 'false', 'off']: + return False + raise ValueError('"{}" is not a valid boolean value'.format(s)) + + +def to_int(s): + if not isinstance(s, str): + return s + + # We must be able to handle octal representation for `mode` values notably + if re.match('^0[0-9]+$', s.strip()): + s = '0o' + s[1:] + try: + return int(s, base=0) + except ValueError: + raise ValueError('"{}" is not a valid integer'.format(s)) + + +def to_float(s): + if not isinstance(s, str): + return s + + try: + return float(s) + except ValueError: + raise ValueError('"{}" is not a valid float'.format(s)) + + +def to_str(o): + if isinstance(o, (bool, float, int)): + return '{}'.format(o) + return o + + +def bytes_to_int(s): + v = parse_bytes(s) + if v is None: + raise ValueError('"{}" is not a valid byte value'.format(s)) + return v + + +def to_microseconds(v): + if not isinstance(v, str): + return v + return int(parse_nanoseconds_int(v) / 1000) + + +class ConversionMap: + map = { + service_path('blkio_config', 'weight'): to_int, + service_path('blkio_config', 'weight_device', 'weight'): to_int, + service_path('build', 'labels', FULL_JOKER): to_str, + service_path('cpus'): to_float, + service_path('cpu_count'): to_int, + service_path('cpu_quota'): to_microseconds, + service_path('cpu_period'): to_microseconds, + service_path('cpu_rt_period'): to_microseconds, + service_path('cpu_rt_runtime'): to_microseconds, + service_path('configs', 'mode'): to_int, + service_path('secrets', 'mode'): to_int, + service_path('healthcheck', 'retries'): to_int, + service_path('healthcheck', 'disable'): to_boolean, + service_path('deploy', 'labels', PATH_JOKER): to_str, + service_path('deploy', 'replicas'): to_int, + service_path('deploy', 'resources', 'limits', "cpus"): to_float, + service_path('deploy', 'update_config', 'parallelism'): to_int, + service_path('deploy', 'update_config', 'max_failure_ratio'): to_float, + service_path('deploy', 'rollback_config', 'parallelism'): to_int, + service_path('deploy', 'rollback_config', 'max_failure_ratio'): to_float, + service_path('deploy', 'restart_policy', 'max_attempts'): to_int, + service_path('mem_swappiness'): to_int, + service_path('labels', FULL_JOKER): to_str, + service_path('oom_kill_disable'): to_boolean, + service_path('oom_score_adj'): to_int, + service_path('ports', 'target'): to_int, + service_path('ports', 'published'): to_int, + service_path('scale'): to_int, + service_path('ulimits', PATH_JOKER): to_int, + service_path('ulimits', PATH_JOKER, 'soft'): to_int, + service_path('ulimits', PATH_JOKER, 'hard'): to_int, + service_path('privileged'): to_boolean, + service_path('read_only'): to_boolean, + service_path('stdin_open'): to_boolean, + service_path('tty'): to_boolean, + service_path('volumes', 'read_only'): to_boolean, + service_path('volumes', 'volume', 'nocopy'): to_boolean, + service_path('volumes', 'tmpfs', 'size'): bytes_to_int, + re_path_basic('network', 'attachable'): to_boolean, + re_path_basic('network', 'external'): to_boolean, + re_path_basic('network', 'internal'): to_boolean, + re_path('network', PATH_JOKER, 'labels', FULL_JOKER): to_str, + re_path_basic('volume', 'external'): to_boolean, + re_path('volume', PATH_JOKER, 'labels', FULL_JOKER): to_str, + re_path_basic('secret', 'external'): to_boolean, + re_path('secret', PATH_JOKER, 'labels', FULL_JOKER): to_str, + re_path_basic('config', 'external'): to_boolean, + re_path('config', PATH_JOKER, 'labels', FULL_JOKER): to_str, + } + + def convert(self, path, value): + for rexp in self.map.keys(): + if rexp.match(path): + try: + return self.map[rexp](value) + except ValueError as e: + raise ConfigurationError( + 'Error while attempting to convert {} to appropriate type: {}'.format( + path.replace('/', '.'), e + ) + ) + return value + + +converter = ConversionMap() diff --git a/compose/config/serialize.py b/compose/config/serialize.py new file mode 100644 index 00000000000..e3295df78ea --- /dev/null +++ b/compose/config/serialize.py @@ -0,0 +1,149 @@ +import yaml + +from compose.config import types +from compose.const import COMPOSE_SPEC as VERSION +from compose.const import COMPOSEFILE_V1 as V1 + + +def serialize_config_type(dumper, data): + representer = dumper.represent_str + return representer(data.repr()) + + +def serialize_dict_type(dumper, data): + return dumper.represent_dict(data.repr()) + + +def serialize_string(dumper, data): + """ Ensure boolean-like strings are quoted in the output """ + representer = dumper.represent_str + + if isinstance(data, bytes): + data = data.decode('utf-8') + + if data.lower() in ('y', 'n', 'yes', 'no', 'on', 'off', 'true', 'false'): + # Empirically only y/n appears to be an issue, but this might change + # depending on which PyYaml version is being used. Err on safe side. + return dumper.represent_scalar('tag:yaml.org,2002:str', data, style='"') + return representer(data) + + +def serialize_string_escape_dollar(dumper, data): + """ Ensure boolean-like strings are quoted in the output and escape $ characters """ + data = data.replace('$', '$$') + return serialize_string(dumper, data) + + +yaml.SafeDumper.add_representer(types.MountSpec, serialize_dict_type) +yaml.SafeDumper.add_representer(types.VolumeFromSpec, serialize_config_type) +yaml.SafeDumper.add_representer(types.VolumeSpec, serialize_config_type) +yaml.SafeDumper.add_representer(types.SecurityOpt, serialize_config_type) +yaml.SafeDumper.add_representer(types.ServiceSecret, serialize_dict_type) +yaml.SafeDumper.add_representer(types.ServiceConfig, serialize_dict_type) +yaml.SafeDumper.add_representer(types.ServicePort, serialize_dict_type) + + +def denormalize_config(config, image_digests=None): + result = {'version': str(config.config_version)} + denormalized_services = [ + denormalize_service_dict( + service_dict, + config.version, + image_digests[service_dict['name']] if image_digests else None) + for service_dict in config.services + ] + result['services'] = { + service_dict.pop('name'): service_dict + for service_dict in denormalized_services + } + + for key in ('networks', 'volumes', 'secrets', 'configs'): + config_dict = getattr(config, key) + if not config_dict: + continue + result[key] = config_dict.copy() + for name, conf in result[key].items(): + if 'external_name' in conf: + del conf['external_name'] + + if 'name' in conf: + if 'external' in conf: + conf['external'] = bool(conf['external']) + return result + + +def serialize_config(config, image_digests=None, escape_dollar=True): + if escape_dollar: + yaml.SafeDumper.add_representer(str, serialize_string_escape_dollar) + yaml.SafeDumper.add_representer(str, serialize_string_escape_dollar) + else: + yaml.SafeDumper.add_representer(str, serialize_string) + yaml.SafeDumper.add_representer(str, serialize_string) + return yaml.safe_dump( + denormalize_config(config, image_digests), + default_flow_style=False, + indent=2, + width=80, + allow_unicode=True + ) + + +def serialize_ns_time_value(value): + result = (value, 'ns') + table = [ + (1000., 'us'), + (1000., 'ms'), + (1000., 's'), + (60., 'm'), + (60., 'h') + ] + for stage in table: + tmp = value / stage[0] + if tmp == int(value / stage[0]): + value = tmp + result = (int(value), stage[1]) + else: + break + return '{}{}'.format(*result) + + +def denormalize_service_dict(service_dict, version, image_digest=None): + service_dict = service_dict.copy() + + if image_digest: + service_dict['image'] = image_digest + + if 'restart' in service_dict: + service_dict['restart'] = types.serialize_restart_spec( + service_dict['restart'] + ) + + if version == V1 and 'network_mode' not in service_dict: + service_dict['network_mode'] = 'bridge' + + if 'healthcheck' in service_dict: + if 'interval' in service_dict['healthcheck']: + service_dict['healthcheck']['interval'] = serialize_ns_time_value( + service_dict['healthcheck']['interval'] + ) + if 'timeout' in service_dict['healthcheck']: + service_dict['healthcheck']['timeout'] = serialize_ns_time_value( + service_dict['healthcheck']['timeout'] + ) + + if 'start_period' in service_dict['healthcheck']: + service_dict['healthcheck']['start_period'] = serialize_ns_time_value( + service_dict['healthcheck']['start_period'] + ) + + if 'ports' in service_dict: + service_dict['ports'] = [ + p.legacy_repr() if p.external_ip or version < VERSION else p + for p in service_dict['ports'] + ] + if 'volumes' in service_dict and (version == V1): + service_dict['volumes'] = [ + v.legacy_repr() if isinstance(v, types.MountSpec) else v for v in service_dict['volumes'] + ] + + return service_dict diff --git a/compose/config/sort_services.py b/compose/config/sort_services.py new file mode 100644 index 00000000000..0a7eb2b4fda --- /dev/null +++ b/compose/config/sort_services.py @@ -0,0 +1,71 @@ +from compose.config.errors import DependencyError + + +def get_service_name_from_network_mode(network_mode): + return get_source_name_from_network_mode(network_mode, 'service') + + +def get_container_name_from_network_mode(network_mode): + return get_source_name_from_network_mode(network_mode, 'container') + + +def get_source_name_from_network_mode(network_mode, source_type): + if not network_mode: + return + + if not network_mode.startswith(source_type+':'): + return + + _, net_name = network_mode.split(':', 1) + return net_name + + +def get_service_names(links): + return [link.split(':', 1)[0] for link in links] + + +def get_service_names_from_volumes_from(volumes_from): + return [volume_from.source for volume_from in volumes_from] + + +def get_service_dependents(service_dict, services): + name = service_dict['name'] + return [ + service for service in services + if (name in get_service_names(service.get('links', [])) or + name in get_service_names_from_volumes_from(service.get('volumes_from', [])) or + name == get_service_name_from_network_mode(service.get('network_mode')) or + name == get_service_name_from_network_mode(service.get('pid')) or + name == get_service_name_from_network_mode(service.get('ipc')) or + name in service.get('depends_on', [])) + ] + + +def sort_service_dicts(services): + # Topological sort (Cormen/Tarjan algorithm). + unmarked = services[:] + temporary_marked = set() + sorted_services = [] + + def visit(n): + if n['name'] in temporary_marked: + if n['name'] in get_service_names(n.get('links', [])): + raise DependencyError('A service can not link to itself: %s' % n['name']) + if n['name'] in n.get('volumes_from', []): + raise DependencyError('A service can not mount itself as volume: %s' % n['name']) + if n['name'] in n.get('depends_on', []): + raise DependencyError('A service can not depend on itself: %s' % n['name']) + raise DependencyError('Circular dependency between %s' % ' and '.join(temporary_marked)) + + if n in unmarked: + temporary_marked.add(n['name']) + for m in get_service_dependents(n, services): + visit(m) + temporary_marked.remove(n['name']) + unmarked.remove(n) + sorted_services.insert(0, n) + + while unmarked: + visit(unmarked[-1]) + + return sorted_services diff --git a/compose/config/types.py b/compose/config/types.py new file mode 100644 index 00000000000..f52b5654139 --- /dev/null +++ b/compose/config/types.py @@ -0,0 +1,500 @@ +""" +Types for objects parsed from the configuration. +""" +import json +import ntpath +import os +import re +from collections import namedtuple + +from docker.utils.ports import build_port_bindings + +from ..const import COMPOSEFILE_V1 as V1 +from ..utils import unquote_path +from .errors import ConfigurationError +from compose.const import IS_WINDOWS_PLATFORM +from compose.utils import splitdrive + +win32_root_path_pattern = re.compile(r'^[A-Za-z]\:\\.*') + + +class VolumeFromSpec(namedtuple('_VolumeFromSpec', 'source mode type')): + + # TODO: drop service_names arg when v1 is removed + @classmethod + def parse(cls, volume_from_config, service_names, version): + func = cls.parse_v1 if version == V1 else cls.parse_v2 + return func(service_names, volume_from_config) + + @classmethod + def parse_v1(cls, service_names, volume_from_config): + parts = volume_from_config.split(':') + if len(parts) > 2: + raise ConfigurationError( + "volume_from {} has incorrect format, should be " + "service[:mode]".format(volume_from_config)) + + if len(parts) == 1: + source = parts[0] + mode = 'rw' + else: + source, mode = parts + + type = 'service' if source in service_names else 'container' + return cls(source, mode, type) + + @classmethod + def parse_v2(cls, service_names, volume_from_config): + parts = volume_from_config.split(':') + if len(parts) > 3: + raise ConfigurationError( + "volume_from {} has incorrect format, should be one of " + "'<service name>[:<mode>]' or " + "'container:<container name>[:<mode>]'".format(volume_from_config)) + + if len(parts) == 1: + source = parts[0] + return cls(source, 'rw', 'service') + + if len(parts) == 2: + if parts[0] == 'container': + type, source = parts + return cls(source, 'rw', type) + + source, mode = parts + return cls(source, mode, 'service') + + if len(parts) == 3: + type, source, mode = parts + if type not in ('service', 'container'): + raise ConfigurationError( + "Unknown volumes_from type '{}' in '{}'".format( + type, + volume_from_config)) + + return cls(source, mode, type) + + def repr(self): + return '{v.type}:{v.source}:{v.mode}'.format(v=self) + + +def parse_restart_spec(restart_config): + if not restart_config: + return None + parts = restart_config.split(':') + if len(parts) > 2: + raise ConfigurationError( + "Restart %s has incorrect format, should be " + "mode[:max_retry]" % restart_config) + if len(parts) == 2: + name, max_retry_count = parts + else: + name, = parts + max_retry_count = 0 + + return {'Name': name, 'MaximumRetryCount': int(max_retry_count)} + + +def serialize_restart_spec(restart_spec): + if not restart_spec: + return '' + parts = [restart_spec['Name']] + if restart_spec['MaximumRetryCount']: + parts.append(str(restart_spec['MaximumRetryCount'])) + return ':'.join(parts) + + +def parse_extra_hosts(extra_hosts_config): + if not extra_hosts_config: + return {} + + if isinstance(extra_hosts_config, dict): + return dict(extra_hosts_config) + + if isinstance(extra_hosts_config, list): + extra_hosts_dict = {} + for extra_hosts_line in extra_hosts_config: + # TODO: validate string contains ':' ? + host, ip = extra_hosts_line.split(':', 1) + extra_hosts_dict[host.strip()] = ip.strip() + return extra_hosts_dict + + +def normalize_path_for_engine(path): + """Windows paths, c:\\my\\path\\shiny, need to be changed to be compatible with + the Engine. Volume paths are expected to be linux style /c/my/path/shiny/ + """ + drive, tail = splitdrive(path) + + if drive: + path = '/' + drive.lower().rstrip(':') + tail + + return path.replace('\\', '/') + + +def normpath(path, win_host=False): + """ Custom path normalizer that handles Compose-specific edge cases like + UNIX paths on Windows hosts and vice-versa. """ + + sysnorm = ntpath.normpath if win_host else os.path.normpath + # If a path looks like a UNIX absolute path on Windows, it probably is; + # we'll need to revert the backslashes to forward slashes after normalization + flip_slashes = path.startswith('/') and IS_WINDOWS_PLATFORM + path = sysnorm(path) + if flip_slashes: + path = path.replace('\\', '/') + return path + + +class MountSpec: + options_map = { + 'volume': { + 'nocopy': 'no_copy' + }, + 'bind': { + 'propagation': 'propagation' + }, + 'tmpfs': { + 'size': 'tmpfs_size' + } + } + _fields = ['type', 'source', 'target', 'read_only', 'consistency'] + + @classmethod + def parse(cls, mount_dict, normalize=False, win_host=False): + if mount_dict.get('source'): + if mount_dict['type'] == 'tmpfs': + raise ConfigurationError('tmpfs mounts can not specify a source') + + mount_dict['source'] = normpath(mount_dict['source'], win_host) + if normalize: + mount_dict['source'] = normalize_path_for_engine(mount_dict['source']) + + return cls(**mount_dict) + + def __init__(self, type, source=None, target=None, read_only=None, consistency=None, **kwargs): + self.type = type + self.source = source + self.target = target + self.read_only = read_only + self.consistency = consistency + self.options = None + if self.type in kwargs: + self.options = kwargs[self.type] + + def as_volume_spec(self): + mode = 'ro' if self.read_only else 'rw' + return VolumeSpec(external=self.source, internal=self.target, mode=mode) + + def legacy_repr(self): + return self.as_volume_spec().repr() + + def repr(self): + res = {} + for field in self._fields: + if getattr(self, field, None): + res[field] = getattr(self, field) + if self.options: + res[self.type] = self.options + return res + + @property + def is_named_volume(self): + return self.type == 'volume' and self.source + + @property + def is_tmpfs(self): + return self.type == 'tmpfs' + + @property + def external(self): + return self.source + + +class VolumeSpec(namedtuple('_VolumeSpec', 'external internal mode')): + win32 = False + + @classmethod + def _parse_unix(cls, volume_config): + parts = volume_config.split(':') + + if len(parts) > 3: + raise ConfigurationError( + "Volume %s has incorrect format, should be " + "external:internal[:mode]" % volume_config) + + if len(parts) == 1: + external = None + internal = os.path.normpath(parts[0]) + else: + external = os.path.normpath(parts[0]) + internal = os.path.normpath(parts[1]) + + mode = 'rw' + if len(parts) == 3: + mode = parts[2] + + return cls(external, internal, mode) + + @classmethod + def _parse_win32(cls, volume_config, normalize): + # relative paths in windows expand to include the drive, eg C:\ + # so we join the first 2 parts back together to count as one + mode = 'rw' + + def separate_next_section(volume_config): + drive, tail = splitdrive(volume_config) + parts = tail.split(':', 1) + if drive: + parts[0] = drive + parts[0] + return parts + + parts = separate_next_section(volume_config) + if len(parts) == 1: + internal = parts[0] + external = None + else: + external = parts[0] + parts = separate_next_section(parts[1]) + external = normpath(external, True) + internal = parts[0] + if len(parts) > 1: + if ':' in parts[1]: + raise ConfigurationError( + "Volume %s has incorrect format, should be " + "external:internal[:mode]" % volume_config + ) + mode = parts[1] + + if normalize: + external = normalize_path_for_engine(external) if external else None + + result = cls(external, internal, mode) + result.win32 = True + return result + + @classmethod + def parse(cls, volume_config, normalize=False, win_host=False): + """Parse a volume_config path and split it into external:internal[:mode] + parts to be returned as a valid VolumeSpec. + """ + if IS_WINDOWS_PLATFORM or win_host: + return cls._parse_win32(volume_config, normalize) + else: + return cls._parse_unix(volume_config) + + def repr(self): + external = self.external + ':' if self.external else '' + mode = ':' + self.mode if self.external else '' + return '{ext}{v.internal}{mode}'.format(mode=mode, ext=external, v=self) + + @property + def is_named_volume(self): + res = self.external and not self.external.startswith(('.', '/', '~')) + if not self.win32: + return res + + return ( + res and not self.external.startswith('\\') and + not win32_root_path_pattern.match(self.external) + ) + + +class ServiceLink(namedtuple('_ServiceLink', 'target alias')): + + @classmethod + def parse(cls, link_spec): + target, _, alias = link_spec.partition(':') + if not alias: + alias = target + return cls(target, alias) + + def repr(self): + if self.target == self.alias: + return self.target + return '{s.target}:{s.alias}'.format(s=self) + + @property + def merge_field(self): + return self.alias + + +class ServiceConfigBase(namedtuple('_ServiceConfigBase', 'source target uid gid mode name')): + @classmethod + def parse(cls, spec): + if isinstance(spec, str): + return cls(spec, None, None, None, None, None) + return cls( + spec.get('source'), + spec.get('target'), + spec.get('uid'), + spec.get('gid'), + spec.get('mode'), + spec.get('name') + ) + + @property + def merge_field(self): + return self.source + + def repr(self): + return { + k: v for k, v in zip(self._fields, self) if v is not None + } + + +class ServiceSecret(ServiceConfigBase): + pass + + +class ServiceConfig(ServiceConfigBase): + pass + + +class ServicePort(namedtuple('_ServicePort', 'target published protocol mode external_ip')): + def __new__(cls, target, published, *args, **kwargs): + try: + if target: + target = int(target) + except ValueError: + raise ConfigurationError('Invalid target port: {}'.format(target)) + + if published: + if isinstance(published, str) and '-' in published: # "x-y:z" format + a, b = published.split('-', 1) + if not a.isdigit() or not b.isdigit(): + raise ConfigurationError('Invalid published port: {}'.format(published)) + else: + try: + published = int(published) + except ValueError: + raise ConfigurationError('Invalid published port: {}'.format(published)) + + return super().__new__( + cls, target, published, *args, **kwargs + ) + + @classmethod + def parse(cls, spec): + if isinstance(spec, cls): + # When extending a service with ports, the port definitions have already been parsed + return [spec] + + if not isinstance(spec, dict): + result = [] + try: + for k, v in build_port_bindings([spec]).items(): + if '/' in k: + target, proto = k.split('/', 1) + else: + target, proto = (k, None) + for pub in v: + if pub is None: + result.append( + cls(target, None, proto, None, None) + ) + elif isinstance(pub, tuple): + result.append( + cls(target, pub[1], proto, None, pub[0]) + ) + else: + result.append( + cls(target, pub, proto, None, None) + ) + except ValueError as e: + raise ConfigurationError(str(e)) + + return result + + return [cls( + spec.get('target'), + spec.get('published'), + spec.get('protocol'), + spec.get('mode'), + None + )] + + @property + def merge_field(self): + return (self.target, self.published, self.external_ip, self.protocol) + + def repr(self): + return { + k: v for k, v in zip(self._fields, self) if v is not None + } + + def legacy_repr(self): + return normalize_port_dict(self.repr()) + + +class GenericResource(namedtuple('_GenericResource', 'kind value')): + @classmethod + def parse(cls, dct): + if 'discrete_resource_spec' not in dct: + raise ConfigurationError( + 'generic_resource entry must include a discrete_resource_spec key' + ) + if 'kind' not in dct['discrete_resource_spec']: + raise ConfigurationError( + 'generic_resource entry must include a discrete_resource_spec.kind subkey' + ) + return cls( + dct['discrete_resource_spec']['kind'], + dct['discrete_resource_spec'].get('value') + ) + + def repr(self): + return { + 'discrete_resource_spec': { + 'kind': self.kind, + 'value': self.value, + } + } + + @property + def merge_field(self): + return self.kind + + +def normalize_port_dict(port): + return '{external_ip}{has_ext_ip}{published}{is_pub}{target}/{protocol}'.format( + published=port.get('published', ''), + is_pub=(':' if port.get('published') is not None or port.get('external_ip') else ''), + target=port.get('target'), + protocol=port.get('protocol', 'tcp'), + external_ip=port.get('external_ip', ''), + has_ext_ip=(':' if port.get('external_ip') else ''), + ) + + +class SecurityOpt(namedtuple('_SecurityOpt', 'value src_file')): + @classmethod + def parse(cls, value): + if not isinstance(value, str): + return value + # based on https://github.com/docker/cli/blob/9de1b162f/cli/command/container/opts.go#L673-L697 + con = value.split('=', 2) + if len(con) == 1 and con[0] != 'no-new-privileges': + if ':' not in value: + raise ConfigurationError('Invalid security_opt: {}'.format(value)) + con = value.split(':', 2) + + if con[0] == 'seccomp' and con[1] != 'unconfined': + try: + with open(unquote_path(con[1])) as f: + seccomp_data = json.load(f) + except (OSError, ValueError) as e: + raise ConfigurationError('Error reading seccomp profile: {}'.format(e)) + return cls( + 'seccomp={}'.format(json.dumps(seccomp_data)), con[1] + ) + return cls(value, None) + + def repr(self): + if self.src_file is not None: + return 'seccomp:{}'.format(self.src_file) + return self.value + + @property + def merge_field(self): + return self.value diff --git a/compose/config/validation.py b/compose/config/validation.py new file mode 100644 index 00000000000..d9aaeda4bd2 --- /dev/null +++ b/compose/config/validation.py @@ -0,0 +1,569 @@ +import json +import logging +import os +import re +import sys + +from docker.utils.ports import split_port +from jsonschema import Draft4Validator +from jsonschema import FormatChecker +from jsonschema import RefResolver +from jsonschema import ValidationError + +from ..const import COMPOSEFILE_V1 as V1 +from ..const import NANOCPUS_SCALE +from .errors import ConfigurationError +from .errors import VERSION_EXPLANATION +from .sort_services import get_service_name_from_network_mode + + +log = logging.getLogger(__name__) + + +DOCKER_CONFIG_HINTS = { + 'cpu_share': 'cpu_shares', + 'add_host': 'extra_hosts', + 'hosts': 'extra_hosts', + 'extra_host': 'extra_hosts', + 'device': 'devices', + 'link': 'links', + 'memory_swap': 'memswap_limit', + 'port': 'ports', + 'privilege': 'privileged', + 'priviliged': 'privileged', + 'privilige': 'privileged', + 'volume': 'volumes', + 'workdir': 'working_dir', +} + + +VALID_NAME_CHARS = r'[a-zA-Z0-9\._\-]' +VALID_EXPOSE_FORMAT = r'^\d+(\-\d+)?(\/[a-zA-Z]+)?$' + +VALID_IPV4_SEG = r'(\d{1,2}|1\d{2}|2[0-4]\d|25[0-5])' +VALID_IPV4_ADDR = r"({IPV4_SEG}\.){{3}}{IPV4_SEG}".format(IPV4_SEG=VALID_IPV4_SEG) +VALID_REGEX_IPV4_CIDR = r"^{IPV4_ADDR}/(\d|[1-2]\d|3[0-2])$".format(IPV4_ADDR=VALID_IPV4_ADDR) + +VALID_IPV6_SEG = r'[0-9a-fA-F]{1,4}' +VALID_REGEX_IPV6_CIDR = "".join(r""" +^ +( + (({IPV6_SEG}:){{7}}{IPV6_SEG})| + (({IPV6_SEG}:){{1,7}}:)| + (({IPV6_SEG}:){{1,6}}(:{IPV6_SEG}){{1,1}})| + (({IPV6_SEG}:){{1,5}}(:{IPV6_SEG}){{1,2}})| + (({IPV6_SEG}:){{1,4}}(:{IPV6_SEG}){{1,3}})| + (({IPV6_SEG}:){{1,3}}(:{IPV6_SEG}){{1,4}})| + (({IPV6_SEG}:){{1,2}}(:{IPV6_SEG}){{1,5}})| + (({IPV6_SEG}:){{1,1}}(:{IPV6_SEG}){{1,6}})| + (:((:{IPV6_SEG}){{1,7}}|:))| + (fe80:(:{IPV6_SEG}){{0,4}}%[0-9a-zA-Z]{{1,}})| + (::(ffff(:0{{1,4}}){{0,1}}:){{0,1}}{IPV4_ADDR})| + (({IPV6_SEG}:){{1,4}}:{IPV4_ADDR}) +) +/(\d|[1-9]\d|1[0-1]\d|12[0-8]) +$ +""".format(IPV6_SEG=VALID_IPV6_SEG, IPV4_ADDR=VALID_IPV4_ADDR).split()) + + +@FormatChecker.cls_checks(format="ports", raises=ValidationError) +def format_ports(instance): + try: + split_port(instance) + except ValueError as e: + raise ValidationError(str(e)) + return True + + +@FormatChecker.cls_checks(format="expose", raises=ValidationError) +def format_expose(instance): + if isinstance(instance, str): + if not re.match(VALID_EXPOSE_FORMAT, instance): + raise ValidationError( + "should be of the format 'PORT[/PROTOCOL]'") + + return True + + +@FormatChecker.cls_checks("subnet_ip_address", raises=ValidationError) +def format_subnet_ip_address(instance): + if isinstance(instance, str): + if not re.match(VALID_REGEX_IPV4_CIDR, instance) and \ + not re.match(VALID_REGEX_IPV6_CIDR, instance): + raise ValidationError("should use the CIDR format") + + return True + + +def match_named_volumes(service_dict, project_volumes): + service_volumes = service_dict.get('volumes', []) + for volume_spec in service_volumes: + if volume_spec.is_named_volume and volume_spec.external not in project_volumes: + raise ConfigurationError( + 'Named volume "{}" is used in service "{}" but no' + ' declaration was found in the volumes section.'.format( + volume_spec.repr(), service_dict.get('name') + ) + ) + + +def python_type_to_yaml_type(type_): + type_name = type(type_).__name__ + return { + 'dict': 'mapping', + 'list': 'array', + 'int': 'number', + 'float': 'number', + 'bool': 'boolean', + 'unicode': 'string', + 'str': 'string', + 'bytes': 'string', + }.get(type_name, type_name) + + +def validate_config_section(filename, config, section): + """Validate the structure of a configuration section. This must be done + before interpolation so it's separate from schema validation. + """ + if not isinstance(config, dict): + raise ConfigurationError( + "In file '{filename}', {section} must be a mapping, not " + "{type}.".format( + filename=filename, + section=section, + type=anglicize_json_type(python_type_to_yaml_type(config)))) + + for key, value in config.items(): + if not isinstance(key, str): + raise ConfigurationError( + "In file '{filename}', the {section} name {name} must be a " + "quoted string, i.e. '{name}'.".format( + filename=filename, + section=section, + name=key)) + + if not isinstance(value, (dict, type(None))): + raise ConfigurationError( + "In file '{filename}', {section} '{name}' must be a mapping not " + "{type}.".format( + filename=filename, + section=section, + name=key, + type=anglicize_json_type(python_type_to_yaml_type(value)))) + + +def validate_top_level_object(config_file): + if not isinstance(config_file.config, dict): + raise ConfigurationError( + "Top level object in '{}' needs to be an object not '{}'.".format( + config_file.filename, + type(config_file.config))) + + +def validate_ulimits(service_config): + ulimit_config = service_config.config.get('ulimits', {}) + for limit_name, soft_hard_values in ulimit_config.items(): + if isinstance(soft_hard_values, dict): + if not soft_hard_values['soft'] <= soft_hard_values['hard']: + raise ConfigurationError( + "Service '{s.name}' has invalid ulimit '{ulimit}'. " + "'soft' value can not be greater than 'hard' value ".format( + s=service_config, + ulimit=ulimit_config)) + + +def validate_extends_file_path(service_name, extends_options, filename): + """ + The service to be extended must either be defined in the config key 'file', + or within 'filename'. + """ + error_prefix = "Invalid 'extends' configuration for %s:" % service_name + + if 'file' not in extends_options and filename is None: + raise ConfigurationError( + "%s you need to specify a 'file', e.g. 'file: something.yml'" % error_prefix + ) + + +def validate_network_mode(service_config, service_names): + network_mode = service_config.config.get('network_mode') + if not network_mode: + return + + if 'networks' in service_config.config: + raise ConfigurationError("'network_mode' and 'networks' cannot be combined") + + dependency = get_service_name_from_network_mode(network_mode) + if not dependency: + return + + if dependency not in service_names: + raise ConfigurationError( + "Service '{s.name}' uses the network stack of service '{dep}' which " + "is undefined.".format(s=service_config, dep=dependency)) + + +def validate_pid_mode(service_config, service_names): + pid_mode = service_config.config.get('pid') + if not pid_mode: + return + + dependency = get_service_name_from_network_mode(pid_mode) + if not dependency: + return + if dependency not in service_names: + raise ConfigurationError( + "Service '{s.name}' uses the PID namespace of service '{dep}' which " + "is undefined.".format(s=service_config, dep=dependency) + ) + + +def validate_ipc_mode(service_config, service_names): + ipc_mode = service_config.config.get('ipc') + if not ipc_mode: + return + + dependency = get_service_name_from_network_mode(ipc_mode) + if not dependency: + return + if dependency not in service_names: + raise ConfigurationError( + "Service '{s.name}' uses the IPC namespace of service '{dep}' which " + "is undefined.".format(s=service_config, dep=dependency) + ) + + +def validate_links(service_config, service_names): + for link in service_config.config.get('links', []): + if link.split(':')[0] not in service_names: + raise ConfigurationError( + "Service '{s.name}' has a link to service '{link}' which is " + "undefined.".format(s=service_config, link=link)) + + +def validate_depends_on(service_config, service_names): + deps = service_config.config.get('depends_on', {}) + for dependency in deps.keys(): + if dependency not in service_names: + raise ConfigurationError( + "Service '{s.name}' depends on service '{dep}' which is " + "undefined.".format(s=service_config, dep=dependency) + ) + + +def validate_credential_spec(service_config): + credential_spec = service_config.config.get('credential_spec') + if not credential_spec: + return + + if 'registry' not in credential_spec and 'file' not in credential_spec: + raise ConfigurationError( + "Service '{s.name}' is missing 'credential_spec.file' or " + "credential_spec.registry'".format(s=service_config) + ) + + +def get_unsupported_config_msg(path, error_key): + msg = "Unsupported config option for {}: '{}'".format(path_string(path), error_key) + if error_key in DOCKER_CONFIG_HINTS: + msg += " (did you mean '{}'?)".format(DOCKER_CONFIG_HINTS[error_key]) + return msg + + +def anglicize_json_type(json_type): + if json_type.startswith(('a', 'e', 'i', 'o', 'u')): + return 'an ' + json_type + return 'a ' + json_type + + +def is_service_dict_schema(schema_id): + return schema_id in ('config_schema_v1.json', '#/properties/services') + + +def handle_error_for_schema_with_id(error, path): + schema_id = error.schema['id'] + + if is_service_dict_schema(schema_id) and error.validator == 'additionalProperties': + return "Invalid service name '{}' - only {} characters are allowed".format( + # The service_name is one of the keys in the json object + [i for i in list(error.instance) if not i or any(filter( + lambda c: not re.match(VALID_NAME_CHARS, c), i + ))][0], + VALID_NAME_CHARS + ) + + if error.validator == 'additionalProperties': + if schema_id == '#/definitions/service': + invalid_config_key = parse_key_from_error_msg(error) + return get_unsupported_config_msg(path, invalid_config_key) + + if schema_id.startswith('config_schema_'): + invalid_config_key = parse_key_from_error_msg(error) + return ('Invalid top-level property "{key}". Valid top-level ' + 'sections for this Compose file are: {properties}, and ' + 'extensions starting with "x-".\n\n{explanation}').format( + key=invalid_config_key, + properties=', '.join(error.schema['properties'].keys()), + explanation=VERSION_EXPLANATION + ) + + if not error.path: + return '{}\n\n{}'.format(error.message, VERSION_EXPLANATION) + + +def handle_generic_error(error, path): + msg_format = None + error_msg = error.message + + if error.validator == 'oneOf': + msg_format = "{path} {msg}" + config_key, error_msg = _parse_oneof_validator(error) + if config_key: + path.append(config_key) + + elif error.validator == 'type': + msg_format = "{path} contains an invalid type, it should be {msg}" + error_msg = _parse_valid_types_from_validator(error.validator_value) + + elif error.validator == 'required': + error_msg = ", ".join(error.validator_value) + msg_format = "{path} is invalid, {msg} is required." + + elif error.validator == 'dependencies': + config_key = list(error.validator_value.keys())[0] + required_keys = ",".join(error.validator_value[config_key]) + + msg_format = "{path} is invalid: {msg}" + path.append(config_key) + error_msg = "when defining '{}' you must set '{}' as well".format( + config_key, + required_keys) + + elif error.cause: + error_msg = str(error.cause) + msg_format = "{path} is invalid: {msg}" + + elif error.path: + msg_format = "{path} value {msg}" + + if msg_format: + return msg_format.format(path=path_string(path), msg=error_msg) + + return error.message + + +def parse_key_from_error_msg(error): + try: + return error.message.split("'")[1] + except IndexError: + return error.message.split('(')[1].split(' ')[0].strip("'") + + +def path_string(path): + return ".".join(c for c in path if isinstance(c, str)) + + +def _parse_valid_types_from_validator(validator): + """A validator value can be either an array of valid types or a string of + a valid type. Parse the valid types and prefix with the correct article. + """ + if not isinstance(validator, list): + return anglicize_json_type(validator) + + if len(validator) == 1: + return anglicize_json_type(validator[0]) + + return "{}, or {}".format( + ", ".join([anglicize_json_type(validator[0])] + validator[1:-1]), + anglicize_json_type(validator[-1])) + + +def _parse_oneof_validator(error): + """oneOf has multiple schemas, so we need to reason about which schema, sub + schema or constraint the validation is failing on. + Inspecting the context value of a ValidationError gives us information about + which sub schema failed and which kind of error it is. + """ + types = [] + for context in error.context: + if context.validator == 'oneOf': + _, error_msg = _parse_oneof_validator(context) + return path_string(context.path), error_msg + + if context.validator == 'required': + return (None, context.message) + + if context.validator == 'additionalProperties': + invalid_config_key = parse_key_from_error_msg(context) + return (None, "contains unsupported option: '{}'".format(invalid_config_key)) + + if context.validator == 'uniqueItems': + return ( + path_string(context.path) if context.path else None, + "contains non-unique items, please remove duplicates from {}".format( + context.instance), + ) + + if context.path: + return ( + path_string(context.path), + "contains {}, which is an invalid type, it should be {}".format( + json.dumps(context.instance), + _parse_valid_types_from_validator(context.validator_value)), + ) + + if context.validator == 'type': + types.append(context.validator_value) + + valid_types = _parse_valid_types_from_validator(types) + return (None, "contains an invalid type, it should be {}".format(valid_types)) + + +def process_service_constraint_errors(error, service_name, version): + if version == V1: + if 'image' in error.instance and 'build' in error.instance: + return ( + "Service {} has both an image and build path specified. " + "A service can either be built to image or use an existing " + "image, not both.".format(service_name)) + + if 'image' in error.instance and 'dockerfile' in error.instance: + return ( + "Service {} has both an image and alternate Dockerfile. " + "A service can either be built to image or use an existing " + "image, not both.".format(service_name)) + + if 'image' not in error.instance and 'build' not in error.instance: + return ( + "Service {} has neither an image nor a build context specified. " + "At least one must be provided.".format(service_name)) + + +def process_config_schema_errors(error): + path = list(error.path) + + if 'id' in error.schema: + error_msg = handle_error_for_schema_with_id(error, path) + if error_msg: + return error_msg + + return handle_generic_error(error, path) + + +def keys_to_str(config_file): + """ + Non-string keys may break validator with patterned fields. + """ + d = {} + for k, v in config_file.items(): + d[str(k)] = v + if isinstance(v, dict): + d[str(k)] = keys_to_str(v) + return d + + +def validate_against_config_schema(config_file, version): + schema = load_jsonschema(version) + config = keys_to_str(config_file.config) + + format_checker = FormatChecker(["ports", "expose", "subnet_ip_address"]) + validator = Draft4Validator( + schema, + resolver=RefResolver(get_resolver_path(), schema), + format_checker=format_checker) + handle_errors( + validator.iter_errors(config), + process_config_schema_errors, + config_file.filename) + + +def validate_service_constraints(config, service_name, config_file): + def handler(errors): + return process_service_constraint_errors( + errors, service_name, config_file.version) + + schema = load_jsonschema(config_file.version) + validator = Draft4Validator(schema['definitions']['constraints']['service']) + handle_errors(validator.iter_errors(config), handler, None) + + +def validate_cpu(service_config): + cpus = service_config.config.get('cpus') + if not cpus: + return + nano_cpus = cpus * NANOCPUS_SCALE + if isinstance(nano_cpus, float) and not nano_cpus.is_integer(): + raise ConfigurationError( + "cpus must have nine or less digits after decimal point") + + +def get_schema_path(): + return os.path.dirname(os.path.abspath(__file__)) + + +def load_jsonschema(version): + name = "compose_spec" + if version == V1: + name = "config_schema_v1" + + filename = os.path.join( + get_schema_path(), + "{}.json".format(name)) + + if not os.path.exists(filename): + raise ConfigurationError( + 'Version in "{}" is unsupported. {}' + .format(filename, VERSION_EXPLANATION)) + with open(filename) as fh: + return json.load(fh) + + +def get_resolver_path(): + schema_path = get_schema_path() + if sys.platform == "win32": + scheme = "///" + # TODO: why is this necessary? + schema_path = schema_path.replace('\\', '/') + else: + scheme = "//" + return "file:{}{}/".format(scheme, schema_path) + + +def handle_errors(errors, format_error_func, filename): + """jsonschema returns an error tree full of information to explain what has + gone wrong. Process each error and pull out relevant information and re-write + helpful error messages that are relevant. + """ + errors = sorted(errors, key=str) + if not errors: + return + + error_msg = '\n'.join(format_error_func(error) for error in errors) + raise ConfigurationError( + "The Compose file{file_msg} is invalid because:\n{error_msg}".format( + file_msg=" '{}'".format(filename) if filename else "", + error_msg=error_msg)) + + +def validate_healthcheck(service_config): + healthcheck = service_config.config.get('healthcheck', {}) + + if 'test' in healthcheck and isinstance(healthcheck['test'], list): + if len(healthcheck['test']) == 0: + raise ConfigurationError( + 'Service "{}" defines an invalid healthcheck: ' + '"test" is an empty list' + .format(service_config.name)) + + # when disable is true config.py::process_healthcheck adds "test: ['NONE']" to service_config + elif healthcheck['test'][0] == 'NONE' and len(healthcheck) > 1: + raise ConfigurationError( + 'Service "{}" defines an invalid healthcheck: ' + '"disable: true" cannot be combined with other options' + .format(service_config.name)) + + elif healthcheck['test'][0] not in ('NONE', 'CMD', 'CMD-SHELL'): + raise ConfigurationError( + 'Service "{}" defines an invalid healthcheck: ' + 'when "test" is a list the first item must be either NONE, CMD or CMD-SHELL' + .format(service_config.name)) diff --git a/compose/const.py b/compose/const.py new file mode 100644 index 00000000000..04342980255 --- /dev/null +++ b/compose/const.py @@ -0,0 +1,39 @@ +import sys + +from .version import ComposeVersion + +DEFAULT_TIMEOUT = 10 +HTTP_TIMEOUT = 60 +IS_WINDOWS_PLATFORM = (sys.platform == "win32") +LABEL_CONTAINER_NUMBER = 'com.docker.compose.container-number' +LABEL_ONE_OFF = 'com.docker.compose.oneoff' +LABEL_PROJECT = 'com.docker.compose.project' +LABEL_WORKING_DIR = 'com.docker.compose.project.working_dir' +LABEL_CONFIG_FILES = 'com.docker.compose.project.config_files' +LABEL_ENVIRONMENT_FILE = 'com.docker.compose.project.environment_file' +LABEL_SERVICE = 'com.docker.compose.service' +LABEL_NETWORK = 'com.docker.compose.network' +LABEL_VERSION = 'com.docker.compose.version' +LABEL_SLUG = 'com.docker.compose.slug' +LABEL_VOLUME = 'com.docker.compose.volume' +LABEL_CONFIG_HASH = 'com.docker.compose.config-hash' +NANOCPUS_SCALE = 1000000000 +PARALLEL_LIMIT = 64 + +SECRETS_PATH = '/run/secrets' +WINDOWS_LONGPATH_PREFIX = '\\\\?\\' + +COMPOSEFILE_V1 = ComposeVersion('1') +COMPOSE_SPEC = ComposeVersion('3.9') + +# minimum DOCKER ENGINE API version needed to support +# features for each compose schema version +API_VERSIONS = { + COMPOSEFILE_V1: '1.21', + COMPOSE_SPEC: '1.38', +} + +API_VERSION_TO_ENGINE_VERSION = { + API_VERSIONS[COMPOSEFILE_V1]: '1.9.0', + API_VERSIONS[COMPOSE_SPEC]: '18.06.0', +} diff --git a/compose/container.py b/compose/container.py new file mode 100644 index 00000000000..00626b6190e --- /dev/null +++ b/compose/container.py @@ -0,0 +1,331 @@ +from functools import reduce + +from docker.errors import ImageNotFound + +from .const import LABEL_CONTAINER_NUMBER +from .const import LABEL_ONE_OFF +from .const import LABEL_PROJECT +from .const import LABEL_SERVICE +from .const import LABEL_SLUG +from .const import LABEL_VERSION +from .utils import truncate_id +from .version import ComposeVersion + + +class Container: + """ + Represents a Docker container, constructed from the output of + GET /containers/:id:/json. + """ + def __init__(self, client, dictionary, has_been_inspected=False): + self.client = client + self.dictionary = dictionary + self.has_been_inspected = has_been_inspected + self.log_stream = None + + @classmethod + def from_ps(cls, client, dictionary, **kwargs): + """ + Construct a container object from the output of GET /containers/json. + """ + name = get_container_name(dictionary) + if name is None: + return None + + new_dictionary = { + 'Id': dictionary['Id'], + 'Image': dictionary['Image'], + 'Name': '/' + name, + } + return cls(client, new_dictionary, **kwargs) + + @classmethod + def from_id(cls, client, id): + return cls(client, client.inspect_container(id), has_been_inspected=True) + + @classmethod + def create(cls, client, **options): + response = client.create_container(**options) + return cls.from_id(client, response['Id']) + + @property + def id(self): + return self.dictionary['Id'] + + @property + def image(self): + return self.dictionary['Image'] + + @property + def image_config(self): + return self.client.inspect_image(self.image) + + @property + def short_id(self): + return self.id[:12] + + @property + def name(self): + return self.dictionary['Name'][1:] + + @property + def project(self): + return self.labels.get(LABEL_PROJECT) + + @property + def service(self): + return self.labels.get(LABEL_SERVICE) + + @property + def name_without_project(self): + if self.name.startswith('{}_{}'.format(self.project, self.service)): + return '{}_{}'.format(self.service, self.number if self.number is not None else self.slug) + else: + return self.name + + @property + def number(self): + if self.one_off: + # One-off containers are no longer assigned numbers and use slugs instead. + return None + + number = self.labels.get(LABEL_CONTAINER_NUMBER) + if not number: + raise ValueError("Container {} does not have a {} label".format( + self.short_id, LABEL_CONTAINER_NUMBER)) + return int(number) + + @property + def slug(self): + if not self.full_slug: + return None + return truncate_id(self.full_slug) + + @property + def full_slug(self): + return self.labels.get(LABEL_SLUG) + + @property + def one_off(self): + return self.labels.get(LABEL_ONE_OFF) == 'True' + + @property + def ports(self): + self.inspect_if_not_inspected() + return self.get('NetworkSettings.Ports') or {} + + @property + def human_readable_ports(self): + def format_port(private, public): + if not public: + return [private] + return [ + '{HostIp}:{HostPort}->{private}'.format(private=private, **pub) + for pub in public + ] + + return ', '.join( + ','.join(format_port(*item)) + for item in sorted(self.ports.items()) + ) + + @property + def labels(self): + return self.get('Config.Labels') or {} + + @property + def stop_signal(self): + return self.get('Config.StopSignal') + + @property + def log_config(self): + return self.get('HostConfig.LogConfig') or None + + @property + def human_readable_state(self): + if self.is_paused: + return 'Paused' + if self.is_restarting: + return 'Restarting' + if self.is_running: + return 'Ghost' if self.get('State.Ghost') else self.human_readable_health_status + else: + return 'Exit %s' % self.get('State.ExitCode') + + @property + def human_readable_command(self): + entrypoint = self.get('Config.Entrypoint') or [] + cmd = self.get('Config.Cmd') or [] + return ' '.join(entrypoint + cmd) + + @property + def environment(self): + def parse_env(var): + if '=' in var: + return var.split("=", 1) + return var, None + return dict(parse_env(var) for var in self.get('Config.Env') or []) + + @property + def exit_code(self): + return self.get('State.ExitCode') + + @property + def is_running(self): + return self.get('State.Running') + + @property + def is_restarting(self): + return self.get('State.Restarting') + + @property + def is_paused(self): + return self.get('State.Paused') + + @property + def log_driver(self): + return self.get('HostConfig.LogConfig.Type') + + @property + def has_api_logs(self): + log_type = self.log_driver + return not log_type or log_type in ('json-file', 'journald', 'local') + + @property + def human_readable_health_status(self): + """ Generate UP status string with up time and health + """ + status_string = 'Up' + container_status = self.get('State.Health.Status') + if container_status == 'starting': + status_string += ' (health: starting)' + elif container_status is not None: + status_string += ' (%s)' % container_status + return status_string + + def attach_log_stream(self): + """A log stream can only be attached if the container uses a + json-file, journald or local log driver. + """ + if self.has_api_logs: + self.log_stream = self.attach(stdout=True, stderr=True, stream=True) + + def get(self, key): + """Return a value from the container or None if the value is not set. + + :param key: a string using dotted notation for nested dictionary + lookups + """ + self.inspect_if_not_inspected() + + def get_value(dictionary, key): + return (dictionary or {}).get(key) + + return reduce(get_value, key.split('.'), self.dictionary) + + def get_local_port(self, port, protocol='tcp'): + port = self.ports.get("{}/{}".format(port, protocol)) + return "{HostIp}:{HostPort}".format(**port[0]) if port else None + + def get_mount(self, mount_dest): + for mount in self.get('Mounts'): + if mount['Destination'] == mount_dest: + return mount + return None + + def start(self, **options): + return self.client.start(self.id, **options) + + def stop(self, **options): + return self.client.stop(self.id, **options) + + def pause(self, **options): + return self.client.pause(self.id, **options) + + def unpause(self, **options): + return self.client.unpause(self.id, **options) + + def kill(self, **options): + return self.client.kill(self.id, **options) + + def restart(self, **options): + return self.client.restart(self.id, **options) + + def remove(self, **options): + return self.client.remove_container(self.id, **options) + + def create_exec(self, command, **options): + return self.client.exec_create(self.id, command, **options) + + def start_exec(self, exec_id, **options): + return self.client.exec_start(exec_id, **options) + + def rename_to_tmp_name(self): + """Rename the container to a hopefully unique temporary container name + by prepending the short id. + """ + if not self.name.startswith(self.short_id): + self.client.rename( + self.id, '{}_{}'.format(self.short_id, self.name) + ) + + def inspect_if_not_inspected(self): + if not self.has_been_inspected: + self.inspect() + + def wait(self): + return self.client.wait(self.id).get('StatusCode', 127) + + def logs(self, *args, **kwargs): + return self.client.logs(self.id, *args, **kwargs) + + def inspect(self): + self.dictionary = self.client.inspect_container(self.id) + self.has_been_inspected = True + return self.dictionary + + def image_exists(self): + try: + self.client.inspect_image(self.image) + except ImageNotFound: + return False + + return True + + def reset_image(self, img_id): + """ If this container's image has been removed, temporarily replace the old image ID + with `img_id`. + """ + if not self.image_exists(): + self.dictionary['Image'] = img_id + + def attach(self, *args, **kwargs): + return self.client.attach(self.id, *args, **kwargs) + + def has_legacy_proj_name(self, project_name): + return ( + ComposeVersion(self.labels.get(LABEL_VERSION)) < ComposeVersion('1.21.0') and + self.project != project_name + ) + + def __repr__(self): + return '<Container: {} ({})>'.format(self.name, self.id[:6]) + + def __eq__(self, other): + if type(self) != type(other): + return False + return self.id == other.id + + def __hash__(self): + return self.id.__hash__() + + +def get_container_name(container): + if not container.get('Name') and not container.get('Names'): + return None + # inspect + if 'Name' in container: + return container['Name'] + # ps + shortest_name = min(container['Names'], key=lambda n: len(n.split('/'))) + return shortest_name.split('/')[-1] diff --git a/compose/errors.py b/compose/errors.py new file mode 100644 index 00000000000..d4fead251ad --- /dev/null +++ b/compose/errors.py @@ -0,0 +1,29 @@ +class OperationFailedError(Exception): + def __init__(self, reason): + self.msg = reason + + +class StreamParseError(RuntimeError): + def __init__(self, reason): + self.msg = reason + + +class HealthCheckException(Exception): + def __init__(self, reason): + self.msg = reason + + +class HealthCheckFailed(HealthCheckException): + def __init__(self, container_id): + super().__init__( + 'Container "{}" is unhealthy.'.format(container_id) + ) + + +class NoHealthCheckConfigured(HealthCheckException): + def __init__(self, service_name): + super().__init__( + 'Service "{}" is missing a healthcheck configuration'.format( + service_name + ) + ) diff --git a/compose/metrics/__init__.py b/compose/metrics/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/compose/metrics/client.py b/compose/metrics/client.py new file mode 100644 index 00000000000..513e22ce61c --- /dev/null +++ b/compose/metrics/client.py @@ -0,0 +1,64 @@ +import os +from enum import Enum + +import requests +from docker import ContextAPI +from docker.transport import UnixHTTPAdapter + +from compose.const import IS_WINDOWS_PLATFORM + +if IS_WINDOWS_PLATFORM: + from docker.transport import NpipeHTTPAdapter + + +class Status(Enum): + SUCCESS = "success" + FAILURE = "failure" + CANCELED = "canceled" + + +class MetricsSource: + CLI = "docker-compose" + + +if IS_WINDOWS_PLATFORM: + METRICS_SOCKET_FILE = 'npipe://\\\\.\\pipe\\docker_cli' +else: + METRICS_SOCKET_FILE = 'http+unix:///var/run/docker-cli.sock' + + +class MetricsCommand(requests.Session): + """ + Representation of a command in the metrics. + """ + + def __init__(self, command, + context_type=None, status=Status.SUCCESS, + source=MetricsSource.CLI, uri=None): + super().__init__() + self.command = "compose " + command if command else "compose --help" + self.context = context_type or ContextAPI.get_current_context().context_type or 'moby' + self.source = source + self.status = status.value + self.uri = uri or os.environ.get("METRICS_SOCKET_FILE", METRICS_SOCKET_FILE) + if IS_WINDOWS_PLATFORM: + self.mount("http+unix://", NpipeHTTPAdapter(self.uri)) + else: + self.mount("http+unix://", UnixHTTPAdapter(self.uri)) + + def send_metrics(self): + try: + return self.post("http+unix://localhost/usage", + json=self.to_map(), + timeout=.05, + headers={'Content-Type': 'application/json'}) + except Exception as e: + return e + + def to_map(self): + return { + 'command': self.command, + 'context': self.context, + 'source': self.source, + 'status': self.status, + } diff --git a/compose/metrics/decorator.py b/compose/metrics/decorator.py new file mode 100644 index 00000000000..3126e6941fa --- /dev/null +++ b/compose/metrics/decorator.py @@ -0,0 +1,21 @@ +import functools + +from compose.metrics.client import MetricsCommand +from compose.metrics.client import Status + + +class metrics: + def __init__(self, command_name=None): + self.command_name = command_name + + def __call__(self, fn): + @functools.wraps(fn, + assigned=functools.WRAPPER_ASSIGNMENTS, + updated=functools.WRAPPER_UPDATES) + def wrapper(*args, **kwargs): + if not self.command_name: + self.command_name = fn.__name__ + result = fn(*args, **kwargs) + MetricsCommand(self.command_name, status=Status.SUCCESS).send_metrics() + return result + return wrapper diff --git a/compose/network.py b/compose/network.py new file mode 100644 index 00000000000..a67c703c01f --- /dev/null +++ b/compose/network.py @@ -0,0 +1,332 @@ +import logging +import re +from collections import OrderedDict +from operator import itemgetter + +from docker.errors import NotFound +from docker.types import IPAMConfig +from docker.types import IPAMPool +from docker.utils import version_gte +from docker.utils import version_lt + +from . import __version__ +from .config import ConfigurationError +from .const import LABEL_NETWORK +from .const import LABEL_PROJECT +from .const import LABEL_VERSION + + +log = logging.getLogger(__name__) + +OPTS_EXCEPTIONS = [ + 'com.docker.network.driver.overlay.vxlanid_list', + 'com.docker.network.windowsshim.hnsid', + 'com.docker.network.windowsshim.networkname' +] + + +class Network: + def __init__(self, client, project, name, driver=None, driver_opts=None, + ipam=None, external=False, internal=False, enable_ipv6=False, + labels=None, custom_name=False): + self.client = client + self.project = project + self.name = name + self.driver = driver + self.driver_opts = driver_opts + self.ipam = create_ipam_config_from_dict(ipam) + self.external = external + self.internal = internal + self.enable_ipv6 = enable_ipv6 + self.labels = labels + self.custom_name = custom_name + self.legacy = None + + def ensure(self): + if self.external: + if self.driver == 'overlay': + # Swarm nodes do not register overlay networks that were + # created on a different node unless they're in use. + # See docker/compose#4399 + return + try: + self.inspect() + log.debug( + 'Network {} declared as external. No new ' + 'network will be created.'.format(self.name) + ) + except NotFound: + raise ConfigurationError( + 'Network {name} declared as external, but could' + ' not be found. Please create the network manually' + ' using `{command} {name}` and try again.'.format( + name=self.full_name, + command='docker network create' + ) + ) + return + + self._set_legacy_flag() + try: + data = self.inspect(legacy=self.legacy) + check_remote_network_config(data, self) + except NotFound: + driver_name = 'the default driver' + if self.driver: + driver_name = 'driver "{}"'.format(self.driver) + + log.info( + 'Creating network "{}" with {}'.format(self.full_name, driver_name) + ) + + self.client.create_network( + name=self.full_name, + driver=self.driver, + options=self.driver_opts, + ipam=self.ipam, + internal=self.internal, + enable_ipv6=self.enable_ipv6, + labels=self._labels, + attachable=version_gte(self.client._version, '1.24') or None, + check_duplicate=True, + ) + + def remove(self): + if self.external: + log.info("Network %s is external, skipping", self.true_name) + return + + log.info("Removing network {}".format(self.true_name)) + self.client.remove_network(self.true_name) + + def inspect(self, legacy=False): + if legacy: + return self.client.inspect_network(self.legacy_full_name) + return self.client.inspect_network(self.full_name) + + @property + def legacy_full_name(self): + if self.custom_name: + return self.name + return '{}_{}'.format( + re.sub(r'[_-]', '', self.project), self.name + ) + + @property + def full_name(self): + if self.custom_name: + return self.name + return '{}_{}'.format(self.project, self.name) + + @property + def true_name(self): + self._set_legacy_flag() + if self.legacy: + return self.legacy_full_name + return self.full_name + + @property + def _labels(self): + if version_lt(self.client._version, '1.23'): + return None + labels = self.labels.copy() if self.labels else {} + labels.update({ + LABEL_PROJECT: self.project, + LABEL_NETWORK: self.name, + LABEL_VERSION: __version__, + }) + return labels + + def _set_legacy_flag(self): + if self.legacy is not None: + return + try: + data = self.inspect(legacy=True) + self.legacy = data is not None + except NotFound: + self.legacy = False + + +def create_ipam_config_from_dict(ipam_dict): + if not ipam_dict: + return None + + return IPAMConfig( + driver=ipam_dict.get('driver') or 'default', + pool_configs=[ + IPAMPool( + subnet=config.get('subnet'), + iprange=config.get('ip_range'), + gateway=config.get('gateway'), + aux_addresses=config.get('aux_addresses'), + ) + for config in ipam_dict.get('config', []) + ], + options=ipam_dict.get('options') + ) + + +class NetworkConfigChangedError(ConfigurationError): + def __init__(self, net_name, property_name): + super().__init__( + 'Network "{}" needs to be recreated - {} has changed'.format( + net_name, property_name + ) + ) + + +def check_remote_ipam_config(remote, local): + remote_ipam = remote.get('IPAM') + ipam_dict = create_ipam_config_from_dict(local.ipam) + if local.ipam.get('driver') and local.ipam.get('driver') != remote_ipam.get('Driver'): + raise NetworkConfigChangedError(local.true_name, 'IPAM driver') + if len(ipam_dict['Config']) != 0: + if len(ipam_dict['Config']) != len(remote_ipam['Config']): + raise NetworkConfigChangedError(local.true_name, 'IPAM configs') + remote_configs = sorted(remote_ipam['Config'], key='Subnet') + local_configs = sorted(ipam_dict['Config'], key='Subnet') + while local_configs: + lc = local_configs.pop() + rc = remote_configs.pop() + if lc.get('Subnet') != rc.get('Subnet'): + raise NetworkConfigChangedError(local.true_name, 'IPAM config subnet') + if lc.get('Gateway') is not None and lc.get('Gateway') != rc.get('Gateway'): + raise NetworkConfigChangedError(local.true_name, 'IPAM config gateway') + if lc.get('IPRange') != rc.get('IPRange'): + raise NetworkConfigChangedError(local.true_name, 'IPAM config ip_range') + if sorted(lc.get('AuxiliaryAddresses')) != sorted(rc.get('AuxiliaryAddresses')): + raise NetworkConfigChangedError(local.true_name, 'IPAM config aux_addresses') + + remote_opts = remote_ipam.get('Options') or {} + local_opts = local.ipam.get('Options') or {} + for k in set.union(set(remote_opts.keys()), set(local_opts.keys())): + if remote_opts.get(k) != local_opts.get(k): + raise NetworkConfigChangedError(local.true_name, 'IPAM option "{}"'.format(k)) + + +def check_remote_network_config(remote, local): + if local.driver and remote.get('Driver') != local.driver: + raise NetworkConfigChangedError(local.true_name, 'driver') + local_opts = local.driver_opts or {} + remote_opts = remote.get('Options') or {} + for k in set.union(set(remote_opts.keys()), set(local_opts.keys())): + if k in OPTS_EXCEPTIONS: + continue + if remote_opts.get(k) != local_opts.get(k): + raise NetworkConfigChangedError(local.true_name, 'option "{}"'.format(k)) + + if local.ipam is not None: + check_remote_ipam_config(remote, local) + + if local.internal is not None and local.internal != remote.get('Internal', False): + raise NetworkConfigChangedError(local.true_name, 'internal') + if local.enable_ipv6 is not None and local.enable_ipv6 != remote.get('EnableIPv6', False): + raise NetworkConfigChangedError(local.true_name, 'enable_ipv6') + + local_labels = local.labels or {} + remote_labels = remote.get('Labels') or {} + for k in set.union(set(remote_labels.keys()), set(local_labels.keys())): + if k.startswith('com.docker.'): # We are only interested in user-specified labels + continue + if remote_labels.get(k) != local_labels.get(k): + log.warning( + 'Network {}: label "{}" has changed. It may need to be' + ' recreated.'.format(local.true_name, k) + ) + + +def build_networks(name, config_data, client): + network_config = config_data.networks or {} + networks = { + network_name: Network( + client=client, project=name, + name=data.get('name', network_name), + driver=data.get('driver'), + driver_opts=data.get('driver_opts'), + ipam=data.get('ipam'), + external=bool(data.get('external', False)), + internal=data.get('internal'), + enable_ipv6=data.get('enable_ipv6'), + labels=data.get('labels'), + custom_name=data.get('name') is not None, + ) + for network_name, data in network_config.items() + } + + if 'default' not in networks: + networks['default'] = Network(client, name, 'default') + + return networks + + +class ProjectNetworks: + + def __init__(self, networks, use_networking): + self.networks = networks or {} + self.use_networking = use_networking + + @classmethod + def from_services(cls, services, networks, use_networking): + service_networks = { + network: networks.get(network) + for service in services + for network in get_network_names_for_service(service) + } + unused = set(networks) - set(service_networks) - {'default'} + if unused: + log.warning( + "Some networks were defined but are not used by any service: " + "{}".format(", ".join(unused))) + return cls(service_networks, use_networking) + + def remove(self): + if not self.use_networking: + return + for network in self.networks.values(): + try: + network.remove() + except NotFound: + log.warning("Network %s not found.", network.true_name) + + def initialize(self): + if not self.use_networking: + return + + for network in self.networks.values(): + network.ensure() + + +def get_network_defs_for_service(service_dict): + if 'network_mode' in service_dict: + return {} + networks = service_dict.get('networks', {'default': None}) + return { + net: (config or {}) + for net, config in networks.items() + } + + +def get_network_names_for_service(service_dict): + return get_network_defs_for_service(service_dict).keys() + + +def get_networks(service_dict, network_definitions): + networks = {} + for name, netdef in get_network_defs_for_service(service_dict).items(): + network = network_definitions.get(name) + if network: + networks[network.true_name] = netdef + else: + raise ConfigurationError( + 'Service "{}" uses an undefined network "{}"' + .format(service_dict['name'], name)) + + if any([v.get('priority') for v in networks.values()]): + return OrderedDict(sorted( + networks.items(), + key=lambda t: t[1].get('priority') or 0, reverse=True + )) + else: + # Ensure Compose will pick a consistent primary network if no + # priority is set + return OrderedDict(sorted(networks.items(), key=itemgetter(0))) diff --git a/compose/parallel.py b/compose/parallel.py new file mode 100644 index 00000000000..acf9e4a84cf --- /dev/null +++ b/compose/parallel.py @@ -0,0 +1,349 @@ +import _thread as thread +import logging +import operator +import sys +from queue import Empty +from queue import Queue +from threading import Lock +from threading import Semaphore +from threading import Thread + +from docker.errors import APIError +from docker.errors import ImageNotFound + +from compose.cli.colors import green +from compose.cli.colors import red +from compose.cli.signals import ShutdownException +from compose.const import PARALLEL_LIMIT +from compose.errors import HealthCheckFailed +from compose.errors import NoHealthCheckConfigured +from compose.errors import OperationFailedError + + +log = logging.getLogger(__name__) + +STOP = object() + + +class GlobalLimit: + """Simple class to hold a global semaphore limiter for a project. This class + should be treated as a singleton that is instantiated when the project is. + """ + + global_limiter = Semaphore(PARALLEL_LIMIT) + + @classmethod + def set_global_limit(cls, value): + if value is None: + value = PARALLEL_LIMIT + cls.global_limiter = Semaphore(value) + + +def parallel_execute_watch(events, writer, errors, results, msg, get_name, fail_check): + """ Watch events from a parallel execution, update status and fill errors and results. + Returns exception to re-raise. + """ + error_to_reraise = None + for obj, result, exception in events: + if exception is None: + if fail_check is not None and fail_check(obj): + writer.write(msg, get_name(obj), 'failed', red) + else: + writer.write(msg, get_name(obj), 'done', green) + results.append(result) + elif isinstance(exception, ImageNotFound): + # This is to bubble up ImageNotFound exceptions to the client so we + # can prompt the user if they want to rebuild. + errors[get_name(obj)] = exception.explanation + writer.write(msg, get_name(obj), 'error', red) + error_to_reraise = exception + elif isinstance(exception, APIError): + errors[get_name(obj)] = exception.explanation + writer.write(msg, get_name(obj), 'error', red) + elif isinstance(exception, (OperationFailedError, HealthCheckFailed, NoHealthCheckConfigured)): + errors[get_name(obj)] = exception.msg + writer.write(msg, get_name(obj), 'error', red) + elif isinstance(exception, UpstreamError): + writer.write(msg, get_name(obj), 'error', red) + else: + errors[get_name(obj)] = exception + error_to_reraise = exception + return error_to_reraise + + +def parallel_execute(objects, func, get_name, msg, get_deps=None, limit=None, fail_check=None): + """Runs func on objects in parallel while ensuring that func is + ran on object only after it is ran on all its dependencies. + + get_deps called on object must return a collection with its dependencies. + get_name called on object must return its name. + fail_check is an additional failure check for cases that should display as a failure + in the CLI logs, but don't raise an exception (such as attempting to start 0 containers) + """ + objects = list(objects) + stream = sys.stderr + + if ParallelStreamWriter.instance: + writer = ParallelStreamWriter.instance + else: + writer = ParallelStreamWriter(stream) + + for obj in objects: + writer.add_object(msg, get_name(obj)) + for obj in objects: + writer.write_initial(msg, get_name(obj)) + + events = parallel_execute_iter(objects, func, get_deps, limit) + + errors = {} + results = [] + error_to_reraise = parallel_execute_watch( + events, writer, errors, results, msg, get_name, fail_check + ) + + for obj_name, error in errors.items(): + stream.write("\nERROR: for {} {}\n".format(obj_name, error)) + + if error_to_reraise: + raise error_to_reraise + + return results, errors + + +def _no_deps(x): + return [] + + +class State: + """ + Holds the state of a partially-complete parallel operation. + + state.started: objects being processed + state.finished: objects which have been processed + state.failed: objects which either failed or whose dependencies failed + """ + def __init__(self, objects): + self.objects = objects + + self.started = set() + self.finished = set() + self.failed = set() + + def is_done(self): + return len(self.finished) + len(self.failed) >= len(self.objects) + + def pending(self): + return set(self.objects) - self.started - self.finished - self.failed + + +class NoLimit: + def __enter__(self): + pass + + def __exit__(self, *ex): + pass + + +def parallel_execute_iter(objects, func, get_deps, limit): + """ + Runs func on objects in parallel while ensuring that func is + ran on object only after it is ran on all its dependencies. + + Returns an iterator of tuples which look like: + + # if func returned normally when run on object + (object, result, None) + + # if func raised an exception when run on object + (object, None, exception) + + # if func raised an exception when run on one of object's dependencies + (object, None, UpstreamError()) + """ + if get_deps is None: + get_deps = _no_deps + + if limit is None: + limiter = NoLimit() + else: + limiter = Semaphore(limit) + + results = Queue() + state = State(objects) + + while True: + feed_queue(objects, func, get_deps, results, state, limiter) + + try: + event = results.get(timeout=0.1) + except Empty: + continue + # See https://github.com/docker/compose/issues/189 + except thread.error: + raise ShutdownException() + + if event is STOP: + break + + obj, _, exception = event + if exception is None: + log.debug('Finished processing: {}'.format(obj)) + state.finished.add(obj) + else: + log.debug('Failed: {}'.format(obj)) + state.failed.add(obj) + + yield event + + +def producer(obj, func, results, limiter): + """ + The entry point for a producer thread which runs func on a single object. + Places a tuple on the results queue once func has either returned or raised. + """ + with limiter, GlobalLimit.global_limiter: + try: + result = func(obj) + results.put((obj, result, None)) + except Exception as e: + results.put((obj, None, e)) + + +def feed_queue(objects, func, get_deps, results, state, limiter): + """ + Starts producer threads for any objects which are ready to be processed + (i.e. they have no dependencies which haven't been successfully processed). + + Shortcuts any objects whose dependencies have failed and places an + (object, None, UpstreamError()) tuple on the results queue. + """ + pending = state.pending() + log.debug('Pending: {}'.format(pending)) + + for obj in pending: + deps = get_deps(obj) + try: + if any(dep[0] in state.failed for dep in deps): + log.debug('{} has upstream errors - not processing'.format(obj)) + results.put((obj, None, UpstreamError())) + state.failed.add(obj) + elif all( + dep not in objects or ( + dep in state.finished and (not ready_check or ready_check(dep)) + ) for dep, ready_check in deps + ): + log.debug('Starting producer thread for {}'.format(obj)) + t = Thread(target=producer, args=(obj, func, results, limiter)) + t.daemon = True + t.start() + state.started.add(obj) + except (HealthCheckFailed, NoHealthCheckConfigured) as e: + log.debug( + 'Healthcheck for service(s) upstream of {} failed - ' + 'not processing'.format(obj) + ) + results.put((obj, None, e)) + + if state.is_done(): + results.put(STOP) + + +class UpstreamError(Exception): + pass + + +class ParallelStreamWriter: + """Write out messages for operations happening in parallel. + + Each operation has its own line, and ANSI code characters are used + to jump to the correct line, and write over the line. + """ + + noansi = False + lock = Lock() + instance = None + + @classmethod + def set_noansi(cls, value=True): + cls.noansi = value + + def __init__(self, stream): + self.stream = stream + self.lines = [] + self.width = 0 + ParallelStreamWriter.instance = self + + def add_object(self, msg, obj_index): + if msg is None: + return + self.lines.append(msg + obj_index) + self.width = max(self.width, len(msg + ' ' + obj_index)) + + def write_initial(self, msg, obj_index): + if msg is None: + return + return self._write_noansi(msg, obj_index, '') + + def _write_ansi(self, msg, obj_index, status): + self.lock.acquire() + position = self.lines.index(msg + obj_index) + diff = len(self.lines) - position + # move up + self.stream.write("%c[%dA" % (27, diff)) + # erase + self.stream.write("%c[2K\r" % 27) + self.stream.write("{:<{width}} ... {}\r".format(msg + ' ' + obj_index, + status, width=self.width)) + # move back down + self.stream.write("%c[%dB" % (27, diff)) + self.stream.flush() + self.lock.release() + + def _write_noansi(self, msg, obj_index, status): + self.stream.write( + "{:<{width}} ... {}\r\n".format( + msg + ' ' + obj_index, status, width=self.width + ) + ) + self.stream.flush() + + def write(self, msg, obj_index, status, color_func): + if msg is None: + return + if self.noansi: + self._write_noansi(msg, obj_index, status) + else: + self._write_ansi(msg, obj_index, color_func(status)) + + +def get_stream_writer(): + instance = ParallelStreamWriter.instance + if instance is None: + raise RuntimeError('ParallelStreamWriter has not yet been instantiated') + return instance + + +def parallel_operation(containers, operation, options, message): + parallel_execute( + containers, + operator.methodcaller(operation, **options), + operator.attrgetter('name'), + message, + ) + + +def parallel_remove(containers, options): + stopped_containers = [c for c in containers if not c.is_running] + parallel_operation(stopped_containers, 'remove', options, 'Removing') + + +def parallel_pause(containers, options): + parallel_operation(containers, 'pause', options, 'Pausing') + + +def parallel_unpause(containers, options): + parallel_operation(containers, 'unpause', options, 'Unpausing') + + +def parallel_kill(containers, options): + parallel_operation(containers, 'kill', options, 'Killing') diff --git a/compose/progress_stream.py b/compose/progress_stream.py new file mode 100644 index 00000000000..3c03cc4b5b9 --- /dev/null +++ b/compose/progress_stream.py @@ -0,0 +1,123 @@ +from compose import utils + + +class StreamOutputError(Exception): + pass + + +def write_to_stream(s, stream): + try: + stream.write(s) + except UnicodeEncodeError: + encoding = getattr(stream, 'encoding', 'ascii') + stream.write(s.encode(encoding, errors='replace').decode(encoding)) + + +def stream_output(output, stream): + is_terminal = hasattr(stream, 'isatty') and stream.isatty() + stream = stream + lines = {} + diff = 0 + + for event in utils.json_stream(output): + yield event + is_progress_event = 'progress' in event or 'progressDetail' in event + + if not is_progress_event: + print_output_event(event, stream, is_terminal) + stream.flush() + continue + + if not is_terminal: + continue + + # if it's a progress event and we have a terminal, then display the progress bars + image_id = event.get('id') + if not image_id: + continue + + if image_id not in lines: + lines[image_id] = len(lines) + write_to_stream("\n", stream) + + diff = len(lines) - lines[image_id] + + # move cursor up `diff` rows + write_to_stream("%c[%dA" % (27, diff), stream) + + print_output_event(event, stream, is_terminal) + + if 'id' in event: + # move cursor back down + write_to_stream("%c[%dB" % (27, diff), stream) + + stream.flush() + + +def print_output_event(event, stream, is_terminal): + if 'errorDetail' in event: + raise StreamOutputError(event['errorDetail']['message']) + + terminator = '' + + if is_terminal and 'stream' not in event: + # erase current line + write_to_stream("%c[2K\r" % 27, stream) + terminator = "\r" + elif 'progressDetail' in event: + return + + if 'time' in event: + write_to_stream("[%s] " % event['time'], stream) + + if 'id' in event: + write_to_stream("%s: " % event['id'], stream) + + if 'from' in event: + write_to_stream("(from %s) " % event['from'], stream) + + status = event.get('status', '') + + if 'progress' in event: + write_to_stream("{} {}{}".format(status, event['progress'], terminator), stream) + elif 'progressDetail' in event: + detail = event['progressDetail'] + total = detail.get('total') + if 'current' in detail and total: + percentage = float(detail['current']) / float(total) * 100 + write_to_stream('{} ({:.1f}%){}'.format(status, percentage, terminator), stream) + else: + write_to_stream('{}{}'.format(status, terminator), stream) + elif 'stream' in event: + write_to_stream("{}{}".format(event['stream'], terminator), stream) + else: + write_to_stream("{}{}\n".format(status, terminator), stream) + + +def get_digest_from_pull(events): + digest = None + for event in events: + status = event.get('status') + if not status or 'Digest' not in status: + continue + else: + digest = status.split(':', 1)[1].strip() + return digest + + +def get_digest_from_push(events): + for event in events: + digest = event.get('aux', {}).get('Digest') + if digest: + return digest + return None + + +def read_status(event): + status = event['status'].lower() + if 'progressDetail' in event: + detail = event['progressDetail'] + if 'current' in detail and 'total' in detail: + percentage = float(detail['current']) / float(detail['total']) + status = '{} ({:.1%})'.format(status, percentage) + return status diff --git a/compose/project.py b/compose/project.py new file mode 100644 index 00000000000..336aaa3fcac --- /dev/null +++ b/compose/project.py @@ -0,0 +1,1168 @@ +import datetime +import enum +import logging +import operator +import re +from functools import reduce +from os import path + +from docker.errors import APIError +from docker.errors import ImageNotFound +from docker.errors import NotFound +from docker.utils import version_lt + +from . import parallel +from .cli.errors import UserError +from .config import ConfigurationError +from .config.config import V1 +from .config.sort_services import get_container_name_from_network_mode +from .config.sort_services import get_service_name_from_network_mode +from .const import LABEL_ONE_OFF +from .const import LABEL_PROJECT +from .const import LABEL_SERVICE +from .container import Container +from .network import build_networks +from .network import get_networks +from .network import ProjectNetworks +from .progress_stream import read_status +from .service import BuildAction +from .service import ContainerIpcMode +from .service import ContainerNetworkMode +from .service import ContainerPidMode +from .service import ConvergenceStrategy +from .service import IpcMode +from .service import NetworkMode +from .service import NoSuchImageError +from .service import parse_repository_tag +from .service import PidMode +from .service import Service +from .service import ServiceIpcMode +from .service import ServiceNetworkMode +from .service import ServicePidMode +from .utils import filter_attached_for_up +from .utils import microseconds_from_time_nano +from .utils import truncate_string +from .volume import ProjectVolumes + +log = logging.getLogger(__name__) + + +@enum.unique +class OneOffFilter(enum.Enum): + include = 0 + exclude = 1 + only = 2 + + @classmethod + def update_labels(cls, value, labels): + if value == cls.only: + labels.append('{}={}'.format(LABEL_ONE_OFF, "True")) + elif value == cls.exclude: + labels.append('{}={}'.format(LABEL_ONE_OFF, "False")) + elif value == cls.include: + pass + else: + raise ValueError("Invalid value for one_off: {}".format(repr(value))) + + +class Project: + """ + A collection of services. + """ + def __init__(self, name, services, client, networks=None, volumes=None, config_version=None, + enabled_profiles=None): + self.name = name + self.services = services + self.client = client + self.volumes = volumes or ProjectVolumes({}) + self.networks = networks or ProjectNetworks({}, False) + self.config_version = config_version + self.enabled_profiles = enabled_profiles or [] + + def labels(self, one_off=OneOffFilter.exclude, legacy=False): + name = self.name + if legacy: + name = re.sub(r'[_-]', '', name) + labels = ['{}={}'.format(LABEL_PROJECT, name)] + + OneOffFilter.update_labels(one_off, labels) + return labels + + @classmethod + def from_config(cls, name, config_data, client, default_platform=None, extra_labels=None, + enabled_profiles=None): + """ + Construct a Project from a config.Config object. + """ + extra_labels = extra_labels or [] + use_networking = (config_data.version and config_data.version != V1) + networks = build_networks(name, config_data, client) + project_networks = ProjectNetworks.from_services( + config_data.services, + networks, + use_networking) + volumes = ProjectVolumes.from_config(name, config_data, client) + project = cls(name, [], client, project_networks, volumes, config_data.version, enabled_profiles) + + for service_dict in config_data.services: + service_dict = dict(service_dict) + if use_networking: + service_networks = get_networks(service_dict, networks) + else: + service_networks = {} + + service_dict.pop('networks', None) + links = project.get_links(service_dict) + ipc_mode = project.get_ipc_mode(service_dict) + network_mode = project.get_network_mode( + service_dict, list(service_networks.keys()) + ) + pid_mode = project.get_pid_mode(service_dict) + volumes_from = get_volumes_from(project, service_dict) + + if config_data.version != V1: + service_dict['volumes'] = [ + volumes.namespace_spec(volume_spec) + for volume_spec in service_dict.get('volumes', []) + ] + + secrets = get_secrets( + service_dict['name'], + service_dict.pop('secrets', None) or [], + config_data.secrets) + + service_dict['scale'] = project.get_service_scale(service_dict) + service_dict['device_requests'] = project.get_device_requests(service_dict) + service_dict = translate_credential_spec_to_security_opt(service_dict) + service_dict, ignored_keys = translate_deploy_keys_to_container_config( + service_dict + ) + if ignored_keys: + log.warning( + 'The following deploy sub-keys are not supported and have' + ' been ignored: {}'.format(', '.join(ignored_keys)) + ) + + project.services.append( + Service( + service_dict.pop('name'), + client=client, + project=name, + use_networking=use_networking, + networks=service_networks, + links=links, + network_mode=network_mode, + volumes_from=volumes_from, + secrets=secrets, + pid_mode=pid_mode, + ipc_mode=ipc_mode, + platform=service_dict.pop('platform', None), + default_platform=default_platform, + extra_labels=extra_labels, + **service_dict) + ) + + return project + + @property + def service_names(self): + return [service.name for service in self.services] + + def get_service(self, name): + """ + Retrieve a service by name. Raises NoSuchService + if the named service does not exist. + """ + for service in self.services: + if service.name == name: + return service + + raise NoSuchService(name) + + def validate_service_names(self, service_names): + """ + Validate that the given list of service names only contains valid + services. Raises NoSuchService if one of the names is invalid. + """ + valid_names = self.service_names + for name in service_names: + if name not in valid_names: + raise NoSuchService(name) + + def get_services(self, service_names=None, include_deps=False, auto_enable_profiles=True): + """ + Returns a list of this project's services filtered + by the provided list of names, or all services if service_names is None + or []. + + If include_deps is specified, returns a list including the dependencies for + service_names, in order of dependency. + + Preserves the original order of self.services where possible, + reordering as needed to resolve dependencies. + + Raises NoSuchService if any of the named services do not exist. + + Raises ConfigurationError if any service depended on is not enabled by active profiles + """ + # create a copy so we can *locally* add auto-enabled profiles later + enabled_profiles = self.enabled_profiles.copy() + + if service_names is None or len(service_names) == 0: + auto_enable_profiles = False + service_names = [ + service.name + for service in self.services + if service.enabled_for_profiles(enabled_profiles) + ] + + unsorted = [self.get_service(name) for name in service_names] + services = [s for s in self.services if s in unsorted] + + if auto_enable_profiles: + # enable profiles of explicitly targeted services + for service in services: + for profile in service.get_profiles(): + if profile not in enabled_profiles: + enabled_profiles.append(profile) + + if include_deps: + services = reduce( + lambda acc, s: self._inject_deps(acc, s, enabled_profiles), + services, + [] + ) + + uniques = [] + [uniques.append(s) for s in services if s not in uniques] + + return uniques + + def get_services_without_duplicate(self, service_names=None, include_deps=False): + services = self.get_services(service_names, include_deps) + for service in services: + service.remove_duplicate_containers() + return services + + def get_links(self, service_dict): + links = [] + if 'links' in service_dict: + for link in service_dict.get('links', []): + if ':' in link: + service_name, link_name = link.split(':', 1) + else: + service_name, link_name = link, None + try: + links.append((self.get_service(service_name), link_name)) + except NoSuchService: + raise ConfigurationError( + 'Service "%s" has a link to service "%s" which does not ' + 'exist.' % (service_dict['name'], service_name)) + del service_dict['links'] + return links + + def get_network_mode(self, service_dict, networks): + network_mode = service_dict.pop('network_mode', None) + if not network_mode: + if self.networks.use_networking: + return NetworkMode(networks[0]) if networks else NetworkMode('none') + return NetworkMode(None) + + service_name = get_service_name_from_network_mode(network_mode) + if service_name: + return ServiceNetworkMode(self.get_service(service_name)) + + container_name = get_container_name_from_network_mode(network_mode) + if container_name: + try: + return ContainerNetworkMode(Container.from_id(self.client, container_name)) + except APIError: + raise ConfigurationError( + "Service '{name}' uses the network stack of container '{dep}' which " + "does not exist.".format(name=service_dict['name'], dep=container_name)) + + return NetworkMode(network_mode) + + def get_pid_mode(self, service_dict): + pid_mode = service_dict.pop('pid', None) + if not pid_mode: + return PidMode(None) + + service_name = get_service_name_from_network_mode(pid_mode) + if service_name: + return ServicePidMode(self.get_service(service_name)) + + container_name = get_container_name_from_network_mode(pid_mode) + if container_name: + try: + return ContainerPidMode(Container.from_id(self.client, container_name)) + except APIError: + raise ConfigurationError( + "Service '{name}' uses the PID namespace of container '{dep}' which " + "does not exist.".format(name=service_dict['name'], dep=container_name) + ) + + return PidMode(pid_mode) + + def get_ipc_mode(self, service_dict): + ipc_mode = service_dict.pop('ipc', None) + if not ipc_mode: + return IpcMode(None) + + service_name = get_service_name_from_network_mode(ipc_mode) + if service_name: + return ServiceIpcMode(self.get_service(service_name)) + + container_name = get_container_name_from_network_mode(ipc_mode) + if container_name: + try: + return ContainerIpcMode(Container.from_id(self.client, container_name)) + except APIError: + raise ConfigurationError( + "Service '{name}' uses the IPC namespace of container '{dep}' which " + "does not exist.".format(name=service_dict['name'], dep=container_name) + ) + + return IpcMode(ipc_mode) + + def get_service_scale(self, service_dict): + # service.scale for v2 and deploy.replicas for v3 + scale = service_dict.get('scale', None) + deploy_dict = service_dict.get('deploy', None) + if not deploy_dict: + return 1 if scale is None else scale + + if deploy_dict.get('mode', 'replicated') != 'replicated': + return 1 if scale is None else scale + + replicas = deploy_dict.get('replicas', None) + if scale is not None and replicas is not None: + raise ConfigurationError( + "Both service.scale and service.deploy.replicas are set." + " Only one of them must be set." + ) + if replicas is not None: + scale = replicas + if scale is None: + return 1 + # deploy may contain placement constraints introduced in v3.8 + max_replicas = deploy_dict.get('placement', {}).get( + 'max_replicas_per_node', + scale) + + scale = min(scale, max_replicas) + if max_replicas < scale: + log.warning("Scale is limited to {} ('max_replicas_per_node' field).".format( + max_replicas)) + return scale + + def get_device_requests(self, service_dict): + deploy_dict = service_dict.get('deploy', None) + if not deploy_dict: + return + + resources = deploy_dict.get('resources', None) + if not resources or not resources.get('reservations', None): + return + devices = resources['reservations'].get('devices') + if not devices: + return + + for dev in devices: + count = dev.get("count", -1) + if not isinstance(count, int): + if count != "all": + raise ConfigurationError( + 'Invalid value "{}" for devices count'.format(dev["count"]), + '(expected integer or "all")') + dev["count"] = -1 + + if 'capabilities' in dev: + dev['capabilities'] = [dev['capabilities']] + return devices + + def start(self, service_names=None, **options): + containers = [] + + def start_service(service): + service_containers = service.start(quiet=True, **options) + containers.extend(service_containers) + + services = self.get_services(service_names) + + def get_deps(service): + return { + (self.get_service(dep), config) + for dep, config in service.get_dependency_configs().items() + } + + parallel.parallel_execute( + services, + start_service, + operator.attrgetter('name'), + 'Starting', + get_deps, + fail_check=lambda obj: not obj.containers(), + ) + + return containers + + def stop(self, service_names=None, one_off=OneOffFilter.exclude, **options): + containers = self.containers(service_names, one_off=one_off) + + def get_deps(container): + # actually returning inversed dependencies + return {(other, None) for other in containers + if container.service in + self.get_service(other.service).get_dependency_names()} + + parallel.parallel_execute( + containers, + self.build_container_operation_with_timeout_func('stop', options), + operator.attrgetter('name'), + 'Stopping', + get_deps, + ) + + def pause(self, service_names=None, **options): + containers = self.containers(service_names) + parallel.parallel_pause(reversed(containers), options) + return containers + + def unpause(self, service_names=None, **options): + containers = self.containers(service_names) + parallel.parallel_unpause(containers, options) + return containers + + def kill(self, service_names=None, **options): + parallel.parallel_kill(self.containers(service_names), options) + + def remove_stopped(self, service_names=None, one_off=OneOffFilter.exclude, **options): + parallel.parallel_remove(self.containers( + service_names, stopped=True, one_off=one_off + ), options) + + def down( + self, + remove_image_type, + include_volumes, + remove_orphans=False, + timeout=None, + ignore_orphans=False): + self.stop(one_off=OneOffFilter.include, timeout=timeout) + if not ignore_orphans: + self.find_orphan_containers(remove_orphans) + self.remove_stopped(v=include_volumes, one_off=OneOffFilter.include) + + self.networks.remove() + + if include_volumes: + self.volumes.remove() + + self.remove_images(remove_image_type) + + def remove_images(self, remove_image_type): + for service in self.services: + service.remove_image(remove_image_type) + + def restart(self, service_names=None, **options): + # filter service_names by enabled profiles + service_names = [s.name for s in self.get_services(service_names)] + containers = self.containers(service_names, stopped=True) + + parallel.parallel_execute( + containers, + self.build_container_operation_with_timeout_func('restart', options), + operator.attrgetter('name'), + 'Restarting', + ) + return containers + + def build(self, service_names=None, no_cache=False, pull=False, force_rm=False, memory=None, + build_args=None, gzip=False, parallel_build=False, rm=True, silent=False, cli=False, + progress=None): + + services = [] + for service in self.get_services(service_names): + if service.can_be_built(): + services.append(service) + elif not silent: + log.info('%s uses an image, skipping' % service.name) + + if cli: + log.info("Building with native build. Learn about native build in Compose here: " + "https://docs.docker.com/go/compose-native-build/") + if parallel_build: + log.warning("Flag '--parallel' is ignored when building with " + "COMPOSE_DOCKER_CLI_BUILD=1") + if gzip: + log.warning("Flag '--compress' is ignored when building with " + "COMPOSE_DOCKER_CLI_BUILD=1") + + def build_service(service): + service.build(no_cache, pull, force_rm, memory, build_args, gzip, rm, silent, cli, progress) + + if parallel_build: + _, errors = parallel.parallel_execute( + services, + build_service, + operator.attrgetter('name'), + 'Building', + limit=5, + ) + if len(errors): + combined_errors = '\n'.join([ + e.decode('utf-8') if isinstance(e, bytes) else e for e in errors.values() + ]) + raise ProjectError(combined_errors) + + else: + for service in services: + build_service(service) + + def create( + self, + service_names=None, + strategy=ConvergenceStrategy.changed, + do_build=BuildAction.none, + ): + services = self.get_services_without_duplicate(service_names, include_deps=True) + + for svc in services: + svc.ensure_image_exists(do_build=do_build) + plans = self._get_convergence_plans(services, strategy) + + for service in services: + service.execute_convergence_plan( + plans[service.name], + detached=True, + start=False) + + def _legacy_event_processor(self, service_names): + # Only for v1 files or when Compose is forced to use an older API version + def build_container_event(event, container): + time = datetime.datetime.fromtimestamp(event['time']) + time = time.replace( + microsecond=microseconds_from_time_nano(event['timeNano']) + ) + return { + 'time': time, + 'type': 'container', + 'action': event['status'], + 'id': container.id, + 'service': container.service, + 'attributes': { + 'name': container.name, + 'image': event['from'], + }, + 'container': container, + } + + service_names = set(service_names or self.service_names) + for event in self.client.events( + filters={'label': self.labels()}, + decode=True + ): + # This is a guard against some events broadcasted by swarm that + # don't have a status field. + # See https://github.com/docker/compose/issues/3316 + if 'status' not in event: + continue + + try: + # this can fail if the container has been removed or if the event + # refers to an image + container = Container.from_id(self.client, event['id']) + except APIError: + continue + if container.service not in service_names: + continue + yield build_container_event(event, container) + + def events(self, service_names=None): + if version_lt(self.client.api_version, '1.22'): + # New, better event API was introduced in 1.22. + return self._legacy_event_processor(service_names) + + def build_container_event(event): + container_attrs = event['Actor']['Attributes'] + time = datetime.datetime.fromtimestamp(event['time']) + time = time.replace( + microsecond=microseconds_from_time_nano(event['timeNano']) + ) + + container = None + try: + container = Container.from_id(self.client, event['id']) + except APIError: + # Container may have been removed (e.g. if this is a destroy event) + pass + + return { + 'time': time, + 'type': 'container', + 'action': event['status'], + 'id': event['Actor']['ID'], + 'service': container_attrs.get(LABEL_SERVICE), + 'attributes': { + k: v for k, v in container_attrs.items() + if not k.startswith('com.docker.compose.') + }, + 'container': container, + } + + def yield_loop(service_names): + for event in self.client.events( + filters={'label': self.labels()}, + decode=True + ): + # TODO: support other event types + if event.get('Type') != 'container': + continue + + try: + if event['Actor']['Attributes'][LABEL_SERVICE] not in service_names: + continue + except KeyError: + continue + yield build_container_event(event) + + return yield_loop(set(service_names) if service_names else self.service_names) + + def up(self, + service_names=None, + start_deps=True, + strategy=ConvergenceStrategy.changed, + do_build=BuildAction.none, + timeout=None, + detached=False, + remove_orphans=False, + ignore_orphans=False, + scale_override=None, + rescale=True, + start=True, + always_recreate_deps=False, + reset_container_image=False, + renew_anonymous_volumes=False, + silent=False, + cli=False, + one_off=False, + attach_dependencies=False, + override_options=None, + ): + + if cli: + log.info("Building with native build. Learn about native build in Compose here: " + "https://docs.docker.com/go/compose-native-build/") + + self.initialize() + if not ignore_orphans: + self.find_orphan_containers(remove_orphans) + + if scale_override is None: + scale_override = {} + + services = self.get_services_without_duplicate( + service_names, + include_deps=start_deps) + + for svc in services: + svc.ensure_image_exists(do_build=do_build, silent=silent, cli=cli) + plans = self._get_convergence_plans( + services, + strategy, + always_recreate_deps=always_recreate_deps, + one_off=service_names if one_off else [], + ) + + services_to_attach = filter_attached_for_up( + services, + service_names, + attach_dependencies, + lambda service: service.name) + + def do(service): + return service.execute_convergence_plan( + plans[service.name], + timeout=timeout, + detached=detached or (service not in services_to_attach), + scale_override=scale_override.get(service.name), + rescale=rescale, + start=start, + reset_container_image=reset_container_image, + renew_anonymous_volumes=renew_anonymous_volumes, + override_options=override_options, + ) + + def get_deps(service): + return { + (self.get_service(dep), config) + for dep, config in service.get_dependency_configs().items() + } + + results, errors = parallel.parallel_execute( + services, + do, + operator.attrgetter('name'), + None, + get_deps, + ) + if errors: + raise ProjectError( + 'Encountered errors while bringing up the project.' + ) + + return [ + container + for svc_containers in results + if svc_containers is not None + for container in svc_containers + ] + + def initialize(self): + self.networks.initialize() + self.volumes.initialize() + + def _get_convergence_plans(self, services, strategy, always_recreate_deps=False, one_off=None): + plans = {} + + for service in services: + updated_dependencies = [ + name + for name in service.get_dependency_names() + if name in plans and + plans[name].action in ('recreate', 'create') + ] + is_one_off = one_off and service.name in one_off + + if updated_dependencies and strategy.allows_recreate: + log.debug('%s has upstream changes (%s)', + service.name, + ", ".join(updated_dependencies)) + containers_stopped = any( + service.containers(stopped=True, filters={'status': ['created', 'exited']})) + service_has_links = any(service.get_link_names()) + container_has_links = any(c.get('HostConfig.Links') for c in service.containers()) + should_recreate_for_links = service_has_links ^ container_has_links + if always_recreate_deps or containers_stopped or should_recreate_for_links: + plan = service.convergence_plan(ConvergenceStrategy.always, is_one_off) + else: + plan = service.convergence_plan(strategy, is_one_off) + else: + plan = service.convergence_plan(strategy, is_one_off) + + plans[service.name] = plan + + return plans + + def pull(self, service_names=None, ignore_pull_failures=False, parallel_pull=True, silent=False, + include_deps=False): + services = self.get_services(service_names, include_deps) + + if parallel_pull: + self.parallel_pull(services, silent=silent) + + else: + must_build = [] + for service in services: + try: + service.pull(ignore_pull_failures, silent=silent) + except (ImageNotFound, NotFound): + if service.can_be_built(): + must_build.append(service.name) + else: + raise + + if len(must_build): + log.warning('Some service image(s) must be built from source by running:\n' + ' docker-compose build {}' + .format(' '.join(must_build))) + + def parallel_pull(self, services, ignore_pull_failures=False, silent=False): + msg = 'Pulling' if not silent else None + must_build = [] + + def pull_service(service): + strm = service.pull(ignore_pull_failures, True, stream=True) + + if strm is None: # Attempting to pull service with no `image` key is a no-op + return + + try: + writer = parallel.get_stream_writer() + for event in strm: + if 'status' not in event: + continue + status = read_status(event) + writer.write( + msg, service.name, truncate_string(status), lambda s: s + ) + except (ImageNotFound, NotFound): + if service.can_be_built(): + must_build.append(service.name) + else: + raise + + _, errors = parallel.parallel_execute( + services, + pull_service, + operator.attrgetter('name'), + msg, + limit=5, + ) + + if len(must_build): + log.warning('Some service image(s) must be built from source by running:\n' + ' docker-compose build {}' + .format(' '.join(must_build))) + if len(errors): + combined_errors = '\n'.join([ + e.decode('utf-8') if isinstance(e, bytes) else e for e in errors.values() + ]) + raise ProjectError(combined_errors) + + def push(self, service_names=None, ignore_push_failures=False): + unique_images = set() + for service in self.get_services(service_names, include_deps=False): + # Considering <image> and <image:latest> as the same + repo, tag, sep = parse_repository_tag(service.image_name) + service_image_name = sep.join((repo, tag)) if tag else sep.join((repo, 'latest')) + + if service_image_name not in unique_images: + service.push(ignore_push_failures) + unique_images.add(service_image_name) + + def _labeled_containers(self, stopped=False, one_off=OneOffFilter.exclude): + ctnrs = list(filter(None, [ + Container.from_ps(self.client, container) + for container in self.client.containers( + all=stopped, + filters={'label': self.labels(one_off=one_off)})]) + ) + if ctnrs: + return ctnrs + + return list(filter(lambda c: c.has_legacy_proj_name(self.name), filter(None, [ + Container.from_ps(self.client, container) + for container in self.client.containers( + all=stopped, + filters={'label': self.labels(one_off=one_off, legacy=True)})]) + )) + + def containers(self, service_names=None, stopped=False, one_off=OneOffFilter.exclude): + if service_names: + self.validate_service_names(service_names) + else: + service_names = self.service_names + + containers = self._labeled_containers(stopped, one_off) + + def matches_service_names(container): + return container.labels.get(LABEL_SERVICE) in service_names + + return [c for c in containers if matches_service_names(c)] + + def find_orphan_containers(self, remove_orphans): + def _find(): + containers = set(self._labeled_containers() + self._labeled_containers(stopped=True)) + for ctnr in containers: + service_name = ctnr.labels.get(LABEL_SERVICE) + if service_name not in self.service_names: + yield ctnr + orphans = list(_find()) + if not orphans: + return + if remove_orphans: + for ctnr in orphans: + log.info('Removing orphan container "{}"'.format(ctnr.name)) + try: + ctnr.kill() + except APIError: + pass + ctnr.remove(force=True) + else: + log.warning( + 'Found orphan containers ({}) for this project. If ' + 'you removed or renamed this service in your compose ' + 'file, you can run this command with the ' + '--remove-orphans flag to clean it up.'.format( + ', '.join(["{}".format(ctnr.name) for ctnr in orphans]) + ) + ) + + def _inject_deps(self, acc, service, enabled_profiles): + dep_names = service.get_dependency_names() + + if len(dep_names) > 0: + dep_services = self.get_services( + service_names=list(set(dep_names)), + include_deps=True, + auto_enable_profiles=False + ) + + for dep in dep_services: + if not dep.enabled_for_profiles(enabled_profiles): + raise ConfigurationError( + 'Service "{dep_name}" was pulled in as a dependency of ' + 'service "{service_name}" but is not enabled by the ' + 'active profiles. ' + 'You may fix this by adding a common profile to ' + '"{dep_name}" and "{service_name}".' + .format(dep_name=dep.name, service_name=service.name) + ) + else: + dep_services = [] + + dep_services.append(service) + return acc + dep_services + + def build_container_operation_with_timeout_func(self, operation, options): + def container_operation_with_timeout(container): + _options = options.copy() + if _options.get('timeout') is None: + service = self.get_service(container.service) + _options['timeout'] = service.stop_timeout(None) + return getattr(container, operation)(**_options) + return container_operation_with_timeout + + +def translate_credential_spec_to_security_opt(service_dict): + result = [] + + if 'credential_spec' in service_dict: + spec = convert_credential_spec_to_security_opt(service_dict['credential_spec']) + result.append('credentialspec={spec}'.format(spec=spec)) + + if result: + service_dict['security_opt'] = result + + return service_dict + + +def translate_resource_keys_to_container_config(resources_dict, service_dict): + if 'limits' in resources_dict: + service_dict['mem_limit'] = resources_dict['limits'].get('memory') + if 'cpus' in resources_dict['limits']: + service_dict['cpus'] = float(resources_dict['limits']['cpus']) + if 'reservations' in resources_dict: + service_dict['mem_reservation'] = resources_dict['reservations'].get('memory') + if 'cpus' in resources_dict['reservations']: + return ['resources.reservations.cpus'] + return [] + + +def convert_restart_policy(name): + try: + return { + 'any': 'always', + 'none': 'no', + 'on-failure': 'on-failure' + }[name] + except KeyError: + raise ConfigurationError('Invalid restart policy "{}"'.format(name)) + + +def convert_credential_spec_to_security_opt(credential_spec): + if 'file' in credential_spec: + return 'file://{file}'.format(file=credential_spec['file']) + return 'registry://{registry}'.format(registry=credential_spec['registry']) + + +def translate_deploy_keys_to_container_config(service_dict): + if 'credential_spec' in service_dict: + del service_dict['credential_spec'] + if 'configs' in service_dict: + del service_dict['configs'] + + if 'deploy' not in service_dict: + return service_dict, [] + + deploy_dict = service_dict['deploy'] + ignored_keys = [ + k for k in ['endpoint_mode', 'labels', 'update_config', 'rollback_config'] + if k in deploy_dict + ] + + if 'restart_policy' in deploy_dict: + service_dict['restart'] = { + 'Name': convert_restart_policy(deploy_dict['restart_policy'].get('condition', 'any')), + 'MaximumRetryCount': deploy_dict['restart_policy'].get('max_attempts', 0) + } + for k in deploy_dict['restart_policy'].keys(): + if k != 'condition' and k != 'max_attempts': + ignored_keys.append('restart_policy.{}'.format(k)) + + ignored_keys.extend( + translate_resource_keys_to_container_config( + deploy_dict.get('resources', {}), service_dict + ) + ) + del service_dict['deploy'] + return service_dict, ignored_keys + + +def get_volumes_from(project, service_dict): + volumes_from = service_dict.pop('volumes_from', None) + if not volumes_from: + return [] + + def build_volume_from(spec): + if spec.type == 'service': + try: + return spec._replace(source=project.get_service(spec.source)) + except NoSuchService: + pass + + if spec.type == 'container': + try: + container = Container.from_id(project.client, spec.source) + return spec._replace(source=container) + except APIError: + pass + + raise ConfigurationError( + "Service \"{}\" mounts volumes from \"{}\", which is not the name " + "of a service or container.".format( + service_dict['name'], + spec.source)) + + return [build_volume_from(vf) for vf in volumes_from] + + +def get_secrets(service, service_secrets, secret_defs): + secrets = [] + + for secret in service_secrets: + secret_def = secret_defs.get(secret.source) + if not secret_def: + raise ConfigurationError( + "Service \"{service}\" uses an undefined secret \"{secret}\" " + .format(service=service, secret=secret.source)) + + if secret_def.get('external'): + log.warning('Service "{service}" uses secret "{secret}" which is external. ' + 'External secrets are not available to containers created by ' + 'docker-compose.'.format(service=service, secret=secret.source)) + continue + + if secret.uid or secret.gid or secret.mode: + log.warning( + 'Service "{service}" uses secret "{secret}" with uid, ' + 'gid, or mode. These fields are not supported by this ' + 'implementation of the Compose file'.format( + service=service, secret=secret.source + ) + ) + + secret_file = secret_def.get('file') + if not path.isfile(str(secret_file)): + log.warning( + 'Service "{service}" uses an undefined secret file "{secret_file}", ' + 'the following file should be created "{secret_file}"'.format( + service=service, secret_file=secret_file + ) + ) + secrets.append({'secret': secret, 'file': secret_file}) + + return secrets + + +def get_image_digests(project): + digests = {} + needs_push = set() + needs_pull = set() + + for service in project.services: + try: + digests[service.name] = get_image_digest(service) + except NeedsPush as e: + needs_push.add(e.image_name) + except NeedsPull as e: + needs_pull.add(e.service_name) + + if needs_push or needs_pull: + raise MissingDigests(needs_push, needs_pull) + + return digests + + +def get_image_digest(service): + if 'image' not in service.options: + raise UserError( + "Service '{s.name}' doesn't define an image tag. An image name is " + "required to generate a proper image digest. Specify an image repo " + "and tag with the 'image' option.".format(s=service)) + + _, _, separator = parse_repository_tag(service.options['image']) + # Compose file already uses a digest, no lookup required + if separator == '@': + return service.options['image'] + + digest = get_digest(service) + + if digest: + return digest + + if 'build' not in service.options: + raise NeedsPull(service.image_name, service.name) + + raise NeedsPush(service.image_name) + + +def get_digest(service): + digest = None + try: + image = service.image() + # TODO: pick a digest based on the image tag if there are multiple + # digests + if image['RepoDigests']: + digest = image['RepoDigests'][0] + except NoSuchImageError: + try: + # Fetch the image digest from the registry + distribution = service.get_image_registry_data() + + if distribution['Descriptor']['digest']: + digest = '{image_name}@{digest}'.format( + image_name=service.image_name, + digest=distribution['Descriptor']['digest'] + ) + except NoSuchImageError: + raise UserError( + "Digest not found for service '{service}'. " + "Repository does not exist or may require 'docker login'" + .format(service=service.name)) + return digest + + +class MissingDigests(Exception): + def __init__(self, needs_push, needs_pull): + self.needs_push = needs_push + self.needs_pull = needs_pull + + +class NeedsPush(Exception): + def __init__(self, image_name): + self.image_name = image_name + + +class NeedsPull(Exception): + def __init__(self, image_name, service_name): + self.image_name = image_name + self.service_name = service_name + + +class NoSuchService(Exception): + def __init__(self, name): + if isinstance(name, bytes): + name = name.decode('utf-8') + self.name = name + self.msg = "No such service: %s" % self.name + + def __str__(self): + return self.msg + + +class ProjectError(Exception): + def __init__(self, msg): + self.msg = msg diff --git a/compose/service.py b/compose/service.py new file mode 100644 index 00000000000..df0d76fb913 --- /dev/null +++ b/compose/service.py @@ -0,0 +1,1930 @@ +import enum +import itertools +import json +import logging +import os +import re +import subprocess +import sys +import tempfile +from collections import namedtuple +from collections import OrderedDict +from operator import attrgetter + +from docker.errors import APIError +from docker.errors import ImageNotFound +from docker.errors import NotFound +from docker.types import LogConfig +from docker.types import Mount +from docker.utils import version_gte +from docker.utils import version_lt +from docker.utils.ports import build_port_bindings +from docker.utils.ports import split_port +from docker.utils.utils import convert_tmpfs_mounts + +from . import __version__ +from . import const +from . import progress_stream +from .config import DOCKER_CONFIG_KEYS +from .config import is_url +from .config import merge_environment +from .config import merge_labels +from .config.errors import DependencyError +from .config.types import MountSpec +from .config.types import ServicePort +from .config.types import VolumeSpec +from .const import DEFAULT_TIMEOUT +from .const import IS_WINDOWS_PLATFORM +from .const import LABEL_CONFIG_HASH +from .const import LABEL_CONTAINER_NUMBER +from .const import LABEL_ONE_OFF +from .const import LABEL_PROJECT +from .const import LABEL_SERVICE +from .const import LABEL_SLUG +from .const import LABEL_VERSION +from .const import NANOCPUS_SCALE +from .const import WINDOWS_LONGPATH_PREFIX +from .container import Container +from .errors import HealthCheckFailed +from .errors import NoHealthCheckConfigured +from .errors import OperationFailedError +from .parallel import parallel_execute +from .progress_stream import stream_output +from .progress_stream import StreamOutputError +from .utils import generate_random_id +from .utils import json_hash +from .utils import parse_bytes +from .utils import parse_seconds_float +from .utils import truncate_id +from .utils import unique_everseen +from compose.cli.utils import binarystr_to_unicode + + +log = logging.getLogger(__name__) + +HOST_CONFIG_KEYS = [ + 'cap_add', + 'cap_drop', + 'cgroup_parent', + 'cpu_count', + 'cpu_percent', + 'cpu_period', + 'cpu_quota', + 'cpu_rt_period', + 'cpu_rt_runtime', + 'cpu_shares', + 'cpus', + 'cpuset', + 'device_cgroup_rules', + 'devices', + 'device_requests', + 'dns', + 'dns_search', + 'dns_opt', + 'env_file', + 'extra_hosts', + 'group_add', + 'init', + 'ipc', + 'isolation', + 'read_only', + 'log_driver', + 'log_opt', + 'mem_limit', + 'mem_reservation', + 'memswap_limit', + 'mem_swappiness', + 'oom_kill_disable', + 'oom_score_adj', + 'pid', + 'pids_limit', + 'privileged', + 'restart', + 'runtime', + 'security_opt', + 'shm_size', + 'storage_opt', + 'sysctls', + 'userns_mode', + 'volumes_from', + 'volume_driver', +] + +CONDITION_STARTED = 'service_started' +CONDITION_HEALTHY = 'service_healthy' + + +class BuildError(Exception): + def __init__(self, service, reason): + self.service = service + self.reason = reason + + +class NeedsBuildError(Exception): + def __init__(self, service): + self.service = service + + +class NoSuchImageError(Exception): + pass + + +ServiceName = namedtuple('ServiceName', 'project service number') + +ConvergencePlan = namedtuple('ConvergencePlan', 'action containers') + + +@enum.unique +class ConvergenceStrategy(enum.Enum): + """Enumeration for all possible convergence strategies. Values refer to + when containers should be recreated. + """ + changed = 1 + always = 2 + never = 3 + + @property + def allows_recreate(self): + return self is not type(self).never + + +@enum.unique +class ImageType(enum.Enum): + """Enumeration for the types of images known to compose.""" + none = 0 + local = 1 + all = 2 + + +@enum.unique +class BuildAction(enum.Enum): + """Enumeration for the possible build actions.""" + none = 0 + force = 1 + skip = 2 + + +class Service: + def __init__( + self, + name, + client=None, + project='default', + use_networking=False, + links=None, + volumes_from=None, + network_mode=None, + networks=None, + secrets=None, + scale=1, + ipc_mode=None, + pid_mode=None, + default_platform=None, + extra_labels=None, + **options + ): + self.name = name + self.client = client + self.project = project + self.use_networking = use_networking + self.links = links or [] + self.volumes_from = volumes_from or [] + self.ipc_mode = ipc_mode or IpcMode(None) + self.network_mode = network_mode or NetworkMode(None) + self.pid_mode = pid_mode or PidMode(None) + self.networks = networks or {} + self.secrets = secrets or [] + self.scale_num = scale + self.default_platform = default_platform + self.options = options + self.extra_labels = extra_labels or [] + + def __repr__(self): + return '<Service: {}>'.format(self.name) + + def containers(self, stopped=False, one_off=False, filters=None, labels=None): + if filters is None: + filters = {} + filters.update({'label': self.labels(one_off=one_off) + (labels or [])}) + + result = list(filter(None, [ + Container.from_ps(self.client, container) + for container in self.client.containers( + all=stopped, + filters=filters)]) + ) + if result: + return result + + filters.update({'label': self.labels(one_off=one_off, legacy=True) + (labels or [])}) + return list( + filter( + lambda c: c.has_legacy_proj_name(self.project), filter(None, [ + Container.from_ps(self.client, container) + for container in self.client.containers( + all=stopped, + filters=filters)]) + ) + ) + + def get_container(self, number=1): + """Return a :class:`compose.container.Container` for this service. The + container must be active, and match `number`. + """ + for container in self.containers(labels=['{}={}'.format(LABEL_CONTAINER_NUMBER, number)]): + return container + + raise ValueError("No container found for {}_{}".format(self.name, number)) + + def start(self, **options): + containers = self.containers(stopped=True) + for c in containers: + self.start_container_if_stopped(c, **options) + return containers + + def show_scale_warnings(self, desired_num): + if self.custom_container_name and desired_num > 1: + log.warning('The "%s" service is using the custom container name "%s". ' + 'Docker requires each container to have a unique name. ' + 'Remove the custom name to scale the service.' + % (self.name, self.custom_container_name)) + + if self.specifies_host_port() and desired_num > 1: + log.warning('The "%s" service specifies a port on the host. If multiple containers ' + 'for this service are created on a single host, the port will clash.' + % self.name) + + def scale(self, desired_num, timeout=None): + """ + Adjusts the number of containers to the specified number and ensures + they are running. + + - creates containers until there are at least `desired_num` + - stops containers until there are at most `desired_num` running + - starts containers until there are at least `desired_num` running + - removes all stopped containers + """ + + self.show_scale_warnings(desired_num) + + running_containers = self.containers(stopped=False) + num_running = len(running_containers) + for c in running_containers: + if not c.has_legacy_proj_name(self.project): + continue + log.info('Recreating container with legacy name %s' % c.name) + self.recreate_container(c, timeout, start_new_container=False) + + if desired_num == num_running: + # do nothing as we already have the desired number + log.info('Desired container number already achieved') + return + + if desired_num > num_running: + all_containers = self.containers(stopped=True) + + if num_running != len(all_containers): + # we have some stopped containers, check for divergences + stopped_containers = [ + c for c in all_containers if not c.is_running + ] + + # Remove containers that have diverged + divergent_containers = [ + c for c in stopped_containers if self._containers_have_diverged([c]) + ] + for c in divergent_containers: + c.remove() + + all_containers = list(set(all_containers) - set(divergent_containers)) + + sorted_containers = sorted(all_containers, key=attrgetter('number')) + self._execute_convergence_start( + sorted_containers, desired_num, timeout, True, True + ) + + if desired_num < num_running: + num_to_stop = num_running - desired_num + + sorted_running_containers = sorted( + running_containers, + key=attrgetter('number')) + + self._downscale(sorted_running_containers[-num_to_stop:], timeout) + + def create_container(self, + one_off=False, + previous_container=None, + number=None, + quiet=False, + **override_options): + """ + Create a container for this service. If the image doesn't exist, attempt to pull + it. + """ + # This is only necessary for `scale` and `volumes_from` + # auto-creating containers to satisfy the dependency. + self.ensure_image_exists() + + container_options = self._get_container_create_options( + override_options, + number or self._next_container_number(one_off=one_off), + one_off=one_off, + previous_container=previous_container, + ) + + if 'name' in container_options and not quiet: + log.info("Creating %s" % container_options['name']) + + try: + return Container.create(self.client, **container_options) + except APIError as ex: + raise OperationFailedError("Cannot create container for service %s: %s" % + (self.name, binarystr_to_unicode(ex.explanation))) + + def ensure_image_exists(self, do_build=BuildAction.none, silent=False, cli=False): + if self.can_be_built() and do_build == BuildAction.force: + self.build(cli=cli) + return + + try: + self.image() + return + except NoSuchImageError: + pass + + if not self.can_be_built(): + self.pull(silent=silent) + return + + if do_build == BuildAction.skip: + raise NeedsBuildError(self) + + self.build(cli=cli) + log.warning( + "Image for service {} was built because it did not already exist. To " + "rebuild this image you must use `docker-compose build` or " + "`docker-compose up --build`.".format(self.name)) + + def get_image_registry_data(self): + try: + return self.client.inspect_distribution(self.image_name) + except APIError: + raise NoSuchImageError("Image '{}' not found".format(self.image_name)) + + def image(self): + try: + return self.client.inspect_image(self.image_name) + except ImageNotFound: + raise NoSuchImageError("Image '{}' not found".format(self.image_name)) + + @property + def image_name(self): + return self.options.get('image', '{project}_{s.name}'.format( + s=self, project=self.project.lstrip('_-') + )) + + @property + def platform(self): + platform = self.options.get('platform') + if not platform and version_gte(self.client.api_version, '1.35'): + platform = self.default_platform + return platform + + def convergence_plan(self, strategy=ConvergenceStrategy.changed, one_off=False): + containers = self.containers(stopped=True) + + if one_off: + return ConvergencePlan('one_off', []) + + if not containers: + return ConvergencePlan('create', []) + + if strategy is ConvergenceStrategy.never: + return ConvergencePlan('start', containers) + + if ( + strategy is ConvergenceStrategy.always or + self._containers_have_diverged(containers) + ): + return ConvergencePlan('recreate', containers) + + stopped = [c for c in containers if not c.is_running] + + if stopped: + return ConvergencePlan('start', containers) + + return ConvergencePlan('noop', containers) + + def _containers_have_diverged(self, containers): + config_hash = None + + try: + config_hash = self.config_hash + except NoSuchImageError as e: + log.debug( + 'Service %s has diverged: %s', + self.name, str(e), + ) + return True + + has_diverged = False + + for c in containers: + if c.has_legacy_proj_name(self.project): + log.debug('%s has diverged: Legacy project name' % c.name) + has_diverged = True + continue + container_config_hash = c.labels.get(LABEL_CONFIG_HASH, None) + if container_config_hash != config_hash: + log.debug( + '%s has diverged: %s != %s', + c.name, container_config_hash, config_hash, + ) + has_diverged = True + + return has_diverged + + def _execute_convergence_create(self, scale, detached, start, one_off=False, override_options=None): + + i = self._next_container_number() + + def create_and_start(service, n): + if one_off: + container = service.create_container(one_off=True, quiet=True, **override_options) + else: + container = service.create_container(number=n, quiet=True) + if not detached: + container.attach_log_stream() + if start and not one_off: + self.start_container(container) + return container + + def get_name(service_name): + if one_off: + return "_".join([ + service_name.project, + service_name.service, + "run", + ]) + return self.get_container_name(service_name.service, service_name.number) + + containers, errors = parallel_execute( + [ + ServiceName(self.project, self.name, index) + for index in range(i, i + scale) + ], + lambda service_name: create_and_start(self, service_name.number), + get_name, + "Creating" + ) + for error in errors.values(): + raise OperationFailedError(error) + + return containers + + def _execute_convergence_recreate(self, containers, scale, timeout, detached, start, + renew_anonymous_volumes): + if scale is not None and len(containers) > scale: + self._downscale(containers[scale:], timeout) + containers = containers[:scale] + + def recreate(container): + return self.recreate_container( + container, timeout=timeout, attach_logs=not detached, + start_new_container=start, renew_anonymous_volumes=renew_anonymous_volumes + ) + + containers, errors = parallel_execute( + containers, + recreate, + lambda c: c.name, + "Recreating", + ) + for error in errors.values(): + raise OperationFailedError(error) + + if scale is not None and len(containers) < scale: + containers.extend(self._execute_convergence_create( + scale - len(containers), detached, start + )) + return containers + + def _execute_convergence_start(self, containers, scale, timeout, detached, start): + if scale is not None and len(containers) > scale: + self._downscale(containers[scale:], timeout) + containers = containers[:scale] + if start: + stopped = [c for c in containers if not c.is_running] + _, errors = parallel_execute( + stopped, + lambda c: self.start_container_if_stopped(c, attach_logs=not detached, quiet=True), + lambda c: c.name, + "Starting", + ) + + for error in errors.values(): + raise OperationFailedError(error) + + if scale is not None and len(containers) < scale: + containers.extend(self._execute_convergence_create( + scale - len(containers), detached, start + )) + return containers + + def _downscale(self, containers, timeout=None): + def stop_and_remove(container): + container.stop(timeout=self.stop_timeout(timeout)) + container.remove() + + parallel_execute( + containers, + stop_and_remove, + lambda c: c.name, + "Stopping and removing", + ) + + def execute_convergence_plan(self, plan, timeout=None, detached=False, + start=True, scale_override=None, + rescale=True, reset_container_image=False, + renew_anonymous_volumes=False, override_options=None): + (action, containers) = plan + scale = scale_override if scale_override is not None else self.scale_num + containers = sorted(containers, key=attrgetter('number')) + + self.show_scale_warnings(scale) + + if action in ['create', 'one_off']: + return self._execute_convergence_create( + scale, + detached, + start, + one_off=(action == 'one_off'), + override_options=override_options + ) + + # The create action needs always needs an initial scale, but otherwise, + # we set scale to none in no-rescale scenarios (`run` dependencies) + if not rescale: + scale = None + + if action == 'recreate': + if reset_container_image: + # Updating the image ID on the container object lets us recover old volumes if + # the new image uses them as well + img_id = self.image()['Id'] + for c in containers: + c.reset_image(img_id) + return self._execute_convergence_recreate( + containers, scale, timeout, detached, start, + renew_anonymous_volumes, + ) + + if action == 'start': + return self._execute_convergence_start( + containers, scale, timeout, detached, start + ) + + if action == 'noop': + if scale != len(containers): + return self._execute_convergence_start( + containers, scale, timeout, detached, start + ) + for c in containers: + log.info("%s is up-to-date" % c.name) + + return containers + + raise Exception("Invalid action: {}".format(action)) + + def recreate_container(self, container, timeout=None, attach_logs=False, start_new_container=True, + renew_anonymous_volumes=False): + """Recreate a container. + + The original container is renamed to a temporary name so that data + volumes can be copied to the new container, before the original + container is removed. + """ + + container.stop(timeout=self.stop_timeout(timeout)) + container.rename_to_tmp_name() + new_container = self.create_container( + previous_container=container if not renew_anonymous_volumes else None, + number=container.number, + quiet=True, + ) + if attach_logs: + new_container.attach_log_stream() + if start_new_container: + self.start_container(new_container) + container.remove() + return new_container + + def stop_timeout(self, timeout): + if timeout is not None: + return timeout + timeout = parse_seconds_float(self.options.get('stop_grace_period')) + if timeout is not None: + return timeout + return DEFAULT_TIMEOUT + + def start_container_if_stopped(self, container, attach_logs=False, quiet=False): + if not container.is_running: + if not quiet: + log.info("Starting %s" % container.name) + if attach_logs: + container.attach_log_stream() + return self.start_container(container) + + def start_container(self, container, use_network_aliases=True): + self.connect_container_to_networks(container, use_network_aliases) + try: + container.start() + except APIError as ex: + expl = binarystr_to_unicode(ex.explanation) + if "driver failed programming external connectivity" in expl: + log.warn("Host is already in use by another container") + raise OperationFailedError("Cannot start service {}: {}".format(self.name, expl)) + return container + + @property + def prioritized_networks(self): + return OrderedDict( + sorted( + self.networks.items(), + key=lambda t: t[1].get('priority') or 0, reverse=True + ) + ) + + def connect_container_to_networks(self, container, use_network_aliases=True): + connected_networks = container.get('NetworkSettings.Networks') + + for network, netdefs in self.prioritized_networks.items(): + if network in connected_networks: + if short_id_alias_exists(container, network): + continue + self.client.disconnect_container_from_network(container.id, network) + + aliases = self._get_aliases(netdefs, container) if use_network_aliases else [] + + self.client.connect_container_to_network( + container.id, network, + aliases=aliases, + ipv4_address=netdefs.get('ipv4_address', None), + ipv6_address=netdefs.get('ipv6_address', None), + links=self._get_links(False), + link_local_ips=netdefs.get('link_local_ips', None), + ) + + def remove_duplicate_containers(self, timeout=None): + for c in self.duplicate_containers(): + log.info('Removing %s' % c.name) + c.stop(timeout=self.stop_timeout(timeout)) + c.remove() + + def duplicate_containers(self): + containers = sorted( + self.containers(stopped=True), + key=lambda c: c.get('Created'), + ) + + numbers = set() + + for c in containers: + if c.number in numbers: + yield c + else: + numbers.add(c.number) + + @property + def config_hash(self): + return json_hash(self.config_dict()) + + def config_dict(self): + def image_id(): + try: + return self.image()['Id'] + except NoSuchImageError: + return None + + return { + 'options': self.options, + 'image_id': image_id(), + 'links': self.get_link_names(), + 'net': self.network_mode.id, + 'networks': self.networks, + 'secrets': self.secrets, + 'volumes_from': [ + (v.source.name, v.mode) + for v in self.volumes_from if isinstance(v.source, Service) + ] + } + + def get_dependency_names(self): + net_name = self.network_mode.service_name + pid_namespace = self.pid_mode.service_name + ipc_namespace = self.ipc_mode.service_name + return ( + self.get_linked_service_names() + + self.get_volumes_from_names() + + ([net_name] if net_name else []) + + ([pid_namespace] if pid_namespace else []) + + ([ipc_namespace] if ipc_namespace else []) + + list(self.options.get('depends_on', {}).keys()) + ) + + def get_dependency_configs(self): + net_name = self.network_mode.service_name + pid_namespace = self.pid_mode.service_name + ipc_namespace = self.ipc_mode.service_name + + configs = { + name: None for name in self.get_linked_service_names() + } + configs.update( + (name, None) for name in self.get_volumes_from_names() + ) + configs.update({net_name: None} if net_name else {}) + configs.update({pid_namespace: None} if pid_namespace else {}) + configs.update({ipc_namespace: None} if ipc_namespace else {}) + configs.update(self.options.get('depends_on', {})) + for svc, config in self.options.get('depends_on', {}).items(): + if config['condition'] == CONDITION_STARTED: + configs[svc] = lambda s: True + elif config['condition'] == CONDITION_HEALTHY: + configs[svc] = lambda s: s.is_healthy() + else: + # The config schema already prevents this, but it might be + # bypassed if Compose is called programmatically. + raise ValueError( + 'depends_on condition "{}" is invalid.'.format( + config['condition'] + ) + ) + + return configs + + def get_linked_service_names(self): + return [service.name for (service, _) in self.links] + + def get_link_names(self): + return [(service.name, alias) for service, alias in self.links] + + def get_volumes_from_names(self): + return [s.source.name for s in self.volumes_from if isinstance(s.source, Service)] + + def _next_container_number(self, one_off=False): + if one_off: + return None + containers = itertools.chain( + self._fetch_containers( + all=True, + filters={'label': self.labels(one_off=False)} + ), self._fetch_containers( + all=True, + filters={'label': self.labels(one_off=False, legacy=True)} + ) + ) + numbers = [c.number for c in containers if c.number is not None] + return 1 if not numbers else max(numbers) + 1 + + def _fetch_containers(self, **fetch_options): + # Account for containers that might have been removed since we fetched + # the list. + def soft_inspect(container): + try: + return Container.from_id(self.client, container['Id']) + except NotFound: + return None + + return filter(None, [ + soft_inspect(container) + for container in self.client.containers(**fetch_options) + ]) + + def _get_aliases(self, network, container=None): + return list( + {self.name} | + ({container.short_id} if container else set()) | + set(network.get('aliases', ())) + ) + + def build_default_networking_config(self): + if not self.networks: + return {} + + network = self.networks[self.network_mode.id] + endpoint = { + 'Aliases': self._get_aliases(network), + 'IPAMConfig': {}, + } + + if network.get('ipv4_address'): + endpoint['IPAMConfig']['IPv4Address'] = network.get('ipv4_address') + if network.get('ipv6_address'): + endpoint['IPAMConfig']['IPv6Address'] = network.get('ipv6_address') + + return {"EndpointsConfig": {self.network_mode.id: endpoint}} + + def _get_links(self, link_to_self): + links = {} + + for service, link_name in self.links: + for container in service.containers(): + links[link_name or service.name] = container.name + links[container.name] = container.name + links[container.name_without_project] = container.name + + if link_to_self: + for container in self.containers(): + links[self.name] = container.name + links[container.name] = container.name + links[container.name_without_project] = container.name + + for external_link in self.options.get('external_links') or []: + if ':' not in external_link: + link_name = external_link + else: + external_link, link_name = external_link.split(':') + links[link_name] = external_link + + return [ + (alias, container_name) + for (container_name, alias) in links.items() + ] + + def _get_volumes_from(self): + return [build_volume_from(spec) for spec in self.volumes_from] + + def _get_container_create_options( + self, + override_options, + number, + one_off=False, + previous_container=None): + add_config_hash = (not one_off and not override_options) + slug = generate_random_id() if one_off else None + + container_options = { + k: self.options[k] + for k in DOCKER_CONFIG_KEYS if k in self.options} + override_volumes = override_options.pop('volumes', []) + container_options.update(override_options) + + if not container_options.get('name'): + container_options['name'] = self.get_container_name(self.name, number, slug) + + container_options.setdefault('detach', True) + + # If a qualified hostname was given, split it into an + # unqualified hostname and a domainname unless domainname + # was also given explicitly. This matches behavior + # until Docker Engine 1.11.0 - Docker API 1.23. + if (version_lt(self.client.api_version, '1.23') and + 'hostname' in container_options and + 'domainname' not in container_options and + '.' in container_options['hostname']): + parts = container_options['hostname'].partition('.') + container_options['hostname'] = parts[0] + container_options['domainname'] = parts[2] + + if (version_gte(self.client.api_version, '1.25') and + 'stop_grace_period' in self.options): + container_options['stop_timeout'] = self.stop_timeout(None) + + if 'ports' in container_options or 'expose' in self.options: + container_options['ports'] = build_container_ports( + formatted_ports(container_options.get('ports', [])), + self.options) + + if 'volumes' in container_options or override_volumes: + container_options['volumes'] = list(set( + container_options.get('volumes', []) + override_volumes + )) + + container_options['environment'] = merge_environment( + self._parse_proxy_config(), + merge_environment( + self.options.get('environment'), + override_options.get('environment') + ) + ) + + container_options['labels'] = merge_labels( + self.options.get('labels'), + override_options.get('labels')) + + container_options, override_options = self._build_container_volume_options( + previous_container, container_options, override_options + ) + + container_options['image'] = self.image_name + + container_options['labels'] = build_container_labels( + container_options.get('labels', {}), + self.labels(one_off=one_off) + self.extra_labels, + number, + self.config_hash if add_config_hash else None, + slug + ) + + # Delete options which are only used in HostConfig + for key in HOST_CONFIG_KEYS: + container_options.pop(key, None) + + container_options['host_config'] = self._get_container_host_config( + override_options, + one_off=one_off) + + networking_config = self.build_default_networking_config() + if networking_config: + container_options['networking_config'] = networking_config + + container_options['environment'] = format_environment( + container_options['environment']) + return container_options + + def _build_container_volume_options(self, previous_container, container_options, override_options): + container_volumes = [] + container_mounts = [] + if 'volumes' in container_options: + container_volumes = [ + v for v in container_options.get('volumes') if isinstance(v, VolumeSpec) + ] + container_mounts = [v for v in container_options.get('volumes') if isinstance(v, MountSpec)] + + binds, affinity = merge_volume_bindings( + container_volumes, self.options.get('tmpfs') or [], previous_container, + container_mounts + ) + container_options['environment'].update(affinity) + + container_options['volumes'] = {v.internal: {} for v in container_volumes or {}} + if version_gte(self.client.api_version, '1.30'): + override_options['mounts'] = [build_mount(v) for v in container_mounts] or None + else: + # Workaround for 3.2 format + override_options['tmpfs'] = self.options.get('tmpfs') or [] + for m in container_mounts: + if m.is_tmpfs: + override_options['tmpfs'].append(m.target) + else: + binds.append(m.legacy_repr()) + container_options['volumes'][m.target] = {} + + secret_volumes = self.get_secret_volumes() + if secret_volumes: + if version_lt(self.client.api_version, '1.30'): + binds.extend(v.legacy_repr() for v in secret_volumes) + container_options['volumes'].update( + (v.target, {}) for v in secret_volumes + ) + else: + override_options['mounts'] = override_options.get('mounts') or [] + override_options['mounts'].extend([build_mount(v) for v in secret_volumes]) + + # Remove possible duplicates (see e.g. https://github.com/docker/compose/issues/5885). + # unique_everseen preserves order. (see https://github.com/docker/compose/issues/6091). + override_options['binds'] = list(unique_everseen(binds)) + return container_options, override_options + + def _get_container_host_config(self, override_options, one_off=False): + options = dict(self.options, **override_options) + + logging_dict = options.get('logging', None) + blkio_config = convert_blkio_config(options.get('blkio_config', None)) + log_config = get_log_config(logging_dict) + init_path = None + if isinstance(options.get('init'), str): + init_path = options.get('init') + options['init'] = True + + security_opt = [ + o.value for o in options.get('security_opt') + ] if options.get('security_opt') else None + + nano_cpus = None + if 'cpus' in options: + nano_cpus = int(options.get('cpus') * NANOCPUS_SCALE) + + return self.client.create_host_config( + links=self._get_links(link_to_self=one_off), + port_bindings=build_port_bindings( + formatted_ports(options.get('ports', [])) + ), + binds=options.get('binds'), + volumes_from=self._get_volumes_from(), + privileged=options.get('privileged', False), + network_mode=self.network_mode.mode, + devices=options.get('devices'), + device_requests=options.get('device_requests'), + dns=options.get('dns'), + dns_opt=options.get('dns_opt'), + dns_search=options.get('dns_search'), + restart_policy=options.get('restart'), + runtime=options.get('runtime'), + cap_add=options.get('cap_add'), + cap_drop=options.get('cap_drop'), + mem_limit=options.get('mem_limit'), + mem_reservation=options.get('mem_reservation'), + memswap_limit=options.get('memswap_limit'), + ulimits=build_ulimits(options.get('ulimits')), + log_config=log_config, + extra_hosts=options.get('extra_hosts'), + read_only=options.get('read_only'), + pid_mode=self.pid_mode.mode, + security_opt=security_opt, + ipc_mode=self.ipc_mode.mode, + cgroup_parent=options.get('cgroup_parent'), + cpu_quota=options.get('cpu_quota'), + shm_size=options.get('shm_size'), + sysctls=options.get('sysctls'), + pids_limit=options.get('pids_limit'), + tmpfs=options.get('tmpfs'), + oom_kill_disable=options.get('oom_kill_disable'), + oom_score_adj=options.get('oom_score_adj'), + mem_swappiness=options.get('mem_swappiness'), + group_add=options.get('group_add'), + userns_mode=options.get('userns_mode'), + init=options.get('init', None), + init_path=init_path, + isolation=options.get('isolation'), + cpu_count=options.get('cpu_count'), + cpu_percent=options.get('cpu_percent'), + nano_cpus=nano_cpus, + volume_driver=options.get('volume_driver'), + cpuset_cpus=options.get('cpuset'), + cpu_shares=options.get('cpu_shares'), + storage_opt=options.get('storage_opt'), + blkio_weight=blkio_config.get('weight'), + blkio_weight_device=blkio_config.get('weight_device'), + device_read_bps=blkio_config.get('device_read_bps'), + device_read_iops=blkio_config.get('device_read_iops'), + device_write_bps=blkio_config.get('device_write_bps'), + device_write_iops=blkio_config.get('device_write_iops'), + mounts=options.get('mounts'), + device_cgroup_rules=options.get('device_cgroup_rules'), + cpu_period=options.get('cpu_period'), + cpu_rt_period=options.get('cpu_rt_period'), + cpu_rt_runtime=options.get('cpu_rt_runtime'), + ) + + def get_secret_volumes(self): + def build_spec(secret): + target = secret['secret'].target + if target is None: + target = '{}/{}'.format(const.SECRETS_PATH, secret['secret'].source) + elif not os.path.isabs(target): + target = '{}/{}'.format(const.SECRETS_PATH, target) + + return MountSpec('bind', secret['file'], target, read_only=True) + + return [build_spec(secret) for secret in self.secrets] + + def build(self, no_cache=False, pull=False, force_rm=False, memory=None, build_args_override=None, + gzip=False, rm=True, silent=False, cli=False, progress=None): + output_stream = open(os.devnull, 'w') + if not silent: + output_stream = sys.stdout + log.info('Building %s' % self.name) + + build_opts = self.options.get('build', {}) + + build_args = build_opts.get('args', {}).copy() + if build_args_override: + build_args.update(build_args_override) + + for k, v in self._parse_proxy_config().items(): + build_args.setdefault(k, v) + + path = rewrite_build_path(build_opts.get('context')) + if self.platform and version_lt(self.client.api_version, '1.35'): + raise OperationFailedError( + 'Impossible to perform platform-targeted builds for API version < 1.35' + ) + + builder = self.client if not cli else _CLIBuilder(progress) + build_output = builder.build( + path=path, + tag=self.image_name, + rm=rm, + forcerm=force_rm, + pull=pull, + nocache=no_cache, + dockerfile=build_opts.get('dockerfile', None), + cache_from=self.get_cache_from(build_opts), + labels=build_opts.get('labels', None), + buildargs=build_args, + network_mode=build_opts.get('network', None), + target=build_opts.get('target', None), + shmsize=parse_bytes(build_opts.get('shm_size')) if build_opts.get('shm_size') else None, + extra_hosts=build_opts.get('extra_hosts', None), + container_limits={ + 'memory': parse_bytes(memory) if memory else None + }, + gzip=gzip, + isolation=build_opts.get('isolation', self.options.get('isolation', None)), + platform=self.platform, + ) + + try: + all_events = list(stream_output(build_output, output_stream)) + except StreamOutputError as e: + raise BuildError(self, str(e)) + + # Ensure the HTTP connection is not reused for another + # streaming command, as the Docker daemon can sometimes + # complain about it + self.client.close() + + image_id = None + + for event in all_events: + if 'stream' in event: + match = re.search(r'Successfully built ([0-9a-f]+)', event.get('stream', '')) + if match: + image_id = match.group(1) + + if image_id is None: + raise BuildError(self, event if all_events else 'Unknown') + + return image_id + + def get_cache_from(self, build_opts): + cache_from = build_opts.get('cache_from', None) + if cache_from is not None: + cache_from = [tag for tag in cache_from if tag] + return cache_from + + def can_be_built(self): + return 'build' in self.options + + def labels(self, one_off=False, legacy=False): + proj_name = self.project if not legacy else re.sub(r'[_-]', '', self.project) + return [ + '{}={}'.format(LABEL_PROJECT, proj_name), + '{}={}'.format(LABEL_SERVICE, self.name), + '{}={}'.format(LABEL_ONE_OFF, "True" if one_off else "False"), + ] + + @property + def custom_container_name(self): + return self.options.get('container_name') + + def get_container_name(self, service_name, number, slug=None): + if self.custom_container_name and slug is None: + return self.custom_container_name + + container_name = build_container_name( + self.project, service_name, number, slug, + ) + ext_links_origins = [link.split(':')[0] for link in self.options.get('external_links', [])] + if container_name in ext_links_origins: + raise DependencyError( + 'Service {} has a self-referential external link: {}'.format( + self.name, container_name + ) + ) + return container_name + + def remove_image(self, image_type): + if not image_type or image_type == ImageType.none: + return False + if image_type == ImageType.local and self.options.get('image'): + return False + + log.info("Removing image %s", self.image_name) + try: + self.client.remove_image(self.image_name) + return True + except ImageNotFound: + log.warning("Image %s not found.", self.image_name) + return False + except APIError as e: + log.error("Failed to remove image for service %s: %s", self.name, e) + return False + + def specifies_host_port(self): + def has_host_port(binding): + if isinstance(binding, dict): + external_bindings = binding.get('published') + else: + _, external_bindings = split_port(binding) + + # there are no external bindings + if external_bindings is None: + return False + + # we only need to check the first binding from the range + external_binding = external_bindings[0] + + # non-tuple binding means there is a host port specified + if not isinstance(external_binding, tuple): + return True + + # extract actual host port from tuple of (host_ip, host_port) + _, host_port = external_binding + if host_port is not None: + return True + + return False + + return any(has_host_port(binding) for binding in self.options.get('ports', [])) + + def _do_pull(self, repo, pull_kwargs, silent, ignore_pull_failures): + try: + output = self.client.pull(repo, **pull_kwargs) + if silent: + with open(os.devnull, 'w') as devnull: + yield from stream_output(output, devnull) + else: + yield from stream_output(output, sys.stdout) + except (StreamOutputError, NotFound) as e: + if not ignore_pull_failures: + raise + else: + log.error(str(e)) + + def pull(self, ignore_pull_failures=False, silent=False, stream=False): + if 'image' not in self.options: + return + + repo, tag, separator = parse_repository_tag(self.options['image']) + kwargs = { + 'tag': tag or 'latest', + 'stream': True, + 'platform': self.platform, + } + if not silent: + log.info('Pulling {} ({}{}{})...'.format(self.name, repo, separator, tag)) + + if kwargs['platform'] and version_lt(self.client.api_version, '1.35'): + raise OperationFailedError( + 'Impossible to perform platform-targeted pulls for API version < 1.35' + ) + + event_stream = self._do_pull(repo, kwargs, silent, ignore_pull_failures) + if stream: + return event_stream + return progress_stream.get_digest_from_pull(event_stream) + + def push(self, ignore_push_failures=False): + if 'image' not in self.options or 'build' not in self.options: + return + + repo, tag, separator = parse_repository_tag(self.options['image']) + tag = tag or 'latest' + log.info('Pushing {} ({}{}{})...'.format(self.name, repo, separator, tag)) + output = self.client.push(repo, tag=tag, stream=True) + + try: + return progress_stream.get_digest_from_push( + stream_output(output, sys.stdout)) + except StreamOutputError as e: + if not ignore_push_failures: + raise + else: + log.error(str(e)) + + def is_healthy(self): + """ Check that all containers for this service report healthy. + Returns false if at least one healthcheck is pending. + If an unhealthy container is detected, raise a HealthCheckFailed + exception. + """ + result = True + for ctnr in self.containers(): + ctnr.inspect() + status = ctnr.get('State.Health.Status') + if status is None: + raise NoHealthCheckConfigured(self.name) + elif status == 'starting': + result = False + elif status == 'unhealthy': + raise HealthCheckFailed(ctnr.short_id) + return result + + def _parse_proxy_config(self): + client = self.client + if 'proxies' not in client._general_configs: + return {} + docker_host = getattr(client, '_original_base_url', client.base_url) + proxy_config = client._general_configs['proxies'].get( + docker_host, client._general_configs['proxies'].get('default') + ) or {} + + permitted = { + 'ftpProxy': 'FTP_PROXY', + 'httpProxy': 'HTTP_PROXY', + 'httpsProxy': 'HTTPS_PROXY', + 'noProxy': 'NO_PROXY', + } + + result = {} + + for k, v in proxy_config.items(): + if k not in permitted: + continue + result[permitted[k]] = result[permitted[k].lower()] = v + + return result + + def get_profiles(self): + if 'profiles' not in self.options: + return [] + + return self.options.get('profiles') + + def enabled_for_profiles(self, enabled_profiles): + # if service has no profiles specified it is always enabled + if 'profiles' not in self.options: + return True + + service_profiles = self.options.get('profiles') + for profile in enabled_profiles: + if profile in service_profiles: + return True + + return False + + +def short_id_alias_exists(container, network): + aliases = container.get( + 'NetworkSettings.Networks.{net}.Aliases'.format(net=network)) or () + return container.short_id in aliases + + +class IpcMode: + def __init__(self, mode): + self._mode = mode + + @property + def mode(self): + return self._mode + + @property + def service_name(self): + return None + + +class ServiceIpcMode(IpcMode): + def __init__(self, service): + self.service = service + + @property + def service_name(self): + return self.service.name + + @property + def mode(self): + containers = self.service.containers() + if containers: + return 'container:' + containers[0].id + + log.warning( + "Service %s is trying to use reuse the IPC namespace " + "of another service that is not running." % (self.service_name) + ) + return None + + +class ContainerIpcMode(IpcMode): + def __init__(self, container): + self.container = container + self._mode = 'container:{}'.format(container.id) + + +class PidMode: + def __init__(self, mode): + self._mode = mode + + @property + def mode(self): + return self._mode + + @property + def service_name(self): + return None + + +class ServicePidMode(PidMode): + def __init__(self, service): + self.service = service + + @property + def service_name(self): + return self.service.name + + @property + def mode(self): + containers = self.service.containers() + if containers: + return 'container:' + containers[0].id + + log.warning( + "Service %s is trying to use reuse the PID namespace " + "of another service that is not running." % (self.service_name) + ) + return None + + +class ContainerPidMode(PidMode): + def __init__(self, container): + self.container = container + self._mode = 'container:{}'.format(container.id) + + +class NetworkMode: + """A `standard` network mode (ex: host, bridge)""" + + service_name = None + + def __init__(self, network_mode): + self.network_mode = network_mode + + @property + def id(self): + return self.network_mode + + mode = id + + +class ContainerNetworkMode: + """A network mode that uses a container's network stack.""" + + service_name = None + + def __init__(self, container): + self.container = container + + @property + def id(self): + return self.container.id + + @property + def mode(self): + return 'container:' + self.container.id + + +class ServiceNetworkMode: + """A network mode that uses a service's network stack.""" + + def __init__(self, service): + self.service = service + + @property + def id(self): + return self.service.name + + service_name = id + + @property + def mode(self): + containers = self.service.containers() + if containers: + return 'container:' + containers[0].id + + log.warning("Service %s is trying to use reuse the network stack " + "of another service that is not running." % (self.id)) + return None + + +# Names + + +def build_container_name(project, service, number, slug=None): + bits = [project.lstrip('-_'), service] + if slug: + bits.extend(['run', truncate_id(slug)]) + else: + bits.append(str(number)) + return '_'.join(bits) + + +# Images + +def parse_repository_tag(repo_path): + """Splits image identification into base image path, tag/digest + and it's separator. + + Example: + + >>> parse_repository_tag('user/repo@sha256:digest') + ('user/repo', 'sha256:digest', '@') + >>> parse_repository_tag('user/repo:v1') + ('user/repo', 'v1', ':') + """ + tag_separator = ":" + digest_separator = "@" + + if digest_separator in repo_path: + repo, tag = repo_path.rsplit(digest_separator, 1) + return repo, tag, digest_separator + + repo, tag = repo_path, "" + if tag_separator in repo_path: + repo, tag = repo_path.rsplit(tag_separator, 1) + if "/" in tag: + repo, tag = repo_path, "" + + return repo, tag, tag_separator + + +# Volumes + + +def merge_volume_bindings(volumes, tmpfs, previous_container, mounts): + """ + Return a list of volume bindings for a container. Container data volumes + are replaced by those from the previous container. + Anonymous mounts are updated in place. + """ + affinity = {} + + volume_bindings = OrderedDict( + build_volume_binding(volume) + for volume in volumes + if volume.external + ) + + if previous_container: + old_volumes, old_mounts = get_container_data_volumes( + previous_container, volumes, tmpfs, mounts + ) + warn_on_masked_volume(volumes, old_volumes, previous_container.service) + volume_bindings.update( + build_volume_binding(volume) for volume in old_volumes + ) + + if old_volumes or old_mounts: + affinity = {'affinity:container': '=' + previous_container.id} + + return list(volume_bindings.values()), affinity + + +def get_container_data_volumes(container, volumes_option, tmpfs_option, mounts_option): + """ + Find the container data volumes that are in `volumes_option`, and return + a mapping of volume bindings for those volumes. + Anonymous volume mounts are updated in place instead. + """ + volumes = [] + volumes_option = volumes_option or [] + + container_mounts = { + mount['Destination']: mount + for mount in container.get('Mounts') or {} + } + + image_volumes = [ + VolumeSpec.parse(volume) + for volume in + container.image_config['ContainerConfig'].get('Volumes') or {} + ] + + for volume in set(volumes_option + image_volumes): + # No need to preserve host volumes + if volume.external: + continue + + # Attempting to rebind tmpfs volumes breaks: https://github.com/docker/compose/issues/4751 + if volume.internal in convert_tmpfs_mounts(tmpfs_option).keys(): + continue + + mount = container_mounts.get(volume.internal) + + # New volume, doesn't exist in the old container + if not mount: + continue + + # Volume was previously a host volume, now it's a container volume + if not mount.get('Name'): + continue + + # Volume (probably an image volume) is overridden by a mount in the service's config + # and would cause a duplicate mountpoint error + if volume.internal in [m.target for m in mounts_option]: + continue + + # Copy existing volume from old container + volume = volume._replace(external=mount['Name']) + volumes.append(volume) + + updated_mounts = False + for mount in mounts_option: + if mount.type != 'volume': + continue + + ctnr_mount = container_mounts.get(mount.target) + if not ctnr_mount or not ctnr_mount.get('Name'): + continue + + mount.source = ctnr_mount['Name'] + updated_mounts = True + + return volumes, updated_mounts + + +def warn_on_masked_volume(volumes_option, container_volumes, service): + container_volumes = { + volume.internal: volume.external + for volume in container_volumes} + + for volume in volumes_option: + if ( + volume.external and + volume.internal in container_volumes and + container_volumes.get(volume.internal) != volume.external + ): + log.warning(( + "Service \"{service}\" is using volume \"{volume}\" from the " + "previous container. Host mapping \"{host_path}\" has no effect. " + "Remove the existing containers (with `docker-compose rm {service}`) " + "to use the host volume mapping." + ).format( + service=service, + volume=volume.internal, + host_path=volume.external)) + + +def build_volume_binding(volume_spec): + return volume_spec.internal, volume_spec.repr() + + +def build_volume_from(volume_from_spec): + """ + volume_from can be either a service or a container. We want to return the + container.id and format it into a string complete with the mode. + """ + if isinstance(volume_from_spec.source, Service): + containers = volume_from_spec.source.containers(stopped=True) + if not containers: + return "{}:{}".format( + volume_from_spec.source.create_container().id, + volume_from_spec.mode) + + container = containers[0] + return "{}:{}".format(container.id, volume_from_spec.mode) + elif isinstance(volume_from_spec.source, Container): + return "{}:{}".format(volume_from_spec.source.id, volume_from_spec.mode) + + +def build_mount(mount_spec): + kwargs = {} + if mount_spec.options: + for option, sdk_name in mount_spec.options_map[mount_spec.type].items(): + if option in mount_spec.options: + kwargs[sdk_name] = mount_spec.options[option] + + return Mount( + type=mount_spec.type, target=mount_spec.target, source=mount_spec.source, + read_only=mount_spec.read_only, consistency=mount_spec.consistency, **kwargs + ) + + +# Labels + + +def build_container_labels(label_options, service_labels, number, config_hash, slug): + labels = dict(label_options or {}) + labels.update(label.split('=', 1) for label in service_labels) + if number is not None: + labels[LABEL_CONTAINER_NUMBER] = str(number) + if slug is not None: + labels[LABEL_SLUG] = slug + labels[LABEL_VERSION] = __version__ + + if config_hash: + log.debug("Added config hash: %s" % config_hash) + labels[LABEL_CONFIG_HASH] = config_hash + + return labels + + +# Ulimits + + +def build_ulimits(ulimit_config): + if not ulimit_config: + return None + ulimits = [] + for limit_name, soft_hard_values in ulimit_config.items(): + if isinstance(soft_hard_values, int): + ulimits.append({'name': limit_name, 'soft': soft_hard_values, 'hard': soft_hard_values}) + elif isinstance(soft_hard_values, dict): + ulimit_dict = {'name': limit_name} + ulimit_dict.update(soft_hard_values) + ulimits.append(ulimit_dict) + + return ulimits + + +def get_log_config(logging_dict): + log_driver = logging_dict.get('driver', "") if logging_dict else "" + log_options = logging_dict.get('options', None) if logging_dict else None + return LogConfig( + type=log_driver, + config=log_options + ) + + +# TODO: remove once fix is available in docker-py +def format_environment(environment): + def format_env(key, value): + if value is None: + return key + if isinstance(value, bytes): + value = value.decode('utf-8') + return '{key}={value}'.format(key=key, value=value) + + return [format_env(*item) for item in environment.items()] + + +# Ports +def formatted_ports(ports): + result = [] + for port in ports: + if isinstance(port, ServicePort): + result.append(port.legacy_repr()) + else: + result.append(port) + return result + + +def build_container_ports(container_ports, options): + ports = [] + all_ports = container_ports + options.get('expose', []) + for port_range in all_ports: + internal_range, _ = split_port(port_range) + for port in internal_range: + port = str(port) + if '/' in port: + port = tuple(port.split('/')) + ports.append(port) + return ports + + +def convert_blkio_config(blkio_config): + result = {} + if blkio_config is None: + return result + + result['weight'] = blkio_config.get('weight') + for field in [ + "device_read_bps", "device_read_iops", "device_write_bps", + "device_write_iops", "weight_device", + ]: + if field not in blkio_config: + continue + arr = [] + for item in blkio_config[field]: + arr.append({k.capitalize(): v for k, v in item.items()}) + result[field] = arr + return result + + +def rewrite_build_path(path): + if IS_WINDOWS_PLATFORM and not is_url(path) and not path.startswith(WINDOWS_LONGPATH_PREFIX): + path = WINDOWS_LONGPATH_PREFIX + os.path.normpath(path) + + return path + + +class _CLIBuilder: + def __init__(self, progress): + self._progress = progress + + def build(self, path, tag=None, quiet=False, fileobj=None, + nocache=False, rm=False, timeout=None, + custom_context=False, encoding=None, pull=False, + forcerm=False, dockerfile=None, container_limits=None, + decode=False, buildargs=None, gzip=False, shmsize=None, + labels=None, cache_from=None, target=None, network_mode=None, + squash=None, extra_hosts=None, platform=None, isolation=None, + use_config_proxy=True): + """ + Args: + path (str): Path to the directory containing the Dockerfile + buildargs (dict): A dictionary of build arguments + cache_from (:py:class:`list`): A list of images used for build + cache resolution + container_limits (dict): A dictionary of limits applied to each + container created by the build process. Valid keys: + - memory (int): set memory limit for build + - memswap (int): Total memory (memory + swap), -1 to disable + swap + - cpushares (int): CPU shares (relative weight) + - cpusetcpus (str): CPUs in which to allow execution, e.g., + ``"0-3"``, ``"0,1"`` + custom_context (bool): Optional if using ``fileobj`` + decode (bool): If set to ``True``, the returned stream will be + decoded into dicts on the fly. Default ``False`` + dockerfile (str): path within the build context to the Dockerfile + encoding (str): The encoding for a stream. Set to ``gzip`` for + compressing + extra_hosts (dict): Extra hosts to add to /etc/hosts in building + containers, as a mapping of hostname to IP address. + fileobj: A file object to use as the Dockerfile. (Or a file-like + object) + forcerm (bool): Always remove intermediate containers, even after + unsuccessful builds + isolation (str): Isolation technology used during build. + Default: `None`. + labels (dict): A dictionary of labels to set on the image + network_mode (str): networking mode for the run commands during + build + nocache (bool): Don't use the cache when set to ``True`` + platform (str): Platform in the format ``os[/arch[/variant]]`` + pull (bool): Downloads any updates to the FROM image in Dockerfiles + quiet (bool): Whether to return the status + rm (bool): Remove intermediate containers. The ``docker build`` + command now defaults to ``--rm=true``, but we have kept the old + default of `False` to preserve backward compatibility + shmsize (int): Size of `/dev/shm` in bytes. The size must be + greater than 0. If omitted the system uses 64MB + squash (bool): Squash the resulting images layers into a + single layer. + tag (str): A tag to add to the final image + target (str): Name of the build-stage to build in a multi-stage + Dockerfile + timeout (int): HTTP timeout + use_config_proxy (bool): If ``True``, and if the docker client + configuration file (``~/.docker/config.json`` by default) + contains a proxy configuration, the corresponding environment + variables will be set in the container being built. + Returns: + A generator for the build output. + """ + if dockerfile: + dockerfile = os.path.join(path, dockerfile) + iidfile = tempfile.mktemp() + + command_builder = _CommandBuilder() + command_builder.add_params("--build-arg", buildargs) + command_builder.add_list("--cache-from", cache_from) + command_builder.add_arg("--file", dockerfile) + command_builder.add_flag("--force-rm", forcerm) + command_builder.add_params("--label", labels) + command_builder.add_arg("--memory", container_limits.get("memory")) + command_builder.add_arg("--network", network_mode) + command_builder.add_flag("--no-cache", nocache) + command_builder.add_arg("--progress", self._progress) + command_builder.add_flag("--pull", pull) + command_builder.add_arg("--tag", tag) + command_builder.add_arg("--target", target) + command_builder.add_arg("--iidfile", iidfile) + args = command_builder.build([path]) + + magic_word = "Successfully built " + appear = False + with subprocess.Popen(args, stdout=subprocess.PIPE, + universal_newlines=True) as p: + while True: + line = p.stdout.readline() + if not line: + break + if line.startswith(magic_word): + appear = True + yield json.dumps({"stream": line}) + + p.communicate() + if p.returncode != 0: + raise StreamOutputError() + + with open(iidfile) as f: + line = f.readline() + image_id = line.split(":")[1].strip() + os.remove(iidfile) + + # In case of `DOCKER_BUILDKIT=1` + # there is no success message already present in the output. + # Since that's the way `Service::build` gets the `image_id` + # it has to be added `manually` + if not appear: + yield json.dumps({"stream": "{}{}\n".format(magic_word, image_id)}) + + +class _CommandBuilder: + def __init__(self): + self._args = ["docker", "build"] + + def add_arg(self, name, value): + if value: + self._args.extend([name, str(value)]) + + def add_flag(self, name, flag): + if flag: + self._args.extend([name]) + + def add_params(self, name, params): + if params: + for key, val in params.items(): + self._args.extend([name, "{}={}".format(key, val)]) + + def add_list(self, name, values): + if values: + for val in values: + self._args.extend([name, val]) + + def build(self, args): + return self._args + args diff --git a/compose/timeparse.py b/compose/timeparse.py new file mode 100644 index 00000000000..47744562519 --- /dev/null +++ b/compose/timeparse.py @@ -0,0 +1,92 @@ +#!/usr/bin/env python +''' +timeparse.py +(c) Will Roberts <wildwilhelm@gmail.com> 1 February, 2014 + +This is a vendored and modified copy of: +github.com/wroberts/pytimeparse @ cc0550d + +It has been modified to mimic the behaviour of +https://golang.org/pkg/time/#ParseDuration +''' +# MIT LICENSE +# +# Permission is hereby granted, free of charge, to any person +# obtaining a copy of this software and associated documentation files +# (the "Software"), to deal in the Software without restriction, +# including without limitation the rights to use, copy, modify, merge, +# publish, distribute, sublicense, and/or sell copies of the Software, +# and to permit persons to whom the Software is furnished to do so, +# subject to the following conditions: +# +# The above copyright notice and this permission notice shall be +# included in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS +# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN +# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. +import re + +HOURS = r'(?P<hours>[\d.]+)h' +MINS = r'(?P<mins>[\d.]+)m' +SECS = r'(?P<secs>[\d.]+)s' +MILLI = r'(?P<milli>[\d.]+)ms' +MICRO = r'(?P<micro>[\d.]+)(?:us|µs)' +NANO = r'(?P<nano>[\d.]+)ns' + + +def opt(x): + return r'(?:{x})?'.format(x=x) + + +TIMEFORMAT = r'{HOURS}{MINS}{SECS}{MILLI}{MICRO}{NANO}'.format( + HOURS=opt(HOURS), + MINS=opt(MINS), + SECS=opt(SECS), + MILLI=opt(MILLI), + MICRO=opt(MICRO), + NANO=opt(NANO), +) + +MULTIPLIERS = { + 'hours': 60 * 60, + 'mins': 60, + 'secs': 1, + 'milli': 1.0 / 1000, + 'micro': 1.0 / 1000.0 / 1000, + 'nano': 1.0 / 1000.0 / 1000.0 / 1000.0, +} + + +def timeparse(sval): + """Parse a time expression, returning it as a number of seconds. If + possible, the return value will be an `int`; if this is not + possible, the return will be a `float`. Returns `None` if a time + expression cannot be parsed from the given string. + + Arguments: + - `sval`: the string value to parse + + >>> timeparse('1m24s') + 84 + >>> timeparse('1.2 minutes') + 72 + >>> timeparse('1.2 seconds') + 1.2 + """ + match = re.match(r'\s*' + TIMEFORMAT + r'\s*$', sval, re.I) + if not match or not match.group(0).strip(): + return + + mdict = match.groupdict() + return sum( + MULTIPLIERS[k] * cast(v) for (k, v) in mdict.items() if v is not None) + + +def cast(value): + return int(value) if value.isdigit() else float(value) diff --git a/compose/utils.py b/compose/utils.py new file mode 100644 index 00000000000..86af8f8852a --- /dev/null +++ b/compose/utils.py @@ -0,0 +1,191 @@ +import hashlib +import json.decoder +import logging +import ntpath +import random + +from docker.errors import DockerException +from docker.utils import parse_bytes as sdk_parse_bytes + +from .errors import StreamParseError +from .timeparse import MULTIPLIERS +from .timeparse import timeparse + + +json_decoder = json.JSONDecoder() +log = logging.getLogger(__name__) + + +def stream_as_text(stream): + """Given a stream of bytes or text, if any of the items in the stream + are bytes convert them to text. + + This function can be removed once docker-py returns text streams instead + of byte streams. + """ + for data in stream: + if not isinstance(data, str): + data = data.decode('utf-8', 'replace') + yield data + + +def line_splitter(buffer, separator='\n'): + index = buffer.find(str(separator)) + if index == -1: + return None + return buffer[:index + 1], buffer[index + 1:] + + +def split_buffer(stream, splitter=None, decoder=lambda a: a): + """Given a generator which yields strings and a splitter function, + joins all input, splits on the separator and yields each chunk. + + Unlike string.split(), each chunk includes the trailing + separator, except for the last one if none was found on the end + of the input. + """ + splitter = splitter or line_splitter + buffered = '' + + for data in stream_as_text(stream): + buffered += data + while True: + buffer_split = splitter(buffered) + if buffer_split is None: + break + + item, buffered = buffer_split + yield item + + if buffered: + try: + yield decoder(buffered) + except Exception as e: + log.error( + 'Compose tried decoding the following data chunk, but failed:' + '\n%s' % repr(buffered) + ) + raise StreamParseError(e) + + +def json_splitter(buffer): + """Attempt to parse a json object from a buffer. If there is at least one + object, return it and the rest of the buffer, otherwise return None. + """ + buffer = buffer.strip() + try: + obj, index = json_decoder.raw_decode(buffer) + rest = buffer[json.decoder.WHITESPACE.match(buffer, index).end():] + return obj, rest + except ValueError: + return None + + +def json_stream(stream): + """Given a stream of text, return a stream of json objects. + This handles streams which are inconsistently buffered (some entries may + be newline delimited, and others are not). + """ + return split_buffer(stream, json_splitter, json_decoder.decode) + + +def json_hash(obj): + dump = json.dumps(obj, sort_keys=True, separators=(',', ':'), default=lambda x: x.repr()) + h = hashlib.sha256() + h.update(dump.encode('utf8')) + return h.hexdigest() + + +def microseconds_from_time_nano(time_nano): + return int(time_nano % 1000000000 / 1000) + + +def nanoseconds_from_time_seconds(time_seconds): + return int(time_seconds / MULTIPLIERS['nano']) + + +def parse_seconds_float(value): + return timeparse(value or '') + + +def parse_nanoseconds_int(value): + parsed = timeparse(value or '') + if parsed is None: + return None + return nanoseconds_from_time_seconds(parsed) + + +def build_string_dict(source_dict): + return {k: str(v if v is not None else '') for k, v in source_dict.items()} + + +def splitdrive(path): + if len(path) == 0: + return ('', '') + if path[0] in ['.', '\\', '/', '~']: + return ('', path) + return ntpath.splitdrive(path) + + +def parse_bytes(n): + try: + return sdk_parse_bytes(n) + except DockerException: + return None + + +def unquote_path(s): + if not s: + return s + if s[0] == '"' and s[-1] == '"': + return s[1:-1] + return s + + +def generate_random_id(): + while True: + val = hex(random.getrandbits(32 * 8))[2:-1] + try: + int(truncate_id(val)) + continue + except ValueError: + return val + + +def truncate_id(value): + if ':' in value: + value = value[value.index(':') + 1:] + if len(value) > 12: + return value[:12] + return value + + +def unique_everseen(iterable, key=lambda x: x): + "List unique elements, preserving order. Remember all elements ever seen." + seen = set() + for element in iterable: + unique_key = key(element) + if unique_key not in seen: + seen.add(unique_key) + yield element + + +def truncate_string(s, max_chars=35): + if len(s) > max_chars: + return s[:max_chars - 2] + '...' + return s + + +def filter_attached_for_up(items, service_names, attach_dependencies=False, + item_to_service_name=lambda x: x): + """This function contains the logic of choosing which services to + attach when doing docker-compose up. It may be used both with containers + and services, and any other entities that map to service names - + this mapping is provided by item_to_service_name.""" + if attach_dependencies or not service_names: + return items + + return [ + item + for item in items if item_to_service_name(item) in service_names + ] diff --git a/compose/version.py b/compose/version.py new file mode 100644 index 00000000000..c039263acb9 --- /dev/null +++ b/compose/version.py @@ -0,0 +1,7 @@ +from distutils.version import LooseVersion + + +class ComposeVersion(LooseVersion): + """ A hashable version object """ + def __hash__(self): + return hash(self.vstring) diff --git a/compose/volume.py b/compose/volume.py new file mode 100644 index 00000000000..5f36e432ba9 --- /dev/null +++ b/compose/volume.py @@ -0,0 +1,213 @@ +import logging +import re +from itertools import chain + +from docker.errors import NotFound +from docker.utils import version_lt + +from . import __version__ +from .config import ConfigurationError +from .config.types import VolumeSpec +from .const import LABEL_PROJECT +from .const import LABEL_VERSION +from .const import LABEL_VOLUME + + +log = logging.getLogger(__name__) + + +class Volume: + def __init__(self, client, project, name, driver=None, driver_opts=None, + external=False, labels=None, custom_name=False): + self.client = client + self.project = project + self.name = name + self.driver = driver + self.driver_opts = driver_opts + self.external = external + self.labels = labels + self.custom_name = custom_name + self.legacy = None + + def create(self): + return self.client.create_volume( + self.full_name, self.driver, self.driver_opts, labels=self._labels + ) + + def remove(self): + if self.external: + log.info("Volume %s is external, skipping", self.true_name) + return + log.info("Removing volume %s", self.true_name) + return self.client.remove_volume(self.true_name) + + def inspect(self, legacy=None): + if legacy: + return self.client.inspect_volume(self.legacy_full_name) + return self.client.inspect_volume(self.full_name) + + def exists(self): + self._set_legacy_flag() + try: + self.inspect(legacy=self.legacy) + except NotFound: + return False + return True + + @property + def full_name(self): + if self.custom_name: + return self.name + return '{}_{}'.format(self.project.lstrip('-_'), self.name) + + @property + def legacy_full_name(self): + if self.custom_name: + return self.name + return '{}_{}'.format( + re.sub(r'[_-]', '', self.project), self.name + ) + + @property + def true_name(self): + self._set_legacy_flag() + if self.legacy: + return self.legacy_full_name + return self.full_name + + @property + def _labels(self): + if version_lt(self.client._version, '1.23'): + return None + labels = self.labels.copy() if self.labels else {} + labels.update({ + LABEL_PROJECT: self.project, + LABEL_VOLUME: self.name, + LABEL_VERSION: __version__, + }) + return labels + + def _set_legacy_flag(self): + if self.legacy is not None: + return + try: + data = self.inspect(legacy=True) + self.legacy = data is not None + except NotFound: + self.legacy = False + + +class ProjectVolumes: + + def __init__(self, volumes): + self.volumes = volumes + + @classmethod + def from_config(cls, name, config_data, client): + config_volumes = config_data.volumes or {} + volumes = { + vol_name: Volume( + client=client, + project=name, + name=data.get('name', vol_name), + driver=data.get('driver'), + driver_opts=data.get('driver_opts'), + custom_name=data.get('name') is not None, + labels=data.get('labels'), + external=bool(data.get('external', False)) + ) + for vol_name, data in config_volumes.items() + } + return cls(volumes) + + def remove(self): + for volume in self.volumes.values(): + try: + volume.remove() + except NotFound: + log.warning("Volume %s not found.", volume.true_name) + + def initialize(self): + try: + for volume in self.volumes.values(): + volume_exists = volume.exists() + if volume.external: + log.debug( + 'Volume {} declared as external. No new ' + 'volume will be created.'.format(volume.name) + ) + if not volume_exists: + raise ConfigurationError( + 'Volume {name} declared as external, but could' + ' not be found. Please create the volume manually' + ' using `{command}{name}` and try again.'.format( + name=volume.full_name, + command='docker volume create --name=' + ) + ) + continue + + if not volume_exists: + log.info( + 'Creating volume "{}" with {} driver'.format( + volume.full_name, volume.driver or 'default' + ) + ) + volume.create() + else: + check_remote_volume_config(volume.inspect(legacy=volume.legacy), volume) + except NotFound: + raise ConfigurationError( + 'Volume {} specifies nonexistent driver {}'.format(volume.name, volume.driver) + ) + + def namespace_spec(self, volume_spec): + if not volume_spec.is_named_volume: + return volume_spec + + if isinstance(volume_spec, VolumeSpec): + volume = self.volumes[volume_spec.external] + return volume_spec._replace(external=volume.true_name) + else: + volume_spec.source = self.volumes[volume_spec.source].true_name + return volume_spec + + +class VolumeConfigChangedError(ConfigurationError): + def __init__(self, local, property_name, local_value, remote_value): + super().__init__( + 'Configuration for volume {vol_name} specifies {property_name} ' + '{local_value}, but a volume with the same name uses a different ' + '{property_name} ({remote_value}). If you wish to use the new ' + 'configuration, please remove the existing volume "{full_name}" ' + 'first:\n$ docker volume rm {full_name}'.format( + vol_name=local.name, property_name=property_name, + local_value=local_value, remote_value=remote_value, + full_name=local.true_name + ) + ) + + +def check_remote_volume_config(remote, local): + if local.driver and remote.get('Driver') != local.driver: + raise VolumeConfigChangedError(local, 'driver', local.driver, remote.get('Driver')) + local_opts = local.driver_opts or {} + remote_opts = remote.get('Options') or {} + for k in set(chain(remote_opts, local_opts)): + if k.startswith('com.docker.'): # These options are set internally + continue + if remote_opts.get(k) != local_opts.get(k): + raise VolumeConfigChangedError( + local, '"{}" driver_opt'.format(k), local_opts.get(k), remote_opts.get(k), + ) + + local_labels = local.labels or {} + remote_labels = remote.get('Labels') or {} + for k in set(chain(remote_labels, local_labels)): + if k.startswith('com.docker.'): # We are only interested in user-specified labels + continue + if remote_labels.get(k) != local_labels.get(k): + log.warning( + 'Volume {}: label "{}" has changed. It may need to be' + ' recreated.'.format(local.name, k) + ) diff --git a/contrib/completion/bash/docker-compose b/contrib/completion/bash/docker-compose new file mode 100644 index 00000000000..04f1a78ae76 --- /dev/null +++ b/contrib/completion/bash/docker-compose @@ -0,0 +1,666 @@ +#!/bin/bash +# +# bash completion for docker-compose +# +# This work is based on the completion for the docker command. +# +# This script provides completion of: +# - commands and their options +# - service names +# - filepaths +# +# To enable the completions either: +# - place this file in /etc/bash_completion.d +# or +# - copy this file to e.g. ~/.docker-compose-completion.sh and add the line +# below to your .bashrc after bash completion features are loaded +# . ~/.docker-compose-completion.sh + +__docker_compose_previous_extglob_setting=$(shopt -p extglob) +shopt -s extglob + +__docker_compose_q() { + docker-compose 2>/dev/null "${top_level_options[@]}" "$@" +} + +# Transforms a multiline list of strings into a single line string +# with the words separated by "|". +__docker_compose_to_alternatives() { + local parts=( $1 ) + local IFS='|' + echo "${parts[*]}" +} + +# Transforms a multiline list of options into an extglob pattern +# suitable for use in case statements. +__docker_compose_to_extglob() { + local extglob=$( __docker_compose_to_alternatives "$1" ) + echo "@($extglob)" +} + +# Determines whether the option passed as the first argument exist on +# the commandline. The option may be a pattern, e.g. `--force|-f`. +__docker_compose_has_option() { + local pattern="$1" + for (( i=2; i < $cword; ++i)); do + if [[ ${words[$i]} =~ ^($pattern)$ ]] ; then + return 0 + fi + done + return 1 +} + +# Returns `key` if we are currently completing the value of a map option (`key=value`) +# which matches the extglob passed in as an argument. +# This function is needed for key-specific completions. +__docker_compose_map_key_of_current_option() { + local glob="$1" + + local key glob_pos + if [ "$cur" = "=" ] ; then # key= case + key="$prev" + glob_pos=$((cword - 2)) + elif [[ $cur == *=* ]] ; then # key=value case (OSX) + key=${cur%=*} + glob_pos=$((cword - 1)) + elif [ "$prev" = "=" ] ; then + key=${words[$cword - 2]} # key=value case + glob_pos=$((cword - 3)) + else + return + fi + + [ "${words[$glob_pos]}" = "=" ] && ((glob_pos--)) # --option=key=value syntax + + [[ ${words[$glob_pos]} == @($glob) ]] && echo "$key" +} + +# suppress trailing whitespace +__docker_compose_nospace() { + # compopt is not available in ancient bash versions + type compopt &>/dev/null && compopt -o nospace +} + + +# Outputs a list of all defined services, regardless of their running state. +# Arguments for `docker-compose ps` may be passed in order to filter the service list, +# e.g. `status=running`. +__docker_compose_services() { + __docker_compose_q ps --services "$@" +} + +# Applies completion of services based on the current value of `$cur`. +# Arguments for `docker-compose ps` may be passed in order to filter the service list, +# see `__docker_compose_services`. +__docker_compose_complete_services() { + COMPREPLY=( $(compgen -W "$(__docker_compose_services "$@")" -- "$cur") ) +} + +# The services for which at least one running container exists +__docker_compose_complete_running_services() { + local names=$(__docker_compose_services --filter status=running) + COMPREPLY=( $(compgen -W "$names" -- "$cur") ) +} + + +_docker_compose_build() { + case "$prev" in + --build-arg) + COMPREPLY=( $( compgen -e -- "$cur" ) ) + __docker_compose_nospace + return + ;; + --memory|-m) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--build-arg --compress --force-rm --help --memory -m --no-cache --no-rm --pull --parallel -q --quiet" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services --filter source=build + ;; + esac +} + + +_docker_compose_config() { + case "$prev" in + --hash) + if [[ $cur == \\* ]] ; then + COMPREPLY=( '\*' ) + else + COMPREPLY=( $(compgen -W "$(__docker_compose_services) \\\* " -- "$cur") ) + fi + return + ;; + esac + + COMPREPLY=( $( compgen -W "--hash --help --no-interpolate --quiet -q --resolve-image-digests --services --volumes" -- "$cur" ) ) +} + + +_docker_compose_create() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--build --force-recreate --help --no-build --no-recreate" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_docker_compose() { + case "$prev" in + --tlscacert|--tlscert|--tlskey) + _filedir + return + ;; + --file|-f) + _filedir "y?(a)ml" + return + ;; + --log-level) + COMPREPLY=( $( compgen -W "debug info warning error critical" -- "$cur" ) ) + return + ;; + --project-directory) + _filedir -d + return + ;; + --env-file) + _filedir + return + ;; + $(__docker_compose_to_extglob "$daemon_options_with_args") ) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "$daemon_boolean_options $daemon_options_with_args $top_level_options_with_args --help -h --no-ansi --verbose --version -v" -- "$cur" ) ) + ;; + *) + COMPREPLY=( $( compgen -W "${commands[*]}" -- "$cur" ) ) + ;; + esac +} + + +_docker_compose_down() { + case "$prev" in + --rmi) + COMPREPLY=( $( compgen -W "all local" -- "$cur" ) ) + return + ;; + --timeout|-t) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --rmi --timeout -t --volumes -v --remove-orphans" -- "$cur" ) ) + ;; + esac +} + + +_docker_compose_events() { + case "$prev" in + --json) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --json" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_exec() { + case "$prev" in + --index|--user|-u|--workdir|-w) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "-d --detach --help --index --privileged -T --user -u --workdir -w" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_running_services + ;; + esac +} + + +_docker_compose_help() { + COMPREPLY=( $( compgen -W "${commands[*]}" -- "$cur" ) ) +} + +_docker_compose_images() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --quiet -q" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + +_docker_compose_kill() { + case "$prev" in + -s) + COMPREPLY=( $( compgen -W "SIGHUP SIGINT SIGKILL SIGUSR1 SIGUSR2" -- "$(echo $cur | tr '[:lower:]' '[:upper:]')" ) ) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help -s" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_running_services + ;; + esac +} + + +_docker_compose_logs() { + case "$prev" in + --tail) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--follow -f --help --no-color --tail --timestamps -t" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_pause() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_running_services + ;; + esac +} + + +_docker_compose_port() { + case "$prev" in + --protocol) + COMPREPLY=( $( compgen -W "tcp udp" -- "$cur" ) ) + return; + ;; + --index) + return; + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --index --protocol" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_ps() { + local key=$(__docker_compose_map_key_of_current_option '--filter') + case "$key" in + source) + COMPREPLY=( $( compgen -W "build image" -- "${cur##*=}" ) ) + return + ;; + status) + COMPREPLY=( $( compgen -W "paused restarting running stopped" -- "${cur##*=}" ) ) + return + ;; + esac + + case "$prev" in + --filter) + COMPREPLY=( $( compgen -W "source status" -S "=" -- "$cur" ) ) + __docker_compose_nospace + return; + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--all -a --filter --help --quiet -q --services" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_pull() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --ignore-pull-failures --include-deps --no-parallel --quiet -q" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services --filter source=image + ;; + esac +} + + +_docker_compose_push() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --ignore-push-failures" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_restart() { + case "$prev" in + --timeout|-t) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --timeout -t" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_running_services + ;; + esac +} + + +_docker_compose_rm() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--force -f --help --stop -s -v" -- "$cur" ) ) + ;; + *) + if __docker_compose_has_option "--stop|-s" ; then + __docker_compose_complete_services + else + __docker_compose_complete_services --filter status=stopped + fi + ;; + esac +} + + +_docker_compose_run() { + case "$prev" in + -e) + COMPREPLY=( $( compgen -e -- "$cur" ) ) + __docker_compose_nospace + return + ;; + --entrypoint|--label|-l|--name|--user|-u|--volume|-v|--workdir|-w) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--detach -d --entrypoint -e --help --label -l --name --no-deps --publish -p --rm --service-ports -T --use-aliases --user -u --volume -v --workdir -w" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_scale() { + case "$prev" in + =) + COMPREPLY=("$cur") + return + ;; + --timeout|-t) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --timeout -t" -- "$cur" ) ) + ;; + *) + COMPREPLY=( $(compgen -S "=" -W "$(__docker_compose_services)" -- "$cur") ) + __docker_compose_nospace + ;; + esac +} + + +_docker_compose_start() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services --filter status=stopped + ;; + esac +} + + +_docker_compose_stop() { + case "$prev" in + --timeout|-t) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help --timeout -t" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_running_services + ;; + esac +} + + +_docker_compose_top() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_running_services + ;; + esac +} + + +_docker_compose_unpause() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--help" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services --filter status=paused + ;; + esac +} + + +_docker_compose_up() { + case "$prev" in + =) + COMPREPLY=("$cur") + return + ;; + --exit-code-from) + __docker_compose_complete_services + return + ;; + --scale) + COMPREPLY=( $(compgen -S "=" -W "$(__docker_compose_services)" -- "$cur") ) + __docker_compose_nospace + return + ;; + --timeout|-t) + return + ;; + esac + + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--abort-on-container-exit --always-recreate-deps --attach-dependencies --build -d --detach --exit-code-from --force-recreate --help --no-build --no-color --no-deps --no-recreate --no-start --renew-anon-volumes -V --remove-orphans --scale --timeout -t" -- "$cur" ) ) + ;; + *) + __docker_compose_complete_services + ;; + esac +} + + +_docker_compose_version() { + case "$cur" in + -*) + COMPREPLY=( $( compgen -W "--short" -- "$cur" ) ) + ;; + esac +} + + +_docker_compose() { + local previous_extglob_setting=$(shopt -p extglob) + shopt -s extglob + + local commands=( + build + config + create + down + events + exec + help + images + kill + logs + pause + port + ps + pull + push + restart + rm + run + scale + start + stop + top + unpause + up + version + ) + + # Options for the docker daemon that have to be passed to secondary calls to + # docker-compose executed by this script. + local daemon_boolean_options=" + --skip-hostname-check + --tls + --tlsverify + " + local daemon_options_with_args=" + --context -c + --env-file + --file -f + --host -H + --project-directory + --project-name -p + --tlscacert + --tlscert + --tlskey + " + + # These options are require special treatment when searching the command. + local top_level_options_with_args=" + --log-level + " + + COMPREPLY=() + local cur prev words cword + _get_comp_words_by_ref -n : cur prev words cword + + # search subcommand and invoke its handler. + # special treatment of some top-level options + local command='docker_compose' + local top_level_options=() + local counter=1 + + while [ $counter -lt $cword ]; do + case "${words[$counter]}" in + $(__docker_compose_to_extglob "$daemon_boolean_options") ) + local opt=${words[counter]} + top_level_options+=($opt) + ;; + $(__docker_compose_to_extglob "$daemon_options_with_args") ) + local opt=${words[counter]} + local arg=${words[++counter]} + top_level_options+=($opt $arg) + ;; + $(__docker_compose_to_extglob "$top_level_options_with_args") ) + (( counter++ )) + ;; + -*) + ;; + *) + command="${words[$counter]}" + break + ;; + esac + (( counter++ )) + done + + local completions_func=_docker_compose_${command//-/_} + declare -F $completions_func >/dev/null && $completions_func + + eval "$previous_extglob_setting" + return 0 +} + +eval "$__docker_compose_previous_extglob_setting" +unset __docker_compose_previous_extglob_setting + +complete -F _docker_compose docker-compose docker-compose.exe diff --git a/contrib/completion/fish/docker-compose.fish b/contrib/completion/fish/docker-compose.fish new file mode 100644 index 00000000000..0566e16ae88 --- /dev/null +++ b/contrib/completion/fish/docker-compose.fish @@ -0,0 +1,25 @@ +# Tab completion for docker-compose (https://github.com/docker/compose). +# Version: 1.9.0 + +complete -e -c docker-compose + +for line in (docker-compose --help | \ + string match -r '^\s+\w+\s+[^\n]+' | \ + string trim) + set -l doc (string split -m 1 ' ' -- $line) + complete -c docker-compose -n '__fish_use_subcommand' -xa $doc[1] --description $doc[2] +end + +complete -c docker-compose -s f -l file -r -d 'Specify an alternate compose file' +complete -c docker-compose -s p -l project-name -x -d 'Specify an alternate project name' +complete -c docker-compose -l env-file -r -d 'Specify an alternate environment file (default: .env)' +complete -c docker-compose -l verbose -d 'Show more output' +complete -c docker-compose -s H -l host -x -d 'Daemon socket to connect to' +complete -c docker-compose -l tls -d 'Use TLS; implied by --tlsverify' +complete -c docker-compose -l tlscacert -r -d 'Trust certs signed only by this CA' +complete -c docker-compose -l tlscert -r -d 'Path to TLS certificate file' +complete -c docker-compose -l tlskey -r -d 'Path to TLS key file' +complete -c docker-compose -l tlsverify -d 'Use TLS and verify the remote' +complete -c docker-compose -l skip-hostname-check -d "Don't check the daemon's hostname against the name specified in the client certificate (for example if your docker host is an IP address)" +complete -c docker-compose -s h -l help -d 'Print usage' +complete -c docker-compose -s v -l version -d 'Print version and exit' diff --git a/contrib/completion/zsh/_docker-compose b/contrib/completion/zsh/_docker-compose new file mode 100755 index 00000000000..de14149844a --- /dev/null +++ b/contrib/completion/zsh/_docker-compose @@ -0,0 +1,420 @@ +#compdef docker-compose + +# Description +# ----------- +# zsh completion for docker-compose +# ------------------------------------------------------------------------- +# Authors +# ------- +# * Steve Durrheimer <s.durrheimer@gmail.com> +# ------------------------------------------------------------------------- +# Inspiration +# ----------- +# * @albers docker-compose bash completion script +# * @felixr docker zsh completion script : https://github.com/felixr/docker-zsh-completion +# ------------------------------------------------------------------------- + +__docker-compose_q() { + docker-compose 2>/dev/null $compose_options "$@" +} + +# All services defined in docker-compose.yml +__docker-compose_all_services_in_compose_file() { + local already_selected + local -a services + already_selected=$(echo $words | tr " " "|") + __docker-compose_q ps --services "$@" \ + | grep -Ev "^(${already_selected})$" +} + +# All services, even those without an existing container +__docker-compose_services_all() { + [[ $PREFIX = -* ]] && return 1 + integer ret=1 + services=$(__docker-compose_all_services_in_compose_file "$@") + _alternative "args:services:($services)" && ret=0 + + return ret +} + +# All services that are defined by a Dockerfile reference +__docker-compose_services_from_build() { + [[ $PREFIX = -* ]] && return 1 + __docker-compose_services_all --filter source=build +} + +# All services that are defined by an image +__docker-compose_services_from_image() { + [[ $PREFIX = -* ]] && return 1 + __docker-compose_services_all --filter source=image +} + +__docker-compose_pausedservices() { + [[ $PREFIX = -* ]] && return 1 + __docker-compose_services_all --filter status=paused +} + +__docker-compose_stoppedservices() { + [[ $PREFIX = -* ]] && return 1 + __docker-compose_services_all --filter status=stopped +} + +__docker-compose_runningservices() { + [[ $PREFIX = -* ]] && return 1 + __docker-compose_services_all --filter status=running +} + +__docker-compose_services() { + [[ $PREFIX = -* ]] && return 1 + __docker-compose_services_all +} + +__docker-compose_caching_policy() { + oldp=( "$1"(Nmh+1) ) # 1 hour + (( $#oldp )) +} + +__docker-compose_commands() { + local cache_policy + + zstyle -s ":completion:${curcontext}:" cache-policy cache_policy + if [[ -z "$cache_policy" ]]; then + zstyle ":completion:${curcontext}:" cache-policy __docker-compose_caching_policy + fi + + if ( [[ ${+_docker_compose_subcommands} -eq 0 ]] || _cache_invalid docker_compose_subcommands) \ + && ! _retrieve_cache docker_compose_subcommands; + then + local -a lines + lines=(${(f)"$(_call_program commands docker-compose 2>&1)"}) + _docker_compose_subcommands=(${${${lines[$((${lines[(i)Commands:]} + 1)),${lines[(I) *]}]}## #}/ ##/:}) + (( $#_docker_compose_subcommands > 0 )) && _store_cache docker_compose_subcommands _docker_compose_subcommands + fi + _describe -t docker-compose-commands "docker-compose command" _docker_compose_subcommands +} + +__docker-compose_subcommand() { + local opts_help opts_force_recreate opts_no_recreate opts_no_build opts_remove_orphans opts_timeout opts_no_color opts_no_deps + + opts_help='(: -)--help[Print usage]' + opts_force_recreate="(--no-recreate)--force-recreate[Recreate containers even if their configuration and image haven't changed. Incompatible with --no-recreate.]" + opts_no_recreate="(--force-recreate)--no-recreate[If containers already exist, don't recreate them. Incompatible with --force-recreate.]" + opts_no_build="(--build)--no-build[Don't build an image, even if it's missing.]" + opts_remove_orphans="--remove-orphans[Remove containers for services not defined in the Compose file]" + opts_timeout=('(-t --timeout)'{-t,--timeout}"[Specify a shutdown timeout in seconds. (default: 10)]:seconds: ") + opts_no_color='--no-color[Produce monochrome output.]' + opts_no_deps="--no-deps[Don't start linked services.]" + + integer ret=1 + + case "$words[1]" in + (build) + _arguments \ + $opts_help \ + "*--build-arg=[Set build-time variables for one service.]:<varname>=<value>: " \ + '--force-rm[Always remove intermediate containers.]' \ + '(--quiet -q)'{--quiet,-q}'[Curb build output]' \ + '(--memory -m)'{--memory,-m}'[Memory limit for the build container.]' \ + '--no-cache[Do not use cache when building the image.]' \ + '--pull[Always attempt to pull a newer version of the image.]' \ + '--compress[Compress the build context using gzip.]' \ + '--parallel[Build images in parallel.]' \ + '*:services:__docker-compose_services_from_build' && ret=0 + ;; + (config) + _arguments \ + $opts_help \ + '(--quiet -q)'{--quiet,-q}"[Only validate the configuration, don't print anything.]" \ + '--resolve-image-digests[Pin image tags to digests.]' \ + '--services[Print the service names, one per line.]' \ + '--volumes[Print the volume names, one per line.]' \ + '--hash[Print the service config hash, one per line. Set "service1,service2" for a list of specified services.]' \ && ret=0 + ;; + (create) + _arguments \ + $opts_help \ + $opts_force_recreate \ + $opts_no_recreate \ + $opts_no_build \ + "(--no-build)--build[Build images before creating containers.]" \ + '*:services:__docker-compose_services' && ret=0 + ;; + (down) + _arguments \ + $opts_help \ + $opts_timeout \ + "--rmi[Remove images. Type must be one of: 'all': Remove all images used by any service. 'local': Remove only images that don't have a custom tag set by the \`image\` field.]:type:(all local)" \ + '(-v --volumes)'{-v,--volumes}"[Remove named volumes declared in the \`volumes\` section of the Compose file and anonymous volumes attached to containers.]" \ + $opts_remove_orphans && ret=0 + ;; + (events) + _arguments \ + $opts_help \ + '--json[Output events as a stream of json objects]' \ + '*:services:__docker-compose_services' && ret=0 + ;; + (exec) + _arguments \ + $opts_help \ + '-d[Detached mode: Run command in the background.]' \ + '--privileged[Give extended privileges to the process.]' \ + '(-u --user)'{-u,--user=}'[Run the command as this user.]:username:_users' \ + '-T[Disable pseudo-tty allocation. By default `docker-compose exec` allocates a TTY.]' \ + '--index=[Index of the container if there are multiple instances of a service \[default: 1\]]:index: ' \ + '*'{-e,--env}'[KEY=VAL Set an environment variable (can be used multiple times)]:environment variable KEY=VAL: ' \ + '(-w --workdir)'{-w,--workdir=}'[Working directory inside the container]:workdir: ' \ + '(-):running services:__docker-compose_runningservices' \ + '(-):command: _command_names -e' \ + '*::arguments: _normal' && ret=0 + ;; + (help) + _arguments ':subcommand:__docker-compose_commands' && ret=0 + ;; + (images) + _arguments \ + $opts_help \ + '-q[Only display IDs]' \ + '*:services:__docker-compose_services' && ret=0 + ;; + (kill) + _arguments \ + $opts_help \ + '-s[SIGNAL to send to the container. Default signal is SIGKILL.]:signal:_signals' \ + '*:running services:__docker-compose_runningservices' && ret=0 + ;; + (logs) + _arguments \ + $opts_help \ + '(-f --follow)'{-f,--follow}'[Follow log output]' \ + $opts_no_color \ + '--tail=[Number of lines to show from the end of the logs for each container.]:number of lines: ' \ + '(-t --timestamps)'{-t,--timestamps}'[Show timestamps]' \ + '*:services:__docker-compose_services' && ret=0 + ;; + (pause) + _arguments \ + $opts_help \ + '*:running services:__docker-compose_runningservices' && ret=0 + ;; + (port) + _arguments \ + $opts_help \ + '--protocol=[tcp or udp \[default: tcp\]]:protocol:(tcp udp)' \ + '--index=[index of the container if there are multiple instances of a service \[default: 1\]]:index: ' \ + '1:running services:__docker-compose_runningservices' \ + '2:port:_ports' && ret=0 + ;; + (ps) + _arguments \ + $opts_help \ + '-q[Only display IDs]' \ + '--filter KEY=VAL[Filter services by a property]:<filtername>=<value>:' \ + '*:services:__docker-compose_services' && ret=0 + ;; + (pull) + _arguments \ + $opts_help \ + '--ignore-pull-failures[Pull what it can and ignores images with pull failures.]' \ + '--no-parallel[Disable parallel pulling]' \ + '(-q --quiet)'{-q,--quiet}'[Pull without printing progress information]' \ + '--include-deps[Also pull services declared as dependencies]' \ + '*:services:__docker-compose_services_from_image' && ret=0 + ;; + (push) + _arguments \ + $opts_help \ + '--ignore-push-failures[Push what it can and ignores images with push failures.]' \ + '*:services:__docker-compose_services' && ret=0 + ;; + (rm) + _arguments \ + $opts_help \ + '(-f --force)'{-f,--force}"[Don't ask to confirm removal]" \ + '-v[Remove any anonymous volumes attached to containers]' \ + '*:stopped services:__docker-compose_stoppedservices' && ret=0 + ;; + (run) + _arguments \ + $opts_help \ + $opts_no_deps \ + '-d[Detached mode: Run container in the background, print new container name.]' \ + '*-e[KEY=VAL Set an environment variable (can be used multiple times)]:environment variable KEY=VAL: ' \ + '*'{-l,--label}'[KEY=VAL Add or override a label (can be used multiple times)]:label KEY=VAL: ' \ + '--entrypoint[Overwrite the entrypoint of the image.]:entry point: ' \ + '--name=[Assign a name to the container]:name: ' \ + '(-p --publish)'{-p,--publish=}"[Publish a container's port(s) to the host]" \ + '--rm[Remove container after run. Ignored in detached mode.]' \ + "--service-ports[Run command with the service's ports enabled and mapped to the host.]" \ + '-T[Disable pseudo-tty allocation. By default `docker-compose run` allocates a TTY.]' \ + '(-u --user)'{-u,--user=}'[Run as specified username or uid]:username or uid:_users' \ + '(-v --volume)*'{-v,--volume=}'[Bind mount a volume]:volume: ' \ + '(-w --workdir)'{-w,--workdir=}'[Working directory inside the container]:workdir: ' \ + "--use-aliases[Use the services network aliases in the network(s) the container connects to]" \ + '(-):services:__docker-compose_services' \ + '(-):command: _command_names -e' \ + '*::arguments: _normal' && ret=0 + ;; + (scale) + _arguments \ + $opts_help \ + $opts_timeout \ + '*:running services:__docker-compose_runningservices' && ret=0 + ;; + (start) + _arguments \ + $opts_help \ + '*:stopped services:__docker-compose_stoppedservices' && ret=0 + ;; + (stop|restart) + _arguments \ + $opts_help \ + $opts_timeout \ + '*:running services:__docker-compose_runningservices' && ret=0 + ;; + (top) + _arguments \ + $opts_help \ + '*:running services:__docker-compose_runningservices' && ret=0 + ;; + (unpause) + _arguments \ + $opts_help \ + '*:paused services:__docker-compose_pausedservices' && ret=0 + ;; + (up) + _arguments \ + $opts_help \ + '(--abort-on-container-exit)-d[Detached mode: Run containers in the background, print new container names. Incompatible with --abort-on-container-exit and --attach-dependencies.]' \ + $opts_no_color \ + $opts_no_deps \ + $opts_force_recreate \ + $opts_no_recreate \ + $opts_no_build \ + "(--no-build)--build[Build images before starting containers.]" \ + "(-d)--abort-on-container-exit[Stops all containers if any container was stopped. Incompatible with -d.]" \ + "(-d)--attach-dependencies[Attach to dependent containers. Incompatible with -d.]" \ + '(-t --timeout)'{-t,--timeout}"[Use this timeout in seconds for container shutdown when attached or when containers are already running. (default: 10)]:seconds: " \ + '--scale[SERVICE=NUM Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present.]:service scale SERVICE=NUM: ' \ + '--exit-code-from=[Return the exit code of the selected service container. Implies --abort-on-container-exit]:service:__docker-compose_services' \ + $opts_remove_orphans \ + '*:services:__docker-compose_services' && ret=0 + ;; + (version) + _arguments \ + $opts_help \ + "--short[Shows only Compose's version number.]" && ret=0 + ;; + (*) + _message 'Unknown sub command' && ret=1 + ;; + esac + + return ret +} + +_docker-compose() { + # Support for subservices, which allows for `compdef _docker docker-shell=_docker_containers`. + # Based on /usr/share/zsh/functions/Completion/Unix/_git without support for `ret`. + if [[ $service != docker-compose ]]; then + _call_function - _$service + return + fi + + local curcontext="$curcontext" state line + integer ret=1 + typeset -A opt_args + + local file_description + + if [[ -n ${words[(r)-f]} || -n ${words[(r)--file]} ]] ; then + file_description="Specify an override docker-compose file (default: docker-compose.override.yml)" + else + file_description="Specify an alternate docker-compose file (default: docker-compose.yml)" + fi + + _arguments -C \ + '(- :)'{-h,--help}'[Get help]' \ + '*'{-f,--file}"[${file_description}]:file:_files -g '*.yml'" \ + '(-p --project-name)'{-p,--project-name}'[Specify an alternate project name (default: directory name)]:project name:' \ + '--env-file[Specify an alternate environment file (default: .env)]:env-file:_files' \ + "--compatibility[If set, Compose will attempt to convert keys in v3 files to their non-Swarm equivalent]" \ + '(- :)'{-v,--version}'[Print version and exit]' \ + '--verbose[Show more output]' \ + '--log-level=[Set log level]:level:(DEBUG INFO WARNING ERROR CRITICAL)' \ + '--no-ansi[Do not print ANSI control characters]' \ + '(-H --host)'{-H,--host}'[Daemon socket to connect to]:host:' \ + '--tls[Use TLS; implied by --tlsverify]' \ + '--tlscacert=[Trust certs signed only by this CA]:ca path:' \ + '--tlscert=[Path to TLS certificate file]:client cert path:' \ + '--tlskey=[Path to TLS key file]:tls key path:' \ + '--tlsverify[Use TLS and verify the remote]' \ + "--skip-hostname-check[Don't check the daemon's hostname against the name specified in the client certificate (for example if your docker host is an IP address)]" \ + '(-): :->command' \ + '(-)*:: :->option-or-argument' && ret=0 + + local -a relevant_compose_flags relevant_compose_repeatable_flags relevant_docker_flags compose_options docker_options + + relevant_compose_flags=( + "--env-file" + "--file" "-f" + "--host" "-H" + "--project-name" "-p" + "--tls" + "--tlscacert" + "--tlscert" + "--tlskey" + "--tlsverify" + "--skip-hostname-check" + ) + + relevant_compose_repeatable_flags=( + "--file" "-f" + ) + + relevant_docker_flags=( + "--host" "-H" + "--tls" + "--tlscacert" + "--tlscert" + "--tlskey" + "--tlsverify" + ) + + for k in "${(@k)opt_args}"; do + if [[ -n "${relevant_docker_flags[(r)$k]}" ]]; then + docker_options+=$k + if [[ -n "$opt_args[$k]" ]]; then + docker_options+=$opt_args[$k] + fi + fi + if [[ -n "${relevant_compose_flags[(r)$k]}" ]]; then + if [[ -n "${relevant_compose_repeatable_flags[(r)$k]}" ]]; then + values=("${(@s/:/)opt_args[$k]}") + for value in $values + do + compose_options+=$k + compose_options+=$value + done + else + compose_options+=$k + if [[ -n "$opt_args[$k]" ]]; then + compose_options+=$opt_args[$k] + fi + fi + fi + done + + case $state in + (command) + __docker-compose_commands && ret=0 + ;; + (option-or-argument) + curcontext=${curcontext%:*:*}:docker-compose-$words[1]: + __docker-compose_subcommand && ret=0 + ;; + esac + + return ret +} + +_docker-compose "$@" diff --git a/contrib/migration/migrate-compose-file-v1-to-v2.py b/contrib/migration/migrate-compose-file-v1-to-v2.py new file mode 100755 index 00000000000..26511206c5f --- /dev/null +++ b/contrib/migration/migrate-compose-file-v1-to-v2.py @@ -0,0 +1,170 @@ +#!/usr/bin/env python +""" +Migrate a Compose file from the V1 format in Compose 1.5 to the V2 format +supported by Compose 1.6+ +""" +import argparse +import logging +import sys + +import ruamel.yaml + +from compose.config.types import VolumeSpec + + +log = logging.getLogger('migrate') + + +def migrate(content): + data = ruamel.yaml.load(content, ruamel.yaml.RoundTripLoader) + + service_names = data.keys() + + for name, service in data.items(): + warn_for_links(name, service) + warn_for_external_links(name, service) + rewrite_net(service, service_names) + rewrite_build(service) + rewrite_logging(service) + rewrite_volumes_from(service, service_names) + + services = {name: data.pop(name) for name in data.keys()} + + data['version'] = "2" + data['services'] = services + create_volumes_section(data) + + return data + + +def warn_for_links(name, service): + links = service.get('links') + if links: + example_service = links[0].partition(':')[0] + log.warning( + "Service {name} has links, which no longer create environment " + "variables such as {example_service_upper}_PORT. " + "If you are using those in your application code, you should " + "instead connect directly to the hostname, e.g. " + "'{example_service}'." + .format(name=name, example_service=example_service, + example_service_upper=example_service.upper())) + + +def warn_for_external_links(name, service): + external_links = service.get('external_links') + if external_links: + log.warning( + "Service {name} has external_links: {ext}, which now work " + "slightly differently. In particular, two containers must be " + "connected to at least one network in common in order to " + "communicate, even if explicitly linked together.\n\n" + "Either connect the external container to your app's default " + "network, or connect both the external container and your " + "service's containers to a pre-existing network. See " + "https://docs.docker.com/compose/networking/ " + "for more on how to do this." + .format(name=name, ext=external_links)) + + +def rewrite_net(service, service_names): + if 'net' in service: + network_mode = service.pop('net') + + # "container:<service name>" is now "service:<service name>" + if network_mode.startswith('container:'): + name = network_mode.partition(':')[2] + if name in service_names: + network_mode = 'service:{}'.format(name) + + service['network_mode'] = network_mode + + +def rewrite_build(service): + if 'dockerfile' in service: + service['build'] = { + 'context': service.pop('build'), + 'dockerfile': service.pop('dockerfile'), + } + + +def rewrite_logging(service): + if 'log_driver' in service: + service['logging'] = {'driver': service.pop('log_driver')} + if 'log_opt' in service: + service['logging']['options'] = service.pop('log_opt') + + +def rewrite_volumes_from(service, service_names): + for idx, volume_from in enumerate(service.get('volumes_from', [])): + if volume_from.split(':', 1)[0] not in service_names: + service['volumes_from'][idx] = 'container:%s' % volume_from + + +def create_volumes_section(data): + named_volumes = get_named_volumes(data['services']) + if named_volumes: + log.warning( + "Named volumes ({names}) must be explicitly declared. Creating a " + "'volumes' section with declarations.\n\n" + "For backwards-compatibility, they've been declared as external. " + "If you don't mind the volume names being prefixed with the " + "project name, you can remove the 'external' option from each one." + .format(names=', '.join(list(named_volumes)))) + + data['volumes'] = named_volumes + + +def get_named_volumes(services): + volume_specs = [ + VolumeSpec.parse(volume) + for service in services.values() + for volume in service.get('volumes', []) + ] + names = { + spec.external + for spec in volume_specs + if spec.is_named_volume + } + return {name: {'external': True} for name in names} + + +def write(stream, new_format, indent, width): + ruamel.yaml.dump( + new_format, + stream, + Dumper=ruamel.yaml.RoundTripDumper, + indent=indent, + width=width) + + +def parse_opts(args): + parser = argparse.ArgumentParser() + parser.add_argument("filename", help="Compose file filename.") + parser.add_argument("-i", "--in-place", action='store_true') + parser.add_argument( + "--indent", type=int, default=2, + help="Number of spaces used to indent the output yaml.") + parser.add_argument( + "--width", type=int, default=80, + help="Number of spaces used as the output width.") + return parser.parse_args() + + +def main(args): + logging.basicConfig(format='\033[33m%(levelname)s:\033[37m %(message)s\033[0m\n') + + opts = parse_opts(args) + + with open(opts.filename) as fh: + new_format = migrate(fh.read()) + + if opts.in_place: + output = open(opts.filename, 'w') + else: + output = sys.stdout + write(output, new_format, opts.indent, opts.width) + + +if __name__ == "__main__": + main(sys.argv) diff --git a/contrib/update/update-docker-compose.ps1 b/contrib/update/update-docker-compose.ps1 new file mode 100644 index 00000000000..bb033b46467 --- /dev/null +++ b/contrib/update/update-docker-compose.ps1 @@ -0,0 +1,116 @@ +# Self-elevate the script if required +# http://www.expta.com/2017/03/how-to-self-elevate-powershell-script.html +If (-Not ([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] 'Administrator')) { + If ([int](Get-CimInstance -Class Win32_OperatingSystem | Select-Object -ExpandProperty BuildNumber) -ge 6000) { + $CommandLine = "-File `"" + $MyInvocation.MyCommand.Path + "`" " + $MyInvocation.UnboundArguments + Start-Process -FilePath PowerShell.exe -Verb Runas -ArgumentList $CommandLine + Exit + } +} + +$SectionSeparator = "--------------------------------------------------" + +# Update docker-compose if required +Function UpdateDockerCompose() { + Write-Host "Updating docker-compose if required..." + Write-Host $SectionSeparator + + # Find the installed docker-compose.exe location + Try { + $DockerComposePath = Get-Command docker-compose.exe -ErrorAction Stop | ` + Select-Object -First 1 -ExpandProperty Definition + } + Catch { + Write-Host "Error: Could not find path to docker-compose.exe" ` + -ForegroundColor Red + Return $false + } + + # Prefer/enable TLS 1.2 + # https://stackoverflow.com/a/48030563/153079 + [Net.ServicePointManager]::SecurityProtocol = "tls12, tls11, tls" + + # Query for the latest release version + Try { + $URI = "https://api.github.com/repos/docker/compose/releases/latest" + $LatestComposeVersion = [System.Version](Invoke-RestMethod -Method Get -Uri $URI).tag_name + } + Catch { + Write-Host "Error: Query for the latest docker-compose release version failed" ` + -ForegroundColor Red + Return $false + } + + # Check the installed version and compare with latest release + $UpdateDockerCompose = $false + Try { + $InstalledComposeVersion = ` + [System.Version]((docker-compose.exe version --short) | Out-String) + + If ($InstalledComposeVersion -eq $LatestComposeVersion) { + Write-Host ("Installed docker-compose version ({0}) same as latest ({1})." ` + -f $InstalledComposeVersion.ToString(), $LatestComposeVersion.ToString()) + } + ElseIf ($InstalledComposeVersion -lt $LatestComposeVersion) { + Write-Host ("Installed docker-compose version ({0}) older than latest ({1})." ` + -f $InstalledComposeVersion.ToString(), $LatestComposeVersion.ToString()) + $UpdateDockerCompose = $true + } + Else { + Write-Host ("Installed docker-compose version ({0}) newer than latest ({1})." ` + -f $InstalledComposeVersion.ToString(), $LatestComposeVersion.ToString()) ` + -ForegroundColor Yellow + } + } + Catch { + Write-Host ` + "Warning: Couldn't get docker-compose version, assuming an update is required..." ` + -ForegroundColor Yellow + $UpdateDockerCompose = $true + } + + If (-Not $UpdateDockerCompose) { + # Nothing to do! + Return $false + } + + # Download the latest version of docker-compose.exe + Try { + $RemoteFileName = "docker-compose-Windows-x86_64.exe" + $URI = ("https://github.com/docker/compose/releases/download/{0}/{1}" ` + -f $LatestComposeVersion.ToString(), $RemoteFileName) + Invoke-WebRequest -UseBasicParsing -Uri $URI ` + -OutFile $DockerComposePath + Return $true + } + Catch { + Write-Host ("Error: Failed to download the latest version of docker-compose`n{0}" ` + -f $_.Exception.Message) -ForegroundColor Red + Return $false + } + + Return $false +} + +If (UpdateDockerCompose) { + Write-Host "Updated to latest-version of docker-compose, running update again to verify.`n" + If (UpdateDockerCompose) { + Write-Host "Error: Should not have updated twice." -ForegroundColor Red + } +} + +# Assuming elevation popped up a new powershell window, pause so the user can see what happened +# https://stackoverflow.com/a/22362868/153079 +Function Pause ($Message = "Press any key to continue . . . ") { + If ((Test-Path variable:psISE) -and $psISE) { + $Shell = New-Object -ComObject "WScript.Shell" + $Shell.Popup("Click OK to continue.", 0, "Script Paused", 0) + } + Else { + Write-Host "`n$SectionSeparator" + Write-Host -NoNewline $Message + [void][System.Console]::ReadKey($true) + Write-Host + } +} +Pause diff --git a/docker-bake.hcl b/docker-bake.hcl deleted file mode 100644 index 5c6522d3a81..00000000000 --- a/docker-bake.hcl +++ /dev/null @@ -1,148 +0,0 @@ -// Copyright 2022 Docker Compose CLI authors -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -variable "GO_VERSION" { - # default ARG value set in Dockerfile - default = null -} - -variable "BUILD_TAGS" { - default = "e2e" -} - -variable "DOCS_FORMATS" { - default = "md,yaml" -} - -# Defines the output folder to override the default behavior. -# See Makefile for details, this is generally only useful for -# the packaging scripts and care should be taken to not break -# them. -variable "DESTDIR" { - default = "" -} -function "outdir" { - params = [defaultdir] - result = DESTDIR != "" ? DESTDIR : "${defaultdir}" -} - -# Special target: https://github.com/docker/metadata-action#bake-definition -target "meta-helper" {} - -target "_common" { - args = { - GO_VERSION = GO_VERSION - BUILD_TAGS = BUILD_TAGS - BUILDKIT_CONTEXT_KEEP_GIT_DIR = 1 - } -} - -group "default" { - targets = ["binary"] -} - -group "validate" { - targets = ["lint", "vendor-validate", "license-validate"] -} - -target "lint" { - inherits = ["_common"] - target = "lint" - output = ["type=cacheonly"] -} - -target "license-validate" { - target = "license-validate" - output = ["type=cacheonly"] -} - -target "license-update" { - target = "license-update" - output = ["."] -} - -target "vendor-validate" { - inherits = ["_common"] - target = "vendor-validate" - output = ["type=cacheonly"] -} - -target "vendor-update" { - inherits = ["_common"] - target = "vendor-update" - output = ["."] -} - -target "test" { - inherits = ["_common"] - target = "test-coverage" - output = [outdir("./bin/coverage/unit")] -} - -target "binary-with-coverage" { - inherits = ["_common"] - target = "binary" - args = { - BUILD_FLAGS = "-cover -covermode=atomic" - } - output = [outdir("./bin/build")] - platforms = ["local"] -} - -target "binary" { - inherits = ["_common"] - target = "binary" - output = [outdir("./bin/build")] - platforms = ["local"] -} - -target "binary-cross" { - inherits = ["binary"] - platforms = [ - "darwin/amd64", - "darwin/arm64", - "linux/amd64", - "linux/arm/v6", - "linux/arm/v7", - "linux/arm64", - "linux/ppc64le", - "linux/riscv64", - "linux/s390x", - "windows/amd64", - "windows/arm64" - ] -} - -target "release" { - inherits = ["binary-cross"] - target = "release" - output = [outdir("./bin/release")] -} - -target "docs-validate" { - inherits = ["_common"] - target = "docs-validate" - output = ["type=cacheonly"] -} - -target "docs-update" { - inherits = ["_common"] - target = "docs-update" - output = ["./docs"] -} - -target "image-cross" { - inherits = ["meta-helper", "binary-cross"] - output = ["type=image"] -} diff --git a/docker-compose-entrypoint.sh b/docker-compose-entrypoint.sh new file mode 100755 index 00000000000..84436fa0778 --- /dev/null +++ b/docker-compose-entrypoint.sh @@ -0,0 +1,20 @@ +#!/bin/sh +set -e + +# first arg is `-f` or `--some-option` +if [ "${1#-}" != "$1" ]; then + set -- docker-compose "$@" +fi + +# if our command is a valid Docker subcommand, let's invoke it through Docker instead +# (this allows for "docker run docker ps", etc) +if docker-compose help "$1" > /dev/null 2>&1; then + set -- docker-compose "$@" +fi + +# if we have "--link some-docker:docker" and not DOCKER_HOST, let's set DOCKER_HOST automatically +if [ -z "$DOCKER_HOST" -a "$DOCKER_PORT_2375_TCP" ]; then + export DOCKER_HOST='tcp://docker:2375' +fi + +exec "$@" diff --git a/docker-compose.spec b/docker-compose.spec new file mode 100644 index 00000000000..0c2fa3dec9f --- /dev/null +++ b/docker-compose.spec @@ -0,0 +1,42 @@ +# -*- mode: python -*- + +block_cipher = None + +a = Analysis(['bin/docker-compose'], + pathex=['.'], + hiddenimports=[], + hookspath=None, + runtime_hooks=None, + cipher=block_cipher) + +pyz = PYZ(a.pure, cipher=block_cipher) + +exe = EXE(pyz, + a.scripts, + a.binaries, + a.zipfiles, + a.datas, + [ + ( + 'compose/config/config_schema_v1.json', + 'compose/config/config_schema_v1.json', + 'DATA' + ), + ( + 'compose/config/compose_spec.json', + 'compose/config/compose_spec.json', + 'DATA' + ), + ( + 'compose/GITSHA', + 'compose/GITSHA', + 'DATA' + ) + ], + + name='docker-compose', + debug=False, + strip=None, + upx=True, + console=True, + bootloader_ignore_signals=True) diff --git a/docker-compose_darwin.spec b/docker-compose_darwin.spec new file mode 100644 index 00000000000..24889475990 --- /dev/null +++ b/docker-compose_darwin.spec @@ -0,0 +1,48 @@ +# -*- mode: python -*- + +block_cipher = None + +a = Analysis(['bin/docker-compose'], + pathex=['.'], + hiddenimports=[], + hookspath=[], + runtime_hooks=[], + cipher=block_cipher) + +pyz = PYZ(a.pure, a.zipped_data, + cipher=block_cipher) + +exe = EXE(pyz, + a.scripts, + exclude_binaries=True, + name='docker-compose', + debug=False, + strip=False, + upx=True, + console=True, + bootloader_ignore_signals=True) +coll = COLLECT(exe, + a.binaries, + a.zipfiles, + a.datas, + [ + ( + 'compose/config/config_schema_v1.json', + 'compose/config/config_schema_v1.json', + 'DATA' + ), + ( + 'compose/config/compose_spec.json', + 'compose/config/compose_spec.json', + 'DATA' + ), + ( + 'compose/GITSHA', + 'compose/GITSHA', + 'DATA' + ) + ], + strip=False, + upx=True, + upx_exclude=[], + name='docker-compose-Darwin-x86_64') diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000000..accc7c23e42 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,14 @@ +# The docs have been moved! + +The documentation for Compose has been merged into +[the general documentation repo](https://github.com/docker/docker.github.io). + +The docs for Compose are now here: +https://github.com/docker/docker.github.io/tree/master/compose + +Please submit pull requests for unreleased features/changes on the `master` branch (https://github.com/docker/docker.github.io/tree/master), please prefix the PR title with `[WIP]` to indicate that it relates to an unreleased change. + +If you submit a PR to this codebase that has a docs impact, create a second docs PR on `docker.github.io`. Use the docs PR template provided. + +As always, the docs remain open-source and we appreciate your feedback and +pull requests! diff --git a/docs/examples/provider.go b/docs/examples/provider.go deleted file mode 100644 index 79fd3256eed..00000000000 --- a/docs/examples/provider.go +++ /dev/null @@ -1,152 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package main - -import ( - "encoding/json" - "fmt" - "os" - "time" - - "github.com/spf13/cobra" - "github.com/spf13/pflag" -) - -func main() { - cmd := &cobra.Command{ - Short: "Compose Provider Example", - Use: "demo", - } - cmd.AddCommand(composeCommand()) - err := cmd.Execute() - if err != nil { - fmt.Fprintln(os.Stderr, err) - os.Exit(1) - } -} - -type options struct { - db string - size int -} - -func composeCommand() *cobra.Command { - c := &cobra.Command{ - Use: "compose EVENT", - TraverseChildren: true, - } - c.PersistentFlags().String("project-name", "", "compose project name") // unused - - var options options - upCmd := &cobra.Command{ - Use: "up", - Run: func(_ *cobra.Command, args []string) { - up(options, args) - }, - Args: cobra.ExactArgs(1), - } - - upCmd.Flags().StringVar(&options.db, "type", "", "Database type (mysql, postgres, etc.)") - _ = upCmd.MarkFlagRequired("type") - upCmd.Flags().IntVar(&options.size, "size", 10, "Database size in GB") - upCmd.Flags().String("name", "", "Name of the database to be created") - _ = upCmd.MarkFlagRequired("name") - - downCmd := &cobra.Command{ - Use: "down", - Run: down, - Args: cobra.ExactArgs(1), - } - downCmd.Flags().String("name", "", "Name of the database to be deleted") - _ = downCmd.MarkFlagRequired("name") - - c.AddCommand(upCmd, downCmd) - c.AddCommand(metadataCommand(upCmd, downCmd)) - return c -} - -const lineSeparator = "\n" - -func up(options options, args []string) { - servicename := args[0] - fmt.Printf(`{ "type": "debug", "message": "Starting %s" }%s`, servicename, lineSeparator) - - for i := 0; i < options.size; i += 10 { - time.Sleep(1 * time.Second) - fmt.Printf(`{ "type": "info", "message": "Processing ... %d%%" }%s`, i*100/options.size, lineSeparator) - } - fmt.Printf(`{ "type": "setenv", "message": "URL=https://magic.cloud/%s" }%s`, servicename, lineSeparator) -} - -func down(_ *cobra.Command, _ []string) { - fmt.Printf(`{ "type": "error", "message": "Permission error" }%s`, lineSeparator) -} - -func metadataCommand(upCmd, downCmd *cobra.Command) *cobra.Command { - return &cobra.Command{ - Use: "metadata", - Run: func(cmd *cobra.Command, _ []string) { - metadata(upCmd, downCmd) - }, - Args: cobra.NoArgs, - } -} - -func metadata(upCmd, downCmd *cobra.Command) { - metadata := ProviderMetadata{} - metadata.Description = "Manage services on AwesomeCloud" - metadata.Up = commandParameters(upCmd) - metadata.Down = commandParameters(downCmd) - jsonMetadata, err := json.Marshal(metadata) - if err != nil { - panic(err) - } - fmt.Println(string(jsonMetadata)) -} - -func commandParameters(cmd *cobra.Command) CommandMetadata { - cmdMetadata := CommandMetadata{} - cmd.Flags().VisitAll(func(f *pflag.Flag) { - _, isRequired := f.Annotations[cobra.BashCompOneRequiredFlag] - cmdMetadata.Parameters = append(cmdMetadata.Parameters, Metadata{ - Name: f.Name, - Description: f.Usage, - Required: isRequired, - Type: f.Value.Type(), - Default: f.DefValue, - }) - }) - return cmdMetadata -} - -type ProviderMetadata struct { - Description string `json:"description"` - Up CommandMetadata `json:"up"` - Down CommandMetadata `json:"down"` -} - -type CommandMetadata struct { - Parameters []Metadata `json:"parameters"` -} - -type Metadata struct { - Name string `json:"name"` - Description string `json:"description"` - Required bool `json:"required"` - Type string `json:"type"` - Default string `json:"default,omitempty"` -} diff --git a/docs/extension.md b/docs/extension.md deleted file mode 100644 index 3682f15655c..00000000000 --- a/docs/extension.md +++ /dev/null @@ -1,176 +0,0 @@ -# About - -The Compose application model defines `service` as an abstraction for a computing unit managing (a subset of) -application needs, which can interact with other services by relying on network(s). Docker Compose is designed -to use the Docker Engine ("Moby") API to manage services as containers, but the abstraction _could_ also cover -many other runtimes, typically cloud services or services natively provided by host. - -The Compose extensibility model has been designed to extend the `service` support to runtimes accessible through -third-party tooling. - -# Architecture - -Compose extensibility relies on the `provider` attribute to select the actual binary responsible for managing -the resource(s) needed to run a service. - -```yaml - database: - provider: - type: awesomecloud - options: - type: mysql - size: 256 - name: myAwesomeCloudDB -``` - -`provider.type` tells Compose the binary to run, which can be either: -- Another Docker CLI plugin (typically, `model` to run `docker-model`) -- An executable in user's `PATH` - -If `provider.type` doesn't resolve into any of those, Compose will report an error and interrupt the `up` command. - -To be a valid Compose extension, provider command *MUST* accept a `compose` command (which can be hidden) -with subcommands `up` and `down`. - -## Up lifecycle - -To execute an application's `up` lifecycle, Compose executes the provider's `compose up` command, passing -the project name, service name, and additional options. The `provider.options` are translated -into command line flags. For example: -```console -awesomecloud compose --project-name <NAME> up --type=mysql --size=256 "database" -``` - -> __Note:__ `project-name` _should_ be used by the provider to tag resources -> set for project, so that later execution with `down` subcommand releases -> all allocated resources set for the project. - -## Communication with Compose - -Providers can interact with Compose using `stdout` as a channel, sending JSON line delimited messages. -JSON messages MUST include a `type` and a `message` attribute. -```json -{ "type": "info", "message": "preparing mysql ..." } -``` - -`type` can be either: -- `info`: Reports status updates to the user. Compose will render message as the service state in the progress UI -- `error`: Lets the user know something went wrong with details about the error. Compose will render the message as the reason for the service failure. -- `setenv`: Lets the plugin tell Compose how dependent services can access the created resource. See next section for further details. -- `debug`: Those messages could help debugging the provider, but are not rendered to the user by default. They are rendered when Compose is started with `--verbose` flag. - -```mermaid -sequenceDiagram - Shell->>Compose: docker compose up - Compose->>Provider: compose up --project-name=xx --foo=bar "database" - Provider--)Compose: json { "info": "pulling 25%" } - Compose-)Shell: pulling 25% - Provider--)Compose: json { "info": "pulling 50%" } - Compose-)Shell: pulling 50% - Provider--)Compose: json { "info": "pulling 75%" } - Compose-)Shell: pulling 75% - Provider--)Compose: json { "setenv": "URL=http://cloud.com/abcd:1234" } - Compose-)Compose: set DATABASE_URL - Provider-)Compose: EOF (command complete) exit 0 - Compose-)Shell: service started -``` - -## Connection to a service managed by a provider - -A service in the Compose application can declare dependency on a service managed by an external provider: - -```yaml -services: - app: - image: myapp - depends_on: - - database - - database: - provider: - type: awesomecloud -``` - -When the provider command sends a `setenv` JSON message, Compose injects the specified variable into any dependent service, -automatically prefixing it with the service name. For example, if `awesomecloud compose up` returns: -```json -{"type": "setenv", "message": "URL=https://awesomecloud.com/db:1234"} -``` -Then the `app` service, which depends on the service managed by the provider, will receive a `DATABASE_URL` environment variable injected -into its runtime environment. - -> __Note:__ The `compose up` provider command _MUST_ be idempotent. If resource is already running, the command _MUST_ set -> the same environment variables to ensure consistent configuration of dependent services. - -## Down lifecycle - -`down` lifecycle is equivalent to `up` with the `<provider> compose --project-name <NAME> down <SERVICE>` command. -The provider is responsible for releasing all resources associated with the service. - -## Provide metadata about options - -Compose extensions *MAY* optionally implement a `metadata` subcommand to provide information about the parameters accepted by the `up` and `down` commands. - -The `metadata` subcommand takes no parameters and returns a JSON structure on the `stdout` channel that describes the parameters accepted by both the `up` and `down` commands, including whether each parameter is mandatory or optional. - -```console -awesomecloud compose metadata -``` - -The expected JSON output format is: -```json -{ - "description": "Manage services on AwesomeCloud", - "up": { - "parameters": [ - { - "name": "type", - "description": "Database type (mysql, postgres, etc.)", - "required": true, - "type": "string" - }, - { - "name": "size", - "description": "Database size in GB", - "required": false, - "type": "integer", - "default": "10" - }, - { - "name": "name", - "description": "Name of the database to be created", - "required": true, - "type": "string" - } - ] - }, - "down": { - "parameters": [ - { - "name": "name", - "description": "Name of the database to be removed", - "required": true, - "type": "string" - } - ] - } -} -``` -The top elements are: -- `description`: Human-readable description of the provider -- `up`: Object describing the parameters accepted by the `up` command -- `down`: Object describing the parameters accepted by the `down` command - -And for each command parameter, you should include the following properties: -- `name`: The parameter name (without `--` prefix) -- `description`: Human-readable description of the parameter -- `required`: Boolean indicating if the parameter is mandatory -- `type`: Parameter type (`string`, `integer`, `boolean`, etc.) -- `default`: Default value (optional, only for non-required parameters) -- `enum`: List of possible values supported by the parameter separated by `,` (optional, only for parameters with a limited set of values) - -This metadata allows Compose and other tools to understand the provider's interface and provide better user experience, such as validation, auto-completion, and documentation generation. - -## Examples - -See [example](examples/provider.go) for illustration on implementing this API in a command line diff --git a/docs/issue_template.md b/docs/issue_template.md new file mode 100644 index 00000000000..774f27e207f --- /dev/null +++ b/docs/issue_template.md @@ -0,0 +1,50 @@ + + +## Description of the issue + +## Context information (for bug reports) + +``` +Output of "docker-compose version" +``` + +``` +Output of "docker version" +``` + +``` +Output of "docker-compose config" +``` + + +## Steps to reproduce the issue + +1. +2. +3. + +### Observed result + +### Expected result + +### Stacktrace / full error message + +``` +(if applicable) +``` + +## Additional information + +OS version / distribution, `docker-compose` install method, etc. diff --git a/docs/pull_request_template.md b/docs/pull_request_template.md new file mode 100644 index 00000000000..15526af0556 --- /dev/null +++ b/docs/pull_request_template.md @@ -0,0 +1,13 @@ + + + +Resolves # diff --git a/docs/reference/compose.md b/docs/reference/compose.md deleted file mode 100644 index d80bb86ec62..00000000000 --- a/docs/reference/compose.md +++ /dev/null @@ -1,264 +0,0 @@ - -# docker compose - -```text -docker compose [-f <arg>...] [options] [COMMAND] [ARGS...] -``` - - -Define and run multi-container applications with Docker - -### Subcommands - -| Name | Description | -|:--------------------------------|:----------------------------------------------------------------------------------------| -| [`attach`](compose_attach.md) | Attach local standard input, output, and error streams to a service's running container | -| [`bridge`](compose_bridge.md) | Convert compose files into another model | -| [`build`](compose_build.md) | Build or rebuild services | -| [`commit`](compose_commit.md) | Create a new image from a service container's changes | -| [`config`](compose_config.md) | Parse, resolve and render compose file in canonical format | -| [`cp`](compose_cp.md) | Copy files/folders between a service container and the local filesystem | -| [`create`](compose_create.md) | Creates containers for a service | -| [`down`](compose_down.md) | Stop and remove containers, networks | -| [`events`](compose_events.md) | Receive real time events from containers | -| [`exec`](compose_exec.md) | Execute a command in a running container | -| [`export`](compose_export.md) | Export a service container's filesystem as a tar archive | -| [`images`](compose_images.md) | List images used by the created containers | -| [`kill`](compose_kill.md) | Force stop service containers | -| [`logs`](compose_logs.md) | View output from containers | -| [`ls`](compose_ls.md) | List running compose projects | -| [`pause`](compose_pause.md) | Pause services | -| [`port`](compose_port.md) | Print the public port for a port binding | -| [`ps`](compose_ps.md) | List containers | -| [`publish`](compose_publish.md) | Publish compose application | -| [`pull`](compose_pull.md) | Pull service images | -| [`push`](compose_push.md) | Push service images | -| [`restart`](compose_restart.md) | Restart service containers | -| [`rm`](compose_rm.md) | Removes stopped service containers | -| [`run`](compose_run.md) | Run a one-off command on a service | -| [`scale`](compose_scale.md) | Scale services | -| [`start`](compose_start.md) | Start services | -| [`stats`](compose_stats.md) | Display a live stream of container(s) resource usage statistics | -| [`stop`](compose_stop.md) | Stop services | -| [`top`](compose_top.md) | Display the running processes | -| [`unpause`](compose_unpause.md) | Unpause services | -| [`up`](compose_up.md) | Create and start containers | -| [`version`](compose_version.md) | Show the Docker Compose version information | -| [`volumes`](compose_volumes.md) | List volumes | -| [`wait`](compose_wait.md) | Block until containers of all (or specified) services stop. | -| [`watch`](compose_watch.md) | Watch build context for service and rebuild/refresh containers when files are updated | - - -### Options - -| Name | Type | Default | Description | -|:-----------------------|:--------------|:--------|:----------------------------------------------------------------------------------------------------| -| `--all-resources` | `bool` | | Include all resources, even those not used by services | -| `--ansi` | `string` | `auto` | Control when to print ANSI control characters ("never"\|"always"\|"auto") | -| `--compatibility` | `bool` | | Run compose in backward compatibility mode | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--env-file` | `stringArray` | | Specify an alternate environment file | -| `-f`, `--file` | `stringArray` | | Compose configuration files | -| `--parallel` | `int` | `-1` | Control max parallelism, -1 for unlimited | -| `--profile` | `stringArray` | | Specify a profile to enable | -| `--progress` | `string` | | Set type of progress output (auto, tty, plain, json, quiet) | -| `--project-directory` | `string` | | Specify an alternate working directory (default: the path of the, first specified, Compose file) | -| `-p`, `--project-name` | `string` | | Project name | - - - - -## Examples - -### Use `-f` to specify the name and path of one or more Compose files -Use the `-f` flag to specify the location of a Compose [configuration file](/reference/compose-file/). - -#### Specifying multiple Compose files -You can supply multiple `-f` configuration files. When you supply multiple files, Compose combines them into a single -configuration. Compose builds the configuration in the order you supply the files. Subsequent files override and add -to their predecessors. - -For example, consider this command line: - -```console -$ docker compose -f compose.yaml -f compose.admin.yaml run backup_db -``` - -The `compose.yaml` file might specify a `webapp` service. - -```yaml -services: - webapp: - image: examples/web - ports: - - "8000:8000" - volumes: - - "/data" -``` -If the `compose.admin.yaml` also specifies this same service, any matching fields override the previous file. -New values, add to the `webapp` service configuration. - -```yaml -services: - webapp: - build: . - environment: - - DEBUG=1 -``` - -When you use multiple Compose files, all paths in the files are relative to the first configuration file specified -with `-f`. You can use the `--project-directory` option to override this base path. - -Use a `-f` with `-` (dash) as the filename to read the configuration from stdin. When stdin is used all paths in the -configuration are relative to the current working directory. - -The `-f` flag is optional. If you don’t provide this flag on the command line, Compose traverses the working directory -and its parent directories looking for a `compose.yaml` or `docker-compose.yaml` file. - -#### Specifying a path to a single Compose file -You can use the `-f` flag to specify a path to a Compose file that is not located in the current directory, either -from the command line or by setting up a `COMPOSE_FILE` environment variable in your shell or in an environment file. - -For an example of using the `-f` option at the command line, suppose you are running the Compose Rails sample, and -have a `compose.yaml` file in a directory called `sandbox/rails`. You can use a command like `docker compose pull` to -get the postgres image for the db service from anywhere by using the `-f` flag as follows: - -```console -$ docker compose -f ~/sandbox/rails/compose.yaml pull db -``` - -#### Using an OCI published artifact -You can use the `-f` flag with the `oci://` prefix to reference a Compose file that has been published to an OCI registry. -This allows you to distribute and version your Compose configurations as OCI artifacts. - -To use a Compose file from an OCI registry: - -```console -$ docker compose -f oci://registry.example.com/my-compose-project:latest up -``` - -You can also combine OCI artifacts with local files: - -```console -$ docker compose -f oci://registry.example.com/my-compose-project:v1.0 -f compose.override.yaml up -``` - -The OCI artifact must contain a valid Compose file. You can publish Compose files to an OCI registry using the -`docker compose publish` command. - -#### Using a git repository -You can use the `-f` flag to reference a Compose file from a git repository. Compose supports various git URL formats: - -Using HTTPS: -```console -$ docker compose -f https://github.com/user/repo.git up -``` - -Using SSH: -```console -$ docker compose -f git@github.com:user/repo.git up -``` - -You can specify a specific branch, tag, or commit: -```console -$ docker compose -f https://github.com/user/repo.git@main up -$ docker compose -f https://github.com/user/repo.git@v1.0.0 up -$ docker compose -f https://github.com/user/repo.git@abc123 up -``` - -You can also specify a subdirectory within the repository: -```console -$ docker compose -f https://github.com/user/repo.git#main:path/to/compose.yaml up -``` - -When using git resources, Compose will clone the repository and use the specified Compose file. You can combine -git resources with local files: - -```console -$ docker compose -f https://github.com/user/repo.git -f compose.override.yaml up -``` - -### Use `-p` to specify a project name - -Each configuration has a project name. Compose sets the project name using -the following mechanisms, in order of precedence: -- The `-p` command line flag -- The `COMPOSE_PROJECT_NAME` environment variable -- The top level `name:` variable from the config file (or the last `name:` -from a series of config files specified using `-f`) -- The `basename` of the project directory containing the config file (or -containing the first config file specified using `-f`) -- The `basename` of the current directory if no config file is specified -Project names must contain only lowercase letters, decimal digits, dashes, -and underscores, and must begin with a lowercase letter or decimal digit. If -the `basename` of the project directory or current directory violates this -constraint, you must use one of the other mechanisms. - -```console -$ docker compose -p my_project ps -a -NAME SERVICE STATUS PORTS -my_project_demo_1 demo running - -$ docker compose -p my_project logs -demo_1 | PING localhost (127.0.0.1): 56 data bytes -demo_1 | 64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.095 ms -``` - -### Use profiles to enable optional services - -Use `--profile` to specify one or more active profiles -Calling `docker compose --profile frontend up` starts the services with the profile `frontend` and services -without any specified profiles. -You can also enable multiple profiles, e.g. with `docker compose --profile frontend --profile debug up` the profiles `frontend` and `debug` is enabled. - -Profiles can also be set by `COMPOSE_PROFILES` environment variable. - -### Configuring parallelism - -Use `--parallel` to specify the maximum level of parallelism for concurrent engine calls. -Calling `docker compose --parallel 1 pull` pulls the pullable images defined in the Compose file -one at a time. This can also be used to control build concurrency. - -Parallelism can also be set by the `COMPOSE_PARALLEL_LIMIT` environment variable. - -### Set up environment variables - -You can set environment variables for various docker compose options, including the `-f`, `-p` and `--profiles` flags. - -Setting the `COMPOSE_FILE` environment variable is equivalent to passing the `-f` flag, -`COMPOSE_PROJECT_NAME` environment variable does the same as the `-p` flag, -`COMPOSE_PROFILES` environment variable is equivalent to the `--profiles` flag -and `COMPOSE_PARALLEL_LIMIT` does the same as the `--parallel` flag. - -If flags are explicitly set on the command line, the associated environment variable is ignored. - -Setting the `COMPOSE_IGNORE_ORPHANS` environment variable to `true` stops docker compose from detecting orphaned -containers for the project. - -Setting the `COMPOSE_MENU` environment variable to `false` disables the helper menu when running `docker compose up` -in attached mode. Alternatively, you can also run `docker compose up --menu=false` to disable the helper menu. - -### Use Dry Run mode to test your command - -Use `--dry-run` flag to test a command without changing your application stack state. -Dry Run mode shows you all the steps Compose applies when executing a command, for example: -```console -$ docker compose --dry-run up --build -d -[+] Pulling 1/1 - ✔ DRY-RUN MODE - db Pulled 0.9s -[+] Running 10/8 - ✔ DRY-RUN MODE - build service backend 0.0s - ✔ DRY-RUN MODE - ==> ==> writing image dryRun-754a08ddf8bcb1cf22f310f09206dd783d42f7dd 0.0s - ✔ DRY-RUN MODE - ==> ==> naming to nginx-golang-mysql-backend 0.0s - ✔ DRY-RUN MODE - Network nginx-golang-mysql_default Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-db-1 Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-backend-1 Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-proxy-1 Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-db-1 Healthy 0.5s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-backend-1 Started 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-proxy-1 Started Started -``` -From the example above, you can see that the first step is to pull the image defined by `db` service, then build the `backend` service. -Next, the containers are created. The `db` service is started, and the `backend` and `proxy` wait until the `db` service is healthy before starting. - -Dry Run mode works with almost all commands. You cannot use Dry Run mode with a command that doesn't change the state of a Compose stack such as `ps`, `ls`, `logs` for example. diff --git a/docs/reference/compose_alpha.md b/docs/reference/compose_alpha.md deleted file mode 100644 index 34485d7deff..00000000000 --- a/docs/reference/compose_alpha.md +++ /dev/null @@ -1,22 +0,0 @@ -# docker compose alpha - - -Experimental commands - -### Subcommands - -| Name | Description | -|:----------------------------------|:-----------------------------------------------------------------------------------------------------| -| [`viz`](compose_alpha_viz.md) | EXPERIMENTAL - Generate a graphviz graph from your compose file | -| [`watch`](compose_alpha_watch.md) | EXPERIMENTAL - Watch build context for service and rebuild/refresh containers when files are updated | - - -### Options - -| Name | Type | Default | Description | -|:------------|:-----|:--------|:--------------------------------| -| `--dry-run` | | | Execute command in dry run mode | - - - - diff --git a/docs/reference/compose_alpha_dry-run.md b/docs/reference/compose_alpha_dry-run.md deleted file mode 100644 index 7c68d94d66b..00000000000 --- a/docs/reference/compose_alpha_dry-run.md +++ /dev/null @@ -1,8 +0,0 @@ -# docker compose alpha dry-run - - -Dry run command allows you to test a command without applying changes - - - - diff --git a/docs/reference/compose_alpha_generate.md b/docs/reference/compose_alpha_generate.md deleted file mode 100644 index f4054627798..00000000000 --- a/docs/reference/compose_alpha_generate.md +++ /dev/null @@ -1,17 +0,0 @@ -# docker compose alpha generate - - -EXPERIMENTAL - Generate a Compose file from existing containers - -### Options - -| Name | Type | Default | Description | -|:----------------|:---------|:--------|:------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--format` | `string` | `yaml` | Format the output. Values: [yaml \| json] | -| `--name` | `string` | | Project name to set in the Compose file | -| `--project-dir` | `string` | | Directory to use for the project | - - - - diff --git a/docs/reference/compose_alpha_publish.md b/docs/reference/compose_alpha_publish.md deleted file mode 100644 index 6e77d714532..00000000000 --- a/docs/reference/compose_alpha_publish.md +++ /dev/null @@ -1,18 +0,0 @@ -# docker compose alpha publish - - -Publish compose application - -### Options - -| Name | Type | Default | Description | -|:--------------------------|:---------|:--------|:-------------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--oci-version` | `string` | | OCI image/artifact specification version (automatically determined by default) | -| `--resolve-image-digests` | `bool` | | Pin image tags to digests | -| `--with-env` | `bool` | | Include environment variables in the published OCI artifact | -| `-y`, `--yes` | `bool` | | Assume "yes" as answer to all prompts | - - - - diff --git a/docs/reference/compose_alpha_scale.md b/docs/reference/compose_alpha_scale.md deleted file mode 100644 index f783f3335c7..00000000000 --- a/docs/reference/compose_alpha_scale.md +++ /dev/null @@ -1,15 +0,0 @@ -# docker compose alpha scale - - -Scale services - -### Options - -| Name | Type | Default | Description | -|:------------|:-----|:--------|:--------------------------------| -| `--dry-run` | | | Execute command in dry run mode | -| `--no-deps` | | | Don't start linked services | - - - - diff --git a/docs/reference/compose_alpha_viz.md b/docs/reference/compose_alpha_viz.md deleted file mode 100644 index 1a05aaac14d..00000000000 --- a/docs/reference/compose_alpha_viz.md +++ /dev/null @@ -1,19 +0,0 @@ -# docker compose alpha viz - - -EXPERIMENTAL - Generate a graphviz graph from your compose file - -### Options - -| Name | Type | Default | Description | -|:---------------------|:-------|:--------|:---------------------------------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--image` | `bool` | | Include service's image name in output graph | -| `--indentation-size` | `int` | `1` | Number of tabs or spaces to use for indentation | -| `--networks` | `bool` | | Include service's attached networks in output graph | -| `--ports` | `bool` | | Include service's exposed ports in output graph | -| `--spaces` | `bool` | | If given, space character ' ' will be used to indent, otherwise tab character '\t' will be used | - - - - diff --git a/docs/reference/compose_alpha_watch.md b/docs/reference/compose_alpha_watch.md deleted file mode 100644 index aa8130e7a02..00000000000 --- a/docs/reference/compose_alpha_watch.md +++ /dev/null @@ -1,16 +0,0 @@ -# docker compose alpha watch - - -Watch build context for service and rebuild/refresh containers when files are updated - -### Options - -| Name | Type | Default | Description | -|:------------|:-----|:--------|:----------------------------------------------| -| `--dry-run` | | | Execute command in dry run mode | -| `--no-up` | | | Do not build & start services before watching | -| `--quiet` | | | hide build output | - - - - diff --git a/docs/reference/compose_attach.md b/docs/reference/compose_attach.md deleted file mode 100644 index 0b9ede1e01a..00000000000 --- a/docs/reference/compose_attach.md +++ /dev/null @@ -1,17 +0,0 @@ -# docker compose attach - - -Attach local standard input, output, and error streams to a service's running container - -### Options - -| Name | Type | Default | Description | -|:----------------|:---------|:--------|:----------------------------------------------------------| -| `--detach-keys` | `string` | | Override the key sequence for detaching from a container. | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--index` | `int` | `0` | index of the container if service has multiple replicas. | -| `--no-stdin` | `bool` | | Do not attach STDIN | -| `--sig-proxy` | `bool` | `true` | Proxy all received signals to the process | - - - diff --git a/docs/reference/compose_bridge.md b/docs/reference/compose_bridge.md deleted file mode 100644 index 78d3da4934c..00000000000 --- a/docs/reference/compose_bridge.md +++ /dev/null @@ -1,22 +0,0 @@ -# docker compose bridge - - -Convert compose files into another model - -### Subcommands - -| Name | Description | -|:-------------------------------------------------------|:-----------------------------------------------------------------------------| -| [`convert`](compose_bridge_convert.md) | Convert compose files to Kubernetes manifests, Helm charts, or another model | -| [`transformations`](compose_bridge_transformations.md) | Manage transformation images | - - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:--------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | - - - - diff --git a/docs/reference/compose_bridge_convert.md b/docs/reference/compose_bridge_convert.md deleted file mode 100644 index d4b91ba172d..00000000000 --- a/docs/reference/compose_bridge_convert.md +++ /dev/null @@ -1,17 +0,0 @@ -# docker compose bridge convert - - -Convert compose files to Kubernetes manifests, Helm charts, or another model - -### Options - -| Name | Type | Default | Description | -|:-------------------------|:--------------|:--------|:-------------------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-o`, `--output` | `string` | `out` | The output directory for the Kubernetes resources | -| `--templates` | `string` | | Directory containing transformation templates | -| `-t`, `--transformation` | `stringArray` | | Transformation to apply to compose model (default: docker/compose-bridge-kubernetes) | - - - - diff --git a/docs/reference/compose_bridge_transformations.md b/docs/reference/compose_bridge_transformations.md deleted file mode 100644 index 1e1c7be392b..00000000000 --- a/docs/reference/compose_bridge_transformations.md +++ /dev/null @@ -1,22 +0,0 @@ -# docker compose bridge transformations - - -Manage transformation images - -### Subcommands - -| Name | Description | -|:-----------------------------------------------------|:-------------------------------| -| [`create`](compose_bridge_transformations_create.md) | Create a new transformation | -| [`list`](compose_bridge_transformations_list.md) | List available transformations | - - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:--------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | - - - - diff --git a/docs/reference/compose_bridge_transformations_create.md b/docs/reference/compose_bridge_transformations_create.md deleted file mode 100644 index 187e8d9eca3..00000000000 --- a/docs/reference/compose_bridge_transformations_create.md +++ /dev/null @@ -1,15 +0,0 @@ -# docker compose bridge transformations create - - -Create a new transformation - -### Options - -| Name | Type | Default | Description | -|:---------------|:---------|:--------|:----------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-f`, `--from` | `string` | | Existing transformation to copy (default: docker/compose-bridge-kubernetes) | - - - - diff --git a/docs/reference/compose_bridge_transformations_list.md b/docs/reference/compose_bridge_transformations_list.md deleted file mode 100644 index ce0a5e6911a..00000000000 --- a/docs/reference/compose_bridge_transformations_list.md +++ /dev/null @@ -1,20 +0,0 @@ -# docker compose bridge transformations list - - -List available transformations - -### Aliases - -`docker compose bridge transformations list`, `docker compose bridge transformations ls` - -### Options - -| Name | Type | Default | Description | -|:----------------|:---------|:--------|:-------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--format` | `string` | `table` | Format the output. Values: [table \| json] | -| `-q`, `--quiet` | `bool` | | Only display transformer names | - - - - diff --git a/docs/reference/compose_build.md b/docs/reference/compose_build.md deleted file mode 100644 index a715974dfa5..00000000000 --- a/docs/reference/compose_build.md +++ /dev/null @@ -1,46 +0,0 @@ -# docker compose build - - -Services are built once and then tagged, by default as `project-service`. - -If the Compose file specifies an -[image](https://github.com/compose-spec/compose-spec/blob/main/spec.md#image) name, -the image is tagged with that name, substituting any variables beforehand. See -[variable interpolation](https://github.com/compose-spec/compose-spec/blob/main/spec.md#interpolation). - -If you change a service's `Dockerfile` or the contents of its build directory, -run `docker compose build` to rebuild it. - -### Options - -| Name | Type | Default | Description | -|:----------------------|:--------------|:--------|:------------------------------------------------------------------------------------------------------------| -| `--build-arg` | `stringArray` | | Set build-time variables for services | -| `--builder` | `string` | | Set builder to use | -| `--check` | `bool` | | Check build configuration | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-m`, `--memory` | `bytes` | `0` | Set memory limit for the build container. Not supported by BuildKit. | -| `--no-cache` | `bool` | | Do not use cache when building the image | -| `--print` | `bool` | | Print equivalent bake file | -| `--provenance` | `string` | | Add a provenance attestation | -| `--pull` | `bool` | | Always attempt to pull a newer version of the image | -| `--push` | `bool` | | Push service images | -| `-q`, `--quiet` | `bool` | | Suppress the build output | -| `--sbom` | `string` | | Add a SBOM attestation | -| `--ssh` | `string` | | Set SSH authentications used when building service images. (use 'default' for using your default SSH Agent) | -| `--with-dependencies` | `bool` | | Also build dependencies (transitively) | - - - - -## Description - -Services are built once and then tagged, by default as `project-service`. - -If the Compose file specifies an -[image](https://github.com/compose-spec/compose-spec/blob/main/spec.md#image) name, -the image is tagged with that name, substituting any variables beforehand. See -[variable interpolation](https://github.com/compose-spec/compose-spec/blob/main/spec.md#interpolation). - -If you change a service's `Dockerfile` or the contents of its build directory, -run `docker compose build` to rebuild it. diff --git a/docs/reference/compose_commit.md b/docs/reference/compose_commit.md deleted file mode 100644 index 1aad40931f9..00000000000 --- a/docs/reference/compose_commit.md +++ /dev/null @@ -1,19 +0,0 @@ -# docker compose commit - - -Create a new image from a service container's changes - -### Options - -| Name | Type | Default | Description | -|:------------------|:---------|:--------|:-----------------------------------------------------------| -| `-a`, `--author` | `string` | | Author (e.g., "John Hannibal Smith <hannibal@a-team.com>") | -| `-c`, `--change` | `list` | | Apply Dockerfile instruction to the created image | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--index` | `int` | `0` | index of the container if service has multiple replicas. | -| `-m`, `--message` | `string` | | Commit message | -| `-p`, `--pause` | `bool` | `true` | Pause container during commit | - - - - diff --git a/docs/reference/compose_config.md b/docs/reference/compose_config.md deleted file mode 100644 index e2e773feae5..00000000000 --- a/docs/reference/compose_config.md +++ /dev/null @@ -1,40 +0,0 @@ -# docker compose convert - - -`docker compose config` renders the actual data model to be applied on the Docker Engine. -It merges the Compose files set by `-f` flags, resolves variables in the Compose file, and expands short-notation into -the canonical format. - -### Options - -| Name | Type | Default | Description | -|:--------------------------|:---------|:--------|:----------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--environment` | `bool` | | Print environment used for interpolation. | -| `--format` | `string` | | Format the output. Values: [yaml \| json] | -| `--hash` | `string` | | Print the service config hash, one per line. | -| `--images` | `bool` | | Print the image names, one per line. | -| `--lock-image-digests` | `bool` | | Produces an override file with image digests | -| `--models` | `bool` | | Print the model names, one per line. | -| `--networks` | `bool` | | Print the network names, one per line. | -| `--no-consistency` | `bool` | | Don't check model consistency - warning: may produce invalid Compose output | -| `--no-env-resolution` | `bool` | | Don't resolve service env files | -| `--no-interpolate` | `bool` | | Don't interpolate environment variables | -| `--no-normalize` | `bool` | | Don't normalize compose model | -| `--no-path-resolution` | `bool` | | Don't resolve file paths | -| `-o`, `--output` | `string` | | Save to file (default to stdout) | -| `--profiles` | `bool` | | Print the profile names, one per line. | -| `-q`, `--quiet` | `bool` | | Only validate the configuration, don't print anything | -| `--resolve-image-digests` | `bool` | | Pin image tags to digests | -| `--services` | `bool` | | Print the service names, one per line. | -| `--variables` | `bool` | | Print model variables and default values. | -| `--volumes` | `bool` | | Print the volume names, one per line. | - - - - -## Description - -`docker compose config` renders the actual data model to be applied on the Docker Engine. -It merges the Compose files set by `-f` flags, resolves variables in the Compose file, and expands short-notation into -the canonical format. diff --git a/docs/reference/compose_cp.md b/docs/reference/compose_cp.md deleted file mode 100644 index 0886bbd9f94..00000000000 --- a/docs/reference/compose_cp.md +++ /dev/null @@ -1,18 +0,0 @@ -# docker compose cp - - -Copy files/folders between a service container and the local filesystem - -### Options - -| Name | Type | Default | Description | -|:----------------------|:-------|:--------|:--------------------------------------------------------| -| `--all` | `bool` | | Include containers created by the run command | -| `-a`, `--archive` | `bool` | | Archive mode (copy all uid/gid information) | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-L`, `--follow-link` | `bool` | | Always follow symbol link in SRC_PATH | -| `--index` | `int` | `0` | Index of the container if service has multiple replicas | - - - - diff --git a/docs/reference/compose_create.md b/docs/reference/compose_create.md deleted file mode 100644 index 4b0b876da91..00000000000 --- a/docs/reference/compose_create.md +++ /dev/null @@ -1,23 +0,0 @@ -# docker compose create - - -Creates containers for a service - -### Options - -| Name | Type | Default | Description | -|:-------------------|:--------------|:---------|:----------------------------------------------------------------------------------------------| -| `--build` | `bool` | | Build images before starting containers | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--force-recreate` | `bool` | | Recreate containers even if their configuration and image haven't changed | -| `--no-build` | `bool` | | Don't build an image, even if it's policy | -| `--no-recreate` | `bool` | | If containers already exist, don't recreate them. Incompatible with --force-recreate. | -| `--pull` | `string` | `policy` | Pull image before running ("always"\|"missing"\|"never"\|"build") | -| `--quiet-pull` | `bool` | | Pull without printing progress information | -| `--remove-orphans` | `bool` | | Remove containers for services not defined in the Compose file | -| `--scale` | `stringArray` | | Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present. | -| `-y`, `--yes` | `bool` | | Assume "yes" as answer to all prompts and run non-interactively | - - - - diff --git a/docs/reference/compose_down.md b/docs/reference/compose_down.md deleted file mode 100644 index 2ac0bf2da42..00000000000 --- a/docs/reference/compose_down.md +++ /dev/null @@ -1,45 +0,0 @@ -# docker compose down - - -Stops containers and removes containers, networks, volumes, and images created by `up`. - -By default, the only things removed are: - -- Containers for services defined in the Compose file. -- Networks defined in the networks section of the Compose file. -- The default network, if one is used. - -Networks and volumes defined as external are never removed. - -Anonymous volumes are not removed by default. However, as they don’t have a stable name, they are not automatically -mounted by a subsequent `up`. For data that needs to persist between updates, use explicit paths as bind mounts or -named volumes. - -### Options - -| Name | Type | Default | Description | -|:-------------------|:---------|:--------|:------------------------------------------------------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--remove-orphans` | `bool` | | Remove containers for services not defined in the Compose file | -| `--rmi` | `string` | | Remove images used by services. "local" remove only images that don't have a custom tag ("local"\|"all") | -| `-t`, `--timeout` | `int` | `0` | Specify a shutdown timeout in seconds | -| `-v`, `--volumes` | `bool` | | Remove named volumes declared in the "volumes" section of the Compose file and anonymous volumes attached to containers | - - - - -## Description - -Stops containers and removes containers, networks, volumes, and images created by `up`. - -By default, the only things removed are: - -- Containers for services defined in the Compose file. -- Networks defined in the networks section of the Compose file. -- The default network, if one is used. - -Networks and volumes defined as external are never removed. - -Anonymous volumes are not removed by default. However, as they don’t have a stable name, they are not automatically -mounted by a subsequent `up`. For data that needs to persist between updates, use explicit paths as bind mounts or -named volumes. diff --git a/docs/reference/compose_events.md b/docs/reference/compose_events.md deleted file mode 100644 index 066b5cf3831..00000000000 --- a/docs/reference/compose_events.md +++ /dev/null @@ -1,56 +0,0 @@ -# docker compose events - - -Stream container events for every container in the project. - -With the `--json` flag, a json object is printed one per line with the format: - -```json -{ - "time": "2015-11-20T18:01:03.615550", - "type": "container", - "action": "create", - "id": "213cf7...5fc39a", - "service": "web", - "attributes": { - "name": "application_web_1", - "image": "alpine:edge" - } -} -``` - -The events that can be received using this can be seen [here](/reference/cli/docker/system/events/#object-types). - -### Options - -| Name | Type | Default | Description | -|:------------|:---------|:--------|:------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--json` | `bool` | | Output events as a stream of json objects | -| `--since` | `string` | | Show all events created since timestamp | -| `--until` | `string` | | Stream events until this timestamp | - - - - -## Description - -Stream container events for every container in the project. - -With the `--json` flag, a json object is printed one per line with the format: - -```json -{ - "time": "2015-11-20T18:01:03.615550", - "type": "container", - "action": "create", - "id": "213cf7...5fc39a", - "service": "web", - "attributes": { - "name": "application_web_1", - "image": "alpine:edge" - } -} -``` - -The events that can be received using this can be seen [here](https://docs.docker.com/reference/cli/docker/system/events/#object-types). diff --git a/docs/reference/compose_exec.md b/docs/reference/compose_exec.md deleted file mode 100644 index 312219e7316..00000000000 --- a/docs/reference/compose_exec.md +++ /dev/null @@ -1,42 +0,0 @@ -# docker compose exec - - -This is the equivalent of `docker exec` targeting a Compose service. - -With this subcommand, you can run arbitrary commands in your services. Commands allocate a TTY by default, so -you can use a command such as `docker compose exec web sh` to get an interactive prompt. - -By default, Compose will enter container in interactive mode and allocate a TTY, while the equivalent `docker exec` -command requires passing `--interactive --tty` flags to get the same behavior. Compose also support those two flags -to offer a smooth migration between commands, whenever they are no-op by default. Still, `interactive` can be used to -force disabling interactive mode (`--interactive=false`), typically when `docker compose exec` command is used inside -a script. - -### Options - -| Name | Type | Default | Description | -|:------------------|:--------------|:--------|:---------------------------------------------------------------------------------| -| `-d`, `--detach` | `bool` | | Detached mode: Run command in the background | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-e`, `--env` | `stringArray` | | Set environment variables | -| `--index` | `int` | `0` | Index of the container if service has multiple replicas | -| `-T`, `--no-tty` | `bool` | `true` | Disable pseudo-TTY allocation. By default 'docker compose exec' allocates a TTY. | -| `--privileged` | `bool` | | Give extended privileges to the process | -| `-u`, `--user` | `string` | | Run the command as this user | -| `-w`, `--workdir` | `string` | | Path to workdir directory for this command | - - - - -## Description - -This is the equivalent of `docker exec` targeting a Compose service. - -With this subcommand, you can run arbitrary commands in your services. Commands allocate a TTY by default, so -you can use a command such as `docker compose exec web sh` to get an interactive prompt. - -By default, Compose will enter container in interactive mode and allocate a TTY, while the equivalent `docker exec` -command requires passing `--interactive --tty` flags to get the same behavior. Compose also support those two flags -to offer a smooth migration between commands, whenever they are no-op by default. Still, `interactive` can be used to -force disabling interactive mode (`--interactive=false`), typically when `docker compose exec` command is used inside -a script. \ No newline at end of file diff --git a/docs/reference/compose_export.md b/docs/reference/compose_export.md deleted file mode 100644 index 942ea6a347f..00000000000 --- a/docs/reference/compose_export.md +++ /dev/null @@ -1,16 +0,0 @@ -# docker compose export - - -Export a service container's filesystem as a tar archive - -### Options - -| Name | Type | Default | Description | -|:-----------------|:---------|:--------|:---------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--index` | `int` | `0` | index of the container if service has multiple replicas. | -| `-o`, `--output` | `string` | | Write to a file, instead of STDOUT | - - - - diff --git a/docs/reference/compose_images.md b/docs/reference/compose_images.md deleted file mode 100644 index 1e4e0259b1d..00000000000 --- a/docs/reference/compose_images.md +++ /dev/null @@ -1,16 +0,0 @@ -# docker compose images - - -List images used by the created containers - -### Options - -| Name | Type | Default | Description | -|:----------------|:---------|:--------|:-------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--format` | `string` | `table` | Format the output. Values: [table \| json] | -| `-q`, `--quiet` | `bool` | | Only display IDs | - - - - diff --git a/docs/reference/compose_kill.md b/docs/reference/compose_kill.md deleted file mode 100644 index 0b6c1d05f01..00000000000 --- a/docs/reference/compose_kill.md +++ /dev/null @@ -1,27 +0,0 @@ -# docker compose kill - - -Forces running containers to stop by sending a `SIGKILL` signal. Optionally the signal can be passed, for example: - -```console -$ docker compose kill -s SIGINT -``` - -### Options - -| Name | Type | Default | Description | -|:-------------------|:---------|:----------|:---------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--remove-orphans` | `bool` | | Remove containers for services not defined in the Compose file | -| `-s`, `--signal` | `string` | `SIGKILL` | SIGNAL to send to the container | - - - - -## Description - -Forces running containers to stop by sending a `SIGKILL` signal. Optionally the signal can be passed, for example: - -```console -$ docker compose kill -s SIGINT -``` diff --git a/docs/reference/compose_logs.md b/docs/reference/compose_logs.md deleted file mode 100644 index 4c8ba7e3486..00000000000 --- a/docs/reference/compose_logs.md +++ /dev/null @@ -1,25 +0,0 @@ -# docker compose logs - - -Displays log output from services - -### Options - -| Name | Type | Default | Description | -|:---------------------|:---------|:--------|:-----------------------------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-f`, `--follow` | `bool` | | Follow log output | -| `--index` | `int` | `0` | index of the container if service has multiple replicas | -| `--no-color` | `bool` | | Produce monochrome output | -| `--no-log-prefix` | `bool` | | Don't print prefix in logs | -| `--since` | `string` | | Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes) | -| `-n`, `--tail` | `string` | `all` | Number of lines to show from the end of the logs for each container | -| `-t`, `--timestamps` | `bool` | | Show timestamps | -| `--until` | `string` | | Show logs before a timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes) | - - - - -## Description - -Displays log output from services diff --git a/docs/reference/compose_ls.md b/docs/reference/compose_ls.md deleted file mode 100644 index 7719d208609..00000000000 --- a/docs/reference/compose_ls.md +++ /dev/null @@ -1,21 +0,0 @@ -# docker compose ls - - -Lists running Compose projects - -### Options - -| Name | Type | Default | Description | -|:----------------|:---------|:--------|:-------------------------------------------| -| `-a`, `--all` | `bool` | | Show all stopped Compose projects | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--filter` | `filter` | | Filter output based on conditions provided | -| `--format` | `string` | `table` | Format the output. Values: [table \| json] | -| `-q`, `--quiet` | `bool` | | Only display project names | - - - - -## Description - -Lists running Compose projects diff --git a/docs/reference/compose_pause.md b/docs/reference/compose_pause.md deleted file mode 100644 index 4a0d5bdcc03..00000000000 --- a/docs/reference/compose_pause.md +++ /dev/null @@ -1,17 +0,0 @@ -# docker compose pause - - -Pauses running containers of a service. They can be unpaused with `docker compose unpause`. - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:--------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | - - - - -## Description - -Pauses running containers of a service. They can be unpaused with `docker compose unpause`. \ No newline at end of file diff --git a/docs/reference/compose_port.md b/docs/reference/compose_port.md deleted file mode 100644 index bbbfbf15616..00000000000 --- a/docs/reference/compose_port.md +++ /dev/null @@ -1,19 +0,0 @@ -# docker compose port - - -Prints the public port for a port binding - -### Options - -| Name | Type | Default | Description | -|:-------------|:---------|:--------|:--------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--index` | `int` | `0` | Index of the container if service has multiple replicas | -| `--protocol` | `string` | `tcp` | tcp or udp | - - - - -## Description - -Prints the public port for a port binding diff --git a/docs/reference/compose_ps.md b/docs/reference/compose_ps.md deleted file mode 100644 index 3572c530556..00000000000 --- a/docs/reference/compose_ps.md +++ /dev/null @@ -1,138 +0,0 @@ -# docker compose ps - - -Lists containers for a Compose project, with current status and exposed ports. - -```console -$ docker compose ps -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp -``` - -By default, only running containers are shown. `--all` flag can be used to include stopped containers. - -```console -$ docker compose ps --all -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp -example-bar-1 alpine "/entrypoint.…" bar 4 seconds ago exited (0) -``` - -### Options - -| Name | Type | Default | Description | -|:----------------------|:--------------|:--------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `-a`, `--all` | `bool` | | Show all stopped containers (including those created by the run command) | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| [`--filter`](#filter) | `string` | | Filter services by a property (supported filters: status) | -| [`--format`](#format) | `string` | `table` | Format output using a custom template: 'table': Print output in table format with column headers (default) 'table TEMPLATE': Print output in table format using the given Go template 'json': Print in JSON format 'TEMPLATE': Print output using the given Go template. Refer to https://docs.docker.com/go/formatting/ for more information about formatting output with templates | -| `--no-trunc` | `bool` | | Don't truncate output | -| `--orphans` | `bool` | `true` | Include orphaned services (not declared by project) | -| `-q`, `--quiet` | `bool` | | Only display IDs | -| `--services` | `bool` | | Display services | -| [`--status`](#status) | `stringArray` | | Filter services by status. Values: [paused \| restarting \| removing \| running \| dead \| created \| exited] | - - - - -## Description - -Lists containers for a Compose project, with current status and exposed ports. - -```console -$ docker compose ps -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp -``` - -By default, only running containers are shown. `--all` flag can be used to include stopped containers. - -```console -$ docker compose ps --all -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp -example-bar-1 alpine "/entrypoint.…" bar 4 seconds ago exited (0) -``` - -## Examples - -### <a name="format"></a> Format the output (--format) - -By default, the `docker compose ps` command uses a table ("pretty") format to -show the containers. The `--format` flag allows you to specify alternative -presentations for the output. Currently, supported options are `pretty` (default), -and `json`, which outputs information about the containers as a JSON array: - -```console -$ docker compose ps --format json -[{"ID":"1553b0236cf4d2715845f053a4ee97042c4f9a2ef655731ee34f1f7940eaa41a","Name":"example-bar-1","Command":"/docker-entrypoint.sh nginx -g 'daemon off;'","Project":"example","Service":"bar","State":"exited","Health":"","ExitCode":0,"Publishers":null},{"ID":"f02a4efaabb67416e1ff127d51c4b5578634a0ad5743bd65225ff7d1909a3fa0","Name":"example-foo-1","Command":"/docker-entrypoint.sh nginx -g 'daemon off;'","Project":"example","Service":"foo","State":"running","Health":"","ExitCode":0,"Publishers":[{"URL":"0.0.0.0","TargetPort":80,"PublishedPort":8080,"Protocol":"tcp"}]}] -``` - -The JSON output allows you to use the information in other tools for further -processing, for example, using the [`jq` utility](https://stedolan.github.io/jq/) -to pretty-print the JSON: - -```console -$ docker compose ps --format json | jq . -[ - { - "ID": "1553b0236cf4d2715845f053a4ee97042c4f9a2ef655731ee34f1f7940eaa41a", - "Name": "example-bar-1", - "Command": "/docker-entrypoint.sh nginx -g 'daemon off;'", - "Project": "example", - "Service": "bar", - "State": "exited", - "Health": "", - "ExitCode": 0, - "Publishers": null - }, - { - "ID": "f02a4efaabb67416e1ff127d51c4b5578634a0ad5743bd65225ff7d1909a3fa0", - "Name": "example-foo-1", - "Command": "/docker-entrypoint.sh nginx -g 'daemon off;'", - "Project": "example", - "Service": "foo", - "State": "running", - "Health": "", - "ExitCode": 0, - "Publishers": [ - { - "URL": "0.0.0.0", - "TargetPort": 80, - "PublishedPort": 8080, - "Protocol": "tcp" - } - ] - } -] -``` - -### <a name="status"></a> Filter containers by status (--status) - -Use the `--status` flag to filter the list of containers by status. For example, -to show only containers that are running or only containers that have exited: - -```console -$ docker compose ps --status=running -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp - -$ docker compose ps --status=exited -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-bar-1 alpine "/entrypoint.…" bar 4 seconds ago exited (0) -``` - -### <a name="filter"></a> Filter containers by status (--filter) - -The [`--status` flag](#status) is a convenient shorthand for the `--filter status=<status>` -flag. The example below is the equivalent to the example from the previous section, -this time using the `--filter` flag: - -```console -$ docker compose ps --filter status=running -NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS -example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp -``` - -The `docker compose ps` command currently only supports the `--filter status=<status>` -option, but additional filter options may be added in the future. diff --git a/docs/reference/compose_publish.md b/docs/reference/compose_publish.md deleted file mode 100644 index 9a82fc260a7..00000000000 --- a/docs/reference/compose_publish.md +++ /dev/null @@ -1,19 +0,0 @@ -# docker compose publish - - -Publish compose application - -### Options - -| Name | Type | Default | Description | -|:--------------------------|:---------|:--------|:-------------------------------------------------------------------------------| -| `--app` | `bool` | | Published compose application (includes referenced images) | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--oci-version` | `string` | | OCI image/artifact specification version (automatically determined by default) | -| `--resolve-image-digests` | `bool` | | Pin image tags to digests | -| `--with-env` | `bool` | | Include environment variables in the published OCI artifact | -| `-y`, `--yes` | `bool` | | Assume "yes" as answer to all prompts | - - - - diff --git a/docs/reference/compose_pull.md b/docs/reference/compose_pull.md deleted file mode 100644 index 6a47f9d509f..00000000000 --- a/docs/reference/compose_pull.md +++ /dev/null @@ -1,68 +0,0 @@ -# docker compose pull - - -Pulls an image associated with a service defined in a `compose.yaml` file, but does not start containers based on those images - -### Options - -| Name | Type | Default | Description | -|:-------------------------|:---------|:--------|:-------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--ignore-buildable` | `bool` | | Ignore images that can be built | -| `--ignore-pull-failures` | `bool` | | Pull what it can and ignores images with pull failures | -| `--include-deps` | `bool` | | Also pull services declared as dependencies | -| `--policy` | `string` | | Apply pull policy ("missing"\|"always") | -| `-q`, `--quiet` | `bool` | | Pull without printing progress information | - - - - -## Description - -Pulls an image associated with a service defined in a `compose.yaml` file, but does not start containers based on those images - - -## Examples - -Consider the following `compose.yaml`: - -```yaml -services: - db: - image: postgres - web: - build: . - command: bundle exec rails s -p 3000 -b '0.0.0.0' - volumes: - - .:/myapp - ports: - - "3000:3000" - depends_on: - - db -``` - -If you run `docker compose pull ServiceName` in the same directory as the `compose.yaml` file that defines the service, -Docker pulls the associated image. For example, to call the postgres image configured as the db service in our example, -you would run `docker compose pull db`. - -```console -$ docker compose pull db -[+] Running 1/15 - ⠸ db Pulling 12.4s - ⠿ 45b42c59be33 Already exists 0.0s - ⠹ 40adec129f1a Downloading 3.374MB/4.178MB 9.3s - ⠹ b4c431d00c78 Download complete 9.3s - ⠹ 2696974e2815 Download complete 9.3s - ⠹ 564b77596399 Downloading 5.622MB/7.965MB 9.3s - ⠹ 5044045cf6f2 Downloading 216.7kB/391.1kB 9.3s - ⠹ d736e67e6ac3 Waiting 9.3s - ⠹ 390c1c9a5ae4 Waiting 9.3s - ⠹ c0e62f172284 Waiting 9.3s - ⠹ ebcdc659c5bf Waiting 9.3s - ⠹ 29be22cb3acc Waiting 9.3s - ⠹ f63c47038e66 Waiting 9.3s - ⠹ 77a0c198cde5 Waiting 9.3s - ⠹ c8752d5b785c Waiting 9.3s -``` - -`docker compose pull` tries to pull image for services with a build section. If pull fails, it lets you know this service image must be built. You can skip this by setting `--ignore-buildable` flag. diff --git a/docs/reference/compose_push.md b/docs/reference/compose_push.md deleted file mode 100644 index 0efc48c46d3..00000000000 --- a/docs/reference/compose_push.md +++ /dev/null @@ -1,54 +0,0 @@ -# docker compose push - - -Pushes images for services to their respective registry/repository. - -The following assumptions are made: -- You are pushing an image you have built locally -- You have access to the build key - -Examples - -```yaml -services: - service1: - build: . - image: localhost:5000/yourimage ## goes to local registry - - service2: - build: . - image: your-dockerid/yourimage ## goes to your repository on Docker Hub -``` - -### Options - -| Name | Type | Default | Description | -|:-------------------------|:-------|:--------|:-------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--ignore-push-failures` | `bool` | | Push what it can and ignores images with push failures | -| `--include-deps` | `bool` | | Also push images of services declared as dependencies | -| `-q`, `--quiet` | `bool` | | Push without printing progress information | - - - - -## Description - -Pushes images for services to their respective registry/repository. - -The following assumptions are made: -- You are pushing an image you have built locally -- You have access to the build key - -Examples - -```yaml -services: - service1: - build: . - image: localhost:5000/yourimage ## goes to local registry - - service2: - build: . - image: your-dockerid/yourimage ## goes to your repository on Docker Hub -``` diff --git a/docs/reference/compose_restart.md b/docs/reference/compose_restart.md deleted file mode 100644 index e57f346a81a..00000000000 --- a/docs/reference/compose_restart.md +++ /dev/null @@ -1,37 +0,0 @@ -# docker compose restart - - -Restarts all stopped and running services, or the specified services only. - -If you make changes to your `compose.yml` configuration, these changes are not reflected -after running this command. For example, changes to environment variables (which are added -after a container is built, but before the container's command is executed) are not updated -after restarting. - -If you are looking to configure a service's restart policy, refer to -[restart](https://github.com/compose-spec/compose-spec/blob/main/spec.md#restart) -or [restart_policy](https://github.com/compose-spec/compose-spec/blob/main/deploy.md#restart_policy). - -### Options - -| Name | Type | Default | Description | -|:------------------|:-------|:--------|:--------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--no-deps` | `bool` | | Don't restart dependent services | -| `-t`, `--timeout` | `int` | `0` | Specify a shutdown timeout in seconds | - - - - -## Description - -Restarts all stopped and running services, or the specified services only. - -If you make changes to your `compose.yml` configuration, these changes are not reflected -after running this command. For example, changes to environment variables (which are added -after a container is built, but before the container's command is executed) are not updated -after restarting. - -If you are looking to configure a service's restart policy, refer to -[restart](https://github.com/compose-spec/compose-spec/blob/main/spec.md#restart) -or [restart_policy](https://github.com/compose-spec/compose-spec/blob/main/deploy.md#restart_policy). diff --git a/docs/reference/compose_rm.md b/docs/reference/compose_rm.md deleted file mode 100644 index 5e84930bac3..00000000000 --- a/docs/reference/compose_rm.md +++ /dev/null @@ -1,48 +0,0 @@ -# docker compose rm - - -Removes stopped service containers. - -By default, anonymous volumes attached to containers are not removed. You can override this with `-v`. To list all -volumes, use `docker volume ls`. - -Any data which is not in a volume is lost. - -Running the command with no options also removes one-off containers created by `docker compose run`: - -```console -$ docker compose rm -Going to remove djangoquickstart_web_run_1 -Are you sure? [yN] y -Removing djangoquickstart_web_run_1 ... done -``` - -### Options - -| Name | Type | Default | Description | -|:------------------|:-------|:--------|:----------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-f`, `--force` | `bool` | | Don't ask to confirm removal | -| `-s`, `--stop` | `bool` | | Stop the containers, if required, before removing | -| `-v`, `--volumes` | `bool` | | Remove any anonymous volumes attached to containers | - - - - -## Description - -Removes stopped service containers. - -By default, anonymous volumes attached to containers are not removed. You can override this with `-v`. To list all -volumes, use `docker volume ls`. - -Any data which is not in a volume is lost. - -Running the command with no options also removes one-off containers created by `docker compose run`: - -```console -$ docker compose rm -Going to remove djangoquickstart_web_run_1 -Are you sure? [yN] y -Removing djangoquickstart_web_run_1 ... done -``` diff --git a/docs/reference/compose_run.md b/docs/reference/compose_run.md deleted file mode 100644 index 25b28d1ded8..00000000000 --- a/docs/reference/compose_run.md +++ /dev/null @@ -1,145 +0,0 @@ -# docker compose run - - -Runs a one-time command against a service. - -The following command starts the `web` service and runs `bash` as its command: - -```console -$ docker compose run web bash -``` - -Commands you use with run start in new containers with configuration defined by that of the service, -including volumes, links, and other details. However, there are two important differences: - -First, the command passed by `run` overrides the command defined in the service configuration. For example, if the -`web` service configuration is started with `bash`, then `docker compose run web python app.py` overrides it with -`python app.py`. - -The second difference is that the `docker compose run` command does not create any of the ports specified in the -service configuration. This prevents port collisions with already-open ports. If you do want the service’s ports -to be created and mapped to the host, specify the `--service-ports` - -```console -$ docker compose run --service-ports web python manage.py shell -``` - -Alternatively, manual port mapping can be specified with the `--publish` or `-p` options, just as when using docker run: - -```console -$ docker compose run --publish 8080:80 -p 2022:22 -p 127.0.0.1:2021:21 web python manage.py shell -``` - -If you start a service configured with links, the run command first checks to see if the linked service is running -and starts the service if it is stopped. Once all the linked services are running, the run executes the command you -passed it. For example, you could run: - -```console -$ docker compose run db psql -h db -U docker -``` - -This opens an interactive PostgreSQL shell for the linked `db` container. - -If you do not want the run command to start linked containers, use the `--no-deps` flag: - -```console -$ docker compose run --no-deps web python manage.py shell -``` - -If you want to remove the container after running while overriding the container’s restart policy, use the `--rm` flag: - -```console -$ docker compose run --rm web python manage.py db upgrade -``` - -This runs a database upgrade script, and removes the container when finished running, even if a restart policy is -specified in the service configuration. - -### Options - -| Name | Type | Default | Description | -|:------------------------|:--------------|:---------|:---------------------------------------------------------------------------------| -| `--build` | `bool` | | Build image before starting container | -| `--cap-add` | `list` | | Add Linux capabilities | -| `--cap-drop` | `list` | | Drop Linux capabilities | -| `-d`, `--detach` | `bool` | | Run container in background and print container ID | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--entrypoint` | `string` | | Override the entrypoint of the image | -| `-e`, `--env` | `stringArray` | | Set environment variables | -| `--env-from-file` | `stringArray` | | Set environment variables from file | -| `-i`, `--interactive` | `bool` | `true` | Keep STDIN open even if not attached | -| `-l`, `--label` | `stringArray` | | Add or override a label | -| `--name` | `string` | | Assign a name to the container | -| `-T`, `--no-TTY` | `bool` | `true` | Disable pseudo-TTY allocation (default: auto-detected) | -| `--no-deps` | `bool` | | Don't start linked services | -| `-p`, `--publish` | `stringArray` | | Publish a container's port(s) to the host | -| `--pull` | `string` | `policy` | Pull image before running ("always"\|"missing"\|"never") | -| `-q`, `--quiet` | `bool` | | Don't print anything to STDOUT | -| `--quiet-build` | `bool` | | Suppress progress output from the build process | -| `--quiet-pull` | `bool` | | Pull without printing progress information | -| `--remove-orphans` | `bool` | | Remove containers for services not defined in the Compose file | -| `--rm` | `bool` | | Automatically remove the container when it exits | -| `-P`, `--service-ports` | `bool` | | Run command with all service's ports enabled and mapped to the host | -| `--use-aliases` | `bool` | | Use the service's network useAliases in the network(s) the container connects to | -| `-u`, `--user` | `string` | | Run as specified username or uid | -| `-v`, `--volume` | `stringArray` | | Bind mount a volume | -| `-w`, `--workdir` | `string` | | Working directory inside the container | - - - - -## Description - -Runs a one-time command against a service. - -The following command starts the `web` service and runs `bash` as its command: - -```console -$ docker compose run web bash -``` - -Commands you use with run start in new containers with configuration defined by that of the service, -including volumes, links, and other details. However, there are two important differences: - -First, the command passed by `run` overrides the command defined in the service configuration. For example, if the -`web` service configuration is started with `bash`, then `docker compose run web python app.py` overrides it with -`python app.py`. - -The second difference is that the `docker compose run` command does not create any of the ports specified in the -service configuration. This prevents port collisions with already-open ports. If you do want the service’s ports -to be created and mapped to the host, specify the `--service-ports` - -```console -$ docker compose run --service-ports web python manage.py shell -``` - -Alternatively, manual port mapping can be specified with the `--publish` or `-p` options, just as when using docker run: - -```console -$ docker compose run --publish 8080:80 -p 2022:22 -p 127.0.0.1:2021:21 web python manage.py shell -``` - -If you start a service configured with links, the run command first checks to see if the linked service is running -and starts the service if it is stopped. Once all the linked services are running, the run executes the command you -passed it. For example, you could run: - -```console -$ docker compose run db psql -h db -U docker -``` - -This opens an interactive PostgreSQL shell for the linked `db` container. - -If you do not want the run command to start linked containers, use the `--no-deps` flag: - -```console -$ docker compose run --no-deps web python manage.py shell -``` - -If you want to remove the container after running while overriding the container’s restart policy, use the `--rm` flag: - -```console -$ docker compose run --rm web python manage.py db upgrade -``` - -This runs a database upgrade script, and removes the container when finished running, even if a restart policy is -specified in the service configuration. diff --git a/docs/reference/compose_scale.md b/docs/reference/compose_scale.md deleted file mode 100644 index 3d0dbdb04a2..00000000000 --- a/docs/reference/compose_scale.md +++ /dev/null @@ -1,15 +0,0 @@ -# docker compose scale - - -Scale services - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:--------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--no-deps` | `bool` | | Don't start linked services | - - - - diff --git a/docs/reference/compose_start.md b/docs/reference/compose_start.md deleted file mode 100644 index 06229e5940e..00000000000 --- a/docs/reference/compose_start.md +++ /dev/null @@ -1,19 +0,0 @@ -# docker compose start - - -Starts existing containers for a service - -### Options - -| Name | Type | Default | Description | -|:-----------------|:-------|:--------|:---------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--wait` | `bool` | | Wait for services to be running\|healthy. Implies detached mode. | -| `--wait-timeout` | `int` | `0` | Maximum duration in seconds to wait for the project to be running\|healthy | - - - - -## Description - -Starts existing containers for a service diff --git a/docs/reference/compose_stats.md b/docs/reference/compose_stats.md deleted file mode 100644 index 78d44b89350..00000000000 --- a/docs/reference/compose_stats.md +++ /dev/null @@ -1,18 +0,0 @@ -# docker compose stats - - -Display a live stream of container(s) resource usage statistics - -### Options - -| Name | Type | Default | Description | -|:--------------|:---------|:--------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `-a`, `--all` | `bool` | | Show all containers (default shows just running) | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--format` | `string` | | Format output using a custom template: 'table': Print output in table format with column headers (default) 'table TEMPLATE': Print output in table format using the given Go template 'json': Print in JSON format 'TEMPLATE': Print output using the given Go template. Refer to https://docs.docker.com/engine/cli/formatting/ for more information about formatting output with templates | -| `--no-stream` | `bool` | | Disable streaming stats and only pull the first result | -| `--no-trunc` | `bool` | | Do not truncate output | - - - - diff --git a/docs/reference/compose_stop.md b/docs/reference/compose_stop.md deleted file mode 100644 index fe84f24f8f5..00000000000 --- a/docs/reference/compose_stop.md +++ /dev/null @@ -1,18 +0,0 @@ -# docker compose stop - - -Stops running containers without removing them. They can be started again with `docker compose start`. - -### Options - -| Name | Type | Default | Description | -|:------------------|:-------|:--------|:--------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-t`, `--timeout` | `int` | `0` | Specify a shutdown timeout in seconds | - - - - -## Description - -Stops running containers without removing them. They can be started again with `docker compose start`. diff --git a/docs/reference/compose_top.md b/docs/reference/compose_top.md deleted file mode 100644 index eeacb3866aa..00000000000 --- a/docs/reference/compose_top.md +++ /dev/null @@ -1,26 +0,0 @@ -# docker compose top - - -Displays the running processes - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:--------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | - - - - -## Description - -Displays the running processes - -## Examples - -```console -$ docker compose top -example_foo_1 -UID PID PPID C STIME TTY TIME CMD -root 142353 142331 2 15:33 ? 00:00:00 ping localhost -c 5 -``` diff --git a/docs/reference/compose_unpause.md b/docs/reference/compose_unpause.md deleted file mode 100644 index 92841ceade7..00000000000 --- a/docs/reference/compose_unpause.md +++ /dev/null @@ -1,17 +0,0 @@ -# docker compose unpause - - -Unpauses paused containers of a service - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:--------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | - - - - -## Description - -Unpauses paused containers of a service diff --git a/docs/reference/compose_up.md b/docs/reference/compose_up.md deleted file mode 100644 index b7f17a0fac9..00000000000 --- a/docs/reference/compose_up.md +++ /dev/null @@ -1,82 +0,0 @@ -# docker compose up - - -Builds, (re)creates, starts, and attaches to containers for a service. - -Unless they are already running, this command also starts any linked services. - -The `docker compose up` command aggregates the output of each container (like `docker compose logs --follow` does). -One can optionally select a subset of services to attach to using `--attach` flag, or exclude some services using -`--no-attach` to prevent output to be flooded by some verbose services. - -When the command exits, all containers are stopped. Running `docker compose up --detach` starts the containers in the -background and leaves them running. - -If there are existing containers for a service, and the service’s configuration or image was changed after the -container’s creation, `docker compose up` picks up the changes by stopping and recreating the containers -(preserving mounted volumes). To prevent Compose from picking up changes, use the `--no-recreate` flag. - -If you want to force Compose to stop and recreate all containers, use the `--force-recreate` flag. - -If the process encounters an error, the exit code for this command is `1`. -If the process is interrupted using `SIGINT` (ctrl + C) or `SIGTERM`, the containers are stopped, and the exit code is `0`. - -### Options - -| Name | Type | Default | Description | -|:-------------------------------|:--------------|:---------|:----------------------------------------------------------------------------------------------------------------------------------------------------| -| `--abort-on-container-exit` | `bool` | | Stops all containers if any container was stopped. Incompatible with -d | -| `--abort-on-container-failure` | `bool` | | Stops all containers if any container exited with failure. Incompatible with -d | -| `--always-recreate-deps` | `bool` | | Recreate dependent containers. Incompatible with --no-recreate. | -| `--attach` | `stringArray` | | Restrict attaching to the specified services. Incompatible with --attach-dependencies. | -| `--attach-dependencies` | `bool` | | Automatically attach to log output of dependent services | -| `--build` | `bool` | | Build images before starting containers | -| `-d`, `--detach` | `bool` | | Detached mode: Run containers in the background | -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--exit-code-from` | `string` | | Return the exit code of the selected service container. Implies --abort-on-container-exit | -| `--force-recreate` | `bool` | | Recreate containers even if their configuration and image haven't changed | -| `--menu` | `bool` | | Enable interactive shortcuts when running attached. Incompatible with --detach. Can also be enable/disable by setting COMPOSE_MENU environment var. | -| `--no-attach` | `stringArray` | | Do not attach (stream logs) to the specified services | -| `--no-build` | `bool` | | Don't build an image, even if it's policy | -| `--no-color` | `bool` | | Produce monochrome output | -| `--no-deps` | `bool` | | Don't start linked services | -| `--no-log-prefix` | `bool` | | Don't print prefix in logs | -| `--no-recreate` | `bool` | | If containers already exist, don't recreate them. Incompatible with --force-recreate. | -| `--no-start` | `bool` | | Don't start the services after creating them | -| `--pull` | `string` | `policy` | Pull image before running ("always"\|"missing"\|"never") | -| `--quiet-build` | `bool` | | Suppress the build output | -| `--quiet-pull` | `bool` | | Pull without printing progress information | -| `--remove-orphans` | `bool` | | Remove containers for services not defined in the Compose file | -| `-V`, `--renew-anon-volumes` | `bool` | | Recreate anonymous volumes instead of retrieving data from the previous containers | -| `--scale` | `stringArray` | | Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present. | -| `-t`, `--timeout` | `int` | `0` | Use this timeout in seconds for container shutdown when attached or when containers are already running | -| `--timestamps` | `bool` | | Show timestamps | -| `--wait` | `bool` | | Wait for services to be running\|healthy. Implies detached mode. | -| `--wait-timeout` | `int` | `0` | Maximum duration in seconds to wait for the project to be running\|healthy | -| `-w`, `--watch` | `bool` | | Watch source code and rebuild/refresh containers when files are updated. | -| `-y`, `--yes` | `bool` | | Assume "yes" as answer to all prompts and run non-interactively | - - - - -## Description - -Builds, (re)creates, starts, and attaches to containers for a service. - -Unless they are already running, this command also starts any linked services. - -The `docker compose up` command aggregates the output of each container (like `docker compose logs --follow` does). -One can optionally select a subset of services to attach to using `--attach` flag, or exclude some services using -`--no-attach` to prevent output to be flooded by some verbose services. - -When the command exits, all containers are stopped. Running `docker compose up --detach` starts the containers in the -background and leaves them running. - -If there are existing containers for a service, and the service’s configuration or image was changed after the -container’s creation, `docker compose up` picks up the changes by stopping and recreating the containers -(preserving mounted volumes). To prevent Compose from picking up changes, use the `--no-recreate` flag. - -If you want to force Compose to stop and recreate all containers, use the `--force-recreate` flag. - -If the process encounters an error, the exit code for this command is `1`. -If the process is interrupted using `SIGINT` (ctrl + C) or `SIGTERM`, the containers are stopped, and the exit code is `0`. diff --git a/docs/reference/compose_version.md b/docs/reference/compose_version.md deleted file mode 100644 index 3a6329dadb4..00000000000 --- a/docs/reference/compose_version.md +++ /dev/null @@ -1,15 +0,0 @@ -# docker compose version - - -Show the Docker Compose version information - -### Options - -| Name | Type | Default | Description | -|:-----------------|:---------|:--------|:---------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `-f`, `--format` | `string` | | Format the output. Values: [pretty \| json]. (Default: pretty) | -| `--short` | `bool` | | Shows only Compose's version number | - - - diff --git a/docs/reference/compose_volumes.md b/docs/reference/compose_volumes.md deleted file mode 100644 index 6bad874f187..00000000000 --- a/docs/reference/compose_volumes.md +++ /dev/null @@ -1,16 +0,0 @@ -# docker compose volumes - - -List volumes - -### Options - -| Name | Type | Default | Description | -|:----------------|:---------|:--------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--format` | `string` | `table` | Format output using a custom template: 'table': Print output in table format with column headers (default) 'table TEMPLATE': Print output in table format using the given Go template 'json': Print in JSON format 'TEMPLATE': Print output using the given Go template. Refer to https://docs.docker.com/go/formatting/ for more information about formatting output with templates | -| `-q`, `--quiet` | `bool` | | Only display volume names | - - - - diff --git a/docs/reference/compose_wait.md b/docs/reference/compose_wait.md deleted file mode 100644 index 59474c9b509..00000000000 --- a/docs/reference/compose_wait.md +++ /dev/null @@ -1,15 +0,0 @@ -# docker compose wait - - -Block until containers of all (or specified) services stop. - -### Options - -| Name | Type | Default | Description | -|:-----------------|:-------|:--------|:---------------------------------------------| -| `--down-project` | `bool` | | Drops project when the first container stops | -| `--dry-run` | `bool` | | Execute command in dry run mode | - - - - diff --git a/docs/reference/compose_watch.md b/docs/reference/compose_watch.md deleted file mode 100644 index f6040c9094f..00000000000 --- a/docs/reference/compose_watch.md +++ /dev/null @@ -1,17 +0,0 @@ -# docker compose watch - - -Watch build context for service and rebuild/refresh containers when files are updated - -### Options - -| Name | Type | Default | Description | -|:------------|:-------|:--------|:----------------------------------------------| -| `--dry-run` | `bool` | | Execute command in dry run mode | -| `--no-up` | `bool` | | Do not build & start services before watching | -| `--prune` | `bool` | `true` | Prune dangling images on rebuild | -| `--quiet` | `bool` | | hide build output | - - - - diff --git a/docs/reference/docker_compose.yaml b/docs/reference/docker_compose.yaml deleted file mode 100644 index c5fdb937510..00000000000 --- a/docs/reference/docker_compose.yaml +++ /dev/null @@ -1,445 +0,0 @@ -command: docker compose -short: Docker Compose -long: Define and run multi-container applications with Docker -usage: docker compose -pname: docker -plink: docker.yaml -cname: - - docker compose attach - - docker compose bridge - - docker compose build - - docker compose commit - - docker compose config - - docker compose cp - - docker compose create - - docker compose down - - docker compose events - - docker compose exec - - docker compose export - - docker compose images - - docker compose kill - - docker compose logs - - docker compose ls - - docker compose pause - - docker compose port - - docker compose ps - - docker compose publish - - docker compose pull - - docker compose push - - docker compose restart - - docker compose rm - - docker compose run - - docker compose scale - - docker compose start - - docker compose stats - - docker compose stop - - docker compose top - - docker compose unpause - - docker compose up - - docker compose version - - docker compose volumes - - docker compose wait - - docker compose watch -clink: - - docker_compose_attach.yaml - - docker_compose_bridge.yaml - - docker_compose_build.yaml - - docker_compose_commit.yaml - - docker_compose_config.yaml - - docker_compose_cp.yaml - - docker_compose_create.yaml - - docker_compose_down.yaml - - docker_compose_events.yaml - - docker_compose_exec.yaml - - docker_compose_export.yaml - - docker_compose_images.yaml - - docker_compose_kill.yaml - - docker_compose_logs.yaml - - docker_compose_ls.yaml - - docker_compose_pause.yaml - - docker_compose_port.yaml - - docker_compose_ps.yaml - - docker_compose_publish.yaml - - docker_compose_pull.yaml - - docker_compose_push.yaml - - docker_compose_restart.yaml - - docker_compose_rm.yaml - - docker_compose_run.yaml - - docker_compose_scale.yaml - - docker_compose_start.yaml - - docker_compose_stats.yaml - - docker_compose_stop.yaml - - docker_compose_top.yaml - - docker_compose_unpause.yaml - - docker_compose_up.yaml - - docker_compose_version.yaml - - docker_compose_volumes.yaml - - docker_compose_wait.yaml - - docker_compose_watch.yaml -options: - - option: all-resources - value_type: bool - default_value: "false" - description: Include all resources, even those not used by services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: ansi - value_type: string - default_value: auto - description: | - Control when to print ANSI control characters ("never"|"always"|"auto") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: compatibility - value_type: bool - default_value: "false" - description: Run compose in backward compatibility mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: env-file - value_type: stringArray - default_value: '[]' - description: Specify an alternate environment file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: file - shorthand: f - value_type: stringArray - default_value: '[]' - description: Compose configuration files - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: insecure-registry - value_type: stringArray - default_value: '[]' - description: | - Use insecure registry to pull Compose OCI artifacts. Doesn't apply to images - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-ansi - value_type: bool - default_value: "false" - description: Do not print ANSI control characters (DEPRECATED) - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: parallel - value_type: int - default_value: "-1" - description: Control max parallelism, -1 for unlimited - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: profile - value_type: stringArray - default_value: '[]' - description: Specify a profile to enable - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: progress - value_type: string - description: Set type of progress output (auto, tty, plain, json, quiet) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: project-directory - value_type: string - description: |- - Specify an alternate working directory - (default: the path of the, first specified, Compose file) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: project-name - shorthand: p - value_type: string - description: Project name - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: verbose - value_type: bool - default_value: "false" - description: Show more output - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: version - shorthand: v - value_type: bool - default_value: "false" - description: Show the Docker Compose version information - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: workdir - value_type: string - description: |- - DEPRECATED! USE --project-directory INSTEAD. - Specify an alternate working directory - (default: the path of the, first specified, Compose file) - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -examples: |- - ### Use `-f` to specify the name and path of one or more Compose files - Use the `-f` flag to specify the location of a Compose [configuration file](/reference/compose-file/). - - #### Specifying multiple Compose files - You can supply multiple `-f` configuration files. When you supply multiple files, Compose combines them into a single - configuration. Compose builds the configuration in the order you supply the files. Subsequent files override and add - to their predecessors. - - For example, consider this command line: - - ```console - $ docker compose -f compose.yaml -f compose.admin.yaml run backup_db - ``` - - The `compose.yaml` file might specify a `webapp` service. - - ```yaml - services: - webapp: - image: examples/web - ports: - - "8000:8000" - volumes: - - "/data" - ``` - If the `compose.admin.yaml` also specifies this same service, any matching fields override the previous file. - New values, add to the `webapp` service configuration. - - ```yaml - services: - webapp: - build: . - environment: - - DEBUG=1 - ``` - - When you use multiple Compose files, all paths in the files are relative to the first configuration file specified - with `-f`. You can use the `--project-directory` option to override this base path. - - Use a `-f` with `-` (dash) as the filename to read the configuration from stdin. When stdin is used all paths in the - configuration are relative to the current working directory. - - The `-f` flag is optional. If you don’t provide this flag on the command line, Compose traverses the working directory - and its parent directories looking for a `compose.yaml` or `docker-compose.yaml` file. - - #### Specifying a path to a single Compose file - You can use the `-f` flag to specify a path to a Compose file that is not located in the current directory, either - from the command line or by setting up a `COMPOSE_FILE` environment variable in your shell or in an environment file. - - For an example of using the `-f` option at the command line, suppose you are running the Compose Rails sample, and - have a `compose.yaml` file in a directory called `sandbox/rails`. You can use a command like `docker compose pull` to - get the postgres image for the db service from anywhere by using the `-f` flag as follows: - - ```console - $ docker compose -f ~/sandbox/rails/compose.yaml pull db - ``` - - #### Using an OCI published artifact - You can use the `-f` flag with the `oci://` prefix to reference a Compose file that has been published to an OCI registry. - This allows you to distribute and version your Compose configurations as OCI artifacts. - - To use a Compose file from an OCI registry: - - ```console - $ docker compose -f oci://registry.example.com/my-compose-project:latest up - ``` - - You can also combine OCI artifacts with local files: - - ```console - $ docker compose -f oci://registry.example.com/my-compose-project:v1.0 -f compose.override.yaml up - ``` - - The OCI artifact must contain a valid Compose file. You can publish Compose files to an OCI registry using the - `docker compose publish` command. - - #### Using a git repository - You can use the `-f` flag to reference a Compose file from a git repository. Compose supports various git URL formats: - - Using HTTPS: - ```console - $ docker compose -f https://github.com/user/repo.git up - ``` - - Using SSH: - ```console - $ docker compose -f git@github.com:user/repo.git up - ``` - - You can specify a specific branch, tag, or commit: - ```console - $ docker compose -f https://github.com/user/repo.git@main up - $ docker compose -f https://github.com/user/repo.git@v1.0.0 up - $ docker compose -f https://github.com/user/repo.git@abc123 up - ``` - - You can also specify a subdirectory within the repository: - ```console - $ docker compose -f https://github.com/user/repo.git#main:path/to/compose.yaml up - ``` - - When using git resources, Compose will clone the repository and use the specified Compose file. You can combine - git resources with local files: - - ```console - $ docker compose -f https://github.com/user/repo.git -f compose.override.yaml up - ``` - - ### Use `-p` to specify a project name - - Each configuration has a project name. Compose sets the project name using - the following mechanisms, in order of precedence: - - The `-p` command line flag - - The `COMPOSE_PROJECT_NAME` environment variable - - The top level `name:` variable from the config file (or the last `name:` - from a series of config files specified using `-f`) - - The `basename` of the project directory containing the config file (or - containing the first config file specified using `-f`) - - The `basename` of the current directory if no config file is specified - Project names must contain only lowercase letters, decimal digits, dashes, - and underscores, and must begin with a lowercase letter or decimal digit. If - the `basename` of the project directory or current directory violates this - constraint, you must use one of the other mechanisms. - - ```console - $ docker compose -p my_project ps -a - NAME SERVICE STATUS PORTS - my_project_demo_1 demo running - - $ docker compose -p my_project logs - demo_1 | PING localhost (127.0.0.1): 56 data bytes - demo_1 | 64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.095 ms - ``` - - ### Use profiles to enable optional services - - Use `--profile` to specify one or more active profiles - Calling `docker compose --profile frontend up` starts the services with the profile `frontend` and services - without any specified profiles. - You can also enable multiple profiles, e.g. with `docker compose --profile frontend --profile debug up` the profiles `frontend` and `debug` is enabled. - - Profiles can also be set by `COMPOSE_PROFILES` environment variable. - - ### Configuring parallelism - - Use `--parallel` to specify the maximum level of parallelism for concurrent engine calls. - Calling `docker compose --parallel 1 pull` pulls the pullable images defined in the Compose file - one at a time. This can also be used to control build concurrency. - - Parallelism can also be set by the `COMPOSE_PARALLEL_LIMIT` environment variable. - - ### Set up environment variables - - You can set environment variables for various docker compose options, including the `-f`, `-p` and `--profiles` flags. - - Setting the `COMPOSE_FILE` environment variable is equivalent to passing the `-f` flag, - `COMPOSE_PROJECT_NAME` environment variable does the same as the `-p` flag, - `COMPOSE_PROFILES` environment variable is equivalent to the `--profiles` flag - and `COMPOSE_PARALLEL_LIMIT` does the same as the `--parallel` flag. - - If flags are explicitly set on the command line, the associated environment variable is ignored. - - Setting the `COMPOSE_IGNORE_ORPHANS` environment variable to `true` stops docker compose from detecting orphaned - containers for the project. - - Setting the `COMPOSE_MENU` environment variable to `false` disables the helper menu when running `docker compose up` - in attached mode. Alternatively, you can also run `docker compose up --menu=false` to disable the helper menu. - - ### Use Dry Run mode to test your command - - Use `--dry-run` flag to test a command without changing your application stack state. - Dry Run mode shows you all the steps Compose applies when executing a command, for example: - ```console - $ docker compose --dry-run up --build -d - [+] Pulling 1/1 - ✔ DRY-RUN MODE - db Pulled 0.9s - [+] Running 10/8 - ✔ DRY-RUN MODE - build service backend 0.0s - ✔ DRY-RUN MODE - ==> ==> writing image dryRun-754a08ddf8bcb1cf22f310f09206dd783d42f7dd 0.0s - ✔ DRY-RUN MODE - ==> ==> naming to nginx-golang-mysql-backend 0.0s - ✔ DRY-RUN MODE - Network nginx-golang-mysql_default Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-db-1 Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-backend-1 Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-proxy-1 Created 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-db-1 Healthy 0.5s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-backend-1 Started 0.0s - ✔ DRY-RUN MODE - Container nginx-golang-mysql-proxy-1 Started Started - ``` - From the example above, you can see that the first step is to pull the image defined by `db` service, then build the `backend` service. - Next, the containers are created. The `db` service is started, and the `backend` and `proxy` wait until the `db` service is healthy before starting. - - Dry Run mode works with almost all commands. You cannot use Dry Run mode with a command that doesn't change the state of a Compose stack such as `ps`, `ls`, `logs` for example. -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha.yaml b/docs/reference/docker_compose_alpha.yaml deleted file mode 100644 index e6b6b6e6b6f..00000000000 --- a/docs/reference/docker_compose_alpha.yaml +++ /dev/null @@ -1,31 +0,0 @@ -command: docker compose alpha -short: Experimental commands -long: Experimental commands -pname: docker compose -plink: docker_compose.yaml -cname: - - docker compose alpha generate - - docker compose alpha publish - - docker compose alpha viz -clink: - - docker_compose_alpha_generate.yaml - - docker_compose_alpha_publish.yaml - - docker_compose_alpha_viz.yaml -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: true -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha_dry-run.yaml b/docs/reference/docker_compose_alpha_dry-run.yaml deleted file mode 100644 index d489d39aeba..00000000000 --- a/docs/reference/docker_compose_alpha_dry-run.yaml +++ /dev/null @@ -1,14 +0,0 @@ -command: docker compose alpha dry-run -short: | - EXPERIMENTAL - Dry run command allow you to test a command without applying changes -long: | - EXPERIMENTAL - Dry run command allow you to test a command without applying changes -usage: docker compose alpha dry-run -- [COMMAND...] -pname: docker compose alpha -plink: docker_compose_alpha.yaml -deprecated: false -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha_generate.yaml b/docs/reference/docker_compose_alpha_generate.yaml deleted file mode 100644 index f31429c2d72..00000000000 --- a/docs/reference/docker_compose_alpha_generate.yaml +++ /dev/null @@ -1,53 +0,0 @@ -command: docker compose alpha generate -short: EXPERIMENTAL - Generate a Compose file from existing containers -long: EXPERIMENTAL - Generate a Compose file from existing containers -usage: docker compose alpha generate [OPTIONS] [CONTAINERS...] -pname: docker compose alpha -plink: docker_compose_alpha.yaml -options: - - option: format - value_type: string - default_value: yaml - description: 'Format the output. Values: [yaml | json]' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: name - value_type: string - description: Project name to set in the Compose file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: project-dir - value_type: string - description: Directory to use for the project - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: true -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha_publish.yaml b/docs/reference/docker_compose_alpha_publish.yaml deleted file mode 100644 index 9059cbf4869..00000000000 --- a/docs/reference/docker_compose_alpha_publish.yaml +++ /dev/null @@ -1,86 +0,0 @@ -command: docker compose alpha publish -short: Publish compose application -long: Publish compose application -usage: docker compose alpha publish [OPTIONS] REPOSITORY[:TAG] -pname: docker compose alpha -plink: docker_compose_alpha.yaml -options: - - option: app - value_type: bool - default_value: "false" - description: Published compose application (includes referenced images) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: insecure-registry - value_type: bool - default_value: "false" - description: Use insecure registry - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: oci-version - value_type: string - description: | - OCI image/artifact specification version (automatically determined by default) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: resolve-image-digests - value_type: bool - default_value: "false" - description: Pin image tags to digests - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: with-env - value_type: bool - default_value: "false" - description: Include environment variables in the published OCI artifact - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: "yes" - shorthand: "y" - value_type: bool - default_value: "false" - description: Assume "yes" as answer to all prompts - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: true -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha_scale.yaml b/docs/reference/docker_compose_alpha_scale.yaml deleted file mode 100644 index cc381493fa3..00000000000 --- a/docs/reference/docker_compose_alpha_scale.yaml +++ /dev/null @@ -1,35 +0,0 @@ -command: docker compose alpha scale -short: Scale services -long: Scale services -usage: docker compose alpha scale [SERVICE=REPLICAS...] -pname: docker compose alpha -plink: docker_compose_alpha.yaml -options: - - option: no-deps - value_type: bool - default_value: "false" - description: Don't start linked services. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha_viz.yaml b/docs/reference/docker_compose_alpha_viz.yaml deleted file mode 100644 index c07475caac8..00000000000 --- a/docs/reference/docker_compose_alpha_viz.yaml +++ /dev/null @@ -1,77 +0,0 @@ -command: docker compose alpha viz -short: EXPERIMENTAL - Generate a graphviz graph from your compose file -long: EXPERIMENTAL - Generate a graphviz graph from your compose file -usage: docker compose alpha viz [OPTIONS] -pname: docker compose alpha -plink: docker_compose_alpha.yaml -options: - - option: image - value_type: bool - default_value: "false" - description: Include service's image name in output graph - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: indentation-size - value_type: int - default_value: "1" - description: Number of tabs or spaces to use for indentation - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: networks - value_type: bool - default_value: "false" - description: Include service's attached networks in output graph - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: ports - value_type: bool - default_value: "false" - description: Include service's exposed ports in output graph - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: spaces - value_type: bool - default_value: "false" - description: |- - If given, space character ' ' will be used to indent, - otherwise tab character '\t' will be used - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: true -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_alpha_watch.yaml b/docs/reference/docker_compose_alpha_watch.yaml deleted file mode 100644 index e8a7d9845ab..00000000000 --- a/docs/reference/docker_compose_alpha_watch.yaml +++ /dev/null @@ -1,47 +0,0 @@ -command: docker compose alpha watch -short: | - Watch build context for service and rebuild/refresh containers when files are updated -long: | - Watch build context for service and rebuild/refresh containers when files are updated -usage: docker compose alpha watch [SERVICE...] -pname: docker compose alpha -plink: docker_compose_alpha.yaml -options: - - option: no-up - value_type: bool - default_value: "false" - description: Do not build & start services before watching - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - value_type: bool - default_value: "false" - description: hide build output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: true -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_attach.yaml b/docs/reference/docker_compose_attach.yaml deleted file mode 100644 index 8fd6957ca18..00000000000 --- a/docs/reference/docker_compose_attach.yaml +++ /dev/null @@ -1,66 +0,0 @@ -command: docker compose attach -short: | - Attach local standard input, output, and error streams to a service's running container -long: | - Attach local standard input, output, and error streams to a service's running container -usage: docker compose attach [OPTIONS] SERVICE -pname: docker compose -plink: docker_compose.yaml -options: - - option: detach-keys - value_type: string - description: Override the key sequence for detaching from a container. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: index - value_type: int - default_value: "0" - description: index of the container if service has multiple replicas. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-stdin - value_type: bool - default_value: "false" - description: Do not attach STDIN - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: sig-proxy - value_type: bool - default_value: "true" - description: Proxy all received signals to the process - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_bridge.yaml b/docs/reference/docker_compose_bridge.yaml deleted file mode 100644 index 5ef9ebf5585..00000000000 --- a/docs/reference/docker_compose_bridge.yaml +++ /dev/null @@ -1,29 +0,0 @@ -command: docker compose bridge -short: Convert compose files into another model -long: Convert compose files into another model -pname: docker compose -plink: docker_compose.yaml -cname: - - docker compose bridge convert - - docker compose bridge transformations -clink: - - docker_compose_bridge_convert.yaml - - docker_compose_bridge_transformations.yaml -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_bridge_convert.yaml b/docs/reference/docker_compose_bridge_convert.yaml deleted file mode 100644 index f55f0b233c3..00000000000 --- a/docs/reference/docker_compose_bridge_convert.yaml +++ /dev/null @@ -1,59 +0,0 @@ -command: docker compose bridge convert -short: | - Convert compose files to Kubernetes manifests, Helm charts, or another model -long: | - Convert compose files to Kubernetes manifests, Helm charts, or another model -usage: docker compose bridge convert -pname: docker compose bridge -plink: docker_compose_bridge.yaml -options: - - option: output - shorthand: o - value_type: string - default_value: out - description: The output directory for the Kubernetes resources - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: templates - value_type: string - description: Directory containing transformation templates - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: transformation - shorthand: t - value_type: stringArray - default_value: '[]' - description: | - Transformation to apply to compose model (default: docker/compose-bridge-kubernetes) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_bridge_transformations.yaml b/docs/reference/docker_compose_bridge_transformations.yaml deleted file mode 100644 index 2ab5661f0b2..00000000000 --- a/docs/reference/docker_compose_bridge_transformations.yaml +++ /dev/null @@ -1,29 +0,0 @@ -command: docker compose bridge transformations -short: Manage transformation images -long: Manage transformation images -pname: docker compose bridge -plink: docker_compose_bridge.yaml -cname: - - docker compose bridge transformations create - - docker compose bridge transformations list -clink: - - docker_compose_bridge_transformations_create.yaml - - docker_compose_bridge_transformations_list.yaml -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_bridge_transformations_create.yaml b/docs/reference/docker_compose_bridge_transformations_create.yaml deleted file mode 100644 index e8dd9e58a51..00000000000 --- a/docs/reference/docker_compose_bridge_transformations_create.yaml +++ /dev/null @@ -1,36 +0,0 @@ -command: docker compose bridge transformations create -short: Create a new transformation -long: Create a new transformation -usage: docker compose bridge transformations create [OPTION] PATH -pname: docker compose bridge transformations -plink: docker_compose_bridge_transformations.yaml -options: - - option: from - shorthand: f - value_type: string - description: | - Existing transformation to copy (default: docker/compose-bridge-kubernetes) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_bridge_transformations_list.yaml b/docs/reference/docker_compose_bridge_transformations_list.yaml deleted file mode 100644 index 3afd3a84b8e..00000000000 --- a/docs/reference/docker_compose_bridge_transformations_list.yaml +++ /dev/null @@ -1,47 +0,0 @@ -command: docker compose bridge transformations list -aliases: docker compose bridge transformations list, docker compose bridge transformations ls -short: List available transformations -long: List available transformations -usage: docker compose bridge transformations list -pname: docker compose bridge transformations -plink: docker_compose_bridge_transformations.yaml -options: - - option: format - value_type: string - default_value: table - description: 'Format the output. Values: [table | json]' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only display transformer names - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_build.yaml b/docs/reference/docker_compose_build.yaml deleted file mode 100644 index e645a40aac2..00000000000 --- a/docs/reference/docker_compose_build.yaml +++ /dev/null @@ -1,214 +0,0 @@ -command: docker compose build -short: Build or rebuild services -long: |- - Services are built once and then tagged, by default as `project-service`. - - If the Compose file specifies an - [image](https://github.com/compose-spec/compose-spec/blob/main/spec.md#image) name, - the image is tagged with that name, substituting any variables beforehand. See - [variable interpolation](https://github.com/compose-spec/compose-spec/blob/main/spec.md#interpolation). - - If you change a service's `Dockerfile` or the contents of its build directory, - run `docker compose build` to rebuild it. -usage: docker compose build [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: build-arg - value_type: stringArray - default_value: '[]' - description: Set build-time variables for services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: builder - value_type: string - description: Set builder to use - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: check - value_type: bool - default_value: "false" - description: Check build configuration - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: compress - value_type: bool - default_value: "true" - description: Compress the build context using gzip. DEPRECATED - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: force-rm - value_type: bool - default_value: "true" - description: Always remove intermediate containers. DEPRECATED - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: memory - shorthand: m - value_type: bytes - default_value: "0" - description: | - Set memory limit for the build container. Not supported by BuildKit. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-cache - value_type: bool - default_value: "false" - description: Do not use cache when building the image - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-rm - value_type: bool - default_value: "false" - description: | - Do not remove intermediate containers after a successful build. DEPRECATED - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: parallel - value_type: bool - default_value: "true" - description: Build images in parallel. DEPRECATED - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: print - value_type: bool - default_value: "false" - description: Print equivalent bake file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: progress - value_type: string - description: Set type of ui output (auto, tty, plain, json, quiet) - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: provenance - value_type: string - description: Add a provenance attestation - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: pull - value_type: bool - default_value: "false" - description: Always attempt to pull a newer version of the image - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: push - value_type: bool - default_value: "false" - description: Push service images - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Suppress the build output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: sbom - value_type: string - description: Add a SBOM attestation - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: ssh - value_type: string - description: | - Set SSH authentications used when building service images. (use 'default' for using your default SSH Agent) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: with-dependencies - value_type: bool - default_value: "false" - description: Also build dependencies (transitively) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_commit.yaml b/docs/reference/docker_compose_commit.yaml deleted file mode 100644 index 95f4834a97b..00000000000 --- a/docs/reference/docker_compose_commit.yaml +++ /dev/null @@ -1,76 +0,0 @@ -command: docker compose commit -short: Create a new image from a service container's changes -long: Create a new image from a service container's changes -usage: docker compose commit [OPTIONS] SERVICE [REPOSITORY[:TAG]] -pname: docker compose -plink: docker_compose.yaml -options: - - option: author - shorthand: a - value_type: string - description: Author (e.g., "John Hannibal Smith <hannibal@a-team.com>") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: change - shorthand: c - value_type: list - description: Apply Dockerfile instruction to the created image - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: index - value_type: int - default_value: "0" - description: index of the container if service has multiple replicas. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: message - shorthand: m - value_type: string - description: Commit message - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: pause - shorthand: p - value_type: bool - default_value: "true" - description: Pause container during commit - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_config.yaml b/docs/reference/docker_compose_config.yaml deleted file mode 100644 index 3efc922b219..00000000000 --- a/docs/reference/docker_compose_config.yaml +++ /dev/null @@ -1,218 +0,0 @@ -command: docker compose config -short: Parse, resolve and render compose file in canonical format -long: |- - `docker compose config` renders the actual data model to be applied on the Docker Engine. - It merges the Compose files set by `-f` flags, resolves variables in the Compose file, and expands short-notation into - the canonical format. -usage: docker compose config [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: environment - value_type: bool - default_value: "false" - description: Print environment used for interpolation. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: format - value_type: string - description: 'Format the output. Values: [yaml | json]' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: hash - value_type: string - description: Print the service config hash, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: images - value_type: bool - default_value: "false" - description: Print the image names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: lock-image-digests - value_type: bool - default_value: "false" - description: Produces an override file with image digests - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: models - value_type: bool - default_value: "false" - description: Print the model names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: networks - value_type: bool - default_value: "false" - description: Print the network names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-consistency - value_type: bool - default_value: "false" - description: | - Don't check model consistency - warning: may produce invalid Compose output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-env-resolution - value_type: bool - default_value: "false" - description: Don't resolve service env files - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-interpolate - value_type: bool - default_value: "false" - description: Don't interpolate environment variables - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-normalize - value_type: bool - default_value: "false" - description: Don't normalize compose model - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-path-resolution - value_type: bool - default_value: "false" - description: Don't resolve file paths - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: output - shorthand: o - value_type: string - description: Save to file (default to stdout) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: profiles - value_type: bool - default_value: "false" - description: Print the profile names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only validate the configuration, don't print anything - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: resolve-image-digests - value_type: bool - default_value: "false" - description: Pin image tags to digests - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: services - value_type: bool - default_value: "false" - description: Print the service names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: variables - value_type: bool - default_value: "false" - description: Print model variables and default values. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: volumes - value_type: bool - default_value: "false" - description: Print the volume names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_convert.yaml b/docs/reference/docker_compose_convert.yaml deleted file mode 100644 index d1913221968..00000000000 --- a/docs/reference/docker_compose_convert.yaml +++ /dev/null @@ -1,140 +0,0 @@ -command: docker compose convert -aliases: docker compose convert, docker compose config -short: Converts the compose file to platform's canonical format -long: |- - `docker compose convert` renders the actual data model to be applied on the target platform. When used with the Docker engine, - it merges the Compose files set by `-f` flags, resolves variables in the Compose file, and expands short-notation into - the canonical format. - - To allow smooth migration from docker-compose, this subcommand declares alias `docker compose config` -usage: docker compose convert [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: format - value_type: string - default_value: yaml - description: 'Format the output. Values: [yaml | json]' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: hash - value_type: string - description: Print the service config hash, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: images - value_type: bool - default_value: "false" - description: Print the image names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-consistency - value_type: bool - default_value: "false" - description: | - Don't check model consistency - warning: may produce invalid Compose output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-interpolate - value_type: bool - default_value: "false" - description: Don't interpolate environment variables. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-normalize - value_type: bool - default_value: "false" - description: Don't normalize compose model. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: output - shorthand: o - value_type: string - description: Save to file (default to stdout) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: profiles - value_type: bool - default_value: "false" - description: Print the profile names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only validate the configuration, don't print anything. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: resolve-image-digests - value_type: bool - default_value: "false" - description: Pin image tags to digests. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: services - value_type: bool - default_value: "false" - description: Print the service names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: volumes - value_type: bool - default_value: "false" - description: Print the volume names, one per line. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_cp.yaml b/docs/reference/docker_compose_cp.yaml deleted file mode 100644 index 24f6aec87f9..00000000000 --- a/docs/reference/docker_compose_cp.yaml +++ /dev/null @@ -1,69 +0,0 @@ -command: docker compose cp -short: Copy files/folders between a service container and the local filesystem -long: Copy files/folders between a service container and the local filesystem -usage: |- - docker compose cp [OPTIONS] SERVICE:SRC_PATH DEST_PATH|- - docker compose cp [OPTIONS] SRC_PATH|- SERVICE:DEST_PATH -pname: docker compose -plink: docker_compose.yaml -options: - - option: all - value_type: bool - default_value: "false" - description: Include containers created by the run command - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: archive - shorthand: a - value_type: bool - default_value: "false" - description: Archive mode (copy all uid/gid information) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: follow-link - shorthand: L - value_type: bool - default_value: "false" - description: Always follow symbol link in SRC_PATH - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: index - value_type: int - default_value: "0" - description: Index of the container if service has multiple replicas - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_create.yaml b/docs/reference/docker_compose_create.yaml deleted file mode 100644 index f6ab1b86824..00000000000 --- a/docs/reference/docker_compose_create.yaml +++ /dev/null @@ -1,119 +0,0 @@ -command: docker compose create -short: Creates containers for a service -long: Creates containers for a service -usage: docker compose create [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: build - value_type: bool - default_value: "false" - description: Build images before starting containers - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: force-recreate - value_type: bool - default_value: "false" - description: | - Recreate containers even if their configuration and image haven't changed - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-build - value_type: bool - default_value: "false" - description: Don't build an image, even if it's policy - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-recreate - value_type: bool - default_value: "false" - description: | - If containers already exist, don't recreate them. Incompatible with --force-recreate. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: pull - value_type: string - default_value: policy - description: Pull image before running ("always"|"missing"|"never"|"build") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet-pull - value_type: bool - default_value: "false" - description: Pull without printing progress information - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: remove-orphans - value_type: bool - default_value: "false" - description: Remove containers for services not defined in the Compose file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: scale - value_type: stringArray - default_value: '[]' - description: | - Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: "yes" - shorthand: "y" - value_type: bool - default_value: "false" - description: Assume "yes" as answer to all prompts and run non-interactively - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_down.yaml b/docs/reference/docker_compose_down.yaml deleted file mode 100644 index 77bf526289b..00000000000 --- a/docs/reference/docker_compose_down.yaml +++ /dev/null @@ -1,81 +0,0 @@ -command: docker compose down -short: Stop and remove containers, networks -long: |- - Stops containers and removes containers, networks, volumes, and images created by `up`. - - By default, the only things removed are: - - - Containers for services defined in the Compose file. - - Networks defined in the networks section of the Compose file. - - The default network, if one is used. - - Networks and volumes defined as external are never removed. - - Anonymous volumes are not removed by default. However, as they don’t have a stable name, they are not automatically - mounted by a subsequent `up`. For data that needs to persist between updates, use explicit paths as bind mounts or - named volumes. -usage: docker compose down [OPTIONS] [SERVICES] -pname: docker compose -plink: docker_compose.yaml -options: - - option: remove-orphans - value_type: bool - default_value: "false" - description: Remove containers for services not defined in the Compose file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: rmi - value_type: string - description: | - Remove images used by services. "local" remove only images that don't have a custom tag ("local"|"all") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: timeout - shorthand: t - value_type: int - default_value: "0" - description: Specify a shutdown timeout in seconds - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: volumes - shorthand: v - value_type: bool - default_value: "false" - description: | - Remove named volumes declared in the "volumes" section of the Compose file and anonymous volumes attached to containers - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_events.yaml b/docs/reference/docker_compose_events.yaml deleted file mode 100644 index 7c4cb4297f9..00000000000 --- a/docs/reference/docker_compose_events.yaml +++ /dev/null @@ -1,72 +0,0 @@ -command: docker compose events -short: Receive real time events from containers -long: |- - Stream container events for every container in the project. - - With the `--json` flag, a json object is printed one per line with the format: - - ```json - { - "time": "2015-11-20T18:01:03.615550", - "type": "container", - "action": "create", - "id": "213cf7...5fc39a", - "service": "web", - "attributes": { - "name": "application_web_1", - "image": "alpine:edge" - } - } - ``` - - The events that can be received using this can be seen [here](/reference/cli/docker/system/events/#object-types). -usage: docker compose events [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: json - value_type: bool - default_value: "false" - description: Output events as a stream of json objects - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: since - value_type: string - description: Show all events created since timestamp - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: until - value_type: string - description: Stream events until this timestamp - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_exec.yaml b/docs/reference/docker_compose_exec.yaml deleted file mode 100644 index 66ecfddab8d..00000000000 --- a/docs/reference/docker_compose_exec.yaml +++ /dev/null @@ -1,131 +0,0 @@ -command: docker compose exec -short: Execute a command in a running container -long: |- - This is the equivalent of `docker exec` targeting a Compose service. - - With this subcommand, you can run arbitrary commands in your services. Commands allocate a TTY by default, so - you can use a command such as `docker compose exec web sh` to get an interactive prompt. - - By default, Compose will enter container in interactive mode and allocate a TTY, while the equivalent `docker exec` - command requires passing `--interactive --tty` flags to get the same behavior. Compose also support those two flags - to offer a smooth migration between commands, whenever they are no-op by default. Still, `interactive` can be used to - force disabling interactive mode (`--interactive=false`), typically when `docker compose exec` command is used inside - a script. -usage: docker compose exec [OPTIONS] SERVICE COMMAND [ARGS...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: detach - shorthand: d - value_type: bool - default_value: "false" - description: 'Detached mode: Run command in the background' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: env - shorthand: e - value_type: stringArray - default_value: '[]' - description: Set environment variables - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: index - value_type: int - default_value: "0" - description: Index of the container if service has multiple replicas - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: interactive - shorthand: i - value_type: bool - default_value: "true" - description: Keep STDIN open even if not attached - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-tty - shorthand: T - value_type: bool - default_value: "true" - description: | - Disable pseudo-TTY allocation. By default 'docker compose exec' allocates a TTY. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: privileged - value_type: bool - default_value: "false" - description: Give extended privileges to the process - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: tty - shorthand: t - value_type: bool - default_value: "true" - description: Allocate a pseudo-TTY - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: user - shorthand: u - value_type: string - description: Run the command as this user - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: workdir - shorthand: w - value_type: string - description: Path to workdir directory for this command - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_export.yaml b/docs/reference/docker_compose_export.yaml deleted file mode 100644 index 5dfb3be0a47..00000000000 --- a/docs/reference/docker_compose_export.yaml +++ /dev/null @@ -1,45 +0,0 @@ -command: docker compose export -short: Export a service container's filesystem as a tar archive -long: Export a service container's filesystem as a tar archive -usage: docker compose export [OPTIONS] SERVICE -pname: docker compose -plink: docker_compose.yaml -options: - - option: index - value_type: int - default_value: "0" - description: index of the container if service has multiple replicas. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: output - shorthand: o - value_type: string - description: Write to a file, instead of STDOUT - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_images.yaml b/docs/reference/docker_compose_images.yaml deleted file mode 100644 index 33187df42d6..00000000000 --- a/docs/reference/docker_compose_images.yaml +++ /dev/null @@ -1,46 +0,0 @@ -command: docker compose images -short: List images used by the created containers -long: List images used by the created containers -usage: docker compose images [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: format - value_type: string - default_value: table - description: 'Format the output. Values: [table | json]' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only display IDs - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_kill.yaml b/docs/reference/docker_compose_kill.yaml deleted file mode 100644 index fbffcc9cb89..00000000000 --- a/docs/reference/docker_compose_kill.yaml +++ /dev/null @@ -1,51 +0,0 @@ -command: docker compose kill -short: Force stop service containers -long: |- - Forces running containers to stop by sending a `SIGKILL` signal. Optionally the signal can be passed, for example: - - ```console - $ docker compose kill -s SIGINT - ``` -usage: docker compose kill [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: remove-orphans - value_type: bool - default_value: "false" - description: Remove containers for services not defined in the Compose file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: signal - shorthand: s - value_type: string - default_value: SIGKILL - description: SIGNAL to send to the container - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_logs.yaml b/docs/reference/docker_compose_logs.yaml deleted file mode 100644 index 92d94dd108c..00000000000 --- a/docs/reference/docker_compose_logs.yaml +++ /dev/null @@ -1,109 +0,0 @@ -command: docker compose logs -short: View output from containers -long: Displays log output from services -usage: docker compose logs [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: follow - shorthand: f - value_type: bool - default_value: "false" - description: Follow log output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: index - value_type: int - default_value: "0" - description: index of the container if service has multiple replicas - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-color - value_type: bool - default_value: "false" - description: Produce monochrome output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-log-prefix - value_type: bool - default_value: "false" - description: Don't print prefix in logs - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: since - value_type: string - description: | - Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: tail - shorthand: "n" - value_type: string - default_value: all - description: | - Number of lines to show from the end of the logs for each container - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: timestamps - shorthand: t - value_type: bool - default_value: "false" - description: Show timestamps - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: until - value_type: string - description: | - Show logs before a timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_ls.yaml b/docs/reference/docker_compose_ls.yaml deleted file mode 100644 index dd6418c652f..00000000000 --- a/docs/reference/docker_compose_ls.yaml +++ /dev/null @@ -1,66 +0,0 @@ -command: docker compose ls -short: List running compose projects -long: Lists running Compose projects -usage: docker compose ls [OPTIONS] -pname: docker compose -plink: docker_compose.yaml -options: - - option: all - shorthand: a - value_type: bool - default_value: "false" - description: Show all stopped Compose projects - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: filter - value_type: filter - description: Filter output based on conditions provided - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: format - value_type: string - default_value: table - description: 'Format the output. Values: [table | json]' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only display project names - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_pause.yaml b/docs/reference/docker_compose_pause.yaml deleted file mode 100644 index 2ae1c402792..00000000000 --- a/docs/reference/docker_compose_pause.yaml +++ /dev/null @@ -1,25 +0,0 @@ -command: docker compose pause -short: Pause services -long: | - Pauses running containers of a service. They can be unpaused with `docker compose unpause`. -usage: docker compose pause [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_port.yaml b/docs/reference/docker_compose_port.yaml deleted file mode 100644 index 8a07f31ea50..00000000000 --- a/docs/reference/docker_compose_port.yaml +++ /dev/null @@ -1,45 +0,0 @@ -command: docker compose port -short: Print the public port for a port binding -long: Prints the public port for a port binding -usage: docker compose port [OPTIONS] SERVICE PRIVATE_PORT -pname: docker compose -plink: docker_compose.yaml -options: - - option: index - value_type: int - default_value: "0" - description: Index of the container if service has multiple replicas - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: protocol - value_type: string - default_value: tcp - description: tcp or udp - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_ps.yaml b/docs/reference/docker_compose_ps.yaml deleted file mode 100644 index d0370275695..00000000000 --- a/docs/reference/docker_compose_ps.yaml +++ /dev/null @@ -1,214 +0,0 @@ -command: docker compose ps -short: List containers -long: |- - Lists containers for a Compose project, with current status and exposed ports. - - ```console - $ docker compose ps - NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS - example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp - ``` - - By default, only running containers are shown. `--all` flag can be used to include stopped containers. - - ```console - $ docker compose ps --all - NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS - example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp - example-bar-1 alpine "/entrypoint.…" bar 4 seconds ago exited (0) - ``` -usage: docker compose ps [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: all - shorthand: a - value_type: bool - default_value: "false" - description: | - Show all stopped containers (including those created by the run command) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: filter - value_type: string - description: 'Filter services by a property (supported filters: status)' - details_url: '#filter' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: format - value_type: string - default_value: table - description: |- - Format output using a custom template: - 'table': Print output in table format with column headers (default) - 'table TEMPLATE': Print output in table format using the given Go template - 'json': Print in JSON format - 'TEMPLATE': Print output using the given Go template. - Refer to https://docs.docker.com/go/formatting/ for more information about formatting output with templates - details_url: '#format' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-trunc - value_type: bool - default_value: "false" - description: Don't truncate output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: orphans - value_type: bool - default_value: "true" - description: Include orphaned services (not declared by project) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only display IDs - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: services - value_type: bool - default_value: "false" - description: Display services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: status - value_type: stringArray - default_value: '[]' - description: | - Filter services by status. Values: [paused | restarting | removing | running | dead | created | exited] - details_url: '#status' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -examples: |- - ### Format the output (--format) {#format} - - By default, the `docker compose ps` command uses a table ("pretty") format to - show the containers. The `--format` flag allows you to specify alternative - presentations for the output. Currently, supported options are `pretty` (default), - and `json`, which outputs information about the containers as a JSON array: - - ```console - $ docker compose ps --format json - [{"ID":"1553b0236cf4d2715845f053a4ee97042c4f9a2ef655731ee34f1f7940eaa41a","Name":"example-bar-1","Command":"/docker-entrypoint.sh nginx -g 'daemon off;'","Project":"example","Service":"bar","State":"exited","Health":"","ExitCode":0,"Publishers":null},{"ID":"f02a4efaabb67416e1ff127d51c4b5578634a0ad5743bd65225ff7d1909a3fa0","Name":"example-foo-1","Command":"/docker-entrypoint.sh nginx -g 'daemon off;'","Project":"example","Service":"foo","State":"running","Health":"","ExitCode":0,"Publishers":[{"URL":"0.0.0.0","TargetPort":80,"PublishedPort":8080,"Protocol":"tcp"}]}] - ``` - - The JSON output allows you to use the information in other tools for further - processing, for example, using the [`jq` utility](https://stedolan.github.io/jq/) - to pretty-print the JSON: - - ```console - $ docker compose ps --format json | jq . - [ - { - "ID": "1553b0236cf4d2715845f053a4ee97042c4f9a2ef655731ee34f1f7940eaa41a", - "Name": "example-bar-1", - "Command": "/docker-entrypoint.sh nginx -g 'daemon off;'", - "Project": "example", - "Service": "bar", - "State": "exited", - "Health": "", - "ExitCode": 0, - "Publishers": null - }, - { - "ID": "f02a4efaabb67416e1ff127d51c4b5578634a0ad5743bd65225ff7d1909a3fa0", - "Name": "example-foo-1", - "Command": "/docker-entrypoint.sh nginx -g 'daemon off;'", - "Project": "example", - "Service": "foo", - "State": "running", - "Health": "", - "ExitCode": 0, - "Publishers": [ - { - "URL": "0.0.0.0", - "TargetPort": 80, - "PublishedPort": 8080, - "Protocol": "tcp" - } - ] - } - ] - ``` - - ### Filter containers by status (--status) {#status} - - Use the `--status` flag to filter the list of containers by status. For example, - to show only containers that are running or only containers that have exited: - - ```console - $ docker compose ps --status=running - NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS - example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp - - $ docker compose ps --status=exited - NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS - example-bar-1 alpine "/entrypoint.…" bar 4 seconds ago exited (0) - ``` - - ### Filter containers by status (--filter) {#filter} - - The [`--status` flag](#status) is a convenient shorthand for the `--filter status=<status>` - flag. The example below is the equivalent to the example from the previous section, - this time using the `--filter` flag: - - ```console - $ docker compose ps --filter status=running - NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS - example-foo-1 alpine "/entrypoint.…" foo 4 seconds ago Up 2 seconds 0.0.0.0:8080->80/tcp - ``` - - The `docker compose ps` command currently only supports the `--filter status=<status>` - option, but additional filter options may be added in the future. -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_publish.yaml b/docs/reference/docker_compose_publish.yaml deleted file mode 100644 index c3189d89c57..00000000000 --- a/docs/reference/docker_compose_publish.yaml +++ /dev/null @@ -1,86 +0,0 @@ -command: docker compose publish -short: Publish compose application -long: Publish compose application -usage: docker compose publish [OPTIONS] REPOSITORY[:TAG] -pname: docker compose -plink: docker_compose.yaml -options: - - option: app - value_type: bool - default_value: "false" - description: Published compose application (includes referenced images) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: insecure-registry - value_type: bool - default_value: "false" - description: Use insecure registry - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: oci-version - value_type: string - description: | - OCI image/artifact specification version (automatically determined by default) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: resolve-image-digests - value_type: bool - default_value: "false" - description: Pin image tags to digests - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: with-env - value_type: bool - default_value: "false" - description: Include environment variables in the published OCI artifact - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: "yes" - shorthand: "y" - value_type: bool - default_value: "false" - description: Assume "yes" as answer to all prompts - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_pull.yaml b/docs/reference/docker_compose_pull.yaml deleted file mode 100644 index 5b1316df132..00000000000 --- a/docs/reference/docker_compose_pull.yaml +++ /dev/null @@ -1,139 +0,0 @@ -command: docker compose pull -short: Pull service images -long: | - Pulls an image associated with a service defined in a `compose.yaml` file, but does not start containers based on those images -usage: docker compose pull [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: ignore-buildable - value_type: bool - default_value: "false" - description: Ignore images that can be built - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: ignore-pull-failures - value_type: bool - default_value: "false" - description: Pull what it can and ignores images with pull failures - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: include-deps - value_type: bool - default_value: "false" - description: Also pull services declared as dependencies - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-parallel - value_type: bool - default_value: "true" - description: DEPRECATED disable parallel pulling - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: parallel - value_type: bool - default_value: "true" - description: DEPRECATED pull multiple images in parallel - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: policy - value_type: string - description: Apply pull policy ("missing"|"always") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Pull without printing progress information - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -examples: |- - Consider the following `compose.yaml`: - - ```yaml - services: - db: - image: postgres - web: - build: . - command: bundle exec rails s -p 3000 -b '0.0.0.0' - volumes: - - .:/myapp - ports: - - "3000:3000" - depends_on: - - db - ``` - - If you run `docker compose pull ServiceName` in the same directory as the `compose.yaml` file that defines the service, - Docker pulls the associated image. For example, to call the postgres image configured as the db service in our example, - you would run `docker compose pull db`. - - ```console - $ docker compose pull db - [+] Running 1/15 - ⠸ db Pulling 12.4s - ⠿ 45b42c59be33 Already exists 0.0s - ⠹ 40adec129f1a Downloading 3.374MB/4.178MB 9.3s - ⠹ b4c431d00c78 Download complete 9.3s - ⠹ 2696974e2815 Download complete 9.3s - ⠹ 564b77596399 Downloading 5.622MB/7.965MB 9.3s - ⠹ 5044045cf6f2 Downloading 216.7kB/391.1kB 9.3s - ⠹ d736e67e6ac3 Waiting 9.3s - ⠹ 390c1c9a5ae4 Waiting 9.3s - ⠹ c0e62f172284 Waiting 9.3s - ⠹ ebcdc659c5bf Waiting 9.3s - ⠹ 29be22cb3acc Waiting 9.3s - ⠹ f63c47038e66 Waiting 9.3s - ⠹ 77a0c198cde5 Waiting 9.3s - ⠹ c8752d5b785c Waiting 9.3s - ``` - - `docker compose pull` tries to pull image for services with a build section. If pull fails, it lets you know this service image must be built. You can skip this by setting `--ignore-buildable` flag. -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_push.yaml b/docs/reference/docker_compose_push.yaml deleted file mode 100644 index be7c116065f..00000000000 --- a/docs/reference/docker_compose_push.yaml +++ /dev/null @@ -1,74 +0,0 @@ -command: docker compose push -short: Push service images -long: |- - Pushes images for services to their respective registry/repository. - - The following assumptions are made: - - You are pushing an image you have built locally - - You have access to the build key - - Examples - - ```yaml - services: - service1: - build: . - image: localhost:5000/yourimage ## goes to local registry - - service2: - build: . - image: your-dockerid/yourimage ## goes to your repository on Docker Hub - ``` -usage: docker compose push [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: ignore-push-failures - value_type: bool - default_value: "false" - description: Push what it can and ignores images with push failures - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: include-deps - value_type: bool - default_value: "false" - description: Also push images of services declared as dependencies - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Push without printing progress information - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_restart.yaml b/docs/reference/docker_compose_restart.yaml deleted file mode 100644 index 3bc0a3ad83a..00000000000 --- a/docs/reference/docker_compose_restart.yaml +++ /dev/null @@ -1,56 +0,0 @@ -command: docker compose restart -short: Restart service containers -long: |- - Restarts all stopped and running services, or the specified services only. - - If you make changes to your `compose.yml` configuration, these changes are not reflected - after running this command. For example, changes to environment variables (which are added - after a container is built, but before the container's command is executed) are not updated - after restarting. - - If you are looking to configure a service's restart policy, refer to - [restart](https://github.com/compose-spec/compose-spec/blob/main/spec.md#restart) - or [restart_policy](https://github.com/compose-spec/compose-spec/blob/main/deploy.md#restart_policy). -usage: docker compose restart [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: no-deps - value_type: bool - default_value: "false" - description: Don't restart dependent services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: timeout - shorthand: t - value_type: int - default_value: "0" - description: Specify a shutdown timeout in seconds - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_rm.yaml b/docs/reference/docker_compose_rm.yaml deleted file mode 100644 index 7ddafae4809..00000000000 --- a/docs/reference/docker_compose_rm.yaml +++ /dev/null @@ -1,84 +0,0 @@ -command: docker compose rm -short: Removes stopped service containers -long: |- - Removes stopped service containers. - - By default, anonymous volumes attached to containers are not removed. You can override this with `-v`. To list all - volumes, use `docker volume ls`. - - Any data which is not in a volume is lost. - - Running the command with no options also removes one-off containers created by `docker compose run`: - - ```console - $ docker compose rm - Going to remove djangoquickstart_web_run_1 - Are you sure? [yN] y - Removing djangoquickstart_web_run_1 ... done - ``` -usage: docker compose rm [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: all - shorthand: a - value_type: bool - default_value: "false" - description: Deprecated - no effect - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: force - shorthand: f - value_type: bool - default_value: "false" - description: Don't ask to confirm removal - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: stop - shorthand: s - value_type: bool - default_value: "false" - description: Stop the containers, if required, before removing - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: volumes - shorthand: v - value_type: bool - default_value: "false" - description: Remove any anonymous volumes attached to containers - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_run.yaml b/docs/reference/docker_compose_run.yaml deleted file mode 100644 index 61c7ca0e8cb..00000000000 --- a/docs/reference/docker_compose_run.yaml +++ /dev/null @@ -1,336 +0,0 @@ -command: docker compose run -short: Run a one-off command on a service -long: |- - Runs a one-time command against a service. - - The following command starts the `web` service and runs `bash` as its command: - - ```console - $ docker compose run web bash - ``` - - Commands you use with run start in new containers with configuration defined by that of the service, - including volumes, links, and other details. However, there are two important differences: - - First, the command passed by `run` overrides the command defined in the service configuration. For example, if the - `web` service configuration is started with `bash`, then `docker compose run web python app.py` overrides it with - `python app.py`. - - The second difference is that the `docker compose run` command does not create any of the ports specified in the - service configuration. This prevents port collisions with already-open ports. If you do want the service’s ports - to be created and mapped to the host, specify the `--service-ports` - - ```console - $ docker compose run --service-ports web python manage.py shell - ``` - - Alternatively, manual port mapping can be specified with the `--publish` or `-p` options, just as when using docker run: - - ```console - $ docker compose run --publish 8080:80 -p 2022:22 -p 127.0.0.1:2021:21 web python manage.py shell - ``` - - If you start a service configured with links, the run command first checks to see if the linked service is running - and starts the service if it is stopped. Once all the linked services are running, the run executes the command you - passed it. For example, you could run: - - ```console - $ docker compose run db psql -h db -U docker - ``` - - This opens an interactive PostgreSQL shell for the linked `db` container. - - If you do not want the run command to start linked containers, use the `--no-deps` flag: - - ```console - $ docker compose run --no-deps web python manage.py shell - ``` - - If you want to remove the container after running while overriding the container’s restart policy, use the `--rm` flag: - - ```console - $ docker compose run --rm web python manage.py db upgrade - ``` - - This runs a database upgrade script, and removes the container when finished running, even if a restart policy is - specified in the service configuration. -usage: docker compose run [OPTIONS] SERVICE [COMMAND] [ARGS...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: build - value_type: bool - default_value: "false" - description: Build image before starting container - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: cap-add - value_type: list - description: Add Linux capabilities - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: cap-drop - value_type: list - description: Drop Linux capabilities - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: detach - shorthand: d - value_type: bool - default_value: "false" - description: Run container in background and print container ID - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: entrypoint - value_type: string - description: Override the entrypoint of the image - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: env - shorthand: e - value_type: stringArray - default_value: '[]' - description: Set environment variables - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: env-from-file - value_type: stringArray - default_value: '[]' - description: Set environment variables from file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: interactive - shorthand: i - value_type: bool - default_value: "true" - description: Keep STDIN open even if not attached - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: label - shorthand: l - value_type: stringArray - default_value: '[]' - description: Add or override a label - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: name - value_type: string - description: Assign a name to the container - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-TTY - shorthand: T - value_type: bool - default_value: "true" - description: 'Disable pseudo-TTY allocation (default: auto-detected)' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-deps - value_type: bool - default_value: "false" - description: Don't start linked services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: publish - shorthand: p - value_type: stringArray - default_value: '[]' - description: Publish a container's port(s) to the host - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: pull - value_type: string - default_value: policy - description: Pull image before running ("always"|"missing"|"never") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Don't print anything to STDOUT - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet-build - value_type: bool - default_value: "false" - description: Suppress progress output from the build process - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet-pull - value_type: bool - default_value: "false" - description: Pull without printing progress information - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: remove-orphans - value_type: bool - default_value: "false" - description: Remove containers for services not defined in the Compose file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: rm - value_type: bool - default_value: "false" - description: Automatically remove the container when it exits - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: service-ports - shorthand: P - value_type: bool - default_value: "false" - description: | - Run command with all service's ports enabled and mapped to the host - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: tty - shorthand: t - value_type: bool - default_value: "true" - description: Allocate a pseudo-TTY - deprecated: false - hidden: true - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: use-aliases - value_type: bool - default_value: "false" - description: | - Use the service's network useAliases in the network(s) the container connects to - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: user - shorthand: u - value_type: string - description: Run as specified username or uid - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: volume - shorthand: v - value_type: stringArray - default_value: '[]' - description: Bind mount a volume - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: workdir - shorthand: w - value_type: string - description: Working directory inside the container - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_scale.yaml b/docs/reference/docker_compose_scale.yaml deleted file mode 100644 index f840a51b4e1..00000000000 --- a/docs/reference/docker_compose_scale.yaml +++ /dev/null @@ -1,35 +0,0 @@ -command: docker compose scale -short: Scale services -long: Scale services -usage: docker compose scale [SERVICE=REPLICAS...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: no-deps - value_type: bool - default_value: "false" - description: Don't start linked services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_start.yaml b/docs/reference/docker_compose_start.yaml deleted file mode 100644 index 56f9bcdff7c..00000000000 --- a/docs/reference/docker_compose_start.yaml +++ /dev/null @@ -1,46 +0,0 @@ -command: docker compose start -short: Start services -long: Starts existing containers for a service -usage: docker compose start [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: wait - value_type: bool - default_value: "false" - description: Wait for services to be running|healthy. Implies detached mode. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: wait-timeout - value_type: int - default_value: "0" - description: | - Maximum duration in seconds to wait for the project to be running|healthy - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_stats.yaml b/docs/reference/docker_compose_stats.yaml deleted file mode 100644 index e6854b05a25..00000000000 --- a/docs/reference/docker_compose_stats.yaml +++ /dev/null @@ -1,71 +0,0 @@ -command: docker compose stats -short: Display a live stream of container(s) resource usage statistics -long: Display a live stream of container(s) resource usage statistics -usage: docker compose stats [OPTIONS] [SERVICE] -pname: docker compose -plink: docker_compose.yaml -options: - - option: all - shorthand: a - value_type: bool - default_value: "false" - description: Show all containers (default shows just running) - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: format - value_type: string - description: |- - Format output using a custom template: - 'table': Print output in table format with column headers (default) - 'table TEMPLATE': Print output in table format using the given Go template - 'json': Print in JSON format - 'TEMPLATE': Print output using the given Go template. - Refer to https://docs.docker.com/engine/cli/formatting/ for more information about formatting output with templates - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-stream - value_type: bool - default_value: "false" - description: Disable streaming stats and only pull the first result - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-trunc - value_type: bool - default_value: "false" - description: Do not truncate output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_stop.yaml b/docs/reference/docker_compose_stop.yaml deleted file mode 100644 index f2ec34ccb3d..00000000000 --- a/docs/reference/docker_compose_stop.yaml +++ /dev/null @@ -1,37 +0,0 @@ -command: docker compose stop -short: Stop services -long: | - Stops running containers without removing them. They can be started again with `docker compose start`. -usage: docker compose stop [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: timeout - shorthand: t - value_type: int - default_value: "0" - description: Specify a shutdown timeout in seconds - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_top.yaml b/docs/reference/docker_compose_top.yaml deleted file mode 100644 index 17cdf7e3818..00000000000 --- a/docs/reference/docker_compose_top.yaml +++ /dev/null @@ -1,31 +0,0 @@ -command: docker compose top -short: Display the running processes -long: Displays the running processes -usage: docker compose top [SERVICES...] -pname: docker compose -plink: docker_compose.yaml -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -examples: |- - ```console - $ docker compose top - example_foo_1 - UID PID PPID C STIME TTY TIME CMD - root 142353 142331 2 15:33 ? 00:00:00 ping localhost -c 5 - ``` -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_unpause.yaml b/docs/reference/docker_compose_unpause.yaml deleted file mode 100644 index e2047720b8f..00000000000 --- a/docs/reference/docker_compose_unpause.yaml +++ /dev/null @@ -1,24 +0,0 @@ -command: docker compose unpause -short: Unpause services -long: Unpauses paused containers of a service -usage: docker compose unpause [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_up.yaml b/docs/reference/docker_compose_up.yaml deleted file mode 100644 index 8c78a8fa683..00000000000 --- a/docs/reference/docker_compose_up.yaml +++ /dev/null @@ -1,350 +0,0 @@ -command: docker compose up -short: Create and start containers -long: |- - Builds, (re)creates, starts, and attaches to containers for a service. - - Unless they are already running, this command also starts any linked services. - - The `docker compose up` command aggregates the output of each container (like `docker compose logs --follow` does). - One can optionally select a subset of services to attach to using `--attach` flag, or exclude some services using - `--no-attach` to prevent output to be flooded by some verbose services. - - When the command exits, all containers are stopped. Running `docker compose up --detach` starts the containers in the - background and leaves them running. - - If there are existing containers for a service, and the service’s configuration or image was changed after the - container’s creation, `docker compose up` picks up the changes by stopping and recreating the containers - (preserving mounted volumes). To prevent Compose from picking up changes, use the `--no-recreate` flag. - - If you want to force Compose to stop and recreate all containers, use the `--force-recreate` flag. - - If the process encounters an error, the exit code for this command is `1`. - If the process is interrupted using `SIGINT` (ctrl + C) or `SIGTERM`, the containers are stopped, and the exit code is `0`. -usage: docker compose up [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: abort-on-container-exit - value_type: bool - default_value: "false" - description: | - Stops all containers if any container was stopped. Incompatible with -d - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: abort-on-container-failure - value_type: bool - default_value: "false" - description: | - Stops all containers if any container exited with failure. Incompatible with -d - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: always-recreate-deps - value_type: bool - default_value: "false" - description: Recreate dependent containers. Incompatible with --no-recreate. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: attach - value_type: stringArray - default_value: '[]' - description: | - Restrict attaching to the specified services. Incompatible with --attach-dependencies. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: attach-dependencies - value_type: bool - default_value: "false" - description: Automatically attach to log output of dependent services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: build - value_type: bool - default_value: "false" - description: Build images before starting containers - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: detach - shorthand: d - value_type: bool - default_value: "false" - description: 'Detached mode: Run containers in the background' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: exit-code-from - value_type: string - description: | - Return the exit code of the selected service container. Implies --abort-on-container-exit - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: force-recreate - value_type: bool - default_value: "false" - description: | - Recreate containers even if their configuration and image haven't changed - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: menu - value_type: bool - default_value: "false" - description: | - Enable interactive shortcuts when running attached. Incompatible with --detach. Can also be enable/disable by setting COMPOSE_MENU environment var. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-attach - value_type: stringArray - default_value: '[]' - description: Do not attach (stream logs) to the specified services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-build - value_type: bool - default_value: "false" - description: Don't build an image, even if it's policy - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-color - value_type: bool - default_value: "false" - description: Produce monochrome output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-deps - value_type: bool - default_value: "false" - description: Don't start linked services - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-log-prefix - value_type: bool - default_value: "false" - description: Don't print prefix in logs - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-recreate - value_type: bool - default_value: "false" - description: | - If containers already exist, don't recreate them. Incompatible with --force-recreate. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: no-start - value_type: bool - default_value: "false" - description: Don't start the services after creating them - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: pull - value_type: string - default_value: policy - description: Pull image before running ("always"|"missing"|"never") - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet-build - value_type: bool - default_value: "false" - description: Suppress the build output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet-pull - value_type: bool - default_value: "false" - description: Pull without printing progress information - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: remove-orphans - value_type: bool - default_value: "false" - description: Remove containers for services not defined in the Compose file - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: renew-anon-volumes - shorthand: V - value_type: bool - default_value: "false" - description: | - Recreate anonymous volumes instead of retrieving data from the previous containers - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: scale - value_type: stringArray - default_value: '[]' - description: | - Scale SERVICE to NUM instances. Overrides the `scale` setting in the Compose file if present. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: timeout - shorthand: t - value_type: int - default_value: "0" - description: | - Use this timeout in seconds for container shutdown when attached or when containers are already running - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: timestamps - value_type: bool - default_value: "false" - description: Show timestamps - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: wait - value_type: bool - default_value: "false" - description: Wait for services to be running|healthy. Implies detached mode. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: wait-timeout - value_type: int - default_value: "0" - description: | - Maximum duration in seconds to wait for the project to be running|healthy - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: watch - shorthand: w - value_type: bool - default_value: "false" - description: | - Watch source code and rebuild/refresh containers when files are updated. - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: "yes" - shorthand: "y" - value_type: bool - default_value: "false" - description: Assume "yes" as answer to all prompts and run non-interactively - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_version.yaml b/docs/reference/docker_compose_version.yaml deleted file mode 100644 index 789e94818ea..00000000000 --- a/docs/reference/docker_compose_version.yaml +++ /dev/null @@ -1,45 +0,0 @@ -command: docker compose version -short: Show the Docker Compose version information -long: Show the Docker Compose version information -usage: docker compose version [OPTIONS] -pname: docker compose -plink: docker_compose.yaml -options: - - option: format - shorthand: f - value_type: string - description: 'Format the output. Values: [pretty | json]. (Default: pretty)' - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: short - value_type: bool - default_value: "false" - description: Shows only Compose's version number - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_volumes.yaml b/docs/reference/docker_compose_volumes.yaml deleted file mode 100644 index 20516db7f13..00000000000 --- a/docs/reference/docker_compose_volumes.yaml +++ /dev/null @@ -1,52 +0,0 @@ -command: docker compose volumes -short: List volumes -long: List volumes -usage: docker compose volumes [OPTIONS] [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: format - value_type: string - default_value: table - description: |- - Format output using a custom template: - 'table': Print output in table format with column headers (default) - 'table TEMPLATE': Print output in table format using the given Go template - 'json': Print in JSON format - 'TEMPLATE': Print output using the given Go template. - Refer to https://docs.docker.com/go/formatting/ for more information about formatting output with templates - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - shorthand: q - value_type: bool - default_value: "false" - description: Only display volume names - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_wait.yaml b/docs/reference/docker_compose_wait.yaml deleted file mode 100644 index 5d8f3013cc0..00000000000 --- a/docs/reference/docker_compose_wait.yaml +++ /dev/null @@ -1,35 +0,0 @@ -command: docker compose wait -short: Block until containers of all (or specified) services stop. -long: Block until containers of all (or specified) services stop. -usage: docker compose wait SERVICE [SERVICE...] [OPTIONS] -pname: docker compose -plink: docker_compose.yaml -options: - - option: down-project - value_type: bool - default_value: "false" - description: Drops project when the first container stops - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/reference/docker_compose_watch.yaml b/docs/reference/docker_compose_watch.yaml deleted file mode 100644 index a3e3e802201..00000000000 --- a/docs/reference/docker_compose_watch.yaml +++ /dev/null @@ -1,57 +0,0 @@ -command: docker compose watch -short: | - Watch build context for service and rebuild/refresh containers when files are updated -long: | - Watch build context for service and rebuild/refresh containers when files are updated -usage: docker compose watch [SERVICE...] -pname: docker compose -plink: docker_compose.yaml -options: - - option: no-up - value_type: bool - default_value: "false" - description: Do not build & start services before watching - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: prune - value_type: bool - default_value: "true" - description: Prune dangling images on rebuild - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false - - option: quiet - value_type: bool - default_value: "false" - description: hide build output - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -inherited_options: - - option: dry-run - value_type: bool - default_value: "false" - description: Execute command in dry run mode - deprecated: false - hidden: false - experimental: false - experimentalcli: false - kubernetes: false - swarm: false -deprecated: false -hidden: false -experimental: false -experimentalcli: false -kubernetes: false -swarm: false - diff --git a/docs/sdk.md b/docs/sdk.md deleted file mode 100644 index 3a03c7fdc1d..00000000000 --- a/docs/sdk.md +++ /dev/null @@ -1,157 +0,0 @@ -# Using the `docker/compose` SDK - -The `docker/compose` package can be used as a Go library by third-party applications to programmatically manage -containerized applications defined in Compose files. This SDK provides a comprehensive API that lets you -integrate Compose functionality directly into your applications, allowing you to load, validate, and manage -multi-container environments without relying on the Compose CLI. - -Whether you need to orchestrate containers as part of -a deployment pipeline, build custom management tools, or embed container orchestration into your application, the -Compose SDK offers the same powerful capabilities that drive the Docker Compose command-line tool. - -## Set up the SDK - -To get started, create an SDK instance using the `NewComposeService()` function, which initializes a service with the -necessary configuration to interact with the Docker daemon and manage Compose projects. This service instance provides -methods for all core Compose operations including creating, starting, stopping, and removing containers, as well as -loading and validating Compose files. The service handles the underlying Docker API interactions and resource -management, allowing you to focus on your application logic. - -## Example usage - -Here's a basic example demonstrating how to load a Compose project and start the services: - -```go -package main - -import ( - "context" - "log" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/flags" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose" -) - -func main() { - ctx := context.Background() - - dockerCLI, err := command.NewDockerCli() - if err != nil { - log.Fatalf("Failed to create docker CLI: %v", err) - } - err = dockerCLI.Initialize(&flags.ClientOptions{}) - if err != nil { - log.Fatalf("Failed to initialize docker CLI: %v", err) - } - - // Create a new Compose service instance - service, err := compose.NewComposeService(dockerCLI) - if err != nil { - log.Fatalf("Failed to create compose service: %v", err) - } - - // Load the Compose project from a compose file - project, err := service.LoadProject(ctx, api.ProjectLoadOptions{ - ConfigPaths: []string{"compose.yaml"}, - ProjectName: "my-app", - }) - if err != nil { - log.Fatalf("Failed to load project: %v", err) - } - - // Start the services defined in the Compose file - err = service.Up(ctx, project, api.UpOptions{ - Create: api.CreateOptions{}, - Start: api.StartOptions{}, - }) - if err != nil { - log.Fatalf("Failed to start services: %v", err) - } - - log.Printf("Successfully started project: %s", project.Name) -} -``` - -This example demonstrates the core workflow - creating a service instance, loading a project from a Compose file, and -starting the services. The SDK provides many additional operations for managing the lifecycle of your containerized -application. - -## Customizing the SDK - -The `NewComposeService()` function accepts optional `compose.Option` parameters to customize the SDK behavior. These -options allow you to configure I/O streams, concurrency limits, dry-run mode, and other advanced features. - -```go - // Create a custom output buffer to capture logs - var outputBuffer bytes.Buffer - - // Create a compose service with custom options - service, err := compose.NewComposeService(dockerCLI, - compose.WithOutputStream(&outputBuffer), // Redirect output to custom writer - compose.WithErrorStream(os.Stderr), // Use stderr for errors - compose.WithMaxConcurrency(4), // Limit concurrent operations - compose.WithPrompt(compose.AlwaysOkPrompt()), // Auto-confirm all prompts - ) -``` - -### Available options - -- `WithOutputStream(io.Writer)` - Redirect standard output to a custom writer -- `WithErrorStream(io.Writer)` - Redirect error output to a custom writer -- `WithInputStream(io.Reader)` - Provide a custom input stream for interactive prompts -- `WithStreams(out, err, in)` - Set all I/O streams at once -- `WithMaxConcurrency(int)` - Limit the number of concurrent operations against the Docker API -- `WithPrompt(Prompt)` - Customize user confirmation behavior (use `AlwaysOkPrompt()` for non-interactive mode) -- `WithDryRun` - Run operations in dry-run mode without actually applying changes -- `WithContextInfo(api.ContextInfo)` - Set custom Docker context information -- `WithProxyConfig(map[string]string)` - Configure HTTP proxy settings for builds -- `WithEventProcessor(progress.EventProcessor)` - Receive progress events and operation notifications - -These options provide fine-grained control over the SDK's behavior, making it suitable for various integration -scenarios including CLI tools, web services, automation scripts, and testing environments. - -## Tracking operations with `EventProcessor` - -The `EventProcessor` interface allows you to monitor Compose operations in real-time by receiving events about changes -applied to Docker resources such as images, containers, volumes, and networks. This is particularly useful for building -user interfaces, logging systems, or monitoring tools that need to track the progress of Compose operations. - -### Understanding `EventProcessor` - -A Compose operation, such as `up`, `down`, `build`, performs a series of changes to Docker resources. The -`EventProcessor` receives notifications about these changes through three key methods: - -- `Start(ctx, operation)` - Called when a Compose operation begins, for example `up` -- `On(events...)` - Called with progress events for individual resource changes, for example, container starting, image - being pulled -- `Done(operation, success)` - Called when the operation completes, indicating success or failure - -Each event contains information about the resource being modified, its current status, and progress indicators when -applicable (such as download progress for image pulls). - -### Event status types - -Events report resource changes with the following status types: - -- Working - Operation is in progress, for example, creating, starting, pulling -- Done - Operation completed successfully -- Warning - Operation completed with warnings -- Error - Operation failed - -Common status text values include: `Creating`, `Created`, `Starting`, `Started`, `Running`, `Stopping`, `Stopped`, -`Removing`, `Removed`, `Building`, `Built`, `Pulling`, `Pulled`, and more. - -### Built-in `EventProcessor` implementations - -The SDK provides three ready-to-use `EventProcessor` implementations: - -- `progress.NewTTYWriter(io.Writer)` - Renders an interactive terminal UI with progress bars and task lists - (similar to the Docker Compose CLI output) -- `progress.NewPlainWriter(io.Writer)` - Outputs simple text-based progress messages suitable for non-interactive - environments or log files -- `progress.NewJSONWriter()` - Render events as JSON objects -- `progress.NewQuietWriter()` - (Default) Silently processes events without producing any output - -Using `EventProcessor`, a custom UI can be plugged into `docker/compose`. diff --git a/docs/yaml/main/generate.go b/docs/yaml/main/generate.go deleted file mode 100644 index 335efe9cc3b..00000000000 --- a/docs/yaml/main/generate.go +++ /dev/null @@ -1,104 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package main - -import ( - "fmt" - "os" - "path/filepath" - - clidocstool "github.com/docker/cli-docs-tool" - "github.com/docker/cli/cli/command" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/cmd/compose" -) - -func generateDocs(opts *options) error { - dockerCLI, err := command.NewDockerCli() - if err != nil { - return err - } - cmd := &cobra.Command{ - Use: "docker", - DisableAutoGenTag: true, - } - cmd.AddCommand(compose.RootCommand(dockerCLI, nil)) - disableFlagsInUseLine(cmd) - - tool, err := clidocstool.New(clidocstool.Options{ - Root: cmd, - SourceDir: opts.source, - TargetDir: opts.target, - Plugin: true, - }) - if err != nil { - return err - } - for _, format := range opts.formats { - switch format { - case "yaml": - if err := tool.GenYamlTree(cmd); err != nil { - return err - } - case "md": - if err := tool.GenMarkdownTree(cmd); err != nil { - return err - } - default: - return fmt.Errorf("unknown format %q", format) - } - } - return nil -} - -func disableFlagsInUseLine(cmd *cobra.Command) { - visitAll(cmd, func(ccmd *cobra.Command) { - // do not add a `[flags]` to the end of the usage line. - ccmd.DisableFlagsInUseLine = true - }) -} - -// visitAll will traverse all commands from the root. -// This is different from the VisitAll of cobra.Command where only parents -// are checked. -func visitAll(root *cobra.Command, fn func(*cobra.Command)) { - for _, cmd := range root.Commands() { - visitAll(cmd, fn) - } - fn(root) -} - -type options struct { - source string - target string - formats []string -} - -func main() { - cwd, _ := os.Getwd() - opts := &options{ - source: filepath.Join(cwd, "docs", "reference"), - target: filepath.Join(cwd, "docs", "reference"), - formats: []string{"yaml", "md"}, - } - fmt.Printf("Project root: %s\n", opts.source) - fmt.Printf("Generating yaml files into %s\n", opts.target) - if err := generateDocs(opts); err != nil { - _, _ = fmt.Fprintf(os.Stderr, "Failed to generate documentation: %s\n", err.Error()) - } -} diff --git a/experimental/compose_swarm_networking.md b/experimental/compose_swarm_networking.md new file mode 100644 index 00000000000..905f52f8007 --- /dev/null +++ b/experimental/compose_swarm_networking.md @@ -0,0 +1,5 @@ +# Experimental: Compose, Swarm and Multi-Host Networking + +Compose now supports multi-host networking as standard. Read more here: + +https://docs.docker.com/compose/networking diff --git a/go.mod b/go.mod deleted file mode 100644 index 466b4e5999c..00000000000 --- a/go.mod +++ /dev/null @@ -1,157 +0,0 @@ -module github.com/docker/compose/v5 - -go 1.24.3 - -require ( - github.com/AlecAivazis/survey/v2 v2.3.7 - github.com/DefangLabs/secret-detector v0.0.0-20250403165618-22662109213e - github.com/Microsoft/go-winio v0.6.2 - github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d - github.com/buger/goterm v1.0.4 - github.com/compose-spec/compose-go/v2 v2.10.1 - github.com/containerd/console v1.0.5 - github.com/containerd/containerd/v2 v2.2.1 - github.com/containerd/errdefs v1.0.0 - github.com/containerd/platforms v1.0.0-rc.2 - github.com/distribution/reference v0.6.0 - github.com/docker/buildx v0.30.1 - github.com/docker/cli v28.5.2+incompatible - github.com/docker/cli-docs-tool v0.11.0 - github.com/docker/docker v28.5.2+incompatible - github.com/docker/go-connections v0.6.0 - github.com/docker/go-units v0.5.0 - github.com/eiannone/keyboard v0.0.0-20220611211555-0d226195f203 - github.com/fsnotify/fsevents v0.2.0 - github.com/go-viper/mapstructure/v2 v2.5.0 - github.com/google/go-cmp v0.7.0 - github.com/google/uuid v1.6.0 - github.com/hashicorp/go-version v1.8.0 - github.com/jonboulle/clockwork v0.5.0 - github.com/mattn/go-shellwords v1.0.12 - github.com/mitchellh/go-ps v1.0.0 - github.com/moby/buildkit v0.26.3 - github.com/moby/go-archive v0.1.0 - github.com/moby/patternmatcher v0.6.0 - github.com/moby/sys/atomicwriter v0.1.0 - github.com/morikuni/aec v1.1.0 - github.com/opencontainers/go-digest v1.0.0 - github.com/opencontainers/image-spec v1.1.1 - github.com/otiai10/copy v1.14.1 - github.com/sirupsen/logrus v1.9.4 - github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 - github.com/spf13/cobra v1.10.2 - github.com/spf13/pflag v1.0.10 - github.com/stretchr/testify v1.11.1 - github.com/tilt-dev/fsnotify v1.4.8-0.20220602155310-fff9c274a375 - go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 - go.opentelemetry.io/otel v1.38.0 - go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0 - go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.38.0 - go.opentelemetry.io/otel/metric v1.38.0 - go.opentelemetry.io/otel/sdk v1.38.0 - go.opentelemetry.io/otel/trace v1.38.0 - go.uber.org/goleak v1.3.0 - go.uber.org/mock v0.6.0 - go.yaml.in/yaml/v4 v4.0.0-rc.4 - golang.org/x/sync v0.19.0 - golang.org/x/sys v0.41.0 - google.golang.org/grpc v1.78.0 - gotest.tools/v3 v3.5.2 - tags.cncf.io/container-device-interface v1.1.0 -) - -require ( - github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect - github.com/beorn7/perks v1.0.1 // indirect - github.com/cenkalti/backoff/v5 v5.0.3 // indirect - github.com/cespare/xxhash/v2 v2.3.0 // indirect - github.com/containerd/containerd/api v1.10.0 // indirect - github.com/containerd/continuity v0.4.5 // indirect - github.com/containerd/errdefs/pkg v0.3.0 // indirect - github.com/containerd/log v0.1.0 // indirect - github.com/containerd/ttrpc v1.2.7 // indirect - github.com/containerd/typeurl/v2 v2.2.3 // indirect - github.com/cpuguy83/go-md2man/v2 v2.0.7 // indirect - github.com/davecgh/go-spew v1.1.1 // indirect - github.com/docker/distribution v2.8.3+incompatible // indirect - github.com/docker/docker-credential-helpers v0.9.3 // indirect - github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c // indirect - github.com/docker/go-metrics v0.0.1 // indirect - github.com/felixge/httpsnoop v1.0.4 // indirect - github.com/fvbommel/sortorder v1.1.0 // indirect - github.com/go-logr/logr v1.4.3 // indirect - github.com/go-logr/stdr v1.2.2 // indirect - github.com/gofrs/flock v0.13.0 // indirect - github.com/gogo/protobuf v1.3.2 // indirect - github.com/golang-jwt/jwt/v5 v5.3.0 // indirect - github.com/golang/protobuf v1.5.4 // indirect - github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect - github.com/gorilla/mux v1.8.1 // indirect - github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2 // indirect - github.com/hashicorp/errwrap v1.1.0 // indirect - github.com/hashicorp/go-multierror v1.1.1 // indirect - github.com/in-toto/in-toto-golang v0.9.0 // indirect - github.com/inconshreveable/mousetrap v1.1.0 // indirect - github.com/inhies/go-bytesize v0.0.0-20220417184213-4913239db9cf // indirect - github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect - github.com/klauspost/compress v1.18.2 // indirect - github.com/magiconair/properties v1.8.9 // indirect - github.com/mattn/go-colorable v0.1.14 // indirect - github.com/mattn/go-isatty v0.0.20 // indirect - github.com/mattn/go-runewidth v0.0.16 // indirect - github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b // indirect - github.com/miekg/pkcs11 v1.1.1 // indirect - github.com/mitchellh/hashstructure/v2 v2.0.2 // indirect - github.com/mitchellh/mapstructure v1.5.0 // indirect - github.com/moby/docker-image-spec v1.3.1 // indirect - github.com/moby/locker v1.0.1 // indirect - github.com/moby/sys/capability v0.4.0 // indirect - github.com/moby/sys/sequential v0.6.0 // indirect - github.com/moby/sys/signal v0.7.1 // indirect - github.com/moby/sys/symlink v0.3.0 // indirect - github.com/moby/sys/user v0.4.0 // indirect - github.com/moby/sys/userns v0.1.0 // indirect - github.com/moby/term v0.5.2 // indirect - github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect - github.com/otiai10/mint v1.6.3 // indirect - github.com/pelletier/go-toml v1.9.5 // indirect - github.com/pkg/errors v0.9.1 // indirect - github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect - github.com/pmezard/go-difflib v1.0.0 // indirect - github.com/prometheus/client_golang v1.23.2 // indirect - github.com/prometheus/client_model v0.6.2 // indirect - github.com/prometheus/common v0.66.1 // indirect - github.com/prometheus/procfs v0.16.1 // indirect - github.com/rivo/uniseg v0.2.0 // indirect - github.com/russross/blackfriday/v2 v2.1.0 // indirect - github.com/santhosh-tekuri/jsonschema/v6 v6.0.1 // indirect - github.com/secure-systems-lab/go-securesystemslib v0.9.1 // indirect - github.com/shibumi/go-pathspec v1.3.0 // indirect - github.com/theupdateframework/notary v0.7.0 // indirect - github.com/tonistiigi/dchapes-mode v0.0.0-20250318174251-73d941a28323 // indirect - github.com/tonistiigi/fsutil v0.0.0-20250605211040-586307ad452f // indirect - github.com/tonistiigi/go-csvvalue v0.0.0-20240814133006-030d3b2625d0 // indirect - github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea // indirect - github.com/tonistiigi/vt100 v0.0.0-20240514184818-90bafcd6abab // indirect - github.com/xhit/go-str2duration/v2 v2.1.0 // indirect - go.opentelemetry.io/auto/sdk v1.2.1 // indirect - go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0 // indirect - go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.63.0 // indirect - go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.38.0 // indirect - go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.38.0 // indirect - go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0 // indirect - go.opentelemetry.io/otel/sdk/metric v1.38.0 // indirect - go.opentelemetry.io/proto/otlp v1.7.1 // indirect - go.yaml.in/yaml/v2 v2.4.2 // indirect - go.yaml.in/yaml/v3 v3.0.4 // indirect - golang.org/x/crypto v0.45.0 // indirect - golang.org/x/net v0.47.0 // indirect - golang.org/x/term v0.37.0 // indirect - golang.org/x/text v0.31.0 // indirect - golang.org/x/time v0.14.0 // indirect - google.golang.org/genproto/googleapis/api v0.0.0-20251029180050-ab9386a59fda // indirect - google.golang.org/genproto/googleapis/rpc v0.0.0-20251029180050-ab9386a59fda // indirect - google.golang.org/protobuf v1.36.10 // indirect - gopkg.in/ini.v1 v1.67.0 // indirect - gopkg.in/yaml.v3 v3.0.1 // indirect -) diff --git a/go.sum b/go.sum deleted file mode 100644 index 75f074fb4e2..00000000000 --- a/go.sum +++ /dev/null @@ -1,568 +0,0 @@ -github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 h1:He8afgbRMd7mFxO99hRNu+6tazq8nFF9lIwo9JFroBk= -github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8= -github.com/AlecAivazis/survey/v2 v2.3.7 h1:6I/u8FvytdGsgonrYsVn2t8t4QiRnh6QSTqkkhIiSjQ= -github.com/AlecAivazis/survey/v2 v2.3.7/go.mod h1:xUTIdE4KCOIjsBAE1JYsUPoCqYdZ1reCfTwbto0Fduo= -github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg= -github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= -github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ= -github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= -github.com/DefangLabs/secret-detector v0.0.0-20250403165618-22662109213e h1:rd4bOvKmDIx0WeTv9Qz+hghsgyjikFiPrseXHlKepO0= -github.com/DefangLabs/secret-detector v0.0.0-20250403165618-22662109213e/go.mod h1:blbwPQh4DTlCZEfk1BLU4oMIhLda2U+A840Uag9DsZw= -github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= -github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= -github.com/Microsoft/hcsshim v0.14.0-rc.1 h1:qAPXKwGOkVn8LlqgBN8GS0bxZ83hOJpcjxzmlQKxKsQ= -github.com/Microsoft/hcsshim v0.14.0-rc.1/go.mod h1:hTKFGbnDtQb1wHiOWv4v0eN+7boSWAHyK/tNAaYZL0c= -github.com/Netflix/go-expect v0.0.0-20220104043353-73e0943537d2 h1:+vx7roKuyA63nhn5WAunQHLTznkw5W8b1Xc0dNjp83s= -github.com/Netflix/go-expect v0.0.0-20220104043353-73e0943537d2/go.mod h1:HBCaDeC1lPdgDeDbhX8XFpy1jqjK0IBG8W5K+xYqA0w= -github.com/Shopify/logrus-bugsnag v0.0.0-20170309145241-6dbc35f2c30d h1:hi6J4K6DKrR4/ljxn6SF6nURyu785wKMuQcjt7H3VCQ= -github.com/Shopify/logrus-bugsnag v0.0.0-20170309145241-6dbc35f2c30d/go.mod h1:HI8ITrYtUY+O+ZhtlqUnD8+KwNPOyugEhfP9fdUIaEQ= -github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d h1:licZJFw2RwpHMqeKTCYkitsPqHNxTmd4SNR5r94FGM8= -github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d/go.mod h1:asat636LX7Bqt5lYEZ27JNDcqxfjdBQuJ/MM4CN/Lzo= -github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= -github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= -github.com/anchore/go-struct-converter v0.0.0-20221118182256-c68fdcfa2092 h1:aM1rlcoLz8y5B2r4tTLMiVTrMtpfY0O8EScKJxaSaEc= -github.com/anchore/go-struct-converter v0.0.0-20221118182256-c68fdcfa2092/go.mod h1:rYqSE9HbjzpHTI74vwPvae4ZVYZd1lue2ta6xHPdblA= -github.com/beorn7/perks v0.0.0-20150223135152-b965b613227f/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= -github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= -github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= -github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= -github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= -github.com/bitly/go-hostpool v0.1.0/go.mod h1:4gOCgp6+NZnVqlKyZ/iBZFTAJKembaVENUpMkpg42fw= -github.com/bitly/go-simplejson v0.5.0/go.mod h1:cXHtHw4XUPsvGaxgjIAn8PhEWG9NfngEKAMDJEczWVA= -github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869/go.mod h1:Ekp36dRnpXw/yCqJaO+ZrUyxD+3VXMFFr56k5XYrpB4= -github.com/buger/goterm v1.0.4 h1:Z9YvGmOih81P0FbVtEYTFF6YsSgxSUKEhf/f9bTMXbY= -github.com/buger/goterm v1.0.4/go.mod h1:HiFWV3xnkolgrBV3mY8m0X0Pumt4zg4QhbdOzQtB8tE= -github.com/bugsnag/bugsnag-go v1.0.5-0.20150529004307-13fd6b8acda0 h1:s7+5BfS4WFJoVF9pnB8kBk03S7pZXRdKamnV0FOl5Sc= -github.com/bugsnag/bugsnag-go v1.0.5-0.20150529004307-13fd6b8acda0/go.mod h1:2oa8nejYd4cQ/b0hMIopN0lCRxU0bueqREvZLWFrtK8= -github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b h1:otBG+dV+YK+Soembjv71DPz3uX/V/6MMlSyD9JBQ6kQ= -github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b/go.mod h1:obH5gd0BsqsP2LwDJ9aOkm/6J86V6lyAXCoQWGw3K50= -github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 h1:nvj0OLI3YqYXer/kZD8Ri1aaunCxIEsOst1BVJswV0o= -github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0/go.mod h1:D/8v3kj0zr8ZAKg1AQ6crr+5VwKN5eIywRkfhyM/+dE= -github.com/cenkalti/backoff/v5 v5.0.3 h1:ZN+IMa753KfX5hd8vVaMixjnqRZ3y8CuJKRKj1xcsSM= -github.com/cenkalti/backoff/v5 v5.0.3/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw= -github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= -github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/cloudflare/cfssl v0.0.0-20180223231731-4e2dcbde5004 h1:lkAMpLVBDaj17e85keuznYcH5rqI438v41pKcBl4ZxQ= -github.com/cloudflare/cfssl v0.0.0-20180223231731-4e2dcbde5004/go.mod h1:yMWuSON2oQp+43nFtAV/uvKQIFpSPerB57DCt9t8sSA= -github.com/codahale/rfc6979 v0.0.0-20141003034818-6a90f24967eb h1:EDmT6Q9Zs+SbUoc7Ik9EfrFqcylYqgPZ9ANSbTAntnE= -github.com/codahale/rfc6979 v0.0.0-20141003034818-6a90f24967eb/go.mod h1:ZjrT6AXHbDs86ZSdt/osfBi5qfexBrKUdONk989Wnk4= -github.com/compose-spec/compose-go/v2 v2.10.1 h1:mFbXobojGRFIVi1UknrvaDAZ+PkJfyjqkA1yseh+vAU= -github.com/compose-spec/compose-go/v2 v2.10.1/go.mod h1:Ohac1SzhO/4fXXrzWIztIVB6ckmKBv1Nt5Z5mGVESUg= -github.com/containerd/cgroups/v3 v3.1.2 h1:OSosXMtkhI6Qove637tg1XgK4q+DhR0mX8Wi8EhrHa4= -github.com/containerd/cgroups/v3 v3.1.2/go.mod h1:PKZ2AcWmSBsY/tJUVhtS/rluX0b1uq1GmPO1ElCmbOw= -github.com/containerd/console v1.0.5 h1:R0ymNeydRqH2DmakFNdmjR2k0t7UPuiOV/N/27/qqsc= -github.com/containerd/console v1.0.5/go.mod h1:YynlIjWYF8myEu6sdkwKIvGQq+cOckRm6So2avqoYAk= -github.com/containerd/containerd/api v1.10.0 h1:5n0oHYVBwN4VhoX9fFykCV9dF1/BvAXeg2F8W6UYq1o= -github.com/containerd/containerd/api v1.10.0/go.mod h1:NBm1OAk8ZL+LG8R0ceObGxT5hbUYj7CzTmR3xh0DlMM= -github.com/containerd/containerd/v2 v2.2.1 h1:TpyxcY4AL5A+07dxETevunVS5zxqzuq7ZqJXknM11yk= -github.com/containerd/containerd/v2 v2.2.1/go.mod h1:NR70yW1iDxe84F2iFWbR9xfAN0N2F0NcjTi1OVth4nU= -github.com/containerd/continuity v0.4.5 h1:ZRoN1sXq9u7V6QoHMcVWGhOwDFqZ4B9i5H6un1Wh0x4= -github.com/containerd/continuity v0.4.5/go.mod h1:/lNJvtJKUQStBzpVQ1+rasXO1LAWtUQssk28EZvJ3nE= -github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI= -github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M= -github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE= -github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk= -github.com/containerd/fifo v1.1.0 h1:4I2mbh5stb1u6ycIABlBw9zgtlK8viPI9QkQNRQEEmY= -github.com/containerd/fifo v1.1.0/go.mod h1:bmC4NWMbXlt2EZ0Hc7Fx7QzTFxgPID13eH0Qu+MAb2o= -github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I= -github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo= -github.com/containerd/nydus-snapshotter v0.15.4 h1:l59kGRVMtwMLDLh322HsWhEsBCkRKMkGWYV5vBeLYCE= -github.com/containerd/nydus-snapshotter v0.15.4/go.mod h1:eRJqnxQDr48HNop15kZdLZpFF5B6vf6Q11Aq1K0E4Ms= -github.com/containerd/platforms v1.0.0-rc.2 h1:0SPgaNZPVWGEi4grZdV8VRYQn78y+nm6acgLGv/QzE4= -github.com/containerd/platforms v1.0.0-rc.2/go.mod h1:J71L7B+aiM5SdIEqmd9wp6THLVRzJGXfNuWCZCllLA4= -github.com/containerd/plugin v1.0.0 h1:c8Kf1TNl6+e2TtMHZt+39yAPDbouRH9WAToRjex483Y= -github.com/containerd/plugin v1.0.0/go.mod h1:hQfJe5nmWfImiqT1q8Si3jLv3ynMUIBB47bQ+KexvO8= -github.com/containerd/stargz-snapshotter v0.17.0 h1:djNS4KU8ztFhLdEDZ1bsfzOiYuVHT6TgSU5qwRk+cNc= -github.com/containerd/stargz-snapshotter/estargz v0.17.0 h1:+TyQIsR/zSFI1Rm31EQBwpAA1ovYgIKHy7kctL3sLcE= -github.com/containerd/stargz-snapshotter/estargz v0.17.0/go.mod h1:s06tWAiJcXQo9/8AReBCIo/QxcXFZ2n4qfsRnpl71SM= -github.com/containerd/ttrpc v1.2.7 h1:qIrroQvuOL9HQ1X6KHe2ohc7p+HP/0VE6XPU7elJRqQ= -github.com/containerd/ttrpc v1.2.7/go.mod h1:YCXHsb32f+Sq5/72xHubdiJRQY9inL4a4ZQrAbN1q9o= -github.com/containerd/typeurl/v2 v2.2.3 h1:yNA/94zxWdvYACdYO8zofhrTVuQY73fFU1y++dYSw40= -github.com/containerd/typeurl/v2 v2.2.3/go.mod h1:95ljDnPfD3bAbDJRugOiShd/DlAAsxGtUBhJxIn7SCk= -github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= -github.com/cpuguy83/go-md2man/v2 v2.0.7 h1:zbFlGlXEAKlwXpmvle3d8Oe3YnkKIK4xSRTd3sHPnBo= -github.com/cpuguy83/go-md2man/v2 v2.0.7/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= -github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= -github.com/creack/pty v1.1.17/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4= -github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s= -github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE= -github.com/cyphar/filepath-securejoin v0.5.1 h1:eYgfMq5yryL4fbWfkLpFFy2ukSELzaJOTaUTuh+oF48= -github.com/cyphar/filepath-securejoin v0.5.1/go.mod h1:Sdj7gXlvMcPZsbhwhQ33GguGLDGQL7h7bg04C/+u9jI= -github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= -github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/denisenkom/go-mssqldb v0.0.0-20191128021309-1d7a30a10f73/go.mod h1:xbL0rPBG9cCiLr28tMa8zpbdarY27NDyej4t/EjAShU= -github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk= -github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= -github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI= -github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8= -github.com/docker/buildx v0.30.1 h1:3vthfaTQOLt5QfN2nl7rKuPLUvx69nL5ZikFIXp//c8= -github.com/docker/buildx v0.30.1/go.mod h1:8nwT0V6UNYNo9rXq6WO/BQd9KrJ0JYcY/QX6x0y1Oro= -github.com/docker/cli v28.5.2+incompatible h1:XmG99IHcBmIAoC1PPg9eLBZPlTrNUAijsHLm8PjhBlg= -github.com/docker/cli v28.5.2+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= -github.com/docker/cli-docs-tool v0.11.0 h1:7d8QARFb7QEobizqxmEM7fOteZEHwH/zWgHQtHZEcfE= -github.com/docker/cli-docs-tool v0.11.0/go.mod h1:ma8BKiisUo8D6W05XEYIh3oa1UbgrZhi1nowyKFJa8Q= -github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= -github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk= -github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= -github.com/docker/docker v28.5.2+incompatible h1:DBX0Y0zAjZbSrm1uzOkdr1onVghKaftjlSWt4AFexzM= -github.com/docker/docker v28.5.2+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= -github.com/docker/docker-credential-helpers v0.9.3 h1:gAm/VtF9wgqJMoxzT3Gj5p4AqIjCBS4wrsOh9yRqcz8= -github.com/docker/docker-credential-helpers v0.9.3/go.mod h1:x+4Gbw9aGmChi3qTLZj8Dfn0TD20M/fuWy0E5+WDeCo= -github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c h1:lzqkGL9b3znc+ZUgi7FlLnqjQhcXxkNM/quxIjBVMD0= -github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c/go.mod h1:CADgU4DSXK5QUlFslkQu2yW2TKzFZcXq/leZfM0UH5Q= -github.com/docker/go-connections v0.4.0/go.mod h1:Gbd7IOopHjR8Iph03tsViu4nIes5XhDvyHbTtUxmeec= -github.com/docker/go-connections v0.6.0 h1:LlMG9azAe1TqfR7sO+NJttz1gy6KO7VJBh+pMmjSD94= -github.com/docker/go-connections v0.6.0/go.mod h1:AahvXYshr6JgfUJGdDCs2b5EZG/vmaMAntpSFH5BFKE= -github.com/docker/go-metrics v0.0.0-20180209012529-399ea8c73916/go.mod h1:/u0gXw0Gay3ceNrsHubL3BtdOL2fHf93USgMTe0W5dI= -github.com/docker/go-metrics v0.0.1 h1:AgB/0SvBxihN0X8OR4SjsblXkbMvalQ8cjmtKQ2rQV8= -github.com/docker/go-metrics v0.0.1/go.mod h1:cG1hvH2utMXtqgqqYE9plW6lDxS3/5ayHzueweSI3Vw= -github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= -github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= -github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7 h1:UhxFibDNY/bfvqU5CAUmr9zpesgbU6SWc8/B4mflAE4= -github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7/go.mod h1:cyGadeNEkKy96OOhEzfZl+yxihPEzKnqJwvfuSUqbZE= -github.com/dvsekhvalnov/jose2go v0.0.0-20170216131308-f21a8cedbbae/go.mod h1:7BvyPhdbLxMXIYTFPLsyJRFMsKmOZnQmzh6Gb+uquuM= -github.com/eiannone/keyboard v0.0.0-20220611211555-0d226195f203 h1:XBBHcIb256gUJtLmY22n99HaZTz+r2Z51xUPi01m3wg= -github.com/eiannone/keyboard v0.0.0-20220611211555-0d226195f203/go.mod h1:E1jcSv8FaEny+OP/5k9UxZVw9YFWGj7eI4KR/iOBqCg= -github.com/erikstmartin/go-testdb v0.0.0-20160219214506-8d10e4a1bae5/go.mod h1:a2zkGnVExMxdzMo3M0Hi/3sEU+cWnZpSni0O6/Yb/P0= -github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= -github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= -github.com/fsnotify/fsevents v0.2.0 h1:BRlvlqjvNTfogHfeBOFvSC9N0Ddy+wzQCQukyoD7o/c= -github.com/fsnotify/fsevents v0.2.0/go.mod h1:B3eEk39i4hz8y1zaWS/wPrAP4O6wkIl7HQwKBr1qH/w= -github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= -github.com/fvbommel/sortorder v1.1.0 h1:fUmoe+HLsBTctBDoaBwpQo5N+nrCp8g/BjKb/6ZQmYw= -github.com/fvbommel/sortorder v1.1.0/go.mod h1:uk88iVf1ovNn1iLfgUVU2F9o5eO30ui720w+kxuqRs0= -github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= -github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= -github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= -github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI= -github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= -github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= -github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= -github.com/go-sql-driver/mysql v1.3.0 h1:pgwjLi/dvffoP9aabwkT3AKpXQM93QARkjFhDDqC1UE= -github.com/go-sql-driver/mysql v1.3.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w= -github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= -github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro= -github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= -github.com/gofrs/flock v0.13.0 h1:95JolYOvGMqeH31+FC7D2+uULf6mG61mEZ/A8dRYMzw= -github.com/gofrs/flock v0.13.0/go.mod h1:jxeyy9R1auM5S6JYDBhDt+E2TCo7DkratH4Pgi8P+Z0= -github.com/gogo/protobuf v1.0.0/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= -github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= -github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= -github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= -github.com/golang-jwt/jwt/v5 v5.3.0 h1:pv4AsKCKKZuqlgs5sUmn4x8UlGa0kEVt/puTpKx9vvo= -github.com/golang-jwt/jwt/v5 v5.3.0/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE= -github.com/golang-sql/civil v0.0.0-20190719163853-cb61b32ac6fe/go.mod h1:8vg3r2VgvsThLBIFL93Qb5yWzgyZWhEmBwUJWevAkK0= -github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 h1:f+oWsMOmNPc8JmEHVZIycC7hBoQxHH9pNKQORJNozsQ= -github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8/go.mod h1:wcDNUvekVysuuOpQKo3191zZyTpiI6se1N1ULghS0sw= -github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= -github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek= -github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= -github.com/google/certificate-transparency-go v1.0.10-0.20180222191210-5ab67e519c93 h1:jc2UWq7CbdszqeH6qu1ougXMIUBfSy8Pbh/anURYbGI= -github.com/google/certificate-transparency-go v1.0.10-0.20180222191210-5ab67e519c93/go.mod h1:QeJfpSbVSfYc7RgB3gJFj9cbuQMMchQxrWXz8Ruopmg= -github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= -github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= -github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= -github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= -github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4= -github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ= -github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= -github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= -github.com/gorilla/mux v1.7.0/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs= -github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY= -github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ= -github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2 h1:8Tjv8EJ+pM1xP8mK6egEbD1OgnVTyacbefKhmbLhIhU= -github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2/go.mod h1:pkJQ2tZHJ0aFOVEEot6oZmaVEZcRme73eIFmhiVuRWs= -github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed h1:5upAirOpQc1Q53c0bnx2ufif5kANL7bfZWcc6VJWJd8= -github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed/go.mod h1:tMWxXQ9wFIaZeTI9F+hmhFiGpFmhOHzyShyFUhRm0H4= -github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= -github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I= -github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= -github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo= -github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM= -github.com/hashicorp/go-version v1.8.0 h1:KAkNb1HAiZd1ukkxDFGmokVZe1Xy9HG6NUp+bPle2i4= -github.com/hashicorp/go-version v1.8.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA= -github.com/hinshun/vt10x v0.0.0-20220119200601-820417d04eec h1:qv2VnGeEQHchGaZ/u7lxST/RaJw+cv273q79D81Xbog= -github.com/hinshun/vt10x v0.0.0-20220119200601-820417d04eec/go.mod h1:Q48J4R4DvxnHolD5P8pOtXigYlRuPLGl6moFx3ulM68= -github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= -github.com/in-toto/in-toto-golang v0.9.0 h1:tHny7ac4KgtsfrG6ybU8gVOZux2H8jN05AXJ9EBM1XU= -github.com/in-toto/in-toto-golang v0.9.0/go.mod h1:xsBVrVsHNsB61++S6Dy2vWosKhuA3lUTQd+eF9HdeMo= -github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= -github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= -github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= -github.com/inhies/go-bytesize v0.0.0-20220417184213-4913239db9cf h1:FtEj8sfIcaaBfAKrE1Cwb61YDtYq9JxChK1c7AKce7s= -github.com/inhies/go-bytesize v0.0.0-20220417184213-4913239db9cf/go.mod h1:yrqSXGoD/4EKfF26AOGzscPOgTTJcyAwM2rpixWT+t4= -github.com/jinzhu/gorm v0.0.0-20170222002820-5409931a1bb8 h1:CZkYfurY6KGhVtlalI4QwQ6T0Cu6iuY3e0x5RLu96WE= -github.com/jinzhu/gorm v0.0.0-20170222002820-5409931a1bb8/go.mod h1:Vla75njaFJ8clLU1W44h34PjIkijhjHIYnZxMqCdxqo= -github.com/jinzhu/inflection v0.0.0-20170102125226-1c35d901db3d h1:jRQLvyVGL+iVtDElaEIDdKwpPqUIZJfzkNLV34htpEc= -github.com/jinzhu/inflection v0.0.0-20170102125226-1c35d901db3d/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc= -github.com/jinzhu/now v1.1.1/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8= -github.com/jonboulle/clockwork v0.5.0 h1:Hyh9A8u51kptdkR+cqRpT1EebBwTn1oK9YfGYbdFz6I= -github.com/jonboulle/clockwork v0.5.0/go.mod h1:3mZlmanh0g2NDKO5TWZVJAfofYk64M7XN3SzBPjZF60= -github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= -github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= -github.com/juju/loggo v0.0.0-20190526231331-6e530bcce5d8/go.mod h1:vgyd7OREkbtVEN/8IXZe5Ooef3LQePvuBm9UWj6ZL8U= -github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= -github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs= -github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8= -github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= -github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= -github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk= -github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4= -github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= -github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= -github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= -github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= -github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= -github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= -github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= -github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= -github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= -github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= -github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= -github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= -github.com/lib/pq v0.0.0-20150723085316-0dad96c0b94f/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo= -github.com/magiconair/properties v1.5.3/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= -github.com/magiconair/properties v1.8.9 h1:nWcCbLq1N2v/cpNsy5WvQ37Fb+YElfq20WJ/a8RkpQM= -github.com/magiconair/properties v1.8.9/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0= -github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE= -github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE= -github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8= -github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s= -github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= -github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= -github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc= -github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= -github.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk= -github.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y= -github.com/mattn/go-sqlite3 v1.6.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc= -github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= -github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b h1:j7+1HpAFS1zy5+Q4qx1fWh90gTKwiN4QCGoY9TWyyO4= -github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE= -github.com/miekg/pkcs11 v1.0.2/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs= -github.com/miekg/pkcs11 v1.1.1 h1:Ugu9pdy6vAYku5DEpVWVFPYnzV+bxB+iRdbuFSu7TvU= -github.com/miekg/pkcs11 v1.1.1/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs= -github.com/mitchellh/go-ps v1.0.0 h1:i6ampVEEF4wQFF+bkYfwYgY+F/uYJDktmvLPf7qIgjc= -github.com/mitchellh/go-ps v1.0.0/go.mod h1:J4lOc8z8yJs6vUwklHw2XEIiT4z4C40KtWVN3nvg8Pg= -github.com/mitchellh/hashstructure/v2 v2.0.2 h1:vGKWl0YJqUNxE8d+h8f6NJLcCJrgbhC4NcD46KavDd4= -github.com/mitchellh/hashstructure/v2 v2.0.2/go.mod h1:MG3aRVU/N29oo/V/IhBX8GR/zz4kQkprJgF2EVszyDE= -github.com/mitchellh/mapstructure v0.0.0-20150613213606-2caf8efc9366/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= -github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY= -github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= -github.com/moby/buildkit v0.26.3 h1:D+ruZVAk/3ipRq5XRxBH9/DIFpRjSlTtMbghT5gQP9g= -github.com/moby/buildkit v0.26.3/go.mod h1:4T4wJzQS4kYWIfFRjsbJry4QoxDBjK+UGOEOs1izL7w= -github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0= -github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo= -github.com/moby/go-archive v0.1.0 h1:Kk/5rdW/g+H8NHdJW2gsXyZ7UnzvJNOy6VKJqueWdcQ= -github.com/moby/go-archive v0.1.0/go.mod h1:G9B+YoujNohJmrIYFBpSd54GTUB4lt9S+xVQvsJyFuo= -github.com/moby/locker v1.0.1 h1:fOXqR41zeveg4fFODix+1Ch4mj/gT0NE1XJbp/epuBg= -github.com/moby/locker v1.0.1/go.mod h1:S7SDdo5zpBK84bzzVlKr2V0hz+7x9hWbYC/kq7oQppc= -github.com/moby/patternmatcher v0.6.0 h1:GmP9lR19aU5GqSSFko+5pRqHi+Ohk1O69aFiKkVGiPk= -github.com/moby/patternmatcher v0.6.0/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc= -github.com/moby/sys/atomicwriter v0.1.0 h1:kw5D/EqkBwsBFi0ss9v1VG3wIkVhzGvLklJ+w3A14Sw= -github.com/moby/sys/atomicwriter v0.1.0/go.mod h1:Ul8oqv2ZMNHOceF643P6FKPXeCmYtlQMvpizfsSoaWs= -github.com/moby/sys/capability v0.4.0 h1:4D4mI6KlNtWMCM1Z/K0i7RV1FkX+DBDHKVJpCndZoHk= -github.com/moby/sys/capability v0.4.0/go.mod h1:4g9IK291rVkms3LKCDOoYlnV8xKwoDTpIrNEE35Wq0I= -github.com/moby/sys/mountinfo v0.7.2 h1:1shs6aH5s4o5H2zQLn796ADW1wMrIwHsyJ2v9KouLrg= -github.com/moby/sys/mountinfo v0.7.2/go.mod h1:1YOa8w8Ih7uW0wALDUgT1dTTSBrZ+HiBLGws92L2RU4= -github.com/moby/sys/sequential v0.6.0 h1:qrx7XFUd/5DxtqcoH1h438hF5TmOvzC/lspjy7zgvCU= -github.com/moby/sys/sequential v0.6.0/go.mod h1:uyv8EUTrca5PnDsdMGXhZe6CCe8U/UiTWd+lL+7b/Ko= -github.com/moby/sys/signal v0.7.1 h1:PrQxdvxcGijdo6UXXo/lU/TvHUWyPhj7UOpSo8tuvk0= -github.com/moby/sys/signal v0.7.1/go.mod h1:Se1VGehYokAkrSQwL4tDzHvETwUZlnY7S5XtQ50mQp8= -github.com/moby/sys/symlink v0.3.0 h1:GZX89mEZ9u53f97npBy4Rc3vJKj7JBDj/PN2I22GrNU= -github.com/moby/sys/symlink v0.3.0/go.mod h1:3eNdhduHmYPcgsJtZXW1W4XUJdZGBIkttZ8xKqPUJq0= -github.com/moby/sys/user v0.4.0 h1:jhcMKit7SA80hivmFJcbB1vqmw//wU61Zdui2eQXuMs= -github.com/moby/sys/user v0.4.0/go.mod h1:bG+tYYYJgaMtRKgEmuueC0hJEAZWwtIbZTB+85uoHjs= -github.com/moby/sys/userns v0.1.0 h1:tVLXkFOxVu9A64/yh59slHVv9ahO9UIev4JZusOLG/g= -github.com/moby/sys/userns v0.1.0/go.mod h1:IHUYgu/kao6N8YZlp9Cf444ySSvCmDlmzUcYfDHOl28= -github.com/moby/term v0.5.2 h1:6qk3FJAFDs6i/q3W/pQ97SX192qKfZgGjCQqfCJkgzQ= -github.com/moby/term v0.5.2/go.mod h1:d3djjFCrjnB+fl8NJux+EJzu0msscUP+f8it8hPkFLc= -github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= -github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= -github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= -github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= -github.com/morikuni/aec v1.1.0 h1:vBBl0pUnvi/Je71dsRrhMBtreIqNMYErSAbEeb8jrXQ= -github.com/morikuni/aec v1.1.0/go.mod h1:xDRgiq/iw5l+zkao76YTKzKttOp2cwPEne25HDkJnBw= -github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= -github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= -github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= -github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= -github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= -github.com/onsi/ginkgo v1.12.0/go.mod h1:oUhWkIvk5aDxtKvDDuw8gItl8pKl42LzjC9KZE0HfGg= -github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= -github.com/onsi/gomega v1.9.0/go.mod h1:Ho0h+IUsWyvy1OpqCwxlQ/21gkhVunqlU8fDGcoTdcA= -github.com/opencontainers/go-digest v0.0.0-20170106003457-a6d0ee40d420/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s= -github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= -github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= -github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0= -github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040= -github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M= -github.com/opencontainers/runtime-spec v1.3.0 h1:YZupQUdctfhpZy3TM39nN9Ika5CBWT5diQ8ibYCRkxg= -github.com/opencontainers/runtime-spec v1.3.0/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= -github.com/opencontainers/selinux v1.13.1 h1:A8nNeceYngH9Ow++M+VVEwJVpdFmrlxsN22F+ISDCJE= -github.com/opencontainers/selinux v1.13.1/go.mod h1:S10WXZ/osk2kWOYKy1x2f/eXF5ZHJoUs8UU/2caNRbg= -github.com/opentracing/opentracing-go v1.1.0 h1:pWlfV3Bxv7k65HYwkikxat0+s3pV4bsqf19k25Ur8rU= -github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= -github.com/otiai10/copy v1.14.1 h1:5/7E6qsUMBaH5AnQ0sSLzzTg1oTECmcCmT6lvF45Na8= -github.com/otiai10/copy v1.14.1/go.mod h1:oQwrEDDOci3IM8dJF0d8+jnbfPDllW6vUjNc3DoZm9I= -github.com/otiai10/mint v1.6.3 h1:87qsV/aw1F5as1eH1zS/yqHY85ANKVMgkDrf9rcxbQs= -github.com/otiai10/mint v1.6.3/go.mod h1:MJm72SBthJjz8qhefc4z1PYEieWmy8Bku7CjcAqyUSM= -github.com/pelletier/go-toml v1.9.5 h1:4yBQzkHv+7BHq2PQUZF3Mx0IYxG7LsP222s7Agd3ve8= -github.com/pelletier/go-toml v1.9.5/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c= -github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= -github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 h1:GFCKgmp0tecUJ0sJuv4pzYCqS9+RGSn52M3FUwPs+uo= -github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10/go.mod h1:t/avpk3KcrXxUnYOhZhMXJlSEyie6gQbtLq5NM3loB8= -github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= -github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_golang v0.9.0-pre1.0.20180209125602-c332b6f63c06/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= -github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= -github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= -github.com/prometheus/client_golang v1.1.0/go.mod h1:I1FGZT9+L76gKKOs5djB6ezCbFQP1xR9D75/vuwEF3g= -github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= -github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= -github.com/prometheus/client_model v0.0.0-20171117100541-99fa1f4be8e5/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= -github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= -github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= -github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= -github.com/prometheus/common v0.0.0-20180110214958-89604d197083/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= -github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= -github.com/prometheus/common v0.6.0/go.mod h1:eBmuwkDJBwy6iBfxCBob6t6dR6ENT/y+J+Zk0j9GMYc= -github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= -github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= -github.com/prometheus/procfs v0.0.0-20180125133057-cb4147076ac7/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= -github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= -github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= -github.com/prometheus/procfs v0.0.3/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDaekg4FpcdQ= -github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= -github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= -github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY= -github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc= -github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= -github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= -github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk= -github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= -github.com/santhosh-tekuri/jsonschema/v6 v6.0.1 h1:PKK9DyHxif4LZo+uQSgXNqs0jj5+xZwwfKHgph2lxBw= -github.com/santhosh-tekuri/jsonschema/v6 v6.0.1/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU= -github.com/secure-systems-lab/go-securesystemslib v0.9.1 h1:nZZaNz4DiERIQguNy0cL5qTdn9lR8XKHf4RUyG1Sx3g= -github.com/secure-systems-lab/go-securesystemslib v0.9.1/go.mod h1:np53YzT0zXGMv6x4iEWc9Z59uR+x+ndLwCLqPYpLXVU= -github.com/shibumi/go-pathspec v1.3.0 h1:QUyMZhFo0Md5B8zV8x2tesohbb5kfbpTi9rBnKh5dkI= -github.com/shibumi/go-pathspec v1.3.0/go.mod h1:Xutfslp817l2I1cZvgcfeMQJG5QnU2lh5tVaaMCl3jE= -github.com/sirupsen/logrus v1.0.6/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc= -github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= -github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q= -github.com/sirupsen/logrus v1.9.4 h1:TsZE7l11zFCLZnZ+teH4Umoq5BhEIfIzfRDZ1Uzql2w= -github.com/sirupsen/logrus v1.9.4/go.mod h1:ftWc9WdOfJ0a92nsE2jF5u5ZwH8Bv2zdeOC42RjbV2g= -github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 h1:JIAuq3EEf9cgbU6AtGPK4CTG3Zf6CKMNqf0MHTggAUA= -github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966/go.mod h1:sUM3LWHvSMaG192sy56D9F7CNvL7jUJVXoqM1QKLnog= -github.com/spdx/tools-golang v0.5.5 h1:61c0KLfAcNqAjlg6UNMdkwpMernhw3zVRwDZ2x9XOmk= -github.com/spdx/tools-golang v0.5.5/go.mod h1:MVIsXx8ZZzaRWNQpUDhC4Dud34edUYJYecciXgrw5vE= -github.com/spf13/cast v0.0.0-20150508191742-4d07383ffe94 h1:JmfC365KywYwHB946TTiQWEb8kqPY+pybPLoGE9GgVk= -github.com/spf13/cast v0.0.0-20150508191742-4d07383ffe94/go.mod h1:r2rcYCSwa1IExKTDiTfzaxqT2FNHs8hODu4LnUfgKEg= -github.com/spf13/cobra v0.0.1/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= -github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU= -github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4= -github.com/spf13/jwalterweatherman v0.0.0-20141219030609-3d60171a6431 h1:XTHrT015sxHyJ5FnQ0AeemSspZWaDq7DoTRW0EVsDCE= -github.com/spf13/jwalterweatherman v0.0.0-20141219030609-3d60171a6431/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= -github.com/spf13/pflag v1.0.0/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= -github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= -github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk= -github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= -github.com/spf13/viper v0.0.0-20150530192845-be5ff3e4840c h1:2EejZtjFjKJGk71ANb+wtFK5EjUzUkEM3R0xnp559xg= -github.com/spf13/viper v0.0.0-20150530192845-be5ff3e4840c/go.mod h1:A8kyI5cUJhb8N+3pkfONlcEcZbueH6nhAm0Fq7SrnBM= -github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE= -github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= -github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= -github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= -github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= -github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= -github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= -github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= -github.com/theupdateframework/notary v0.7.0 h1:QyagRZ7wlSpjT5N2qQAh/pN+DVqgekv4DzbAiAiEL3c= -github.com/theupdateframework/notary v0.7.0/go.mod h1:c9DRxcmhHmVLDay4/2fUYdISnHqbFDGRSlXPO0AhYWw= -github.com/tilt-dev/fsnotify v1.4.8-0.20220602155310-fff9c274a375 h1:QB54BJwA6x8QU9nHY3xJSZR2kX9bgpZekRKGkLTmEXA= -github.com/tilt-dev/fsnotify v1.4.8-0.20220602155310-fff9c274a375/go.mod h1:xRroudyp5iVtxKqZCrA6n2TLFRBf8bmnjr1UD4x+z7g= -github.com/tonistiigi/dchapes-mode v0.0.0-20250318174251-73d941a28323 h1:r0p7fK56l8WPequOaR3i9LBqfPtEdXIQbUTzT55iqT4= -github.com/tonistiigi/dchapes-mode v0.0.0-20250318174251-73d941a28323/go.mod h1:3Iuxbr0P7D3zUzBMAZB+ois3h/et0shEz0qApgHYGpY= -github.com/tonistiigi/fsutil v0.0.0-20250605211040-586307ad452f h1:MoxeMfHAe5Qj/ySSBfL8A7l1V+hxuluj8owsIEEZipI= -github.com/tonistiigi/fsutil v0.0.0-20250605211040-586307ad452f/go.mod h1:BKdcez7BiVtBvIcef90ZPc6ebqIWr4JWD7+EvLm6J98= -github.com/tonistiigi/go-csvvalue v0.0.0-20240814133006-030d3b2625d0 h1:2f304B10LaZdB8kkVEaoXvAMVan2tl9AiK4G0odjQtE= -github.com/tonistiigi/go-csvvalue v0.0.0-20240814133006-030d3b2625d0/go.mod h1:278M4p8WsNh3n4a1eqiFcV2FGk7wE5fwUpUom9mK9lE= -github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea h1:SXhTLE6pb6eld/v/cCndK0AMpt1wiVFb/YYmqB3/QG0= -github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea/go.mod h1:WPnis/6cRcDZSUvVmezrxJPkiO87ThFYsoUiMwWNDJk= -github.com/tonistiigi/vt100 v0.0.0-20240514184818-90bafcd6abab h1:H6aJ0yKQ0gF49Qb2z5hI1UHxSQt4JMyxebFR15KnApw= -github.com/tonistiigi/vt100 v0.0.0-20240514184818-90bafcd6abab/go.mod h1:ulncasL3N9uLrVann0m+CDlJKWsIAP34MPcOJF6VRvc= -github.com/vbatts/tar-split v0.12.2 h1:w/Y6tjxpeiFMR47yzZPlPj/FcPLpXbTUi/9H7d3CPa4= -github.com/vbatts/tar-split v0.12.2/go.mod h1:eF6B6i6ftWQcDqEn3/iGFRFRo8cBIMSJVOpnNdfTMFA= -github.com/xhit/go-str2duration/v2 v2.1.0 h1:lxklc02Drh6ynqX+DdPyp5pCKLUQpRT8bp8Ydu2Bstc= -github.com/xhit/go-str2duration/v2 v2.1.0/go.mod h1:ohY8p+0f07DiV6Em5LKB0s2YpLtXVyJfNt1+BlmyAsU= -github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= -go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0= -go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo= -go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64= -go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= -go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0 h1:YH4g8lQroajqUwWbq/tr2QX1JFmEXaDLgG+ew9bLMWo= -go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0/go.mod h1:fvPi2qXDqFs8M4B4fmJhE92TyQs9Ydjlg3RvfUp+NbQ= -go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.63.0 h1:2pn7OzMewmYRiNtv1doZnLo3gONcnMHlFnmOR8Vgt+8= -go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.63.0/go.mod h1:rjbQTDEPQymPE0YnRQp9/NuPwwtL0sesz/fnqRW/v84= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 h1:RbKq8BG0FI8OiXhBfcRtqqHcZcka+gU3cskNuf05R18= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0/go.mod h1:h06DGIukJOevXaj/xrNjhi/2098RZzcLTbc0jDAUbsg= -go.opentelemetry.io/otel v1.38.0 h1:RkfdswUDRimDg0m2Az18RKOsnI8UDzppJAtj01/Ymk8= -go.opentelemetry.io/otel v1.38.0/go.mod h1:zcmtmQ1+YmQM9wrNsTGV/q/uyusom3P8RxwExxkZhjM= -go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.38.0 h1:vl9obrcoWVKp/lwl8tRE33853I8Xru9HFbw/skNeLs8= -go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.38.0/go.mod h1:GAXRxmLJcVM3u22IjTg74zWBrRCKq8BnOqUVLodpcpw= -go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.38.0 h1:Oe2z/BCg5q7k4iXC3cqJxKYg0ieRiOqF0cecFYdPTwk= -go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.38.0/go.mod h1:ZQM5lAJpOsKnYagGg/zV2krVqTtaVdYdDkhMoX6Oalg= -go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0 h1:GqRJVj7UmLjCVyVJ3ZFLdPRmhDUp2zFmQe3RHIOsw24= -go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0/go.mod h1:ri3aaHSmCTVYu2AWv44YMauwAQc0aqI9gHKIcSbI1pU= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.38.0 h1:lwI4Dc5leUqENgGuQImwLo4WnuXFPetmPpkLi2IrX54= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.38.0/go.mod h1:Kz/oCE7z5wuyhPxsXDuaPteSWqjSBD5YaSdbxZYGbGk= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0 h1:aTL7F04bJHUlztTsNGJ2l+6he8c+y/b//eR0jjjemT4= -go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0/go.mod h1:kldtb7jDTeol0l3ewcmd8SDvx3EmIE7lyvqbasU3QC4= -go.opentelemetry.io/otel/metric v1.38.0 h1:Kl6lzIYGAh5M159u9NgiRkmoMKjvbsKtYRwgfrA6WpA= -go.opentelemetry.io/otel/metric v1.38.0/go.mod h1:kB5n/QoRM8YwmUahxvI3bO34eVtQf2i4utNVLr9gEmI= -go.opentelemetry.io/otel/sdk v1.38.0 h1:l48sr5YbNf2hpCUj/FoGhW9yDkl+Ma+LrVl8qaM5b+E= -go.opentelemetry.io/otel/sdk v1.38.0/go.mod h1:ghmNdGlVemJI3+ZB5iDEuk4bWA3GkTpW+DOoZMYBVVg= -go.opentelemetry.io/otel/sdk/metric v1.38.0 h1:aSH66iL0aZqo//xXzQLYozmWrXxyFkBJ6qT5wthqPoM= -go.opentelemetry.io/otel/sdk/metric v1.38.0/go.mod h1:dg9PBnW9XdQ1Hd6ZnRz689CbtrUp0wMMs9iPcgT9EZA= -go.opentelemetry.io/otel/trace v1.38.0 h1:Fxk5bKrDZJUH+AMyyIXGcFAPah0oRcT+LuNtJrmcNLE= -go.opentelemetry.io/otel/trace v1.38.0/go.mod h1:j1P9ivuFsTceSWe1oY+EeW3sc+Pp42sO++GHkg4wwhs= -go.opentelemetry.io/proto/otlp v1.7.1 h1:gTOMpGDb0WTBOP8JaO72iL3auEZhVmAQg4ipjOVAtj4= -go.opentelemetry.io/proto/otlp v1.7.1/go.mod h1:b2rVh6rfI/s2pHWNlB7ILJcRALpcNDzKhACevjI+ZnE= -go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= -go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= -go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y= -go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU= -go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= -go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= -go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= -go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= -go.yaml.in/yaml/v4 v4.0.0-rc.4 h1:UP4+v6fFrBIb1l934bDl//mmnoIZEDK0idg1+AIvX5U= -go.yaml.in/yaml/v4 v4.0.0-rc.4/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0= -golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= -golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= -golang.org/x/crypto v0.0.0-20190325154230-a5d413f7728c/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= -golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= -golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20201117144127-c1f2f97bffc9/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I= -golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= -golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q= -golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4= -golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= -golang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA= -golang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w= -golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= -golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= -golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= -golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY= -golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU= -golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= -golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= -golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190801041406-cbf593c0f2f3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210331175145-43e1dd70ce54/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k= -golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= -golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw= -golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= -golang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU= -golang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254= -golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= -golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= -golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= -golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM= -golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM= -golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI= -golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= -golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= -golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= -golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk= -gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E= -google.golang.org/genproto/googleapis/api v0.0.0-20251029180050-ab9386a59fda h1:+2XxjfsAu6vqFxwGBRcHiMaDCuZiqXGDUDVWVtrFAnE= -google.golang.org/genproto/googleapis/api v0.0.0-20251029180050-ab9386a59fda/go.mod h1:fDMmzKV90WSg1NbozdqrE64fkuTv6mlq2zxo9ad+3yo= -google.golang.org/genproto/googleapis/rpc v0.0.0-20251029180050-ab9386a59fda h1:i/Q+bfisr7gq6feoJnS/DlpdwEL4ihp41fvRiM3Ork0= -google.golang.org/genproto/googleapis/rpc v0.0.0-20251029180050-ab9386a59fda/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk= -google.golang.org/grpc v1.0.5/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw= -google.golang.org/grpc v1.78.0 h1:K1XZG/yGDJnzMdd/uZHAkVqJE+xIDOcmdSFZkBUicNc= -google.golang.org/grpc v1.78.0/go.mod h1:I47qjTo4OKbMkjA/aOOwxDIiPSBofUtQUI5EfpWvW7U= -google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= -google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= -gopkg.in/airbrake/gobrake.v2 v2.0.9/go.mod h1:/h5ZAUhDkGaJfjzjKLSjv6zCL6O0LLBxU4K+aSYdM/U= -gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= -gopkg.in/cenkalti/backoff.v2 v2.2.1 h1:eJ9UAg01/HIHG987TwxvnzK2MgxXq97YY6rYDpY9aII= -gopkg.in/cenkalti/backoff.v2 v2.2.1/go.mod h1:S0QdOvT2AlerfSBkp0O+dk+bbIMaNbEmVk876gPCthU= -gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= -gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= -gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= -gopkg.in/gemnasium/logrus-airbrake-hook.v2 v2.1.2/go.mod h1:Xk6kEKp8OKb+X14hQBKWaSkCsqBpgog8nAV2xsGOxlo= -gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA= -gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= -gopkg.in/rethinkdb/rethinkdb-go.v6 v6.2.1 h1:d4KQkxAaAiRY2h5Zqis161Pv91A37uZyJOx73duwUwM= -gopkg.in/rethinkdb/rethinkdb-go.v6 v6.2.1/go.mod h1:WbjuEoo1oadwzQ4apSDU+JTvmllEHtsNHS6y7vFc7iw= -gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= -gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10= -gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= -gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -gotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q= -gotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA= -tags.cncf.io/container-device-interface v1.1.0 h1:RnxNhxF1JOu6CJUVpetTYvrXHdxw9j9jFYgZpI+anSY= -tags.cncf.io/container-device-interface v1.1.0/go.mod h1:76Oj0Yqp9FwTx/pySDc8Bxjpg+VqXfDb50cKAXVJ34Q= diff --git a/internal/desktop/client.go b/internal/desktop/client.go deleted file mode 100644 index 1e28899370d..00000000000 --- a/internal/desktop/client.go +++ /dev/null @@ -1,146 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package desktop - -import ( - "context" - "encoding/json" - "fmt" - "io" - "net" - "net/http" - "strings" - - "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp" - - "github.com/docker/compose/v5/internal" - "github.com/docker/compose/v5/internal/memnet" -) - -// identify this client in the logs -var userAgent = "compose/" + internal.Version - -// Client for integration with Docker Desktop features. -type Client struct { - apiEndpoint string - client *http.Client -} - -// NewClient creates a Desktop integration client for the provided in-memory -// socket address (AF_UNIX or named pipe). -func NewClient(apiEndpoint string) *Client { - var transport http.RoundTripper = &http.Transport{ - DisableCompression: true, - DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) { - return memnet.DialEndpoint(ctx, apiEndpoint) - }, - } - transport = otelhttp.NewTransport(transport) - - return &Client{ - apiEndpoint: apiEndpoint, - client: &http.Client{Transport: transport}, - } -} - -func (c *Client) Endpoint() string { - return c.apiEndpoint -} - -// Close releases any open connections. -func (c *Client) Close() error { - c.client.CloseIdleConnections() - return nil -} - -type PingResponse struct { - ServerTime int64 `json:"serverTime"` -} - -// Ping is a minimal API used to ensure that the server is available. -func (c *Client) Ping(ctx context.Context) (*PingResponse, error) { - req, err := c.newRequest(ctx, http.MethodGet, "/ping", http.NoBody) - if err != nil { - return nil, err - } - - resp, err := c.client.Do(req) - if err != nil { - return nil, err - } - defer func() { - _ = resp.Body.Close() - }() - - if resp.StatusCode != http.StatusOK { - return nil, fmt.Errorf("unexpected status code: %d", resp.StatusCode) - } - - var ret PingResponse - if err := json.NewDecoder(resp.Body).Decode(&ret); err != nil { - return nil, err - } - return &ret, nil -} - -type FeatureFlagResponse map[string]FeatureFlagValue - -type FeatureFlagValue struct { - Enabled bool `json:"enabled"` -} - -func (c *Client) FeatureFlags(ctx context.Context) (FeatureFlagResponse, error) { - req, err := c.newRequest(ctx, http.MethodGet, "/features", http.NoBody) - if err != nil { - return nil, err - } - - resp, err := c.client.Do(req) - if err != nil { - return nil, err - } - defer func() { - _ = resp.Body.Close() - }() - - if resp.StatusCode != http.StatusOK { - return nil, fmt.Errorf("unexpected status code: %d", resp.StatusCode) - } - - var ret FeatureFlagResponse - if err := json.NewDecoder(resp.Body).Decode(&ret); err != nil { - return nil, err - } - return ret, nil -} - -func (c *Client) newRequest(ctx context.Context, method, path string, body io.Reader) (*http.Request, error) { - req, err := http.NewRequestWithContext(ctx, method, backendURL(path), body) - if err != nil { - return nil, err - } - req.Header.Set("User-Agent", userAgent) - return req, nil -} - -// backendURL generates a URL for the given API path. -// -// NOTE: Custom transport handles communication. The host is to create a valid -// URL for the Go http.Client that is also descriptive in error/logs. -func backendURL(path string) string { - return "http://docker-desktop/" + strings.TrimPrefix(path, "/") -} diff --git a/internal/desktop/client_test.go b/internal/desktop/client_test.go deleted file mode 100644 index abc4f33122b..00000000000 --- a/internal/desktop/client_test.go +++ /dev/null @@ -1,48 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package desktop - -import ( - "os" - "testing" - "time" - - "github.com/stretchr/testify/require" -) - -func TestClientPing(t *testing.T) { - if testing.Short() { - t.Skip("Skipped in short mode - test connects to Docker Desktop") - } - desktopEndpoint := os.Getenv("COMPOSE_TEST_DESKTOP_ENDPOINT") - if desktopEndpoint == "" { - t.Skip("Skipping - COMPOSE_TEST_DESKTOP_ENDPOINT not defined") - } - - client := NewClient(desktopEndpoint) - t.Cleanup(func() { - _ = client.Close() - }) - - now := time.Now() - - ret, err := client.Ping(t.Context()) - require.NoError(t, err) - - serverTime := time.Unix(0, ret.ServerTime) - require.True(t, now.Before(serverTime)) -} diff --git a/internal/experimental/experimental.go b/internal/experimental/experimental.go deleted file mode 100644 index d6378024307..00000000000 --- a/internal/experimental/experimental.go +++ /dev/null @@ -1,68 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package experimental - -import ( - "context" - "os" - "strconv" - - "github.com/docker/compose/v5/internal/desktop" -) - -// envComposeExperimentalGlobal can be set to a falsy value (e.g. 0, false) to -// globally opt-out of any experimental features in Compose. -const envComposeExperimentalGlobal = "COMPOSE_EXPERIMENTAL" - -// State of experiments (enabled/disabled) based on environment and local config. -type State struct { - // active is false if experiments have been opted-out of globally. - active bool - desktopValues desktop.FeatureFlagResponse -} - -func NewState() *State { - // experimental features have individual controls, but users can opt out - // of ALL experiments easily if desired - experimentsActive := true - if v := os.Getenv(envComposeExperimentalGlobal); v != "" { - experimentsActive, _ = strconv.ParseBool(v) - } - return &State{ - active: experimentsActive, - } -} - -func (s *State) Load(ctx context.Context, client *desktop.Client) error { - if !s.active { - // user opted out of experiments globally, no need to load state from - // Desktop - return nil - } - - if client == nil { - // not running under Docker Desktop - return nil - } - - desktopValues, err := client.FeatureFlags(ctx) - if err != nil { - return err - } - s.desktopValues = desktopValues - return nil -} diff --git a/internal/locker/pidfile.go b/internal/locker/pidfile.go deleted file mode 100644 index 08dcea1f3a5..00000000000 --- a/internal/locker/pidfile.go +++ /dev/null @@ -1,35 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "fmt" - "path/filepath" -) - -type Pidfile struct { - path string -} - -func NewPidfile(projectName string) (*Pidfile, error) { - run, err := runDir() - if err != nil { - return nil, err - } - path := filepath.Join(run, fmt.Sprintf("%s.pid", projectName)) - return &Pidfile{path: path}, nil -} diff --git a/internal/locker/pidfile_unix.go b/internal/locker/pidfile_unix.go deleted file mode 100644 index 484b65d8250..00000000000 --- a/internal/locker/pidfile_unix.go +++ /dev/null @@ -1,29 +0,0 @@ -//go:build !windows - -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "os" - - "github.com/docker/docker/pkg/pidfile" -) - -func (f *Pidfile) Lock() error { - return pidfile.Write(f.path, os.Getpid()) -} diff --git a/internal/locker/pidfile_windows.go b/internal/locker/pidfile_windows.go deleted file mode 100644 index adc151827dc..00000000000 --- a/internal/locker/pidfile_windows.go +++ /dev/null @@ -1,48 +0,0 @@ -//go:build windows - -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "os" - - "github.com/docker/docker/pkg/pidfile" - "github.com/mitchellh/go-ps" -) - -func (f *Pidfile) Lock() error { - newPID := os.Getpid() - err := pidfile.Write(f.path, newPID) - if err != nil { - // Get PID registered in the file - pid, errPid := pidfile.Read(f.path) - if errPid != nil { - return err - } - // Some users faced issues on Windows where the process written in the pidfile was identified as still existing - // So we used a 2nd process library to verify if this not a false positive feedback - // Check if the process exists - process, errPid := ps.FindProcess(pid) - if process == nil && errPid == nil { - // If the process does not exist, remove the pidfile and try to lock again - _ = os.Remove(f.path) - return pidfile.Write(f.path, newPID) - } - } - return err -} diff --git a/internal/locker/runtime.go b/internal/locker/runtime.go deleted file mode 100644 index e60db5cc15e..00000000000 --- a/internal/locker/runtime.go +++ /dev/null @@ -1,35 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "os" -) - -func runDir() (string, error) { - run, ok := os.LookupEnv("XDG_RUNTIME_DIR") - if ok { - return run, nil - } - - path, err := osDependentRunDir() - if err != nil { - return "", err - } - err = os.MkdirAll(path, 0o700) - return path, err -} diff --git a/internal/locker/runtime_darwin.go b/internal/locker/runtime_darwin.go deleted file mode 100644 index 127a5fb048d..00000000000 --- a/internal/locker/runtime_darwin.go +++ /dev/null @@ -1,34 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "os" - "path/filepath" -) - -// Based on https://github.com/adrg/xdg -// Licensed under MIT License (MIT) -// Copyright (c) 2014 Adrian-George Bostan <adrg@epistack.com> - -func osDependentRunDir() (string, error) { - home, err := os.UserHomeDir() - if err != nil { - return "", err - } - return filepath.Join(home, "Library", "Application Support", "com.docker.compose"), nil -} diff --git a/internal/locker/runtime_unix.go b/internal/locker/runtime_unix.go deleted file mode 100644 index 5daf61aac52..00000000000 --- a/internal/locker/runtime_unix.go +++ /dev/null @@ -1,43 +0,0 @@ -//go:build linux || openbsd || freebsd - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "os" - "path/filepath" - "strconv" -) - -// Based on https://github.com/adrg/xdg -// Licensed under MIT License (MIT) -// Copyright (c) 2014 Adrian-George Bostan <adrg@epistack.com> - -func osDependentRunDir() (string, error) { - run := filepath.Join("run", "user", strconv.Itoa(os.Getuid())) - if _, err := os.Stat(run); err == nil { - return run, nil - } - - // /run/user/$uid is set by pam_systemd, but might not be present, especially in containerized environments - home, err := os.UserHomeDir() - if err != nil { - return "", err - } - return filepath.Join(home, ".docker", "docker-compose"), nil -} diff --git a/internal/locker/runtime_windows.go b/internal/locker/runtime_windows.go deleted file mode 100644 index 4d4451b61c6..00000000000 --- a/internal/locker/runtime_windows.go +++ /dev/null @@ -1,49 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package locker - -import ( - "os" - "path/filepath" - - "golang.org/x/sys/windows" -) - -// Based on https://github.com/adrg/xdg -// Licensed under MIT License (MIT) -// Copyright (c) 2014 Adrian-George Bostan <adrg@epistack.com> - -func osDependentRunDir() (string, error) { - flags := []uint32{windows.KF_FLAG_DEFAULT, windows.KF_FLAG_DEFAULT_PATH} - for _, flag := range flags { - p, _ := windows.KnownFolderPath(windows.FOLDERID_LocalAppData, flag|windows.KF_FLAG_DONT_VERIFY) - if p != "" { - return filepath.Join(p, "docker-compose"), nil - } - } - - appData, ok := os.LookupEnv("LOCALAPPDATA") - if ok { - return filepath.Join(appData, "docker-compose"), nil - } - - home, err := os.UserHomeDir() - if err != nil { - return "", err - } - return filepath.Join(home, "AppData", "Local", "docker-compose"), nil -} diff --git a/internal/memnet/conn.go b/internal/memnet/conn.go deleted file mode 100644 index 224bec78830..00000000000 --- a/internal/memnet/conn.go +++ /dev/null @@ -1,50 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package memnet - -import ( - "context" - "fmt" - "net" - "strings" -) - -func DialEndpoint(ctx context.Context, endpoint string) (net.Conn, error) { - if addr, ok := strings.CutPrefix(endpoint, "unix://"); ok { - return Dial(ctx, "unix", addr) - } - if addr, ok := strings.CutPrefix(endpoint, "npipe://"); ok { - return Dial(ctx, "npipe", addr) - } - return nil, fmt.Errorf("unsupported protocol for address: %s", endpoint) -} - -func Dial(ctx context.Context, network, addr string) (net.Conn, error) { - var d net.Dialer - switch network { - case "unix": - if err := validateSocketPath(addr); err != nil { - return nil, err - } - return d.DialContext(ctx, "unix", addr) - case "npipe": - // N.B. this will return an error on non-Windows - return dialNamedPipe(ctx, addr) - default: - return nil, fmt.Errorf("unsupported network: %s", network) - } -} diff --git a/internal/memnet/conn_unix.go b/internal/memnet/conn_unix.go deleted file mode 100644 index e151984848a..00000000000 --- a/internal/memnet/conn_unix.go +++ /dev/null @@ -1,39 +0,0 @@ -//go:build !windows - -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package memnet - -import ( - "context" - "fmt" - "net" - "syscall" -) - -const maxUnixSocketPathSize = len(syscall.RawSockaddrUnix{}.Path) - -func dialNamedPipe(_ context.Context, _ string) (net.Conn, error) { - return nil, fmt.Errorf("named pipes are only available on Windows") -} - -func validateSocketPath(addr string) error { - if len(addr) > maxUnixSocketPathSize { - return fmt.Errorf("socket address is too long: %s", addr) - } - return nil -} diff --git a/internal/memnet/conn_windows.go b/internal/memnet/conn_windows.go deleted file mode 100644 index b7f7d9ea8fa..00000000000 --- a/internal/memnet/conn_windows.go +++ /dev/null @@ -1,33 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package memnet - -import ( - "context" - "net" - - "github.com/Microsoft/go-winio" -) - -func dialNamedPipe(ctx context.Context, addr string) (net.Conn, error) { - return winio.DialPipeContext(ctx, addr) -} - -func validateSocketPath(addr string) error { - // AF_UNIX sockets do not have strict path limits on Windows - return nil -} diff --git a/internal/oci/push.go b/internal/oci/push.go deleted file mode 100644 index 1e2d0f2e95b..00000000000 --- a/internal/oci/push.go +++ /dev/null @@ -1,223 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package oci - -import ( - "context" - "encoding/json" - "errors" - "fmt" - "net/http" - "path/filepath" - "slices" - "time" - - "github.com/containerd/containerd/v2/core/remotes" - pusherrors "github.com/containerd/containerd/v2/core/remotes/errors" - "github.com/distribution/reference" - "github.com/opencontainers/go-digest" - "github.com/opencontainers/image-spec/specs-go" - v1 "github.com/opencontainers/image-spec/specs-go/v1" - - "github.com/docker/compose/v5/pkg/api" -) - -const ( - // ComposeProjectArtifactType is the OCI 1.1-compliant artifact type value - // for the generated image manifest. - ComposeProjectArtifactType = "application/vnd.docker.compose.project" - // ComposeYAMLMediaType is the media type for each layer (Compose file) - // in the image manifest. - ComposeYAMLMediaType = "application/vnd.docker.compose.file+yaml" - // ComposeEmptyConfigMediaType is a media type used for the config descriptor - // when doing OCI 1.0-style pushes. - // - // The content is always `{}`, the same as a normal empty descriptor, but - // the specific media type allows clients to fall back to the config media - // type to recognize the manifest as a Compose project since the artifact - // type field is not available in OCI 1.0. - // - // This is based on guidance from the OCI 1.1 spec: - // > Implementers note: artifacts have historically been created without - // > an artifactType field, and tooling to work with artifacts should - // > fallback to the config.mediaType value. - ComposeEmptyConfigMediaType = "application/vnd.docker.compose.config.empty.v1+json" - // ComposeEnvFileMediaType is the media type for each Env File layer in the image manifest. - ComposeEnvFileMediaType = "application/vnd.docker.compose.envfile" -) - -// clientAuthStatusCodes are client (4xx) errors that are authentication -// related. -var clientAuthStatusCodes = []int{ - http.StatusUnauthorized, - http.StatusForbidden, - http.StatusProxyAuthRequired, -} - -func DescriptorForComposeFile(path string, content []byte) v1.Descriptor { - return v1.Descriptor{ - MediaType: ComposeYAMLMediaType, - Digest: digest.FromString(string(content)), - Size: int64(len(content)), - Annotations: map[string]string{ - "com.docker.compose.version": api.ComposeVersion, - "com.docker.compose.file": filepath.Base(path), - }, - Data: content, - } -} - -func DescriptorForEnvFile(path string, content []byte) v1.Descriptor { - return v1.Descriptor{ - MediaType: ComposeEnvFileMediaType, - Digest: digest.FromString(string(content)), - Size: int64(len(content)), - Annotations: map[string]string{ - "com.docker.compose.version": api.ComposeVersion, - "com.docker.compose.envfile": filepath.Base(path), - }, - Data: content, - } -} - -func PushManifest(ctx context.Context, resolver remotes.Resolver, named reference.Named, layers []v1.Descriptor, ociVersion api.OCIVersion) (v1.Descriptor, error) { - // Check if we need an extra empty layer for the manifest config - if ociVersion == api.OCIVersion1_1 || ociVersion == "" { - err := push(ctx, resolver, named, v1.DescriptorEmptyJSON) - if err != nil { - return v1.Descriptor{}, err - } - } - // prepare to push the manifest by pushing the layers - layerDescriptors := make([]v1.Descriptor, len(layers)) - for i := range layers { - layerDescriptors[i] = layers[i] - if err := push(ctx, resolver, named, layers[i]); err != nil { - return v1.Descriptor{}, err - } - } - - if ociVersion != "" { - // if a version was explicitly specified, use it - return createAndPushManifest(ctx, resolver, named, layerDescriptors, ociVersion) - } - - // try to push in the OCI 1.1 format but fallback to OCI 1.0 on 4xx errors - // (other than auth) since it's most likely the result of the registry not - // having support - descriptor, err := createAndPushManifest(ctx, resolver, named, layerDescriptors, api.OCIVersion1_1) - var pushErr pusherrors.ErrUnexpectedStatus - if errors.As(err, &pushErr) && isNonAuthClientError(pushErr.StatusCode) { - // TODO(milas): show a warning here (won't work with logrus) - return createAndPushManifest(ctx, resolver, named, layerDescriptors, api.OCIVersion1_0) - } - return descriptor, err -} - -func push(ctx context.Context, resolver remotes.Resolver, ref reference.Named, descriptor v1.Descriptor) error { - fullRef, err := reference.WithDigest(reference.TagNameOnly(ref), descriptor.Digest) - if err != nil { - return err - } - - return Push(ctx, resolver, fullRef, descriptor) -} - -func createAndPushManifest(ctx context.Context, resolver remotes.Resolver, named reference.Named, layers []v1.Descriptor, ociVersion api.OCIVersion) (v1.Descriptor, error) { - descriptor, toPush, err := generateManifest(layers, ociVersion) - if err != nil { - return v1.Descriptor{}, err - } - for _, p := range toPush { - err = push(ctx, resolver, named, p) - if err != nil { - return v1.Descriptor{}, err - } - } - return descriptor, nil -} - -func isNonAuthClientError(statusCode int) bool { - if statusCode < 400 || statusCode >= 500 { - // not a client error - return false - } - return !slices.Contains(clientAuthStatusCodes, statusCode) -} - -func generateManifest(layers []v1.Descriptor, ociCompat api.OCIVersion) (v1.Descriptor, []v1.Descriptor, error) { - var toPush []v1.Descriptor - var config v1.Descriptor - var artifactType string - switch ociCompat { - case api.OCIVersion1_0: - // "Content other than OCI container images MAY be packaged using the image manifest. - // When this is done, the config.mediaType value MUST be set to a value specific to - // the artifact type or the empty value." - // Source: https://github.com/opencontainers/image-spec/blob/main/manifest.md#guidelines-for-artifact-usage - // - // The `ComposeEmptyConfigMediaType` is used specifically for this purpose: - // there is no config, and an empty descriptor is used for OCI 1.1 in - // conjunction with the `ArtifactType`, but for OCI 1.0 compatibility, - // tooling falls back to the config media type, so this is used to - // indicate that it's not a container image but custom content. - configData := []byte("{}") - config = v1.Descriptor{ - MediaType: ComposeEmptyConfigMediaType, - Digest: digest.FromBytes(configData), - Size: int64(len(configData)), - Data: configData, - } - // N.B. OCI 1.0 does NOT support specifying the artifact type, so it's - // left as an empty string to omit it from the marshaled JSON - artifactType = "" - toPush = append(toPush, config) - case api.OCIVersion1_1: - config = v1.DescriptorEmptyJSON - artifactType = ComposeProjectArtifactType - toPush = append(toPush, config) - default: - return v1.Descriptor{}, nil, fmt.Errorf("unsupported OCI version: %s", ociCompat) - } - - manifest, err := json.Marshal(v1.Manifest{ - Versioned: specs.Versioned{SchemaVersion: 2}, - MediaType: v1.MediaTypeImageManifest, - ArtifactType: artifactType, - Config: config, - Layers: layers, - Annotations: map[string]string{ - "org.opencontainers.image.created": time.Now().Format(time.RFC3339), - }, - }) - if err != nil { - return v1.Descriptor{}, nil, err - } - - manifestDescriptor := v1.Descriptor{ - MediaType: v1.MediaTypeImageManifest, - Digest: digest.FromString(string(manifest)), - Size: int64(len(manifest)), - Annotations: map[string]string{ - "com.docker.compose.version": api.ComposeVersion, - }, - ArtifactType: artifactType, - Data: manifest, - } - toPush = append(toPush, manifestDescriptor) - return manifestDescriptor, toPush, nil -} diff --git a/internal/oci/resolver.go b/internal/oci/resolver.go deleted file mode 100644 index f18c474de57..00000000000 --- a/internal/oci/resolver.go +++ /dev/null @@ -1,143 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package oci - -import ( - "context" - "io" - "net/url" - "slices" - "strings" - - "github.com/containerd/containerd/v2/core/remotes" - "github.com/containerd/containerd/v2/core/remotes/docker" - "github.com/containerd/containerd/v2/pkg/labels" - "github.com/containerd/errdefs" - "github.com/distribution/reference" - "github.com/docker/cli/cli/config/configfile" - "github.com/moby/buildkit/util/contentutil" - spec "github.com/opencontainers/image-spec/specs-go/v1" - - "github.com/docker/compose/v5/internal/registry" -) - -// NewResolver setup an OCI Resolver based on docker/cli config to provide registry credentials -func NewResolver(config *configfile.ConfigFile, insecureRegistries ...string) remotes.Resolver { - return docker.NewResolver(docker.ResolverOptions{ - Hosts: docker.ConfigureDefaultRegistries( - docker.WithAuthorizer(docker.NewDockerAuthorizer( - docker.WithAuthCreds(func(host string) (string, string, error) { - host = registry.GetAuthConfigKey(host) - auth, err := config.GetAuthConfig(host) - if err != nil { - return "", "", err - } - if auth.IdentityToken != "" { - return "", auth.IdentityToken, nil - } - return auth.Username, auth.Password, nil - }), - )), - docker.WithPlainHTTP(func(domain string) (bool, error) { - // Should be used for testing **only** - return slices.Contains(insecureRegistries, domain), nil - }), - ), - }) -} - -// Get retrieves a Named OCI resource and returns OCI Descriptor and Manifest -func Get(ctx context.Context, resolver remotes.Resolver, ref reference.Named) (spec.Descriptor, []byte, error) { - _, descriptor, err := resolver.Resolve(ctx, ref.String()) - if err != nil { - return spec.Descriptor{}, nil, err - } - - fetcher, err := resolver.Fetcher(ctx, ref.String()) - if err != nil { - return spec.Descriptor{}, nil, err - } - fetch, err := fetcher.Fetch(ctx, descriptor) - if err != nil { - return spec.Descriptor{}, nil, err - } - content, err := io.ReadAll(fetch) - if err != nil { - return spec.Descriptor{}, nil, err - } - return descriptor, content, nil -} - -func Copy(ctx context.Context, resolver remotes.Resolver, image reference.Named, named reference.Named) (spec.Descriptor, error) { - src, desc, err := resolver.Resolve(ctx, image.String()) - if err != nil { - return spec.Descriptor{}, err - } - if desc.Annotations == nil { - desc.Annotations = make(map[string]string) - } - // set LabelDistributionSource so push will actually use a registry mount - refspec := reference.TrimNamed(image).String() - u, err := url.Parse("dummy://" + refspec) - if err != nil { - return spec.Descriptor{}, err - } - source, repo := u.Hostname(), strings.TrimPrefix(u.Path, "/") - desc.Annotations[labels.LabelDistributionSource+"."+source] = repo - - p, err := resolver.Pusher(ctx, named.Name()) - if err != nil { - return spec.Descriptor{}, err - } - f, err := resolver.Fetcher(ctx, src) - if err != nil { - return spec.Descriptor{}, err - } - - err = contentutil.CopyChain(ctx, - contentutil.FromPusher(p), - contentutil.FromFetcher(f), desc) - return desc, err -} - -func Push(ctx context.Context, resolver remotes.Resolver, ref reference.Named, descriptor spec.Descriptor) error { - pusher, err := resolver.Pusher(ctx, ref.String()) - if err != nil { - return err - } - ctx = remotes.WithMediaTypeKeyPrefix(ctx, ComposeYAMLMediaType, "artifact-") - ctx = remotes.WithMediaTypeKeyPrefix(ctx, ComposeEnvFileMediaType, "artifact-") - ctx = remotes.WithMediaTypeKeyPrefix(ctx, ComposeEmptyConfigMediaType, "config-") - ctx = remotes.WithMediaTypeKeyPrefix(ctx, spec.MediaTypeEmptyJSON, "config-") - - push, err := pusher.Push(ctx, descriptor) - if errdefs.IsAlreadyExists(err) { - return nil - } - if err != nil { - return err - } - - _, err = push.Write(descriptor.Data) - if err != nil { - // Close the writer on error since Commit won't be called - _ = push.Close() - return err - } - // Commit will close the writer - return push.Commit(ctx, int64(len(descriptor.Data)), descriptor.Digest) -} diff --git a/internal/paths/paths.go b/internal/paths/paths.go deleted file mode 100644 index 4e4c01b8cc4..00000000000 --- a/internal/paths/paths.go +++ /dev/null @@ -1,120 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package paths - -import ( - "os" - "path/filepath" - "strings" -) - -func IsChild(dir string, file string) bool { - if dir == "" { - return false - } - - dir = filepath.Clean(dir) - current := filepath.Clean(file) - child := "." - for { - if strings.EqualFold(dir, current) { - // If the two paths are exactly equal, then they must be the same. - if dir == current { - return true - } - - // If the two paths are equal under case-folding, but not exactly equal, - // then the only way to check if they're truly "equal" is to check - // to see if we're on a case-insensitive file system. - // - // This is a notoriously tricky problem. See how dep solves it here: - // https://github.com/golang/dep/blob/v0.5.4/internal/fs/fs.go#L33 - // - // because you can mount case-sensitive filesystems onto case-insensitive - // file-systems, and vice versa :scream: - // - // We want to do as much of this check as possible with strings-only - // (to avoid a file system read and error handling), so we only - // do this check if we have no other choice. - dirInfo, err := os.Stat(dir) - if err != nil { - return false - } - - currentInfo, err := os.Stat(current) - if err != nil { - return false - } - - if !os.SameFile(dirInfo, currentInfo) { - return false - } - return true - } - - if len(current) <= len(dir) || current == "." { - return false - } - - cDir := filepath.Dir(current) - cBase := filepath.Base(current) - child = filepath.Join(cBase, child) - current = cDir - } -} - -// EncompassingPaths returns the minimal set of paths that root all paths -// from the original collection. -// -// For example, ["/foo", "/foo/bar", "/foo", "/baz"] -> ["/foo", "/baz]. -func EncompassingPaths(paths []string) []string { - result := []string{} - for _, current := range paths { - isCovered := false - hasRemovals := false - - for i, existing := range result { - if IsChild(existing, current) { - // The path is already covered, so there's no need to include it - isCovered = true - break - } - - if IsChild(current, existing) { - // Mark the element empty for removal. - result[i] = "" - hasRemovals = true - } - } - - if !isCovered { - result = append(result, current) - } - - if hasRemovals { - // Remove all the empties - newResult := []string{} - for _, r := range result { - if r != "" { - newResult = append(newResult, r) - } - } - result = newResult - } - } - return result -} diff --git a/internal/registry/registry.go b/internal/registry/registry.go deleted file mode 100644 index 0ee73883070..00000000000 --- a/internal/registry/registry.go +++ /dev/null @@ -1,44 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package registry - -const ( - // DefaultNamespace is the default namespace - DefaultNamespace = "docker.io" - // DefaultRegistryHost is the hostname for the default (Docker Hub) registry - // used for pushing and pulling images. This hostname is hard-coded to handle - // the conversion from image references without registry name (e.g. "ubuntu", - // or "ubuntu:latest"), as well as references using the "docker.io" domain - // name, which is used as canonical reference for images on Docker Hub, but - // does not match the domain-name of Docker Hub's registry. - DefaultRegistryHost = "registry-1.docker.io" - // IndexHostname is the index hostname, used for authentication and image search. - IndexHostname = "index.docker.io" - // IndexServer is used for user auth and image search - IndexServer = "https://" + IndexHostname + "/v1/" - // IndexName is the name of the index - IndexName = "docker.io" -) - -// GetAuthConfigKey special-cases using the full index address of the official -// index as the AuthConfig key, and uses the (host)name[:port] for private indexes. -func GetAuthConfigKey(indexName string) string { - if indexName == IndexName || indexName == IndexHostname || indexName == DefaultRegistryHost { - return IndexServer - } - return indexName -} diff --git a/internal/sync/shared.go b/internal/sync/shared.go deleted file mode 100644 index 4fd9df37719..00000000000 --- a/internal/sync/shared.go +++ /dev/null @@ -1,40 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package sync - -import ( - "context" -) - -// PathMapping contains the Compose service and modified host system path. -type PathMapping struct { - // HostPath that was created/modified/deleted outside the container. - // - // This is the path as seen from the user's perspective, e.g. - // - C:\Users\moby\Documents\hello-world\main.go (file on Windows) - // - /Users/moby/Documents/hello-world (directory on macOS) - HostPath string - // ContainerPath for the target file inside the container (only populated - // for sync events, not rebuild). - // - // This is the path as used in Docker CLI commands, e.g. - // - /workdir/main.go - // - /workdir/subdir - ContainerPath string -} - -type Syncer interface { - Sync(ctx context.Context, service string, paths []*PathMapping) error -} diff --git a/internal/sync/tar.go b/internal/sync/tar.go deleted file mode 100644 index 4250b6afc4a..00000000000 --- a/internal/sync/tar.go +++ /dev/null @@ -1,352 +0,0 @@ -/* - Copyright 2018 The Tilt Dev Authors - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package sync - -import ( - "archive/tar" - "bytes" - "context" - "errors" - "fmt" - "io" - "io/fs" - "os" - "path" - "path/filepath" - "strings" - "sync" - - "github.com/docker/docker/api/types/container" - "github.com/moby/go-archive" - "golang.org/x/sync/errgroup" -) - -type archiveEntry struct { - path string - info os.FileInfo - header *tar.Header -} - -type LowLevelClient interface { - ContainersForService(ctx context.Context, projectName string, serviceName string) ([]container.Summary, error) - - Exec(ctx context.Context, containerID string, cmd []string, in io.Reader) error - Untar(ctx context.Context, id string, reader io.ReadCloser) error -} - -type Tar struct { - client LowLevelClient - - projectName string -} - -var _ Syncer = &Tar{} - -func NewTar(projectName string, client LowLevelClient) *Tar { - return &Tar{ - projectName: projectName, - client: client, - } -} - -func (t *Tar) Sync(ctx context.Context, service string, paths []*PathMapping) error { - containers, err := t.client.ContainersForService(ctx, t.projectName, service) - if err != nil { - return err - } - - var pathsToCopy []PathMapping - var pathsToDelete []string - for _, p := range paths { - if _, err := os.Stat(p.HostPath); err != nil && errors.Is(err, fs.ErrNotExist) { - pathsToDelete = append(pathsToDelete, p.ContainerPath) - } else { - pathsToCopy = append(pathsToCopy, *p) - } - } - - var deleteCmd []string - if len(pathsToDelete) != 0 { - deleteCmd = append([]string{"rm", "-rf"}, pathsToDelete...) - } - - var ( - eg errgroup.Group - errMu sync.Mutex - errs = make([]error, 0, len(containers)*2) // max 2 errs per container - ) - - eg.SetLimit(16) // arbitrary limit, adjust to taste :D - for i := range containers { - containerID := containers[i].ID - tarReader := tarArchive(pathsToCopy) - - eg.Go(func() error { - if len(deleteCmd) != 0 { - if err := t.client.Exec(ctx, containerID, deleteCmd, nil); err != nil { - errMu.Lock() - errs = append(errs, fmt.Errorf("deleting paths in %s: %w", containerID, err)) - errMu.Unlock() - } - } - - if err := t.client.Untar(ctx, containerID, tarReader); err != nil { - errMu.Lock() - errs = append(errs, fmt.Errorf("copying files to %s: %w", containerID, err)) - errMu.Unlock() - } - return nil // don't fail-fast; collect all errors - }) - } - - _ = eg.Wait() - return errors.Join(errs...) -} - -type ArchiveBuilder struct { - tw *tar.Writer - // A shared I/O buffer to help with file copying. - copyBuf *bytes.Buffer -} - -func NewArchiveBuilder(writer io.Writer) *ArchiveBuilder { - tw := tar.NewWriter(writer) - return &ArchiveBuilder{ - tw: tw, - copyBuf: &bytes.Buffer{}, - } -} - -func (a *ArchiveBuilder) Close() error { - return a.tw.Close() -} - -// ArchivePathsIfExist creates a tar archive of all local files in `paths`. It quietly skips any paths that don't exist. -func (a *ArchiveBuilder) ArchivePathsIfExist(paths []PathMapping) error { - // In order to handle overlapping syncs, we - // 1) collect all the entries, - // 2) de-dupe them, with last-one-wins semantics - // 3) write all the entries - // - // It's not obvious that this is the correct behavior. A better approach - // (that's more in-line with how syncs work) might ignore files in earlier - // path mappings when we know they're going to be "synced" over. - // There's a bunch of subtle product decisions about how overlapping path - // mappings work that we're not sure about. - var entries []archiveEntry - for _, p := range paths { - newEntries, err := a.entriesForPath(p.HostPath, p.ContainerPath) - if err != nil { - return fmt.Errorf("inspecting %q: %w", p.HostPath, err) - } - - entries = append(entries, newEntries...) - } - - entries = dedupeEntries(entries) - for _, entry := range entries { - err := a.writeEntry(entry) - if err != nil { - return fmt.Errorf("archiving %q: %w", entry.path, err) - } - } - return nil -} - -func (a *ArchiveBuilder) writeEntry(entry archiveEntry) error { - pathInTar := entry.path - header := entry.header - - if header.Typeflag != tar.TypeReg { - // anything other than a regular file (e.g. dir, symlink) just needs the header - if err := a.tw.WriteHeader(header); err != nil { - return fmt.Errorf("writing %q header: %w", pathInTar, err) - } - return nil - } - - file, err := os.Open(pathInTar) - if err != nil { - // In case the file has been deleted since we last looked at it. - if os.IsNotExist(err) { - return nil - } - return err - } - - defer func() { - _ = file.Close() - }() - - // The size header must match the number of contents bytes. - // - // There is room for a race condition here if something writes to the file - // after we've read the file size. - // - // For small files, we avoid this by first copying the file into a buffer, - // and using the size of the buffer to populate the header. - // - // For larger files, we don't want to copy the whole thing into a buffer, - // because that would blow up heap size. There is some danger that this - // will lead to a spurious error when the tar writer validates the sizes. - // That error will be disruptive but will be handled as best as we - // can downstream. - useBuf := header.Size < 5000000 - if useBuf { - a.copyBuf.Reset() - _, err = io.Copy(a.copyBuf, file) - if err != nil && !errors.Is(err, io.EOF) { - return fmt.Errorf("copying %q: %w", pathInTar, err) - } - header.Size = int64(len(a.copyBuf.Bytes())) - } - - // wait to write the header until _after_ the file is successfully opened - // to avoid generating an invalid tar entry that has a header but no contents - // in the case the file has been deleted - err = a.tw.WriteHeader(header) - if err != nil { - return fmt.Errorf("writing %q header: %w", pathInTar, err) - } - - if useBuf { - _, err = io.Copy(a.tw, a.copyBuf) - } else { - _, err = io.Copy(a.tw, file) - } - - if err != nil && !errors.Is(err, io.EOF) { - return fmt.Errorf("copying %q: %w", pathInTar, err) - } - - // explicitly flush so that if the entry is invalid we will detect it now and - // provide a more meaningful error - if err := a.tw.Flush(); err != nil { - return fmt.Errorf("finalizing %q: %w", pathInTar, err) - } - return nil -} - -// entriesForPath writes the given source path into tarWriter at the given dest (recursively for directories). -// e.g. tarring my_dir --> dest d: d/file_a, d/file_b -// If source path does not exist, quietly skips it and returns no err -func (a *ArchiveBuilder) entriesForPath(localPath, containerPath string) ([]archiveEntry, error) { - localInfo, err := os.Stat(localPath) - if err != nil { - if os.IsNotExist(err) { - return nil, nil - } - return nil, err - } - - localPathIsDir := localInfo.IsDir() - if localPathIsDir { - // Make sure we can trim this off filenames to get valid relative filepaths - if !strings.HasSuffix(localPath, string(filepath.Separator)) { - localPath += string(filepath.Separator) - } - } - - containerPath = strings.TrimPrefix(containerPath, "/") - - result := make([]archiveEntry, 0) - err = filepath.Walk(localPath, func(curLocalPath string, info os.FileInfo, err error) error { - if err != nil { - return fmt.Errorf("walking %q: %w", curLocalPath, err) - } - - linkname := "" - if info.Mode()&os.ModeSymlink != 0 { - var err error - linkname, err = os.Readlink(curLocalPath) - if err != nil { - return err - } - } - - var name string - //nolint:gocritic - if localPathIsDir { - // Name of file in tar should be relative to source directory... - tmp, err := filepath.Rel(localPath, curLocalPath) - if err != nil { - return fmt.Errorf("making %q relative to %q: %w", curLocalPath, localPath, err) - } - // ...and live inside `dest` - name = path.Join(containerPath, filepath.ToSlash(tmp)) - } else if strings.HasSuffix(containerPath, "/") { - name = containerPath + filepath.Base(curLocalPath) - } else { - name = containerPath - } - - header, err := archive.FileInfoHeader(name, info, linkname) - if err != nil { - // Not all types of files are allowed in a tarball. That's OK. - // Mimic the Docker behavior and just skip the file. - return nil - } - - result = append(result, archiveEntry{ - path: curLocalPath, - info: info, - header: header, - }) - - return nil - }) - if err != nil { - return nil, err - } - return result, nil -} - -func tarArchive(ops []PathMapping) io.ReadCloser { - pr, pw := io.Pipe() - go func() { - ab := NewArchiveBuilder(pw) - err := ab.ArchivePathsIfExist(ops) - if err != nil { - _ = pw.CloseWithError(fmt.Errorf("adding files to tar: %w", err)) - } else { - // propagate errors from the TarWriter::Close() because it performs a final - // Flush() and any errors mean the tar is invalid - if err := ab.Close(); err != nil { - _ = pw.CloseWithError(fmt.Errorf("closing tar: %w", err)) - } else { - _ = pw.Close() - } - } - }() - return pr -} - -// Dedupe the entries with last-entry-wins semantics. -func dedupeEntries(entries []archiveEntry) []archiveEntry { - seenIndex := make(map[string]int, len(entries)) - result := make([]archiveEntry, 0, len(entries)) - for i, entry := range entries { - seenIndex[entry.header.Name] = i - } - for i, entry := range entries { - if seenIndex[entry.header.Name] == i { - result = append(result, entry) - } - } - return result -} diff --git a/internal/tracing/attributes.go b/internal/tracing/attributes.go deleted file mode 100644 index 2c8779bc86c..00000000000 --- a/internal/tracing/attributes.go +++ /dev/null @@ -1,196 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "context" - "crypto/sha256" - "encoding/json" - "fmt" - "strings" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/trace" -) - -// SpanOptions is a small helper type to make it easy to share the options helpers between -// downstream functions that accept slices of trace.SpanStartOption and trace.EventOption. -type SpanOptions []trace.SpanStartEventOption - -type MetricsKey struct{} - -type Metrics struct { - CountExtends int - CountIncludesLocal int - CountIncludesRemote int -} - -func (s SpanOptions) SpanStartOptions() []trace.SpanStartOption { - out := make([]trace.SpanStartOption, len(s)) - for i := range s { - out[i] = s[i] - } - return out -} - -func (s SpanOptions) EventOptions() []trace.EventOption { - out := make([]trace.EventOption, len(s)) - for i := range s { - out[i] = s[i] - } - return out -} - -// ProjectOptions returns common attributes from a Compose project. -// -// For convenience, it's returned as a SpanOptions object to allow it to be -// passed directly to the wrapping helper methods in this package such as -// SpanWrapFunc. -func ProjectOptions(ctx context.Context, proj *types.Project) SpanOptions { - if proj == nil { - return nil - } - capabilities, gpu, tpu := proj.ServicesWithCapabilities() - attrs := []attribute.KeyValue{ - attribute.String("project.name", proj.Name), - attribute.String("project.dir", proj.WorkingDir), - attribute.StringSlice("project.compose_files", proj.ComposeFiles), - attribute.StringSlice("project.profiles", proj.Profiles), - attribute.StringSlice("project.volumes", proj.VolumeNames()), - attribute.StringSlice("project.networks", proj.NetworkNames()), - attribute.StringSlice("project.secrets", proj.SecretNames()), - attribute.StringSlice("project.configs", proj.ConfigNames()), - attribute.StringSlice("project.models", proj.ModelNames()), - attribute.StringSlice("project.extensions", keys(proj.Extensions)), - attribute.StringSlice("project.services.active", proj.ServiceNames()), - attribute.StringSlice("project.services.disabled", proj.DisabledServiceNames()), - attribute.StringSlice("project.services.build", proj.ServicesWithBuild()), - attribute.StringSlice("project.services.depends_on", proj.ServicesWithDependsOn()), - attribute.StringSlice("project.services.models", proj.ServicesWithModels()), - attribute.StringSlice("project.services.capabilities", capabilities), - attribute.StringSlice("project.services.capabilities.gpu", gpu), - attribute.StringSlice("project.services.capabilities.tpu", tpu), - } - if metrics, ok := ctx.Value(MetricsKey{}).(Metrics); ok { - attrs = append(attrs, attribute.Int("project.services.extends", metrics.CountExtends)) - attrs = append(attrs, attribute.Int("project.includes.local", metrics.CountIncludesLocal)) - attrs = append(attrs, attribute.Int("project.includes.remote", metrics.CountIncludesRemote)) - } - - if projHash, ok := projectHash(proj); ok { - attrs = append(attrs, attribute.String("project.hash", projHash)) - } - return []trace.SpanStartEventOption{ - trace.WithAttributes(attrs...), - } -} - -// ServiceOptions returns common attributes from a Compose service. -// -// For convenience, it's returned as a SpanOptions object to allow it to be -// passed directly to the wrapping helper methods in this package such as -// SpanWrapFunc. -func ServiceOptions(service types.ServiceConfig) SpanOptions { - attrs := []attribute.KeyValue{ - attribute.String("service.name", service.Name), - attribute.String("service.image", service.Image), - attribute.StringSlice("service.networks", keys(service.Networks)), - attribute.StringSlice("service.models", keys(service.Models)), - } - - configNames := make([]string, len(service.Configs)) - for i := range service.Configs { - configNames[i] = service.Configs[i].Source - } - attrs = append(attrs, attribute.StringSlice("service.configs", configNames)) - - secretNames := make([]string, len(service.Secrets)) - for i := range service.Secrets { - secretNames[i] = service.Secrets[i].Source - } - attrs = append(attrs, attribute.StringSlice("service.secrets", secretNames)) - - volNames := make([]string, len(service.Volumes)) - for i := range service.Volumes { - volNames[i] = service.Volumes[i].Source - } - attrs = append(attrs, attribute.StringSlice("service.volumes", volNames)) - - return []trace.SpanStartEventOption{ - trace.WithAttributes(attrs...), - } -} - -// ContainerOptions returns common attributes from a Moby container. -// -// For convenience, it's returned as a SpanOptions object to allow it to be -// passed directly to the wrapping helper methods in this package such as -// SpanWrapFunc. -func ContainerOptions(ctr container.Summary) SpanOptions { - attrs := []attribute.KeyValue{ - attribute.String("container.id", ctr.ID), - attribute.String("container.image", ctr.Image), - unixTimeAttr("container.created_at", ctr.Created), - } - - if len(ctr.Names) != 0 { - attrs = append(attrs, attribute.String("container.name", strings.TrimPrefix(ctr.Names[0], "/"))) - } - - return []trace.SpanStartEventOption{ - trace.WithAttributes(attrs...), - } -} - -func keys[T any](m map[string]T) []string { - out := make([]string, 0, len(m)) - for k := range m { - out = append(out, k) - } - return out -} - -func timeAttr(key string, value time.Time) attribute.KeyValue { - return attribute.String(key, value.Format(time.RFC3339)) -} - -func unixTimeAttr(key string, value int64) attribute.KeyValue { - return timeAttr(key, time.Unix(value, 0).UTC()) -} - -// projectHash returns a checksum from the JSON encoding of the project. -func projectHash(p *types.Project) (string, bool) { - if p == nil { - return "", false - } - // disabled services aren't included in the output, so make a copy with - // all the services active for hashing - var err error - p, err = p.WithServicesEnabled(append(p.ServiceNames(), p.DisabledServiceNames()...)...) - if err != nil { - return "", false - } - projData, err := json.Marshal(p) - if err != nil { - return "", false - } - d := sha256.Sum256(projData) - return fmt.Sprintf("%x", d), true -} diff --git a/internal/tracing/attributes_test.go b/internal/tracing/attributes_test.go deleted file mode 100644 index d4277a940ab..00000000000 --- a/internal/tracing/attributes_test.go +++ /dev/null @@ -1,67 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/stretchr/testify/require" -) - -func TestProjectHash(t *testing.T) { - projA := &types.Project{ - Name: "fake-proj", - WorkingDir: "/tmp", - Services: map[string]types.ServiceConfig{ - "foo": {Image: "fake-image"}, - }, - DisabledServices: map[string]types.ServiceConfig{ - "bar": {Image: "diff-image"}, - }, - } - projB := &types.Project{ - Name: "fake-proj", - WorkingDir: "/tmp", - Services: map[string]types.ServiceConfig{ - "foo": {Image: "fake-image"}, - "bar": {Image: "diff-image"}, - }, - } - projC := &types.Project{ - Name: "fake-proj", - WorkingDir: "/tmp", - Services: map[string]types.ServiceConfig{ - "foo": {Image: "fake-image"}, - "bar": {Image: "diff-image"}, - "baz": {Image: "yet-another-image"}, - }, - } - - hashA, ok := projectHash(projA) - require.True(t, ok) - require.NotEmpty(t, hashA) - hashB, ok := projectHash(projB) - require.True(t, ok) - require.NotEmpty(t, hashB) - require.Equal(t, hashA, hashB) - - hashC, ok := projectHash(projC) - require.True(t, ok) - require.NotEmpty(t, hashC) - require.NotEqual(t, hashC, hashA) -} diff --git a/internal/tracing/docker_context.go b/internal/tracing/docker_context.go deleted file mode 100644 index 5d5367b0231..00000000000 --- a/internal/tracing/docker_context.go +++ /dev/null @@ -1,123 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "fmt" - "os" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/context/store" - "go.opentelemetry.io/otel/exporters/otlp/otlptrace" - "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc" - "google.golang.org/grpc" - "google.golang.org/grpc/credentials/insecure" - - "github.com/docker/compose/v5/internal/memnet" -) - -const otelConfigFieldName = "otel" - -// traceClientFromDockerContext creates a gRPC OTLP client based on metadata -// from the active Docker CLI context. -func traceClientFromDockerContext(dockerCli command.Cli, otelEnv envMap) (otlptrace.Client, error) { - // attempt to extract an OTEL config from the Docker context to enable - // automatic integration with Docker Desktop; - cfg, err := ConfigFromDockerContext(dockerCli.ContextStore(), dockerCli.CurrentContext()) - if err != nil { - return nil, fmt.Errorf("loading otel config from docker context metadata: %w", err) - } - - if cfg.Endpoint == "" { - return nil, nil - } - - // HACK: unfortunately _all_ public OTEL initialization functions - // implicitly read from the OS env, so temporarily unset them all and - // restore afterwards - defer func() { - for k, v := range otelEnv { - if err := os.Setenv(k, v); err != nil { - panic(fmt.Errorf("restoring env for %q: %w", k, err)) - } - } - }() - for k := range otelEnv { - if err := os.Unsetenv(k); err != nil { - return nil, fmt.Errorf("stashing env for %q: %w", k, err) - } - } - - conn, err := grpc.NewClient(cfg.Endpoint, - grpc.WithContextDialer(memnet.DialEndpoint), - // this dial is restricted to using a local Unix socket / named pipe, - // so there is no need for TLS - grpc.WithTransportCredentials(insecure.NewCredentials()), - ) - if err != nil { - return nil, fmt.Errorf("initializing otel connection from docker context metadata: %w", err) - } - - client := otlptracegrpc.NewClient(otlptracegrpc.WithGRPCConn(conn)) - return client, nil -} - -// ConfigFromDockerContext inspects extra metadata included as part of the -// specified Docker context to try and extract a valid OTLP client configuration. -func ConfigFromDockerContext(st store.Store, name string) (OTLPConfig, error) { - meta, err := st.GetMetadata(name) - if err != nil { - return OTLPConfig{}, err - } - - var otelCfg any - switch m := meta.Metadata.(type) { - case command.DockerContext: - otelCfg = m.AdditionalFields[otelConfigFieldName] - case map[string]any: - otelCfg = m[otelConfigFieldName] - } - if otelCfg == nil { - return OTLPConfig{}, nil - } - - otelMap, ok := otelCfg.(map[string]any) - if !ok { - return OTLPConfig{}, fmt.Errorf( - "unexpected type for field %q: %T (expected: %T)", - otelConfigFieldName, - otelCfg, - otelMap, - ) - } - - // keys from https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/ - cfg := OTLPConfig{ - Endpoint: valueOrDefault[string](otelMap, "OTEL_EXPORTER_OTLP_ENDPOINT"), - } - return cfg, nil -} - -// valueOrDefault returns the type-cast value at the specified key in the map -// if present and the correct type; otherwise, it returns the default value for -// T. -func valueOrDefault[T any](m map[string]any, key string) T { - if v, ok := m[key].(T); ok { - return v - } - return *new(T) -} diff --git a/internal/tracing/errors.go b/internal/tracing/errors.go deleted file mode 100644 index 9fa615054c0..00000000000 --- a/internal/tracing/errors.go +++ /dev/null @@ -1,29 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "go.opentelemetry.io/otel" -) - -// skipErrors is a no-op otel.ErrorHandler. -type skipErrors struct{} - -// Handle does nothing, ignoring any errors passed to it. -func (skipErrors) Handle(_ error) {} - -var _ otel.ErrorHandler = skipErrors{} diff --git a/internal/tracing/keyboard_metrics.go b/internal/tracing/keyboard_metrics.go deleted file mode 100644 index 2e5120fbea7..00000000000 --- a/internal/tracing/keyboard_metrics.go +++ /dev/null @@ -1,35 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "context" - - "go.opentelemetry.io/otel/attribute" -) - -func KeyboardMetrics(ctx context.Context, enabled, isDockerDesktopActive bool) { - commandAvailable := []string{} - if isDockerDesktopActive { - commandAvailable = append(commandAvailable, "gui") - commandAvailable = append(commandAvailable, "gui/composeview") - } - - AddAttributeToSpan(ctx, - attribute.Bool("navmenu.enabled", enabled), - attribute.StringSlice("navmenu.command_available", commandAvailable)) -} diff --git a/internal/tracing/mux.go b/internal/tracing/mux.go deleted file mode 100644 index 1a09ef1d96a..00000000000 --- a/internal/tracing/mux.go +++ /dev/null @@ -1,73 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "context" - "errors" - "sync" - - sdktrace "go.opentelemetry.io/otel/sdk/trace" -) - -type MuxExporter struct { - exporters []sdktrace.SpanExporter -} - -func (m MuxExporter) ExportSpans(ctx context.Context, spans []sdktrace.ReadOnlySpan) error { - var ( - wg sync.WaitGroup - errMu sync.Mutex - errs = make([]error, 0, len(m.exporters)) - ) - - for _, exporter := range m.exporters { - wg.Add(1) - go func() { - defer wg.Done() - if err := exporter.ExportSpans(ctx, spans); err != nil { - errMu.Lock() - errs = append(errs, err) - errMu.Unlock() - } - }() - } - wg.Wait() - return errors.Join(errs...) -} - -func (m MuxExporter) Shutdown(ctx context.Context) error { - var ( - wg sync.WaitGroup - errMu sync.Mutex - errs = make([]error, 0, len(m.exporters)) - ) - - for _, exporter := range m.exporters { - wg.Add(1) - go func() { - defer wg.Done() - if err := exporter.Shutdown(ctx); err != nil { - errMu.Lock() - errs = append(errs, err) - errMu.Unlock() - } - }() - } - wg.Wait() - return errors.Join(errs...) -} diff --git a/internal/tracing/tracing.go b/internal/tracing/tracing.go deleted file mode 100644 index ec09f61e8bd..00000000000 --- a/internal/tracing/tracing.go +++ /dev/null @@ -1,145 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "context" - "errors" - "fmt" - "os" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/moby/buildkit/util/tracing/detect" - _ "github.com/moby/buildkit/util/tracing/env" //nolint:blank-imports - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/exporters/otlp/otlptrace" - "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc" - "go.opentelemetry.io/otel/propagation" - "go.opentelemetry.io/otel/sdk/resource" - sdktrace "go.opentelemetry.io/otel/sdk/trace" - semconv "go.opentelemetry.io/otel/semconv/v1.21.0" - - "github.com/docker/compose/v5/internal" -) - -func init() { - detect.ServiceName = "compose" - // do not log tracing errors to stdio - otel.SetErrorHandler(skipErrors{}) -} - -// OTLPConfig contains the necessary values to initialize an OTLP client -// manually. -// -// This supports a minimal set of options based on what is necessary for -// automatic OTEL configuration from Docker context metadata. -type OTLPConfig struct { - Endpoint string -} - -// ShutdownFunc flushes and stops an OTEL exporter. -type ShutdownFunc func(ctx context.Context) error - -// envMap is a convenience type for OS environment variables. -type envMap map[string]string - -func InitTracing(dockerCli command.Cli) (ShutdownFunc, error) { - // set global propagator to tracecontext (the default is no-op). - otel.SetTextMapPropagator(propagation.TraceContext{}) - return InitProvider(dockerCli) -} - -func InitProvider(dockerCli command.Cli) (ShutdownFunc, error) { - ctx := context.Background() - - var errs []error - var exporters []sdktrace.SpanExporter - - envClient, otelEnv := traceClientFromEnv() - if envClient != nil { - if envExporter, err := otlptrace.New(ctx, envClient); err != nil { - errs = append(errs, err) - } else if envExporter != nil { - exporters = append(exporters, envExporter) - } - } - - if dcClient, err := traceClientFromDockerContext(dockerCli, otelEnv); err != nil { - errs = append(errs, err) - } else if dcClient != nil { - if dcExporter, err := otlptrace.New(ctx, dcClient); err != nil { - errs = append(errs, err) - } else if dcExporter != nil { - exporters = append(exporters, dcExporter) - } - } - if len(errs) != 0 { - return nil, errors.Join(errs...) - } - - res, err := resource.New( - ctx, - resource.WithAttributes( - semconv.ServiceName("compose"), - semconv.ServiceVersion(internal.Version), - attribute.String("docker.context", dockerCli.CurrentContext()), - ), - ) - if err != nil { - return nil, fmt.Errorf("failed to create resource: %w", err) - } - - muxExporter := MuxExporter{exporters: exporters} - tracerProvider := sdktrace.NewTracerProvider( - sdktrace.WithResource(res), - sdktrace.WithBatcher(muxExporter), - ) - otel.SetTracerProvider(tracerProvider) - - // Shutdown will flush any remaining spans and shut down the exporter. - return tracerProvider.Shutdown, nil -} - -// traceClientFromEnv creates a GRPC OTLP client based on OS environment -// variables. -// -// https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/ -func traceClientFromEnv() (otlptrace.Client, envMap) { - hasOtelEndpointInEnv := false - otelEnv := make(map[string]string) - for _, kv := range os.Environ() { - k, v, ok := strings.Cut(kv, "=") - if !ok { - continue - } - if strings.HasPrefix(k, "OTEL_") { - otelEnv[k] = v - if strings.HasSuffix(k, "ENDPOINT") { - hasOtelEndpointInEnv = true - } - } - } - - if !hasOtelEndpointInEnv { - return nil, nil - } - - client := otlptracegrpc.NewClient() - return client, otelEnv -} diff --git a/internal/tracing/tracing_test.go b/internal/tracing/tracing_test.go deleted file mode 100644 index 3d262631921..00000000000 --- a/internal/tracing/tracing_test.go +++ /dev/null @@ -1,60 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing_test - -import ( - "testing" - - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/context/store" - "github.com/stretchr/testify/require" - - "github.com/docker/compose/v5/internal/tracing" -) - -var testStoreCfg = store.NewConfig( - func() any { - return &map[string]any{} - }, -) - -func TestExtractOtelFromContext(t *testing.T) { - if testing.Short() { - t.Skip("Requires filesystem access") - } - - dir := t.TempDir() - - st := store.New(dir, testStoreCfg) - err := st.CreateOrUpdate(store.Metadata{ - Name: "test", - Metadata: command.DockerContext{ - Description: t.Name(), - AdditionalFields: map[string]any{ - "otel": map[string]any{ - "OTEL_EXPORTER_OTLP_ENDPOINT": "localhost:1234", - }, - }, - }, - Endpoints: make(map[string]any), - }) - require.NoError(t, err) - - cfg, err := tracing.ConfigFromDockerContext(st, "test") - require.NoError(t, err) - require.Equal(t, "localhost:1234", cfg.Endpoint) -} diff --git a/internal/tracing/wrap.go b/internal/tracing/wrap.go deleted file mode 100644 index e525b738b42..00000000000 --- a/internal/tracing/wrap.go +++ /dev/null @@ -1,98 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package tracing - -import ( - "context" - - "github.com/acarl005/stripansi" - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/codes" - semconv "go.opentelemetry.io/otel/semconv/v1.19.0" - "go.opentelemetry.io/otel/trace" -) - -// SpanWrapFunc wraps a function that takes a context with a trace.Span, marking the status as codes.Error if the -// wrapped function returns an error. -// -// The context passed to the function is created from the span to ensure correct propagation. -// -// NOTE: This function is nearly identical to SpanWrapFuncForErrGroup, except the latter is designed specially for -// convenience with errgroup.Group due to its prevalence throughout the codebase. The code is duplicated to avoid -// adding even more levels of function wrapping/indirection. -func SpanWrapFunc(spanName string, opts SpanOptions, fn func(ctx context.Context) error) func(context.Context) error { - return func(ctx context.Context) error { - ctx, span := otel.Tracer("").Start(ctx, spanName, opts.SpanStartOptions()...) - defer span.End() - - if err := fn(ctx); err != nil { - span.SetStatus(codes.Error, err.Error()) - return err - } - - span.SetStatus(codes.Ok, "") - return nil - } -} - -// SpanWrapFuncForErrGroup wraps a function that takes a context with a trace.Span, marking the status as codes.Error -// if the wrapped function returns an error. -// -// The context passed to the function is created from the span to ensure correct propagation. -// -// NOTE: This function is nearly identical to SpanWrapFunc, except this function is designed specially for -// convenience with errgroup.Group due to its prevalence throughout the codebase. The code is duplicated to avoid -// adding even more levels of function wrapping/indirection. -func SpanWrapFuncForErrGroup(ctx context.Context, spanName string, opts SpanOptions, fn func(ctx context.Context) error) func() error { - return func() error { - ctx, span := otel.Tracer("").Start(ctx, spanName, opts.SpanStartOptions()...) - defer span.End() - - if err := fn(ctx); err != nil { - span.SetStatus(codes.Error, err.Error()) - return err - } - - span.SetStatus(codes.Ok, "") - return nil - } -} - -// EventWrapFuncForErrGroup invokes a function and records an event, optionally including the returned -// error as the "exception message" on the event. -// -// This is intended for lightweight usage to wrap errgroup.Group calls where a full span is not desired. -func EventWrapFuncForErrGroup(ctx context.Context, eventName string, opts SpanOptions, fn func(ctx context.Context) error) func() error { - return func() error { - span := trace.SpanFromContext(ctx) - eventOpts := opts.EventOptions() - - err := fn(ctx) - if err != nil { - eventOpts = append(eventOpts, trace.WithAttributes(semconv.ExceptionMessage(stripansi.Strip(err.Error())))) - } - span.AddEvent(eventName, eventOpts...) - - return err - } -} - -func AddAttributeToSpan(ctx context.Context, attr ...attribute.KeyValue) { - span := trace.SpanFromContext(ctx) - span.SetAttributes(attr...) -} diff --git a/internal/variables.go b/internal/variables.go deleted file mode 100644 index 28b701cc2db..00000000000 --- a/internal/variables.go +++ /dev/null @@ -1,20 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package internal - -// Version is the version of the CLI injected in compilation time -var Version = "dev" diff --git a/pkg/api/api.go b/pkg/api/api.go deleted file mode 100644 index aed77af1523..00000000000 --- a/pkg/api/api.go +++ /dev/null @@ -1,759 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "context" - "fmt" - "io" - "slices" - "strings" - "time" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/platforms" - "github.com/docker/cli/opts" - "github.com/docker/docker/api/types/volume" -) - -// LoadListener receives events during project loading. -// Events include: -// - "extends": when a service extends another (metadata: service info) -// - "include": when including external compose files (metadata: {"path": StringList}) -// -// Multiple listeners can be registered, and all will be notified of events. -type LoadListener func(event string, metadata map[string]any) - -// ProjectLoadOptions configures how a Compose project should be loaded -type ProjectLoadOptions struct { - // ProjectName to use, or empty to infer from directory - ProjectName string - // ConfigPaths are paths to compose files - ConfigPaths []string - // WorkingDir is the project directory - WorkingDir string - // EnvFiles are paths to .env files - EnvFiles []string - // Profiles to activate - Profiles []string - // Services to select (empty = all) - Services []string - // Offline mode disables remote resource loading - Offline bool - // All includes all resources (not just those used by services) - All bool - // Compatibility enables v1 compatibility mode - Compatibility bool - - // ProjectOptionsFns are compose-go project options to apply. - // Use cli.WithInterpolation(false), cli.WithNormalization(false), etc. - // This is optional - pass nil or empty slice to use defaults. - ProjectOptionsFns []cli.ProjectOptionsFn - - // LoadListeners receive events during project loading. - // All registered listeners will be notified of events. - // This is optional - pass nil or empty slice if not needed. - LoadListeners []LoadListener - - OCI OCIOptions -} - -type OCIOptions struct { - InsecureRegistries []string -} - -// Compose is the API interface one can use to programmatically use docker/compose in a third-party software -// Use [compose.NewComposeService] to get an actual instance -type Compose interface { - // Build executes the equivalent to a `compose build` - Build(ctx context.Context, project *types.Project, options BuildOptions) error - // Push executes the equivalent to a `compose push` - Push(ctx context.Context, project *types.Project, options PushOptions) error - // Pull executes the equivalent of a `compose pull` - Pull(ctx context.Context, project *types.Project, options PullOptions) error - // Create executes the equivalent to a `compose create` - Create(ctx context.Context, project *types.Project, options CreateOptions) error - // Start executes the equivalent to a `compose start` - Start(ctx context.Context, projectName string, options StartOptions) error - // Restart restarts containers - Restart(ctx context.Context, projectName string, options RestartOptions) error - // Stop executes the equivalent to a `compose stop` - Stop(ctx context.Context, projectName string, options StopOptions) error - // Up executes the equivalent to a `compose up` - Up(ctx context.Context, project *types.Project, options UpOptions) error - // Down executes the equivalent to a `compose down` - Down(ctx context.Context, projectName string, options DownOptions) error - // Logs executes the equivalent to a `compose logs` - Logs(ctx context.Context, projectName string, consumer LogConsumer, options LogOptions) error - // Ps executes the equivalent to a `compose ps` - Ps(ctx context.Context, projectName string, options PsOptions) ([]ContainerSummary, error) - // List executes the equivalent to a `docker stack ls` - List(ctx context.Context, options ListOptions) ([]Stack, error) - // Kill executes the equivalent to a `compose kill` - Kill(ctx context.Context, projectName string, options KillOptions) error - // RunOneOffContainer creates a service oneoff container and starts its dependencies - RunOneOffContainer(ctx context.Context, project *types.Project, opts RunOptions) (int, error) - // Remove executes the equivalent to a `compose rm` - Remove(ctx context.Context, projectName string, options RemoveOptions) error - // Exec executes a command in a running service container - Exec(ctx context.Context, projectName string, options RunOptions) (int, error) - // Attach STDIN,STDOUT,STDERR to a running service container - Attach(ctx context.Context, projectName string, options AttachOptions) error - // Copy copies a file/folder between a service container and the local filesystem - Copy(ctx context.Context, projectName string, options CopyOptions) error - // Pause executes the equivalent to a `compose pause` - Pause(ctx context.Context, projectName string, options PauseOptions) error - // UnPause executes the equivalent to a `compose unpause` - UnPause(ctx context.Context, projectName string, options PauseOptions) error - // Top executes the equivalent to a `compose top` - Top(ctx context.Context, projectName string, services []string) ([]ContainerProcSummary, error) - // Events executes the equivalent to a `compose events` - Events(ctx context.Context, projectName string, options EventsOptions) error - // Port executes the equivalent to a `compose port` - Port(ctx context.Context, projectName string, service string, port uint16, options PortOptions) (string, int, error) - // Publish executes the equivalent to a `compose publish` - Publish(ctx context.Context, project *types.Project, repository string, options PublishOptions) error - // Images executes the equivalent of a `compose images` - Images(ctx context.Context, projectName string, options ImagesOptions) (map[string]ImageSummary, error) - // Watch services' development context and sync/notify/rebuild/restart on changes - Watch(ctx context.Context, project *types.Project, options WatchOptions) error - // Viz generates a graphviz graph of the project services - Viz(ctx context.Context, project *types.Project, options VizOptions) (string, error) - // Wait blocks until at least one of the services' container exits - Wait(ctx context.Context, projectName string, options WaitOptions) (int64, error) - // Scale manages numbers of container instances running per service - Scale(ctx context.Context, project *types.Project, options ScaleOptions) error - // Export a service container's filesystem as a tar archive - Export(ctx context.Context, projectName string, options ExportOptions) error - // Create a new image from a service container's changes - Commit(ctx context.Context, projectName string, options CommitOptions) error - // Generate generates a Compose Project from existing containers - Generate(ctx context.Context, options GenerateOptions) (*types.Project, error) - // Volumes executes the equivalent to a `docker volume ls` - Volumes(ctx context.Context, project string, options VolumesOptions) ([]VolumesSummary, error) - // LoadProject loads and validates a Compose project from configuration files. - LoadProject(ctx context.Context, options ProjectLoadOptions) (*types.Project, error) -} - -type VolumesOptions struct { - Services []string -} - -type VolumesSummary = *volume.Volume - -type ScaleOptions struct { - Services []string -} - -type WaitOptions struct { - // Services passed in the command line to be waited - Services []string - // Executes a down when a container exits - DownProjectOnContainerExit bool -} - -type VizOptions struct { - // IncludeNetworks if true, network names a container is attached to should appear in the graph node - IncludeNetworks bool - // IncludePorts if true, ports a container exposes should appear in the graph node - IncludePorts bool - // IncludeImageName if true, name of the image used to create a container should appear in the graph node - IncludeImageName bool - // Indentation string to be used to indent graphviz code, e.g. "\t", " " - Indentation string -} - -// WatchLogger is a reserved name to log watch events -const WatchLogger = "#watch" - -// WatchOptions group options of the Watch API -type WatchOptions struct { - Build *BuildOptions - LogTo LogConsumer - Prune bool - Services []string -} - -// BuildOptions group options of the Build API -type BuildOptions struct { - // Pull always attempt to pull a newer version of the image - Pull bool - // Push pushes service images - Push bool - // Progress set type of progress output ("auto", "plain", "tty") - Progress string - // Args set build-time args - Args types.MappingWithEquals - // NoCache disables cache use - NoCache bool - // Quiet make the build process not output to the console - Quiet bool - // Services passed in the command line to be built - Services []string - // Deps also build selected services dependencies - Deps bool - // Ssh authentications passed in the command line - SSHs []types.SSHKey - // Memory limit for the build container - Memory int64 - // Builder name passed in the command line - Builder string - // Print don't actually run builder but print equivalent build config - Print bool - // Check let builder validate build configuration - Check bool - // Attestations allows to enable attestations generation - Attestations bool - // Provenance generate a provenance attestation - Provenance string - // SBOM generate a SBOM attestation - SBOM string - // Out is the stream to write build progress - Out io.Writer -} - -// Apply mutates project according to build options -func (o BuildOptions) Apply(project *types.Project) error { - platform := project.Environment["DOCKER_DEFAULT_PLATFORM"] - for name, service := range project.Services { - if service.Provider == nil && service.Image == "" && service.Build == nil { - return fmt.Errorf("invalid service %q. Must specify either image or build", name) - } - - if service.Build == nil { - continue - } - if platform != "" { - if len(service.Build.Platforms) > 0 && !slices.Contains(service.Build.Platforms, platform) { - return fmt.Errorf("service %q build.platforms does not support value set by DOCKER_DEFAULT_PLATFORM: %s", name, platform) - } - service.Platform = platform - } - if service.Platform != "" { - if len(service.Build.Platforms) > 0 && !slices.Contains(service.Build.Platforms, service.Platform) { - return fmt.Errorf("service %q build configuration does not support platform: %s", name, service.Platform) - } - } - - service.Build.Pull = service.Build.Pull || o.Pull - service.Build.NoCache = service.Build.NoCache || o.NoCache - - project.Services[name] = service - } - return nil -} - -// CreateOptions group options of the Create API -type CreateOptions struct { - Build *BuildOptions - // Services defines the services user interacts with - Services []string - // Remove legacy containers for services that are not defined in the project - RemoveOrphans bool - // Ignore legacy containers for services that are not defined in the project - IgnoreOrphans bool - // Recreate define the strategy to apply on existing containers - Recreate string - // RecreateDependencies define the strategy to apply on dependencies services - RecreateDependencies string - // Inherit reuse anonymous volumes from previous container - Inherit bool - // Timeout set delay to wait for container to gracefully stop before sending SIGKILL - Timeout *time.Duration - // QuietPull makes the pulling process quiet - QuietPull bool -} - -// StartOptions group options of the Start API -type StartOptions struct { - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project - // Attach to container and forward logs if not nil - Attach LogConsumer - // AttachTo set the services to attach to - AttachTo []string - // OnExit defines behavior when a container stops - OnExit Cascade - // ExitCodeFrom return exit code from specified service - ExitCodeFrom string - // Wait won't return until containers reached the running|healthy state - Wait bool - WaitTimeout time.Duration - // Services passed in the command line to be started - Services []string - Watch bool - NavigationMenu bool -} - -type Cascade int - -const ( - CascadeIgnore Cascade = iota - CascadeStop Cascade = iota - CascadeFail Cascade = iota -) - -// RestartOptions group options of the Restart API -type RestartOptions struct { - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project - // Timeout override container restart timeout - Timeout *time.Duration - // Services passed in the command line to be restarted - Services []string - // NoDeps ignores services dependencies - NoDeps bool -} - -// StopOptions group options of the Stop API -type StopOptions struct { - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project - // Timeout override container stop timeout - Timeout *time.Duration - // Services passed in the command line to be stopped - Services []string -} - -// UpOptions group options of the Up API -type UpOptions struct { - Create CreateOptions - Start StartOptions -} - -// DownOptions group options of the Down API -type DownOptions struct { - // RemoveOrphans will cleanup containers that are not declared on the compose model but own the same labels - RemoveOrphans bool - // Project is the compose project used to define this app. Might be nil if user ran `down` just with project name - Project *types.Project - // Timeout override container stop timeout - Timeout *time.Duration - // Images remove image used by services. 'all': Remove all images. 'local': Remove only images that don't have a tag - Images string - // Volumes remove volumes, both declared in the `volumes` section and anonymous ones - Volumes bool - // Services passed in the command line to be stopped - Services []string -} - -// ConfigOptions group options of the Config API -type ConfigOptions struct { - // Format define the output format used to dump converted application model (json|yaml) - Format string - // Output defines the path to save the application model - Output string - // Resolve image reference to digests - ResolveImageDigests bool -} - -// PushOptions group options of the Push API -type PushOptions struct { - Quiet bool - IgnoreFailures bool - ImageMandatory bool -} - -// PullOptions group options of the Pull API -type PullOptions struct { - Quiet bool - IgnoreFailures bool - IgnoreBuildable bool -} - -// ImagesOptions group options of the Images API -type ImagesOptions struct { - Services []string -} - -// KillOptions group options of the Kill API -type KillOptions struct { - // RemoveOrphans will cleanup containers that are not declared on the compose model but own the same labels - RemoveOrphans bool - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project - // Services passed in the command line to be killed - Services []string - // Signal to send to containers - Signal string - // All can be set to true to try to kill all found containers, independently of their state - All bool -} - -// RemoveOptions group options of the Remove API -type RemoveOptions struct { - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project - // Stop option passed in the command line - Stop bool - // Volumes remove anonymous volumes - Volumes bool - // Force don't ask to confirm removal - Force bool - // Services passed in the command line to be removed - Services []string -} - -// RunOptions group options of the Run API -type RunOptions struct { - CreateOptions - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project - Name string - Service string - Command []string - Entrypoint []string - Detach bool - AutoRemove bool - Tty bool - Interactive bool - WorkingDir string - User string - Environment []string - CapAdd []string - CapDrop []string - Labels types.Labels - Privileged bool - UseNetworkAliases bool - NoDeps bool - // used by exec - Index int -} - -// AttachOptions group options of the Attach API -type AttachOptions struct { - Project *types.Project - Service string - Index int - DetachKeys string - NoStdin bool - Proxy bool -} - -// EventsOptions group options of the Events API -type EventsOptions struct { - Services []string - Consumer func(event Event) error - Since string - Until string -} - -// Event is a container runtime event served by Events API -type Event struct { - Timestamp time.Time - Service string - Container string - Status string - Attributes map[string]string -} - -// PortOptions group options of the Port API -type PortOptions struct { - Protocol string - Index int -} - -// OCIVersion controls manifest generation to ensure compatibility -// with different registries. -// -// Currently, this is not exposed as an option to the user – Compose uses -// OCI 1.0 mode automatically for ECR registries based on domain and OCI 1.1 -// for all other registries. -// -// There are likely other popular registries that do not support the OCI 1.1 -// format, so it might make sense to expose this as a CLI flag or see if -// there's a way to generically probe the registry for support level. -type OCIVersion string - -const ( - OCIVersion1_0 OCIVersion = "1.0" - OCIVersion1_1 OCIVersion = "1.1" -) - -// PublishOptions group options of the Publish API -type PublishOptions struct { - ResolveImageDigests bool - Application bool - WithEnvironment bool - OCIVersion OCIVersion - // Use plain HTTP to access registry. Should only be used for testing purpose - InsecureRegistry bool -} - -func (e Event) String() string { - t := e.Timestamp.Format("2006-01-02 15:04:05.000000") - var attr []string - for k, v := range e.Attributes { - attr = append(attr, fmt.Sprintf("%s=%s", k, v)) - } - return fmt.Sprintf("%s container %s %s (%s)\n", t, e.Status, e.Container, strings.Join(attr, ", ")) -} - -// ListOptions group options of the ls API -type ListOptions struct { - All bool -} - -// PsOptions group options of the Ps API -type PsOptions struct { - Project *types.Project - All bool - Services []string -} - -// CopyOptions group options of the cp API -type CopyOptions struct { - Source string - Destination string - All bool - Index int - FollowLink bool - CopyUIDGID bool -} - -// PortPublisher hold status about published port -type PortPublisher struct { - URL string - TargetPort int - PublishedPort int - Protocol string -} - -// ContainerSummary hold high-level description of a container -type ContainerSummary struct { - ID string - Name string - Names []string - Image string - Command string - Project string - Service string - Created int64 - State string - Status string - Health string - ExitCode int - Publishers PortPublishers - Labels map[string]string - SizeRw int64 `json:",omitempty"` - SizeRootFs int64 `json:",omitempty"` - Mounts []string - Networks []string - LocalVolumes int -} - -// PortPublishers is a slice of PortPublisher -type PortPublishers []PortPublisher - -// Len implements sort.Interface -func (p PortPublishers) Len() int { - return len(p) -} - -// Less implements sort.Interface -func (p PortPublishers) Less(i, j int) bool { - left := p[i] - right := p[j] - if left.URL != right.URL { - return left.URL < right.URL - } - if left.TargetPort != right.TargetPort { - return left.TargetPort < right.TargetPort - } - if left.PublishedPort != right.PublishedPort { - return left.PublishedPort < right.PublishedPort - } - return left.Protocol < right.Protocol -} - -// Swap implements sort.Interface -func (p PortPublishers) Swap(i, j int) { - p[i], p[j] = p[j], p[i] -} - -// ContainerProcSummary holds container processes top data -type ContainerProcSummary struct { - ID string - Name string - Processes [][]string - Titles []string - Service string - Replica string -} - -// ImageSummary holds container image description -type ImageSummary struct { - ID string - Repository string - Tag string - Platform platforms.Platform - Size int64 - Created *time.Time - LastTagTime time.Time -} - -// ServiceStatus hold status about a service -type ServiceStatus struct { - ID string - Name string - Replicas int - Desired int - Ports []string - Publishers []PortPublisher -} - -// LogOptions defines optional parameters for the `Log` API -type LogOptions struct { - Project *types.Project - Index int - Services []string - Tail string - Since string - Until string - Follow bool - Timestamps bool -} - -// PauseOptions group options of the Pause API -type PauseOptions struct { - // Services passed in the command line to be started - Services []string - // Project is the compose project used to define this app. Might be nil if user ran command just with project name - Project *types.Project -} - -// ExportOptions group options of the Export API -type ExportOptions struct { - Service string - Index int - Output string -} - -// CommitOptions group options of the Commit API -type CommitOptions struct { - Service string - Reference string - - Pause bool - Comment string - Author string - Changes opts.ListOpts - - Index int -} - -type GenerateOptions struct { - // ProjectName to set in the Compose file - ProjectName string - // Containers passed in the command line to be used as reference for service definition - Containers []string -} - -const ( - // STARTING indicates that stack is being deployed - STARTING string = "Starting" - // RUNNING indicates that stack is deployed and services are running - RUNNING string = "Running" - // UPDATING indicates that some stack resources are being recreated - UPDATING string = "Updating" - // REMOVING indicates that stack is being deleted - REMOVING string = "Removing" - // UNKNOWN indicates unknown stack state - UNKNOWN string = "Unknown" - // FAILED indicates that stack deployment failed - FAILED string = "Failed" -) - -const ( - // RecreateDiverged to recreate services which configuration diverges from compose model - RecreateDiverged = "diverged" - // RecreateForce to force service container being recreated - RecreateForce = "force" - // RecreateNever to never recreate existing service containers - RecreateNever = "never" -) - -// Stack holds the name and state of a compose application/stack -type Stack struct { - ID string - Name string - Status string - ConfigFiles string - Reason string -} - -// LogConsumer is a callback to process log messages from services -type LogConsumer interface { - Log(containerName, message string) - Err(containerName, message string) - Status(container, msg string) -} - -// ContainerEventListener is a callback to process ContainerEvent from services -type ContainerEventListener func(event ContainerEvent) - -// ContainerEvent notify an event has been collected on source container implementing Service -type ContainerEvent struct { - Type int - Time int64 - Container *ContainerSummary - // Source is the name of the container _without the project prefix_. - // - // This is only suitable for display purposes within Compose, as it's - // not guaranteed to be unique across services. - Source string - ID string - Service string - Line string - // ExitCode is only set on ContainerEventExited events - ExitCode int - Restarting bool -} - -const ( - // ContainerEventLog is a ContainerEvent of type log on stdout. Line is set - ContainerEventLog = iota - // ContainerEventErr is a ContainerEvent of type log on stderr. Line is set - ContainerEventErr - // ContainerEventStarted let consumer know a container has been started - ContainerEventStarted - // ContainerEventRestarted let consumer know a container has been restarted - ContainerEventRestarted - // ContainerEventStopped is a ContainerEvent of type stopped. - ContainerEventStopped - // ContainerEventCreated let consumer know a new container has been created - ContainerEventCreated - // ContainerEventRecreated let consumer know container stopped but his being replaced - ContainerEventRecreated - // ContainerEventExited is a ContainerEvent of type exit. ExitCode is set - ContainerEventExited - // UserCancel user canceled compose up, we are stopping containers - HookEventLog -) - -// Separator is used for naming components -var Separator = "-" - -// GetImageNameOrDefault computes the default image name for a service, used to tag built images -func GetImageNameOrDefault(service types.ServiceConfig, projectName string) string { - imageName := service.Image - if imageName == "" { - imageName = projectName + Separator + service.Name - } - return imageName -} diff --git a/pkg/api/api_test.go b/pkg/api/api_test.go deleted file mode 100644 index fc44abe7f1a..00000000000 --- a/pkg/api/api_test.go +++ /dev/null @@ -1,38 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "gotest.tools/v3/assert" -) - -func TestRunOptionsEnvironmentMap(t *testing.T) { - opts := RunOptions{ - Environment: []string{ - "FOO=BAR", - "ZOT=", - "QIX", - }, - } - env := types.NewMappingWithEquals(opts.Environment) - assert.Equal(t, *env["FOO"], "BAR") - assert.Equal(t, *env["ZOT"], "") - assert.Check(t, env["QIX"] == nil) -} diff --git a/pkg/api/context.go b/pkg/api/context.go deleted file mode 100644 index af49c5d2433..00000000000 --- a/pkg/api/context.go +++ /dev/null @@ -1,32 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -// ContextInfo provides Docker context information for advanced scenarios -type ContextInfo interface { - // CurrentContext returns the name of the current Docker context - // Returns "default" for simple clients without context support - CurrentContext() string - - // ServerOSType returns the Docker daemon's operating system (linux/windows/darwin) - // Used for OS-specific compatibility checks - ServerOSType() string - - // BuildKitEnabled determines whether BuildKit should be used for builds - // Checks DOCKER_BUILDKIT env var, config, and daemon capabilities - BuildKitEnabled() (bool, error) -} diff --git a/pkg/api/env.go b/pkg/api/env.go deleted file mode 100644 index 46319342417..00000000000 --- a/pkg/api/env.go +++ /dev/null @@ -1,20 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -// ComposeCompatibility try to mimic compose v1 as much as possible -const ComposeCompatibility = "COMPOSE_COMPATIBILITY" diff --git a/pkg/api/errors.go b/pkg/api/errors.go deleted file mode 100644 index 7bf4d4a02c3..00000000000 --- a/pkg/api/errors.go +++ /dev/null @@ -1,88 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "errors" -) - -const ( - // ExitCodeLoginRequired exit code when command cannot execute because it requires cloud login - // This will be used by VSCode to detect when creating context if the user needs to login first - ExitCodeLoginRequired = 5 -) - -var ( - // ErrNotFound is returned when an object is not found - ErrNotFound = errors.New("not found") - // ErrAlreadyExists is returned when an object already exists - ErrAlreadyExists = errors.New("already exists") - // ErrForbidden is returned when an operation is not permitted - ErrForbidden = errors.New("forbidden") - // ErrUnknown is returned when the error type is unmapped - ErrUnknown = errors.New("unknown") - // ErrNotImplemented is returned when a backend doesn't implement an action - ErrNotImplemented = errors.New("not implemented") - // ErrUnsupportedFlag is returned when a backend doesn't support a flag - ErrUnsupportedFlag = errors.New("unsupported flag") - // ErrCanceled is returned when the command was canceled by user - ErrCanceled = errors.New("canceled") - // ErrParsingFailed is returned when a string cannot be parsed - ErrParsingFailed = errors.New("parsing failed") - // ErrNoResources is returned when operation didn't selected any resource - ErrNoResources = errors.New("no resources") -) - -// IsNotFoundError returns true if the unwrapped error is ErrNotFound -func IsNotFoundError(err error) bool { - return errors.Is(err, ErrNotFound) -} - -// IsAlreadyExistsError returns true if the unwrapped error is ErrAlreadyExists -func IsAlreadyExistsError(err error) bool { - return errors.Is(err, ErrAlreadyExists) -} - -// IsForbiddenError returns true if the unwrapped error is ErrForbidden -func IsForbiddenError(err error) bool { - return errors.Is(err, ErrForbidden) -} - -// IsUnknownError returns true if the unwrapped error is ErrUnknown -func IsUnknownError(err error) bool { - return errors.Is(err, ErrUnknown) -} - -// IsErrUnsupportedFlag returns true if the unwrapped error is ErrUnsupportedFlag -func IsErrUnsupportedFlag(err error) bool { - return errors.Is(err, ErrUnsupportedFlag) -} - -// IsErrNotImplemented returns true if the unwrapped error is ErrNotImplemented -func IsErrNotImplemented(err error) bool { - return errors.Is(err, ErrNotImplemented) -} - -// IsErrParsingFailed returns true if the unwrapped error is ErrParsingFailed -func IsErrParsingFailed(err error) bool { - return errors.Is(err, ErrParsingFailed) -} - -// IsErrCanceled returns true if the unwrapped error is ErrCanceled -func IsErrCanceled(err error) bool { - return errors.Is(err, ErrCanceled) -} diff --git a/pkg/api/errors_test.go b/pkg/api/errors_test.go deleted file mode 100644 index d221db3fdec..00000000000 --- a/pkg/api/errors_test.go +++ /dev/null @@ -1,53 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "errors" - "fmt" - "testing" - - "gotest.tools/v3/assert" -) - -func TestIsNotFound(t *testing.T) { - err := fmt.Errorf(`object "name": %w`, ErrNotFound) - assert.Assert(t, IsNotFoundError(err)) - - assert.Assert(t, !IsNotFoundError(errors.New("another error"))) -} - -func TestIsAlreadyExists(t *testing.T) { - err := fmt.Errorf(`object "name": %w`, ErrAlreadyExists) - assert.Assert(t, IsAlreadyExistsError(err)) - - assert.Assert(t, !IsAlreadyExistsError(errors.New("another error"))) -} - -func TestIsForbidden(t *testing.T) { - err := fmt.Errorf(`object "name": %w`, ErrForbidden) - assert.Assert(t, IsForbiddenError(err)) - - assert.Assert(t, !IsForbiddenError(errors.New("another error"))) -} - -func TestIsUnknown(t *testing.T) { - err := fmt.Errorf(`object "name": %w`, ErrUnknown) - assert.Assert(t, IsUnknownError(err)) - - assert.Assert(t, !IsUnknownError(errors.New("another error"))) -} diff --git a/pkg/api/event.go b/pkg/api/event.go deleted file mode 100644 index 7ccb1138210..00000000000 --- a/pkg/api/event.go +++ /dev/null @@ -1,107 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "context" -) - -// EventStatus indicates the status of an action -type EventStatus int - -const ( - // Working means that the current task is working - Working EventStatus = iota - // Done means that the current task is done - Done - // Warning means that the current task has warning - Warning - // Error means that the current task has errored - Error -) - -// ResourceCompose is a special resource ID used when event applies to all resources in the application -const ResourceCompose = "Compose" - -const ( - StatusError = "Error" - StatusCreating = "Creating" - StatusStarting = "Starting" - StatusStarted = "Started" - StatusWaiting = "Waiting" - StatusHealthy = "Healthy" - StatusExited = "Exited" - StatusRestarting = "Restarting" - StatusRestarted = "Restarted" - StatusRunning = "Running" - StatusCreated = "Created" - StatusStopping = "Stopping" - StatusStopped = "Stopped" - StatusKilling = "Killing" - StatusKilled = "Killed" - StatusRemoving = "Removing" - StatusRemoved = "Removed" - StatusBuilding = "Building" - StatusBuilt = "Built" - StatusPulling = "Pulling" - StatusPulled = "Pulled" - StatusCommitting = "Committing" - StatusCommitted = "Committed" - StatusCopying = "Copying" - StatusCopied = "Copied" - StatusExporting = "Exporting" - StatusExported = "Exported" - StatusDownloading = "Downloading" - StatusDownloadComplete = "Download complete" - StatusConfiguring = "Configuring" - StatusConfigured = "Configured" -) - -// Resource represents status change and progress for a compose resource. -type Resource struct { - ID string - ParentID string - Text string - Details string - Status EventStatus - Current int64 - Percent int - Total int64 -} - -func (e *Resource) StatusText() string { - switch e.Status { - case Working: - return "Working" - case Warning: - return "Warning" - case Done: - return "Done" - default: - return "Error" - } -} - -// EventProcessor is notified about Compose operations and tasks -type EventProcessor interface { - // Start is triggered as a Compose operation is starting with context - Start(ctx context.Context, operation string) - // On notify about (sub)task and progress processing operation - On(events ...Resource) - // Done is triggered as a Compose operation completed - Done(operation string, success bool) -} diff --git a/pkg/api/labels.go b/pkg/api/labels.go deleted file mode 100644 index 3a0f684b98b..00000000000 --- a/pkg/api/labels.go +++ /dev/null @@ -1,68 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "github.com/hashicorp/go-version" - - "github.com/docker/compose/v5/internal" -) - -const ( - // ProjectLabel allow to track resource related to a compose project - ProjectLabel = "com.docker.compose.project" - // ServiceLabel allow to track resource related to a compose service - ServiceLabel = "com.docker.compose.service" - // ConfigHashLabel stores configuration hash for a compose service - ConfigHashLabel = "com.docker.compose.config-hash" - // ContainerNumberLabel stores the container index of a replicated service - ContainerNumberLabel = "com.docker.compose.container-number" - // VolumeLabel allow to track resource related to a compose volume - VolumeLabel = "com.docker.compose.volume" - // NetworkLabel allow to track resource related to a compose network - NetworkLabel = "com.docker.compose.network" - // WorkingDirLabel stores absolute path to compose project working directory - WorkingDirLabel = "com.docker.compose.project.working_dir" - // ConfigFilesLabel stores absolute path to compose project configuration files - ConfigFilesLabel = "com.docker.compose.project.config_files" - // EnvironmentFileLabel stores absolute path to compose project env file set by `--env-file` - EnvironmentFileLabel = "com.docker.compose.project.environment_file" - // OneoffLabel stores value 'True' for one-off containers created by `compose run` - OneoffLabel = "com.docker.compose.oneoff" - // SlugLabel stores unique slug used for one-off container identity - SlugLabel = "com.docker.compose.slug" - // ImageDigestLabel stores digest of the container image used to run service - ImageDigestLabel = "com.docker.compose.image" - // DependenciesLabel stores service dependencies - DependenciesLabel = "com.docker.compose.depends_on" - // VersionLabel stores the compose tool version used to build/run application - VersionLabel = "com.docker.compose.version" - // ImageBuilderLabel stores the builder (classic or BuildKit) used to produce the image. - ImageBuilderLabel = "com.docker.compose.image.builder" - // ContainerReplaceLabel is set when container is created to replace another container (recreated) - ContainerReplaceLabel = "com.docker.compose.replace" -) - -// ComposeVersion is the compose tool version as declared by label VersionLabel -var ComposeVersion string - -func init() { - v, err := version.NewVersion(internal.Version) - if err == nil { - ComposeVersion = v.Core().String() - } -} diff --git a/pkg/api/labels_test.go b/pkg/api/labels_test.go deleted file mode 100644 index c4af2003e9f..00000000000 --- a/pkg/api/labels_test.go +++ /dev/null @@ -1,36 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package api - -import ( - "testing" - - "github.com/hashicorp/go-version" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/internal" -) - -func TestComposeVersionInitialization(t *testing.T) { - v, err := version.NewVersion(internal.Version) - if err != nil { - assert.Equal(t, "", ComposeVersion, "ComposeVersion should be empty for a non-semver internal version (e.g. 'devel')") - } else { - expected := v.Core().String() - assert.Equal(t, expected, ComposeVersion, "ComposeVersion should be the core of internal.Version") - } -} diff --git a/pkg/bridge/convert.go b/pkg/bridge/convert.go deleted file mode 100644 index cceb99fdd4a..00000000000 --- a/pkg/bridge/convert.go +++ /dev/null @@ -1,236 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package bridge - -import ( - "context" - "fmt" - "io" - "os" - "os/user" - "path/filepath" - "runtime" - "strconv" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/docker/cli/cli/command" - cli "github.com/docker/cli/cli/command/container" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/pkg/jsonmessage" - "github.com/docker/go-connections/nat" - "github.com/sirupsen/logrus" - "go.yaml.in/yaml/v4" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -type ConvertOptions struct { - Output string - Templates string - Transformations []string -} - -func Convert(ctx context.Context, dockerCli command.Cli, project *types.Project, opts ConvertOptions) error { - if len(opts.Transformations) == 0 { - opts.Transformations = []string{DefaultTransformerImage} - } - // Load image references, secrets and configs, also expose ports - project, err := LoadAdditionalResources(ctx, dockerCli, project) - if err != nil { - return err - } - // for user to rely on compose.yaml attribute names, not go struct ones, we marshall back into YAML - raw, err := project.MarshalYAML(types.WithSecretContent) - // Marshall to YAML - if err != nil { - return fmt.Errorf("cannot render project into yaml: %w", err) - } - var model map[string]any - err = yaml.Unmarshal(raw, &model) - if err != nil { - return fmt.Errorf("cannot render project into yaml: %w", err) - } - - if opts.Output != "" { - _ = os.RemoveAll(opts.Output) - err := os.MkdirAll(opts.Output, 0o744) - if err != nil && !os.IsExist(err) { - return fmt.Errorf("cannot create output folder: %w", err) - } - } - // Run Transformers images - return convert(ctx, dockerCli, model, opts) -} - -func convert(ctx context.Context, dockerCli command.Cli, model map[string]any, opts ConvertOptions) error { - raw, err := yaml.Marshal(model) - if err != nil { - return err - } - - dir, err := os.MkdirTemp("", "compose-convert-*") - if err != nil { - return err - } - defer func() { - err := os.RemoveAll(dir) - if err != nil { - logrus.Warnf("failed to remove temp dir %s: %v", dir, err) - } - }() - - composeYaml := filepath.Join(dir, "compose.yaml") - err = os.WriteFile(composeYaml, raw, 0o600) - if err != nil { - return err - } - - out, err := filepath.Abs(opts.Output) - if err != nil { - return err - } - binds := []string{ - fmt.Sprintf("%s:%s", dir, "/in"), - fmt.Sprintf("%s:%s", out, "/out"), - } - if opts.Templates != "" { - templateDir, err := filepath.Abs(opts.Templates) - if err != nil { - return err - } - binds = append(binds, fmt.Sprintf("%s:%s", templateDir, "/templates")) - } - - for _, transformation := range opts.Transformations { - _, err = inspectWithPull(ctx, dockerCli, transformation) - if err != nil { - return err - } - - containerConfig := &container.Config{ - Image: transformation, - Env: []string{"LICENSE_AGREEMENT=true"}, - } - // On POSIX systems, this is a decimal number representing the uid. - // On Windows, this is a security identifier (SID) in a string format and the engine isn't able to manage it - if runtime.GOOS != "windows" { - usr, err := user.Current() - if err != nil { - return err - } - containerConfig.User = usr.Uid - } - created, err := dockerCli.Client().ContainerCreate(ctx, containerConfig, &container.HostConfig{ - AutoRemove: true, - Binds: binds, - }, &network.NetworkingConfig{}, nil, "") - if err != nil { - return err - } - - err = cli.RunStart(ctx, dockerCli, &cli.StartOptions{ - Attach: true, - Containers: []string{created.ID}, - }) - if err != nil { - return err - } - } - return nil -} - -// LoadAdditionalResources loads additional resources from the project, such as image references, secrets, configs and exposed ports -func LoadAdditionalResources(ctx context.Context, dockerCLI command.Cli, project *types.Project) (*types.Project, error) { - for name, service := range project.Services { - imageName := api.GetImageNameOrDefault(service, project.Name) - - inspect, err := inspectWithPull(ctx, dockerCLI, imageName) - if err != nil { - return nil, err - } - service.Image = imageName - exposed := utils.Set[string]{} - exposed.AddAll(service.Expose...) - for port := range inspect.Config.ExposedPorts { - exposed.Add(nat.Port(port).Port()) - } - for _, port := range service.Ports { - exposed.Add(strconv.Itoa(int(port.Target))) - } - service.Expose = exposed.Elements() - project.Services[name] = service - } - - for name, secret := range project.Secrets { - f, err := loadFileObject(types.FileObjectConfig(secret)) - if err != nil { - return nil, err - } - project.Secrets[name] = types.SecretConfig(f) - } - - for name, config := range project.Configs { - f, err := loadFileObject(types.FileObjectConfig(config)) - if err != nil { - return nil, err - } - project.Configs[name] = types.ConfigObjConfig(f) - } - - return project, nil -} - -func loadFileObject(conf types.FileObjectConfig) (types.FileObjectConfig, error) { - if !conf.External { - switch { - case conf.Environment != "": - conf.Content = os.Getenv(conf.Environment) - case conf.File != "": - bytes, err := os.ReadFile(conf.File) - if err != nil { - return conf, err - } - conf.Content = string(bytes) - } - } - return conf, nil -} - -func inspectWithPull(ctx context.Context, dockerCli command.Cli, imageName string) (image.InspectResponse, error) { - inspect, err := dockerCli.Client().ImageInspect(ctx, imageName) - if errdefs.IsNotFound(err) { - var stream io.ReadCloser - stream, err = dockerCli.Client().ImagePull(ctx, imageName, image.PullOptions{}) - if err != nil { - return image.InspectResponse{}, err - } - defer func() { _ = stream.Close() }() - - err = jsonmessage.DisplayJSONMessagesToStream(stream, dockerCli.Out(), nil) - if err != nil { - return image.InspectResponse{}, err - } - if inspect, err = dockerCli.Client().ImageInspect(ctx, imageName); err != nil { - return image.InspectResponse{}, err - } - } - return inspect, err -} diff --git a/pkg/bridge/transformers.go b/pkg/bridge/transformers.go deleted file mode 100644 index dbf4fc6d9dc..00000000000 --- a/pkg/bridge/transformers.go +++ /dev/null @@ -1,120 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package bridge - -import ( - "context" - "fmt" - "os" - "path/filepath" - - "github.com/docker/cli/cli/command" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/api/types/network" - "github.com/moby/go-archive" -) - -const ( - TransformerLabel = "com.docker.compose.bridge" - DefaultTransformerImage = "docker/compose-bridge-kubernetes" - - templatesPath = "/templates" -) - -type CreateTransformerOptions struct { - Dest string - From string -} - -func CreateTransformer(ctx context.Context, dockerCli command.Cli, options CreateTransformerOptions) error { - if options.From == "" { - options.From = DefaultTransformerImage - } - out, err := filepath.Abs(options.Dest) - if err != nil { - return err - } - - if _, err := os.Stat(out); err == nil { - return fmt.Errorf("output folder %s already exists", out) - } - - tmpl := filepath.Join(out, "templates") - err = os.MkdirAll(tmpl, 0o744) - if err != nil && !os.IsExist(err) { - return fmt.Errorf("cannot create output folder: %w", err) - } - - if err := command.ValidateOutputPath(out); err != nil { - return err - } - - created, err := dockerCli.Client().ContainerCreate(ctx, &container.Config{ - Image: options.From, - }, &container.HostConfig{}, &network.NetworkingConfig{}, nil, "") - defer func() { - _ = dockerCli.Client().ContainerRemove(context.Background(), created.ID, container.RemoveOptions{Force: true}) - }() - - if err != nil { - return err - } - content, stat, err := dockerCli.Client().CopyFromContainer(ctx, created.ID, templatesPath) - if err != nil { - return err - } - defer func() { - _ = content.Close() - }() - - srcInfo := archive.CopyInfo{ - Path: templatesPath, - Exists: true, - IsDir: stat.Mode.IsDir(), - } - - preArchive := content - if srcInfo.RebaseName != "" { - _, srcBase := archive.SplitPathDirEntry(srcInfo.Path) - preArchive = archive.RebaseArchiveEntries(content, srcBase, srcInfo.RebaseName) - } - - if err := archive.CopyTo(preArchive, srcInfo, out); err != nil { - return err - } - - dockerfile := `FROM docker/compose-bridge-transformer -LABEL com.docker.compose.bridge=transformation -COPY templates /templates -` - if err := os.WriteFile(filepath.Join(out, "Dockerfile"), []byte(dockerfile), 0o700); err != nil { - return err - } - _, err = fmt.Fprintf(dockerCli.Out(), "Transformer created in %q\n", out) - return err -} - -func ListTransformers(ctx context.Context, dockerCli command.Cli) ([]image.Summary, error) { - api := dockerCli.Client() - return api.ImageList(ctx, image.ListOptions{ - Filters: filters.NewArgs( - filters.Arg("label", fmt.Sprintf("%s=%s", TransformerLabel, "transformation")), - ), - }) -} diff --git a/pkg/compose/apiSocket.go b/pkg/compose/apiSocket.go deleted file mode 100644 index ddc7a029030..00000000000 --- a/pkg/compose/apiSocket.go +++ /dev/null @@ -1,92 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "errors" - "fmt" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/config/configfile" -) - -// --use-api-socket is not actually supported by the Docker Engine -// but is a client-side hack (see https://github.com/docker/cli/blob/master/cli/command/container/create.go#L246) -// we replicate here by transforming the project model - -func (s *composeService) useAPISocket(project *types.Project) (*types.Project, error) { - useAPISocket := false - for _, service := range project.Services { - if service.UseAPISocket { - useAPISocket = true - break - } - } - if !useAPISocket { - return project, nil - } - - if s.getContextInfo().ServerOSType() == "windows" { - return nil, errors.New("use_api_socket can't be used with a Windows Docker Engine") - } - - creds, err := s.configFile().GetAllCredentials() - if err != nil { - return nil, fmt.Errorf("resolving credentials failed: %w", err) - } - - newConfig := &configfile.ConfigFile{ - AuthConfigs: creds, - } - var configBuf bytes.Buffer - if err := newConfig.SaveToWriter(&configBuf); err != nil { - return nil, fmt.Errorf("saving creds for API socket: %w", err) - } - - project.Configs["#apisocket"] = types.ConfigObjConfig{ - Content: configBuf.String(), - } - - for name, service := range project.Services { - if !service.UseAPISocket { - continue - } - service.Volumes = append(service.Volumes, types.ServiceVolumeConfig{ - Type: types.VolumeTypeBind, - Source: "/var/run/docker.sock", - Target: "/var/run/docker.sock", - }) - - _, envvarPresent := service.Environment["DOCKER_CONFIG"] - - // If the DOCKER_CONFIG env var is already present, we assume the client knows - // what they're doing and don't inject the creds. - if !envvarPresent { - // Set our special little location for the config file. - path := "/run/secrets/docker" - service.Environment["DOCKER_CONFIG"] = &path - } - - service.Configs = append(service.Configs, types.ServiceConfigObjConfig{ - Source: "#apisocket", - Target: "/run/secrets/docker/config.json", - }) - project.Services[name] = service - } - return project, nil -} diff --git a/pkg/compose/api_versions.go b/pkg/compose/api_versions.go deleted file mode 100644 index 2caa520dbdd..00000000000 --- a/pkg/compose/api_versions.go +++ /dev/null @@ -1,73 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -// Docker Engine API version constants. -// These versions correspond to specific Docker Engine releases and their features. -const ( - // APIVersion144 represents Docker Engine API version 1.44 (Engine v25.0). - // - // New features in this version: - // - Endpoint-specific MAC address configuration - // - Multiple networks can be connected during container creation - // - healthcheck.start_interval parameter support - // - // Before this version: - // - MAC address was container-wide only - // - Extra networks required post-creation NetworkConnect calls - // - healthcheck.start_interval was not available - APIVersion144 = "1.44" - - // APIVersion148 represents Docker Engine API version 1.48 (Engine v28.0). - // - // New features in this version: - // - Volume mounts with type=image support - // - // Before this version: - // - Only bind, volume, and tmpfs mount types were supported - APIVersion148 = "1.48" - - // APIVersion149 represents Docker Engine API version 1.49 (Engine v28.1). - // - // New features in this version: - // - Network interface_name configuration - // - Platform parameter in ImageList API - // - // Before this version: - // - interface_name was not configurable - // - ImageList didn't support platform filtering - APIVersion149 = "1.49" -) - -// Docker Engine version strings for user-facing error messages. -// These should be used in error messages to provide clear version requirements. -const ( - // DockerEngineV25 is the major version string for Docker Engine 25.x - DockerEngineV25 = "v25" - - // DockerEngineV28 is the major version string for Docker Engine 28.x - DockerEngineV28 = "v28" - - // DockerEngineV28_1 is the specific version string for Docker Engine 28.1 - DockerEngineV28_1 = "v28.1" -) - -// Build tool version constants -const ( - // BuildxMinVersion is the minimum required version of buildx for compose build - BuildxMinVersion = "0.17.0" -) diff --git a/pkg/compose/attach.go b/pkg/compose/attach.go deleted file mode 100644 index 2ab07b1e995..00000000000 --- a/pkg/compose/attach.go +++ /dev/null @@ -1,161 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "io" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - containerType "github.com/docker/docker/api/types/container" - "github.com/docker/docker/pkg/stdcopy" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -func (s *composeService) attach(ctx context.Context, project *types.Project, listener api.ContainerEventListener, selectedServices []string) (Containers, error) { - containers, err := s.getContainers(ctx, project.Name, oneOffExclude, true, selectedServices...) - if err != nil { - return nil, err - } - if len(containers) == 0 { - return containers, nil - } - - containers.sorted() // This enforces predictable colors assignment - - var names []string - for _, c := range containers { - names = append(names, getContainerNameWithoutProject(c)) - } - - _, err = fmt.Fprintf(s.stdout(), "Attaching to %s\n", strings.Join(names, ", ")) - if err != nil { - logrus.Debugf("failed to write attach message: %v", err) - } - - for _, ctr := range containers { - err := s.attachContainer(ctx, ctr, listener) - if err != nil { - return nil, err - } - } - return containers, nil -} - -func (s *composeService) attachContainer(ctx context.Context, container containerType.Summary, listener api.ContainerEventListener) error { - service := container.Labels[api.ServiceLabel] - name := getContainerNameWithoutProject(container) - return s.doAttachContainer(ctx, service, container.ID, name, listener) -} - -func (s *composeService) doAttachContainer(ctx context.Context, service, id, name string, listener api.ContainerEventListener) error { - inspect, err := s.apiClient().ContainerInspect(ctx, id) - if err != nil { - return err - } - - wOut := utils.GetWriter(func(line string) { - listener(api.ContainerEvent{ - Type: api.ContainerEventLog, - Source: name, - ID: id, - Service: service, - Line: line, - }) - }) - wErr := utils.GetWriter(func(line string) { - listener(api.ContainerEvent{ - Type: api.ContainerEventErr, - Source: name, - ID: id, - Service: service, - Line: line, - }) - }) - - err = s.attachContainerStreams(ctx, id, inspect.Config.Tty, wOut, wErr) - if err != nil { - return err - } - - return nil -} - -func (s *composeService) attachContainerStreams(ctx context.Context, container string, tty bool, stdout, stderr io.WriteCloser) error { - streamOut, err := s.getContainerStreams(ctx, container) - if err != nil { - return err - } - - if stdout != nil { - go func() { - defer func() { - if err := stdout.Close(); err != nil { - logrus.Debugf("failed to close stdout: %v", err) - } - if err := stderr.Close(); err != nil { - logrus.Debugf("failed to close stderr: %v", err) - } - if err := streamOut.Close(); err != nil { - logrus.Debugf("failed to close stream output: %v", err) - } - }() - - var err error - if tty { - _, err = io.Copy(stdout, streamOut) - } else { - _, err = stdcopy.StdCopy(stdout, stderr, streamOut) - } - if err != nil && !errors.Is(err, io.EOF) { - logrus.Debugf("stream copy error for container %s: %v", container, err) - } - }() - } - return nil -} - -func (s *composeService) getContainerStreams(ctx context.Context, container string) (io.ReadCloser, error) { - cnx, err := s.apiClient().ContainerAttach(ctx, container, containerType.AttachOptions{ - Stream: true, - Stdin: false, - Stdout: true, - Stderr: true, - Logs: false, - }) - if err == nil { - stdout := ContainerStdout{HijackedResponse: cnx} - return stdout, nil - } - - // Fallback to logs API - logs, err := s.apiClient().ContainerLogs(ctx, container, containerType.LogsOptions{ - ShowStdout: true, - ShowStderr: true, - Follow: true, - }) - if err != nil { - return nil, err - } - return logs, nil -} diff --git a/pkg/compose/attach_service.go b/pkg/compose/attach_service.go deleted file mode 100644 index c6209e02244..00000000000 --- a/pkg/compose/attach_service.go +++ /dev/null @@ -1,45 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strings" - - "github.com/docker/cli/cli/command/container" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Attach(ctx context.Context, projectName string, options api.AttachOptions) error { - projectName = strings.ToLower(projectName) - target, err := s.getSpecifiedContainer(ctx, projectName, oneOffInclude, false, options.Service, options.Index) - if err != nil { - return err - } - - detachKeys := options.DetachKeys - if detachKeys == "" { - detachKeys = s.configFile().DetachKeys - } - - var attach container.AttachOptions - attach.DetachKeys = detachKeys - attach.NoStdin = options.NoStdin - attach.Proxy = options.Proxy - return container.RunAttach(ctx, s.dockerCli, target.ID, &attach) -} diff --git a/pkg/compose/build.go b/pkg/compose/build.go deleted file mode 100644 index 7b9cc76f96c..00000000000 --- a/pkg/compose/build.go +++ /dev/null @@ -1,303 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "strings" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/platforms" - specs "github.com/opencontainers/image-spec/specs-go/v1" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -func (s *composeService) Build(ctx context.Context, project *types.Project, options api.BuildOptions) error { - err := options.Apply(project) - if err != nil { - return err - } - return Run(ctx, func(ctx context.Context) error { - return tracing.SpanWrapFunc("project/build", tracing.ProjectOptions(ctx, project), - func(ctx context.Context) error { - builtImages, err := s.build(ctx, project, options, nil) - if err == nil && len(builtImages) == 0 { - logrus.Warn("No services to build") - } - return err - })(ctx) - }, "build", s.events) -} - -func (s *composeService) build(ctx context.Context, project *types.Project, options api.BuildOptions, localImages map[string]api.ImageSummary) (map[string]string, error) { - imageIDs := map[string]string{} - serviceToBuild := types.Services{} - - var policy types.DependencyOption = types.IgnoreDependencies - if options.Deps { - policy = types.IncludeDependencies - } - - if len(options.Services) == 0 { - options.Services = project.ServiceNames() - } - - // also include services used as additional_contexts with service: prefix - options.Services = addBuildDependencies(options.Services, project) - - // Some build dependencies we just introduced may not be enabled - var err error - project, err = project.WithServicesEnabled(options.Services...) - if err != nil { - return nil, err - } - - project, err = project.WithSelectedServices(options.Services) - if err != nil { - return nil, err - } - - err = project.ForEachService(options.Services, func(serviceName string, service *types.ServiceConfig) error { - if service.Build == nil { - return nil - } - image := api.GetImageNameOrDefault(*service, project.Name) - _, localImagePresent := localImages[image] - if localImagePresent && service.PullPolicy != types.PullPolicyBuild { - return nil - } - serviceToBuild[serviceName] = *service - return nil - }, policy) - if err != nil { - return imageIDs, err - } - - if len(serviceToBuild) == 0 { - return imageIDs, nil - } - - bake, err := buildWithBake(s.dockerCli) - if err != nil { - return nil, err - } - if bake { - return s.doBuildBake(ctx, project, serviceToBuild, options) - } - return s.doBuildClassic(ctx, project, serviceToBuild, options) -} - -func (s *composeService) ensureImagesExists(ctx context.Context, project *types.Project, buildOpts *api.BuildOptions, quietPull bool) error { - for name, service := range project.Services { - if service.Provider == nil && service.Image == "" && service.Build == nil { - return fmt.Errorf("invalid service %q. Must specify either image or build", name) - } - } - - images, err := s.getLocalImagesDigests(ctx, project) - if err != nil { - return err - } - - err = tracing.SpanWrapFunc("project/pull", tracing.ProjectOptions(ctx, project), - func(ctx context.Context) error { - return s.pullRequiredImages(ctx, project, images, quietPull) - }, - )(ctx) - if err != nil { - return err - } - - if buildOpts != nil { - err = tracing.SpanWrapFunc("project/build", tracing.ProjectOptions(ctx, project), - func(ctx context.Context) error { - builtImages, err := s.build(ctx, project, *buildOpts, images) - if err != nil { - return err - } - - for name, digest := range builtImages { - images[name] = api.ImageSummary{ - Repository: name, - ID: digest, - LastTagTime: time.Now(), - } - } - return nil - }, - )(ctx) - if err != nil { - return err - } - } - - // set digest as com.docker.compose.image label so we can detect outdated containers - for name, service := range project.Services { - image := api.GetImageNameOrDefault(service, project.Name) - img, ok := images[image] - if ok { - service.CustomLabels.Add(api.ImageDigestLabel, img.ID) - } - - resolveImageVolumes(&service, images, project.Name) - - project.Services[name] = service - } - return nil -} - -func resolveImageVolumes(service *types.ServiceConfig, images map[string]api.ImageSummary, projectName string) { - for i, vol := range service.Volumes { - if vol.Type == types.VolumeTypeImage { - imgName := vol.Source - if _, ok := images[vol.Source]; !ok { - // check if source is another service in the project - imgName = api.GetImageNameOrDefault(types.ServiceConfig{Name: vol.Source}, projectName) - // If we still can't find it, it might be an external image that wasn't pulled yet or doesn't exist - if _, ok := images[imgName]; !ok { - continue - } - } - if img, ok := images[imgName]; ok { - // Use Image ID directly as source. - // Using name@digest format (via reference.WithDigest) fails for local-only images - // that don't have RepoDigests (e.g. built locally in CI). - // Image ID (sha256:...) is always valid and ensures ServiceHash changes on rebuild. - service.Volumes[i].Source = img.ID - } - } - } -} - -func (s *composeService) getLocalImagesDigests(ctx context.Context, project *types.Project) (map[string]api.ImageSummary, error) { - imageNames := utils.Set[string]{} - for _, s := range project.Services { - imageNames.Add(api.GetImageNameOrDefault(s, project.Name)) - for _, volume := range s.Volumes { - if volume.Type == types.VolumeTypeImage { - imageNames.Add(volume.Source) - } - } - } - imgs, err := s.getImageSummaries(ctx, imageNames.Elements()) - if err != nil { - return nil, err - } - - for i, service := range project.Services { - imgName := api.GetImageNameOrDefault(service, project.Name) - img, ok := imgs[imgName] - if !ok { - continue - } - if service.Platform != "" { - platform, err := platforms.Parse(service.Platform) - if err != nil { - return nil, err - } - inspect, err := s.apiClient().ImageInspect(ctx, img.ID) - if err != nil { - return nil, err - } - actual := specs.Platform{ - Architecture: inspect.Architecture, - OS: inspect.Os, - Variant: inspect.Variant, - } - if !platforms.NewMatcher(platform).Match(actual) { - logrus.Debugf("local image %s doesn't match expected platform %s", service.Image, service.Platform) - // there is a local image, but it's for the wrong platform, so - // pretend it doesn't exist so that we can pull/build an image - // for the correct platform instead - delete(imgs, imgName) - } - } - - project.Services[i].CustomLabels.Add(api.ImageDigestLabel, img.ID) - - } - - return imgs, nil -} - -// resolveAndMergeBuildArgs returns the final set of build arguments to use for the service image build. -// -// First, args directly defined via `build.args` in YAML are considered. -// Then, any explicitly passed args in opts (e.g. via `--build-arg` on the CLI) are merged, overwriting any -// keys that already exist. -// Next, any keys without a value are resolved using the project environment. -// -// Finally, standard proxy variables based on the Docker client configuration are added, but will not overwrite -// any values if already present. -func resolveAndMergeBuildArgs(proxyConfig map[string]string, project *types.Project, service types.ServiceConfig, opts api.BuildOptions) types.MappingWithEquals { - result := make(types.MappingWithEquals). - OverrideBy(service.Build.Args). - OverrideBy(opts.Args). - Resolve(envResolver(project.Environment)) - - // proxy arguments do NOT override and should NOT have env resolution applied, - // so they're handled last - for k, v := range proxyConfig { - if _, ok := result[k]; !ok { - v := v - result[k] = &v - } - } - return result -} - -func getImageBuildLabels(project *types.Project, service types.ServiceConfig) types.Labels { - ret := make(types.Labels) - if service.Build != nil { - for k, v := range service.Build.Labels { - ret.Add(k, v) - } - } - - ret.Add(api.VersionLabel, api.ComposeVersion) - ret.Add(api.ProjectLabel, project.Name) - ret.Add(api.ServiceLabel, service.Name) - return ret -} - -func addBuildDependencies(services []string, project *types.Project) []string { - servicesWithDependencies := utils.NewSet(services...) - for _, service := range services { - s, ok := project.Services[service] - if !ok { - s = project.DisabledServices[service] - } - b := s.Build - if b != nil { - for _, target := range b.AdditionalContexts { - if s, found := strings.CutPrefix(target, types.ServicePrefix); found { - servicesWithDependencies.Add(s) - } - } - } - } - if len(servicesWithDependencies) > len(services) { - return addBuildDependencies(servicesWithDependencies.Elements(), project) - } - return servicesWithDependencies.Elements() -} diff --git a/pkg/compose/build_bake.go b/pkg/compose/build_bake.go deleted file mode 100644 index 90bd0d5a353..00000000000 --- a/pkg/compose/build_bake.go +++ /dev/null @@ -1,591 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bufio" - "bytes" - "context" - "crypto/sha1" - "encoding/json" - "errors" - "fmt" - "io" - "io/fs" - "os" - "os/exec" - "path/filepath" - "slices" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/console" - "github.com/containerd/errdefs" - "github.com/docker/cli/cli-plugins/manager" - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/command/image/build" - "github.com/docker/cli/cli/streams" - "github.com/docker/docker/api/types/versions" - "github.com/google/uuid" - "github.com/moby/buildkit/client" - gitutil "github.com/moby/buildkit/frontend/dockerfile/dfgitutil" - "github.com/moby/buildkit/util/progress/progressui" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func buildWithBake(dockerCli command.Cli) (bool, error) { - enabled, err := dockerCli.BuildKitEnabled() - if err != nil { - return false, err - } - if !enabled { - return false, nil - } - - _, err = manager.GetPlugin("buildx", dockerCli, &cobra.Command{}) - if err != nil { - if errdefs.IsNotFound(err) { - logrus.Warnf("Docker Compose requires buildx plugin to be installed") - return false, nil - } - return false, err - } - return true, err -} - -// We _could_ use bake.* types from github.com/docker/buildx but long term plan is to remove buildx as a dependency -type bakeConfig struct { - Groups map[string]bakeGroup `json:"group"` - Targets map[string]bakeTarget `json:"target"` -} - -type bakeGroup struct { - Targets []string `json:"targets"` -} - -type bakeTarget struct { - Context string `json:"context,omitempty"` - Contexts map[string]string `json:"contexts,omitempty"` - Dockerfile string `json:"dockerfile,omitempty"` - DockerfileInline string `json:"dockerfile-inline,omitempty"` - Args map[string]string `json:"args,omitempty"` - Labels map[string]string `json:"labels,omitempty"` - Tags []string `json:"tags,omitempty"` - CacheFrom []string `json:"cache-from,omitempty"` - CacheTo []string `json:"cache-to,omitempty"` - Target string `json:"target,omitempty"` - Secrets []string `json:"secret,omitempty"` - SSH []string `json:"ssh,omitempty"` - Platforms []string `json:"platforms,omitempty"` - Pull bool `json:"pull,omitempty"` - NoCache bool `json:"no-cache,omitempty"` - NetworkMode string `json:"network,omitempty"` - NoCacheFilter []string `json:"no-cache-filter,omitempty"` - ShmSize types.UnitBytes `json:"shm-size,omitempty"` - Ulimits []string `json:"ulimits,omitempty"` - Call string `json:"call,omitempty"` - Entitlements []string `json:"entitlements,omitempty"` - ExtraHosts map[string]string `json:"extra-hosts,omitempty"` - Outputs []string `json:"output,omitempty"` - Attest []string `json:"attest,omitempty"` -} - -type bakeMetadata map[string]buildStatus - -type buildStatus struct { - Digest string `json:"containerimage.digest"` - Image string `json:"image.name"` -} - -func (s *composeService) doBuildBake(ctx context.Context, project *types.Project, serviceToBeBuild types.Services, options api.BuildOptions) (map[string]string, error) { //nolint:gocyclo - eg := errgroup.Group{} - ch := make(chan *client.SolveStatus) - displayMode := progressui.DisplayMode(options.Progress) - if p, ok := os.LookupEnv("BUILDKIT_PROGRESS"); ok && displayMode == progressui.AutoMode { - displayMode = progressui.DisplayMode(p) - } - out := options.Out - if out == nil { - out = s.stdout() - } - display, err := progressui.NewDisplay(makeConsole(out), displayMode) - if err != nil { - return nil, err - } - eg.Go(func() error { - _, err := display.UpdateFrom(ctx, ch) - return err - }) - - cfg := bakeConfig{ - Groups: map[string]bakeGroup{}, - Targets: map[string]bakeTarget{}, - } - var ( - group bakeGroup - privileged bool - read []string - expectedImages = make(map[string]string, len(serviceToBeBuild)) // service name -> expected image - targets = make(map[string]string, len(serviceToBeBuild)) // service name -> build target - ) - - // produce a unique ID for service used as bake target - for serviceName := range project.Services { - t := strings.ReplaceAll(serviceName, ".", "_") - for { - if _, ok := targets[serviceName]; !ok { - targets[serviceName] = t - break - } - t += "_" - } - } - - var secretsEnv []string - for serviceName, service := range project.Services { - if service.Build == nil { - continue - } - buildConfig := *service.Build - labels := getImageBuildLabels(project, service) - - args := resolveAndMergeBuildArgs(s.getProxyConfig(), project, service, options).ToMapping() - for k, v := range args { - args[k] = strings.ReplaceAll(v, "${", "$${") - } - - entitlements := buildConfig.Entitlements - if slices.Contains(buildConfig.Entitlements, "security.insecure") { - privileged = true - } - if buildConfig.Privileged { - entitlements = append(entitlements, "security.insecure") - privileged = true - } - - var outputs []string - var call string - push := options.Push && service.Image != "" - switch { - case options.Check: - call = "lint" - case len(service.Build.Platforms) > 1: - outputs = []string{fmt.Sprintf("type=image,push=%t", push)} - default: - if push { - outputs = []string{"type=registry"} - } else { - outputs = []string{"type=docker"} - } - } - - read = append(read, buildConfig.Context) - for _, path := range buildConfig.AdditionalContexts { - _, _, err := gitutil.ParseGitRef(path) - if !strings.Contains(path, "://") && err != nil { - read = append(read, path) - } - } - - image := api.GetImageNameOrDefault(service, project.Name) - s.events.On(buildingEvent(image)) - - expectedImages[serviceName] = image - - pull := service.Build.Pull || options.Pull - noCache := service.Build.NoCache || options.NoCache - - target := targets[serviceName] - - secrets, env := toBakeSecrets(project, buildConfig.Secrets) - secretsEnv = append(secretsEnv, env...) - - cfg.Targets[target] = bakeTarget{ - Context: buildConfig.Context, - Contexts: additionalContexts(buildConfig.AdditionalContexts, targets), - Dockerfile: dockerFilePath(buildConfig.Context, buildConfig.Dockerfile), - DockerfileInline: strings.ReplaceAll(buildConfig.DockerfileInline, "${", "$${"), - Args: args, - Labels: labels, - Tags: append(buildConfig.Tags, image), - - CacheFrom: buildConfig.CacheFrom, - CacheTo: buildConfig.CacheTo, - NetworkMode: buildConfig.Network, - NoCacheFilter: buildConfig.NoCacheFilter, - Platforms: buildConfig.Platforms, - Target: buildConfig.Target, - Secrets: secrets, - SSH: toBakeSSH(append(buildConfig.SSH, options.SSHs...)), - Pull: pull, - NoCache: noCache, - ShmSize: buildConfig.ShmSize, - Ulimits: toBakeUlimits(buildConfig.Ulimits), - Entitlements: entitlements, - ExtraHosts: toBakeExtraHosts(buildConfig.ExtraHosts), - - Outputs: outputs, - Call: call, - Attest: toBakeAttest(buildConfig), - } - } - - // create a bake group with targets for services to build - for serviceName, service := range serviceToBeBuild { - if service.Build == nil { - continue - } - group.Targets = append(group.Targets, targets[serviceName]) - } - - cfg.Groups["default"] = group - - b, err := json.MarshalIndent(cfg, "", " ") - if err != nil { - return nil, err - } - - if options.Print { - _, err = fmt.Fprintln(s.stdout(), string(b)) - return nil, err - } - logrus.Debugf("bake build config:\n%s", string(b)) - - tmpdir := os.TempDir() - var metadataFile string - for { - // we don't use os.CreateTemp here as we need a temporary file name, but don't want it actually created - // as bake relies on atomicwriter and this creates conflict during rename - metadataFile = filepath.Join(tmpdir, fmt.Sprintf("compose-build-metadataFile-%s.json", uuid.New().String())) - if _, err = os.Stat(metadataFile); err != nil { - if os.IsNotExist(err) { - break - } - var pathError *fs.PathError - if errors.As(err, &pathError) { - return nil, fmt.Errorf("can't access os.tempDir %s: %w", tmpdir, pathError.Err) - } - } - } - defer func() { - _ = os.Remove(metadataFile) - }() - - buildx, err := s.getBuildxPlugin() - if err != nil { - return nil, err - } - - args := []string{"bake", "--file", "-", "--progress", "rawjson", "--metadata-file", metadataFile} - // FIXME we should prompt user about this, but this is a breaking change in UX - for _, path := range read { - args = append(args, "--allow", "fs.read="+path) - } - if privileged { - args = append(args, "--allow", "security.insecure") - } - if options.SBOM != "" { - args = append(args, "--sbom="+options.SBOM) - } - if options.Provenance != "" { - args = append(args, "--provenance="+options.Provenance) - } - - if options.Builder != "" { - args = append(args, "--builder", options.Builder) - } - if options.Quiet { - args = append(args, "--progress=quiet") - } - - logrus.Debugf("Executing bake with args: %v", args) - - if s.dryRun { - return s.dryRunBake(cfg), nil - } - cmd := exec.CommandContext(ctx, buildx.Path, args...) - - err = s.prepareShellOut(ctx, types.NewMapping(os.Environ()), cmd) - if err != nil { - return nil, err - } - endpoint, cleanup, err := s.propagateDockerEndpoint() - if err != nil { - return nil, err - } - cmd.Env = append(cmd.Env, endpoint...) - cmd.Env = append(cmd.Env, secretsEnv...) - defer cleanup() - - cmd.Stdout = s.stdout() - cmd.Stdin = bytes.NewBuffer(b) - pipe, err := cmd.StderrPipe() - if err != nil { - return nil, err - } - - var errMessage []string - reader := bufio.NewReader(pipe) - - err = cmd.Start() - if err != nil { - return nil, err - } - eg.Go(cmd.Wait) - for { - line, readErr := reader.ReadString('\n') - if readErr != nil { - if readErr == io.EOF { - break - } - if errors.Is(readErr, os.ErrClosed) { - logrus.Debugf("bake stopped") - break - } - return nil, fmt.Errorf("failed to execute bake: %w", readErr) - } - decoder := json.NewDecoder(strings.NewReader(line)) - var status client.SolveStatus - err := decoder.Decode(&status) - if err != nil { - if strings.HasPrefix(line, "ERROR: ") { - errMessage = append(errMessage, line[7:]) - } else { - errMessage = append(errMessage, line) - } - continue - } - ch <- &status - } - close(ch) // stop build progress UI - - err = eg.Wait() - if err != nil { - if len(errMessage) > 0 { - return nil, errors.New(strings.Join(errMessage, "\n")) - } - return nil, fmt.Errorf("failed to execute bake: %w", err) - } - - b, err = os.ReadFile(metadataFile) - if err != nil { - return nil, err - } - - var md bakeMetadata - err = json.Unmarshal(b, &md) - if err != nil { - return nil, err - } - - results := map[string]string{} - for name := range serviceToBeBuild { - image := expectedImages[name] - target := targets[name] - built, ok := md[target] - if !ok { - return nil, fmt.Errorf("build result not found in Bake metadata for service %s", name) - } - results[image] = built.Digest - s.events.On(builtEvent(image)) - } - return results, nil -} - -func (s *composeService) getBuildxPlugin() (*manager.Plugin, error) { - buildx, err := manager.GetPlugin("buildx", s.dockerCli, &cobra.Command{}) - if err != nil { - return nil, err - } - - if buildx.Err != nil { - return nil, buildx.Err - } - - if buildx.Version == "" { - return nil, fmt.Errorf("failed to get version of buildx") - } - - if versions.LessThan(buildx.Version[1:], BuildxMinVersion) { - return nil, fmt.Errorf("compose build requires buildx %s or later", BuildxMinVersion) - } - - return buildx, nil -} - -// makeConsole wraps the provided writer to match [containerd.File] interface if it is of type *streams.Out. -// buildkit's NewDisplay doesn't actually require a [io.Reader], it only uses the [containerd.Console] type to -// benefits from ANSI capabilities, but only does writes. -func makeConsole(out io.Writer) io.Writer { - if s, ok := out.(*streams.Out); ok { - return &_console{s} - } - return out -} - -var _ console.File = &_console{} - -type _console struct { - *streams.Out -} - -func (c _console) Read(p []byte) (n int, err error) { - return 0, errors.New("not implemented") -} - -func (c _console) Close() error { - return nil -} - -func (c _console) Fd() uintptr { - return c.FD() -} - -func (c _console) Name() string { - return "compose" -} - -func toBakeExtraHosts(hosts types.HostsList) map[string]string { - m := make(map[string]string) - for k, v := range hosts { - m[k] = strings.Join(v, ",") - } - return m -} - -func additionalContexts(contexts types.Mapping, targets map[string]string) map[string]string { - ac := map[string]string{} - for k, v := range contexts { - if target, found := strings.CutPrefix(v, types.ServicePrefix); found { - v = "target:" + targets[target] - } - ac[k] = v - } - return ac -} - -func toBakeUlimits(ulimits map[string]*types.UlimitsConfig) []string { - s := []string{} - for u, l := range ulimits { - if l.Single > 0 { - s = append(s, fmt.Sprintf("%s=%d", u, l.Single)) - } else { - s = append(s, fmt.Sprintf("%s=%d:%d", u, l.Soft, l.Hard)) - } - } - return s -} - -func toBakeSSH(ssh types.SSHConfig) []string { - var s []string - for _, key := range ssh { - s = append(s, fmt.Sprintf("%s=%s", key.ID, key.Path)) - } - return s -} - -func toBakeSecrets(project *types.Project, secrets []types.ServiceSecretConfig) ([]string, []string) { - var s []string - var env []string - for _, ref := range secrets { - def := project.Secrets[ref.Source] - target := ref.Target - if target == "" { - target = ref.Source - } - switch { - case def.Environment != "": - env = append(env, fmt.Sprintf("%s=%s", def.Environment, project.Environment[def.Environment])) - s = append(s, fmt.Sprintf("id=%s,type=env,env=%s", target, def.Environment)) - case def.File != "": - s = append(s, fmt.Sprintf("id=%s,type=file,src=%s", target, def.File)) - } - } - return s, env -} - -func toBakeAttest(buildConfig types.BuildConfig) []string { - var attests []string - - // Handle per-service provenance configuration (only from build config, not global options) - if buildConfig.Provenance != "" { - if buildConfig.Provenance == "true" { - attests = append(attests, "type=provenance") - } else if buildConfig.Provenance != "false" { - attests = append(attests, fmt.Sprintf("type=provenance,%s", buildConfig.Provenance)) - } - } - - // Handle per-service SBOM configuration (only from build config, not global options) - if buildConfig.SBOM != "" { - if buildConfig.SBOM == "true" { - attests = append(attests, "type=sbom") - } else if buildConfig.SBOM != "false" { - attests = append(attests, fmt.Sprintf("type=sbom,%s", buildConfig.SBOM)) - } - } - - return attests -} - -func dockerFilePath(ctxName string, dockerfile string) string { - if dockerfile == "" { - return "" - } - if contextType, _ := build.DetectContextType(ctxName); contextType == build.ContextTypeGit { - return dockerfile - } - if !filepath.IsAbs(dockerfile) { - dockerfile = filepath.Join(ctxName, dockerfile) - } - dir := filepath.Dir(dockerfile) - symlinks, err := filepath.EvalSymlinks(dir) - if err == nil { - return filepath.Join(symlinks, filepath.Base(dockerfile)) - } - return dockerfile -} - -func (s composeService) dryRunBake(cfg bakeConfig) map[string]string { - bakeResponse := map[string]string{} - for name, target := range cfg.Targets { - dryRunUUID := fmt.Sprintf("dryRun-%x", sha1.Sum([]byte(name))) - s.displayDryRunBuildEvent(name, dryRunUUID, target.Tags[0]) - bakeResponse[name] = dryRunUUID - } - for name := range bakeResponse { - s.events.On(builtEvent(name)) - } - return bakeResponse -} - -func (s composeService) displayDryRunBuildEvent(name, dryRunUUID, tag string) { - s.events.On(api.Resource{ - ID: name + " ==>", - Status: api.Done, - Text: fmt.Sprintf("==> writing image %s", dryRunUUID), - }) - s.events.On(api.Resource{ - ID: name + " ==> ==>", - Status: api.Done, - Text: fmt.Sprintf(`naming to %s`, tag), - }) -} diff --git a/pkg/compose/build_classic.go b/pkg/compose/build_classic.go deleted file mode 100644 index 1cfa2575322..00000000000 --- a/pkg/compose/build_classic.go +++ /dev/null @@ -1,309 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "encoding/json" - "errors" - "fmt" - "io" - "os" - "path/filepath" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command/image/build" - buildtypes "github.com/docker/docker/api/types/build" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/registry" - "github.com/docker/docker/pkg/jsonmessage" - "github.com/docker/docker/pkg/progress" - "github.com/docker/docker/pkg/streamformatter" - "github.com/moby/go-archive" - "github.com/sirupsen/logrus" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/trace" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) doBuildClassic(ctx context.Context, project *types.Project, serviceToBuild types.Services, options api.BuildOptions) (map[string]string, error) { - imageIDs := map[string]string{} - - // Not using bake, additional_context: service:xx is implemented by building images in dependency order - project, err := project.WithServicesTransform(func(serviceName string, service types.ServiceConfig) (types.ServiceConfig, error) { - if service.Build != nil { - for _, c := range service.Build.AdditionalContexts { - if t, found := strings.CutPrefix(c, types.ServicePrefix); found { - if service.DependsOn == nil { - service.DependsOn = map[string]types.ServiceDependency{} - } - service.DependsOn[t] = types.ServiceDependency{ - Condition: "build", // non-canonical, but will force dependency graph ordering - } - } - } - } - return service, nil - }) - if err != nil { - return imageIDs, err - } - - // we use a pre-allocated []string to collect build digest by service index while running concurrent goroutines - builtDigests := make([]string, len(project.Services)) - names := project.ServiceNames() - getServiceIndex := func(name string) int { - for idx, n := range names { - if n == name { - return idx - } - } - return -1 - } - - err = InDependencyOrder(ctx, project, func(ctx context.Context, name string) error { - trace.SpanFromContext(ctx).SetAttributes(attribute.String("builder", "classic")) - service, ok := serviceToBuild[name] - if !ok { - return nil - } - - image := api.GetImageNameOrDefault(service, project.Name) - s.events.On(buildingEvent(image)) - id, err := s.doBuildImage(ctx, project, service, options) - if err != nil { - return err - } - s.events.On(builtEvent(image)) - builtDigests[getServiceIndex(name)] = id - - if options.Push { - return s.push(ctx, project, api.PushOptions{}) - } - return nil - }, func(traversal *graphTraversal) { - traversal.maxConcurrency = s.maxConcurrency - }) - if err != nil { - return nil, err - } - - for i, imageDigest := range builtDigests { - if imageDigest != "" { - service := project.Services[names[i]] - imageRef := api.GetImageNameOrDefault(service, project.Name) - imageIDs[imageRef] = imageDigest - } - } - return imageIDs, err -} - -//nolint:gocyclo -func (s *composeService) doBuildImage(ctx context.Context, project *types.Project, service types.ServiceConfig, options api.BuildOptions) (string, error) { - var ( - buildCtx io.ReadCloser - dockerfileCtx io.ReadCloser - contextDir string - relDockerfile string - ) - - if len(service.Build.Platforms) > 1 { - return "", fmt.Errorf("the classic builder doesn't support multi-arch build, set DOCKER_BUILDKIT=1 to use BuildKit") - } - if service.Build.Privileged { - return "", fmt.Errorf("the classic builder doesn't support privileged mode, set DOCKER_BUILDKIT=1 to use BuildKit") - } - if len(service.Build.AdditionalContexts) > 0 { - return "", fmt.Errorf("the classic builder doesn't support additional contexts, set DOCKER_BUILDKIT=1 to use BuildKit") - } - if len(service.Build.SSH) > 0 { - return "", fmt.Errorf("the classic builder doesn't support SSH keys, set DOCKER_BUILDKIT=1 to use BuildKit") - } - if len(service.Build.Secrets) > 0 { - return "", fmt.Errorf("the classic builder doesn't support secrets, set DOCKER_BUILDKIT=1 to use BuildKit") - } - - if service.Build.Labels == nil { - service.Build.Labels = make(map[string]string) - } - service.Build.Labels[api.ImageBuilderLabel] = "classic" - - dockerfileName := dockerFilePath(service.Build.Context, service.Build.Dockerfile) - specifiedContext := service.Build.Context - progBuff := s.stdout() - buildBuff := s.stdout() - - contextType, err := build.DetectContextType(specifiedContext) - if err != nil { - return "", err - } - - switch contextType { - case build.ContextTypeStdin: - return "", fmt.Errorf("building from STDIN is not supported") - case build.ContextTypeLocal: - contextDir, relDockerfile, err = build.GetContextFromLocalDir(specifiedContext, dockerfileName) - if err != nil { - return "", fmt.Errorf("unable to prepare context: %w", err) - } - if strings.HasPrefix(relDockerfile, ".."+string(filepath.Separator)) { - // Dockerfile is outside build-context; read the Dockerfile and pass it as dockerfileCtx - dockerfileCtx, err = os.Open(dockerfileName) - if err != nil { - return "", fmt.Errorf("unable to open Dockerfile: %w", err) - } - defer dockerfileCtx.Close() //nolint:errcheck - } - case build.ContextTypeGit: - var tempDir string - tempDir, relDockerfile, err = build.GetContextFromGitURL(specifiedContext, dockerfileName) - if err != nil { - return "", fmt.Errorf("unable to prepare context: %w", err) - } - defer func() { - _ = os.RemoveAll(tempDir) - }() - contextDir = tempDir - case build.ContextTypeRemote: - buildCtx, relDockerfile, err = build.GetContextFromURL(progBuff, specifiedContext, dockerfileName) - if err != nil { - return "", fmt.Errorf("unable to prepare context: %w", err) - } - default: - return "", fmt.Errorf("unable to prepare context: path %q not found", specifiedContext) - } - - // read from a directory into tar archive - if buildCtx == nil { - excludes, err := build.ReadDockerignore(contextDir) - if err != nil { - return "", err - } - - if err := build.ValidateContextDirectory(contextDir, excludes); err != nil { - return "", fmt.Errorf("checking context: %w", err) - } - - // And canonicalize dockerfile name to a platform-independent one - relDockerfile = filepath.ToSlash(relDockerfile) - - excludes = build.TrimBuildFilesFromExcludes(excludes, relDockerfile, false) - buildCtx, err = archive.TarWithOptions(contextDir, &archive.TarOptions{ - ExcludePatterns: excludes, - ChownOpts: &archive.ChownOpts{UID: 0, GID: 0}, - }) - if err != nil { - return "", err - } - } - - // replace Dockerfile if it was added from stdin or a file outside the build-context, and there is archive context - if dockerfileCtx != nil && buildCtx != nil { - buildCtx, relDockerfile, err = build.AddDockerfileToBuildContext(dockerfileCtx, buildCtx) - if err != nil { - return "", err - } - } - - buildCtx, err = build.Compress(buildCtx) - if err != nil { - return "", err - } - - // Setup an upload progress bar - progressOutput := streamformatter.NewProgressOutput(progBuff) - body := progress.NewProgressReader(buildCtx, progressOutput, 0, "", "Sending build context to Docker daemon") - - configFile := s.configFile() - creds, err := configFile.GetAllCredentials() - if err != nil { - return "", err - } - authConfigs := make(map[string]registry.AuthConfig, len(creds)) - for k, authConfig := range creds { - authConfigs[k] = registry.AuthConfig{ - Username: authConfig.Username, - Password: authConfig.Password, - ServerAddress: authConfig.ServerAddress, - - // TODO(thaJeztah): Are these expected to be included? See https://github.com/docker/cli/pull/6516#discussion_r2387586472 - Auth: authConfig.Auth, - IdentityToken: authConfig.IdentityToken, - RegistryToken: authConfig.RegistryToken, - } - } - buildOpts := imageBuildOptions(s.getProxyConfig(), project, service, options) - imageName := api.GetImageNameOrDefault(service, project.Name) - buildOpts.Tags = append(buildOpts.Tags, imageName) - buildOpts.Dockerfile = relDockerfile - buildOpts.AuthConfigs = authConfigs - buildOpts.Memory = options.Memory - - ctx, cancel := context.WithCancel(ctx) - defer cancel() - s.events.On(buildingEvent(imageName)) - response, err := s.apiClient().ImageBuild(ctx, body, buildOpts) - if err != nil { - return "", err - } - defer response.Body.Close() //nolint:errcheck - - imageID := "" - aux := func(msg jsonmessage.JSONMessage) { - var result buildtypes.Result - if err := json.Unmarshal(*msg.Aux, &result); err != nil { - logrus.Errorf("Failed to parse aux message: %s", err) - } else { - imageID = result.ID - } - } - - err = jsonmessage.DisplayJSONMessagesStream(response.Body, buildBuff, progBuff.FD(), true, aux) - if err != nil { - var jerr *jsonmessage.JSONError - if errors.As(err, &jerr) { - // If no error code is set, default to 1 - if jerr.Code == 0 { - jerr.Code = 1 - } - return "", cli.StatusError{Status: jerr.Message, StatusCode: jerr.Code} - } - return "", err - } - s.events.On(builtEvent(imageName)) - return imageID, nil -} - -func imageBuildOptions(proxyConfigs map[string]string, project *types.Project, service types.ServiceConfig, options api.BuildOptions) buildtypes.ImageBuildOptions { - config := service.Build - return buildtypes.ImageBuildOptions{ - Version: buildtypes.BuilderV1, - Tags: config.Tags, - NoCache: config.NoCache, - Remove: true, - PullParent: config.Pull, - BuildArgs: resolveAndMergeBuildArgs(proxyConfigs, project, service, options), - Labels: config.Labels, - NetworkMode: config.Network, - ExtraHosts: config.ExtraHosts.AsList(":"), - Target: config.Target, - Isolation: container.Isolation(config.Isolation), - } -} diff --git a/pkg/compose/build_test.go b/pkg/compose/build_test.go deleted file mode 100644 index fa0a9e2c4e5..00000000000 --- a/pkg/compose/build_test.go +++ /dev/null @@ -1,57 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "slices" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "gotest.tools/v3/assert" -) - -func Test_addBuildDependencies(t *testing.T) { - project := &types.Project{Services: types.Services{ - "test": types.ServiceConfig{ - Build: &types.BuildConfig{ - AdditionalContexts: map[string]string{ - "foo": "service:foo", - "bar": "service:bar", - }, - }, - }, - "foo": types.ServiceConfig{ - Build: &types.BuildConfig{ - AdditionalContexts: map[string]string{ - "zot": "service:zot", - }, - }, - }, - "bar": types.ServiceConfig{ - Build: &types.BuildConfig{}, - }, - "zot": types.ServiceConfig{ - Build: &types.BuildConfig{}, - }, - }} - - services := addBuildDependencies([]string{"test"}, project) - expected := []string{"test", "foo", "bar", "zot"} - slices.Sort(services) - slices.Sort(expected) - assert.DeepEqual(t, services, expected) -} diff --git a/pkg/compose/commit.go b/pkg/compose/commit.go deleted file mode 100644 index 70b22d5d783..00000000000 --- a/pkg/compose/commit.go +++ /dev/null @@ -1,79 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "strings" - - "github.com/docker/docker/api/types/container" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Commit(ctx context.Context, projectName string, options api.CommitOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.commit(ctx, projectName, options) - }, "commit", s.events) -} - -func (s *composeService) commit(ctx context.Context, projectName string, options api.CommitOptions) error { - projectName = strings.ToLower(projectName) - - ctr, err := s.getSpecifiedContainer(ctx, projectName, oneOffInclude, false, options.Service, options.Index) - if err != nil { - return err - } - - name := getCanonicalContainerName(ctr) - - s.events.On(api.Resource{ - ID: name, - Status: api.Working, - Text: api.StatusCommitting, - }) - - if s.dryRun { - s.events.On(api.Resource{ - ID: name, - Status: api.Done, - Text: api.StatusCommitted, - }) - - return nil - } - - response, err := s.apiClient().ContainerCommit(ctx, ctr.ID, container.CommitOptions{ - Reference: options.Reference, - Comment: options.Comment, - Author: options.Author, - Changes: options.Changes.GetSlice(), - Pause: options.Pause, - }) - if err != nil { - return err - } - - s.events.On(api.Resource{ - ID: name, - Text: fmt.Sprintf("Committed as %s", response.ID), - Status: api.Done, - }) - - return nil -} diff --git a/pkg/compose/compose.go b/pkg/compose/compose.go deleted file mode 100644 index 45bda0bf3ec..00000000000 --- a/pkg/compose/compose.go +++ /dev/null @@ -1,514 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "io" - "strconv" - "strings" - "sync" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/buildx/store/storeutil" - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/config/configfile" - "github.com/docker/cli/cli/flags" - "github.com/docker/cli/cli/streams" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/api/types/swarm" - "github.com/docker/docker/api/types/volume" - "github.com/docker/docker/client" - "github.com/jonboulle/clockwork" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/dryrun" -) - -type Option func(service *composeService) error - -// NewComposeService creates a Compose service using Docker CLI. -// This is the standard constructor that requires command.Cli for full functionality. -// -// Example usage: -// -// dockerCli, _ := command.NewDockerCli() -// service := NewComposeService(dockerCli) -// -// For advanced configuration with custom overrides, use ServiceOption functions: -// -// service := NewComposeService(dockerCli, -// WithPrompt(prompt.NewPrompt(cli.In(), cli.Out()).Confirm), -// WithOutputStream(customOut), -// WithErrorStream(customErr), -// WithInputStream(customIn)) -// -// Or set all streams at once: -// -// service := NewComposeService(dockerCli, -// WithStreams(customOut, customErr, customIn)) -func NewComposeService(dockerCli command.Cli, options ...Option) (api.Compose, error) { - s := &composeService{ - dockerCli: dockerCli, - clock: clockwork.NewRealClock(), - maxConcurrency: -1, - dryRun: false, - } - for _, option := range options { - if err := option(s); err != nil { - return nil, err - } - } - if s.prompt == nil { - s.prompt = func(message string, defaultValue bool) (bool, error) { - fmt.Println(message) - logrus.Warning("Compose is running without a 'prompt' component to interact with user") - return defaultValue, nil - } - } - if s.events == nil { - s.events = &ignore{} - } - - // If custom streams were provided, wrap the Docker CLI to use them - if s.outStream != nil || s.errStream != nil || s.inStream != nil { - s.dockerCli = s.wrapDockerCliWithStreams(dockerCli) - } - - return s, nil -} - -// WithStreams sets custom I/O streams for output and interaction -func WithStreams(out, err io.Writer, in io.Reader) Option { - return func(s *composeService) error { - s.outStream = out - s.errStream = err - s.inStream = in - return nil - } -} - -// WithOutputStream sets a custom output stream -func WithOutputStream(out io.Writer) Option { - return func(s *composeService) error { - s.outStream = out - return nil - } -} - -// WithErrorStream sets a custom error stream -func WithErrorStream(err io.Writer) Option { - return func(s *composeService) error { - s.errStream = err - return nil - } -} - -// WithInputStream sets a custom input stream -func WithInputStream(in io.Reader) Option { - return func(s *composeService) error { - s.inStream = in - return nil - } -} - -// WithContextInfo sets custom Docker context information -func WithContextInfo(info api.ContextInfo) Option { - return func(s *composeService) error { - s.contextInfo = info - return nil - } -} - -// WithProxyConfig sets custom HTTP proxy configuration for builds -func WithProxyConfig(config map[string]string) Option { - return func(s *composeService) error { - s.proxyConfig = config - return nil - } -} - -// WithPrompt configure a UI component for Compose service to interact with user and confirm actions -func WithPrompt(prompt Prompt) Option { - return func(s *composeService) error { - s.prompt = prompt - return nil - } -} - -// WithMaxConcurrency defines upper limit for concurrent operations against engine API -func WithMaxConcurrency(maxConcurrency int) Option { - return func(s *composeService) error { - s.maxConcurrency = maxConcurrency - return nil - } -} - -// WithDryRun configure Compose to run without actually applying changes -func WithDryRun(s *composeService) error { - s.dryRun = true - cli, err := command.NewDockerCli() - if err != nil { - return err - } - - options := flags.NewClientOptions() - options.Context = s.dockerCli.CurrentContext() - err = cli.Initialize(options, command.WithInitializeClient(func(cli *command.DockerCli) (client.APIClient, error) { - return dryrun.NewDryRunClient(s.apiClient(), s.dockerCli) - })) - if err != nil { - return err - } - s.dockerCli = cli - return nil -} - -type Prompt func(message string, defaultValue bool) (bool, error) - -// AlwaysOkPrompt returns a Prompt implementation that always returns true without user interaction. -func AlwaysOkPrompt() Prompt { - return func(message string, defaultValue bool) (bool, error) { - return true, nil - } -} - -// WithEventProcessor configure component to get notified on Compose operation and progress events. -// Typically used to configure a progress UI -func WithEventProcessor(bus api.EventProcessor) Option { - return func(s *composeService) error { - s.events = bus - return nil - } -} - -type composeService struct { - dockerCli command.Cli - // prompt is used to interact with user and confirm actions - prompt Prompt - // eventBus collects tasks execution events - events api.EventProcessor - - // Optional overrides for specific components (for SDK users) - outStream io.Writer - errStream io.Writer - inStream io.Reader - contextInfo api.ContextInfo - proxyConfig map[string]string - - clock clockwork.Clock - maxConcurrency int - dryRun bool -} - -// Close releases any connections/resources held by the underlying clients. -// -// In practice, this service has the same lifetime as the process, so everything -// will get cleaned up at about the same time regardless even if not invoked. -func (s *composeService) Close() error { - var errs []error - if s.dockerCli != nil { - errs = append(errs, s.apiClient().Close()) - } - return errors.Join(errs...) -} - -func (s *composeService) apiClient() client.APIClient { - return s.dockerCli.Client() -} - -func (s *composeService) configFile() *configfile.ConfigFile { - return s.dockerCli.ConfigFile() -} - -// getContextInfo returns the context info - either custom override or dockerCli adapter -func (s *composeService) getContextInfo() api.ContextInfo { - if s.contextInfo != nil { - return s.contextInfo - } - return &dockerCliContextInfo{cli: s.dockerCli} -} - -// getProxyConfig returns the proxy config - either custom override or environment-based -func (s *composeService) getProxyConfig() map[string]string { - if s.proxyConfig != nil { - return s.proxyConfig - } - return storeutil.GetProxyConfig(s.dockerCli) -} - -func (s *composeService) stdout() *streams.Out { - return s.dockerCli.Out() -} - -func (s *composeService) stdin() *streams.In { - return s.dockerCli.In() -} - -func (s *composeService) stderr() *streams.Out { - return s.dockerCli.Err() -} - -// readCloserAdapter adapts io.Reader to io.ReadCloser -type readCloserAdapter struct { - r io.Reader -} - -func (r *readCloserAdapter) Read(p []byte) (int, error) { - return r.r.Read(p) -} - -func (r *readCloserAdapter) Close() error { - return nil -} - -// wrapDockerCliWithStreams wraps the Docker CLI to intercept and override stream methods -func (s *composeService) wrapDockerCliWithStreams(baseCli command.Cli) command.Cli { - wrapper := &streamOverrideWrapper{ - Cli: baseCli, - } - - // Wrap custom streams in Docker CLI's stream types - if s.outStream != nil { - wrapper.outStream = streams.NewOut(s.outStream) - } - if s.errStream != nil { - wrapper.errStream = streams.NewOut(s.errStream) - } - if s.inStream != nil { - wrapper.inStream = streams.NewIn(&readCloserAdapter{r: s.inStream}) - } - - return wrapper -} - -// streamOverrideWrapper wraps command.Cli to override streams with custom implementations -type streamOverrideWrapper struct { - command.Cli - outStream *streams.Out - errStream *streams.Out - inStream *streams.In -} - -func (w *streamOverrideWrapper) Out() *streams.Out { - if w.outStream != nil { - return w.outStream - } - return w.Cli.Out() -} - -func (w *streamOverrideWrapper) Err() *streams.Out { - if w.errStream != nil { - return w.errStream - } - return w.Cli.Err() -} - -func (w *streamOverrideWrapper) In() *streams.In { - if w.inStream != nil { - return w.inStream - } - return w.Cli.In() -} - -func getCanonicalContainerName(c container.Summary) string { - if len(c.Names) == 0 { - // corner case, sometime happens on removal. return short ID as a safeguard value - return c.ID[:12] - } - // Names return container canonical name /foo + link aliases /linked_by/foo - for _, name := range c.Names { - if strings.LastIndex(name, "/") == 0 { - return name[1:] - } - } - - return strings.TrimPrefix(c.Names[0], "/") -} - -func getContainerNameWithoutProject(c container.Summary) string { - project := c.Labels[api.ProjectLabel] - defaultName := getDefaultContainerName(project, c.Labels[api.ServiceLabel], c.Labels[api.ContainerNumberLabel]) - name := getCanonicalContainerName(c) - if name != defaultName { - // service declares a custom container_name - return name - } - return name[len(project)+1:] -} - -// projectFromName builds a types.Project based on actual resources with compose labels set -func (s *composeService) projectFromName(containers Containers, projectName string, services ...string) (*types.Project, error) { - project := &types.Project{ - Name: projectName, - Services: types.Services{}, - } - if len(containers) == 0 { - return project, fmt.Errorf("no container found for project %q: %w", projectName, api.ErrNotFound) - } - set := types.Services{} - for _, c := range containers { - serviceLabel, ok := c.Labels[api.ServiceLabel] - if !ok { - serviceLabel = getCanonicalContainerName(c) - } - service, ok := set[serviceLabel] - if !ok { - service = types.ServiceConfig{ - Name: serviceLabel, - Image: c.Image, - Labels: c.Labels, - } - } - service.Scale = increment(service.Scale) - set[serviceLabel] = service - } - for name, service := range set { - dependencies := service.Labels[api.DependenciesLabel] - if dependencies != "" { - service.DependsOn = types.DependsOnConfig{} - for dc := range strings.SplitSeq(dependencies, ",") { - dcArr := strings.Split(dc, ":") - condition := ServiceConditionRunningOrHealthy - // Let's restart the dependency by default if we don't have the info stored in the label - restart := true - required := true - dependency := dcArr[0] - - // backward compatibility - if len(dcArr) > 1 { - condition = dcArr[1] - if len(dcArr) > 2 { - restart, _ = strconv.ParseBool(dcArr[2]) - } - } - service.DependsOn[dependency] = types.ServiceDependency{Condition: condition, Restart: restart, Required: required} - } - set[name] = service - } - } - project.Services = set - -SERVICES: - for _, qs := range services { - for _, es := range project.Services { - if es.Name == qs { - continue SERVICES - } - } - return project, fmt.Errorf("no such service: %q: %w", qs, api.ErrNotFound) - } - project, err := project.WithSelectedServices(services) - if err != nil { - return project, err - } - - return project, nil -} - -func increment(scale *int) *int { - i := 1 - if scale != nil { - i = *scale + 1 - } - return &i -} - -func (s *composeService) actualVolumes(ctx context.Context, projectName string) (types.Volumes, error) { - opts := volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(projectName)), - } - volumes, err := s.apiClient().VolumeList(ctx, opts) - if err != nil { - return nil, err - } - - actual := types.Volumes{} - for _, vol := range volumes.Volumes { - actual[vol.Labels[api.VolumeLabel]] = types.VolumeConfig{ - Name: vol.Name, - Driver: vol.Driver, - Labels: vol.Labels, - } - } - return actual, nil -} - -func (s *composeService) actualNetworks(ctx context.Context, projectName string) (types.Networks, error) { - networks, err := s.apiClient().NetworkList(ctx, network.ListOptions{ - Filters: filters.NewArgs(projectFilter(projectName)), - }) - if err != nil { - return nil, err - } - - actual := types.Networks{} - for _, net := range networks { - actual[net.Labels[api.NetworkLabel]] = types.NetworkConfig{ - Name: net.Name, - Driver: net.Driver, - Labels: net.Labels, - } - } - return actual, nil -} - -var swarmEnabled = struct { - once sync.Once - val bool - err error -}{} - -func (s *composeService) isSwarmEnabled(ctx context.Context) (bool, error) { - swarmEnabled.once.Do(func() { - info, err := s.apiClient().Info(ctx) - if err != nil { - swarmEnabled.err = err - } - switch info.Swarm.LocalNodeState { - case swarm.LocalNodeStateInactive, swarm.LocalNodeStateLocked: - swarmEnabled.val = false - default: - swarmEnabled.val = true - } - }) - return swarmEnabled.val, swarmEnabled.err -} - -type runtimeVersionCache struct { - once sync.Once - val string - err error -} - -var runtimeVersion runtimeVersionCache - -func (s *composeService) RuntimeVersion(ctx context.Context) (string, error) { - runtimeVersion.once.Do(func() { - version, err := s.apiClient().ServerVersion(ctx) - if err != nil { - runtimeVersion.err = err - } - runtimeVersion.val = version.APIVersion - }) - return runtimeVersion.val, runtimeVersion.err -} diff --git a/pkg/compose/container.go b/pkg/compose/container.go deleted file mode 100644 index 502547ddc9b..00000000000 --- a/pkg/compose/container.go +++ /dev/null @@ -1,58 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "io" - - moby "github.com/docker/docker/api/types" -) - -var _ io.ReadCloser = ContainerStdout{} - -// ContainerStdout implement ReadCloser for moby.HijackedResponse -type ContainerStdout struct { - moby.HijackedResponse -} - -// Read implement io.ReadCloser -func (l ContainerStdout) Read(p []byte) (n int, err error) { - return l.Reader.Read(p) -} - -// Close implement io.ReadCloser -func (l ContainerStdout) Close() error { - l.HijackedResponse.Close() - return nil -} - -var _ io.WriteCloser = ContainerStdin{} - -// ContainerStdin implement WriteCloser for moby.HijackedResponse -type ContainerStdin struct { - moby.HijackedResponse -} - -// Write implement io.WriteCloser -func (c ContainerStdin) Write(p []byte) (n int, err error) { - return c.Conn.Write(p) -} - -// Close implement io.WriteCloser -func (c ContainerStdin) Close() error { - return c.CloseWrite() -} diff --git a/pkg/compose/containers.go b/pkg/compose/containers.go deleted file mode 100644 index 50e8ad56016..00000000000 --- a/pkg/compose/containers.go +++ /dev/null @@ -1,182 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "slices" - "sort" - "strconv" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - - "github.com/docker/compose/v5/pkg/api" -) - -// Containers is a set of moby Container -type Containers []container.Summary - -type oneOff int - -const ( - oneOffInclude = oneOff(iota) - oneOffExclude - oneOffOnly -) - -func (s *composeService) getContainers(ctx context.Context, project string, oneOff oneOff, all bool, selectedServices ...string) (Containers, error) { - var containers Containers - f := getDefaultFilters(project, oneOff, selectedServices...) - containers, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - Filters: filters.NewArgs(f...), - All: all, - }) - if err != nil { - return nil, err - } - if len(selectedServices) > 1 { - containers = containers.filter(isService(selectedServices...)) - } - return containers, nil -} - -func getDefaultFilters(projectName string, oneOff oneOff, selectedServices ...string) []filters.KeyValuePair { - f := []filters.KeyValuePair{projectFilter(projectName)} - if len(selectedServices) == 1 { - f = append(f, serviceFilter(selectedServices[0])) - } - f = append(f, hasConfigHashLabel()) - switch oneOff { - case oneOffOnly: - f = append(f, oneOffFilter(true)) - case oneOffExclude: - f = append(f, oneOffFilter(false)) - case oneOffInclude: - } - return f -} - -func (s *composeService) getSpecifiedContainer(ctx context.Context, projectName string, oneOff oneOff, all bool, serviceName string, containerIndex int) (container.Summary, error) { - defaultFilters := getDefaultFilters(projectName, oneOff, serviceName) - if containerIndex > 0 { - defaultFilters = append(defaultFilters, containerNumberFilter(containerIndex)) - } - containers, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - Filters: filters.NewArgs( - defaultFilters..., - ), - All: all, - }) - if err != nil { - return container.Summary{}, err - } - if len(containers) < 1 { - if containerIndex > 0 { - return container.Summary{}, fmt.Errorf("service %q is not running container #%d", serviceName, containerIndex) - } - return container.Summary{}, fmt.Errorf("service %q is not running", serviceName) - } - - // Sort by container number first, then put one-off containers at the end - sort.Slice(containers, func(i, j int) bool { - numberLabelX, _ := strconv.Atoi(containers[i].Labels[api.ContainerNumberLabel]) - numberLabelY, _ := strconv.Atoi(containers[j].Labels[api.ContainerNumberLabel]) - IsOneOffLabelTrueX := containers[i].Labels[api.OneoffLabel] == "True" - IsOneOffLabelTrueY := containers[j].Labels[api.OneoffLabel] == "True" - - if IsOneOffLabelTrueX || IsOneOffLabelTrueY { - return !IsOneOffLabelTrueX && IsOneOffLabelTrueY - } - - return numberLabelX < numberLabelY - }) - return containers[0], nil -} - -// containerPredicate define a predicate we want container to satisfy for filtering operations -type containerPredicate func(c container.Summary) bool - -func matches(c container.Summary, predicates ...containerPredicate) bool { - for _, predicate := range predicates { - if !predicate(c) { - return false - } - } - return true -} - -func isService(services ...string) containerPredicate { - return func(c container.Summary) bool { - service := c.Labels[api.ServiceLabel] - return slices.Contains(services, service) - } -} - -// isOrphaned is a predicate to select containers without a matching service definition in compose project -func isOrphaned(project *types.Project) containerPredicate { - services := append(project.ServiceNames(), project.DisabledServiceNames()...) - return func(c container.Summary) bool { - // One-off container - v, ok := c.Labels[api.OneoffLabel] - if ok && v == "True" { - return c.State == container.StateExited || c.State == container.StateDead - } - // Service that is not defined in the compose model - service := c.Labels[api.ServiceLabel] - return !slices.Contains(services, service) - } -} - -func isNotOneOff(c container.Summary) bool { - v, ok := c.Labels[api.OneoffLabel] - return !ok || v == "False" -} - -// filter return Containers with elements to match predicate -func (containers Containers) filter(predicates ...containerPredicate) Containers { - var filtered Containers - for _, c := range containers { - if matches(c, predicates...) { - filtered = append(filtered, c) - } - } - return filtered -} - -func (containers Containers) names() []string { - var names []string - for _, c := range containers { - names = append(names, getCanonicalContainerName(c)) - } - return names -} - -func (containers Containers) forEach(fn func(container.Summary)) { - for _, c := range containers { - fn(c) - } -} - -func (containers Containers) sorted() Containers { - sort.Slice(containers, func(i, j int) bool { - return getCanonicalContainerName(containers[i]) < getCanonicalContainerName(containers[j]) - }) - return containers -} diff --git a/pkg/compose/convergence.go b/pkg/compose/convergence.go deleted file mode 100644 index baf34a227f7..00000000000 --- a/pkg/compose/convergence.go +++ /dev/null @@ -1,930 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "maps" - "slices" - "sort" - "strconv" - "strings" - "sync" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/platforms" - "github.com/docker/docker/api/types/container" - mmount "github.com/docker/docker/api/types/mount" - "github.com/docker/docker/api/types/versions" - specs "github.com/opencontainers/image-spec/specs-go/v1" - "github.com/sirupsen/logrus" - "go.opentelemetry.io/otel/attribute" - "go.opentelemetry.io/otel/trace" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -const ( - doubledContainerNameWarning = "WARNING: The %q service is using the custom container name %q. " + - "Docker requires each container to have a unique name. " + - "Remove the custom name to scale the service" -) - -// convergence manages service's container lifecycle. -// Based on initially observed state, it reconciles the existing container with desired state, which might include -// re-creating container, adding or removing replicas, or starting stopped containers. -// Cross services dependencies are managed by creating services in expected order and updating `service:xx` reference -// when a service has converged, so dependent ones can be managed with resolved containers references. -type convergence struct { - compose *composeService - services map[string]Containers - networks map[string]string - volumes map[string]string - stateMutex sync.Mutex -} - -func (c *convergence) getObservedState(serviceName string) Containers { - c.stateMutex.Lock() - defer c.stateMutex.Unlock() - return c.services[serviceName] -} - -func (c *convergence) setObservedState(serviceName string, containers Containers) { - c.stateMutex.Lock() - defer c.stateMutex.Unlock() - c.services[serviceName] = containers -} - -func newConvergence(services []string, state Containers, networks map[string]string, volumes map[string]string, s *composeService) *convergence { - observedState := map[string]Containers{} - for _, s := range services { - observedState[s] = Containers{} - } - for _, c := range state.filter(isNotOneOff) { - service := c.Labels[api.ServiceLabel] - observedState[service] = append(observedState[service], c) - } - return &convergence{ - compose: s, - services: observedState, - networks: networks, - volumes: volumes, - } -} - -func (c *convergence) apply(ctx context.Context, project *types.Project, options api.CreateOptions) error { - return InDependencyOrder(ctx, project, func(ctx context.Context, name string) error { - service, err := project.GetService(name) - if err != nil { - return err - } - - return tracing.SpanWrapFunc("service/apply", tracing.ServiceOptions(service), func(ctx context.Context) error { - strategy := options.RecreateDependencies - if slices.Contains(options.Services, name) { - strategy = options.Recreate - } - return c.ensureService(ctx, project, service, strategy, options.Inherit, options.Timeout) - })(ctx) - }) -} - -func (c *convergence) ensureService(ctx context.Context, project *types.Project, service types.ServiceConfig, recreate string, inherit bool, timeout *time.Duration) error { //nolint:gocyclo - if service.Provider != nil { - return c.compose.runPlugin(ctx, project, service, "up") - } - expected, err := getScale(service) - if err != nil { - return err - } - containers := c.getObservedState(service.Name) - actual := len(containers) - updated := make(Containers, expected) - - eg, ctx := errgroup.WithContext(ctx) - - err = c.resolveServiceReferences(&service) - if err != nil { - return err - } - - sort.Slice(containers, func(i, j int) bool { - // select obsolete containers first, so they get removed as we scale down - if obsolete, _ := c.mustRecreate(service, containers[i], recreate); obsolete { - // i is obsolete, so must be first in the list - return true - } - if obsolete, _ := c.mustRecreate(service, containers[j], recreate); obsolete { - // j is obsolete, so must be first in the list - return false - } - - // For up-to-date containers, sort by container number to preserve low-values in container numbers - ni, erri := strconv.Atoi(containers[i].Labels[api.ContainerNumberLabel]) - nj, errj := strconv.Atoi(containers[j].Labels[api.ContainerNumberLabel]) - if erri == nil && errj == nil { - return ni > nj - } - - // If we don't get a container number (?) just sort by creation date - return containers[i].Created < containers[j].Created - }) - - slices.Reverse(containers) - for i, ctr := range containers { - if i >= expected { - // Scale Down - // As we sorted containers, obsolete ones and/or highest number will be removed - ctr := ctr - traceOpts := append(tracing.ServiceOptions(service), tracing.ContainerOptions(ctr)...) - eg.Go(tracing.SpanWrapFuncForErrGroup(ctx, "service/scale/down", traceOpts, func(ctx context.Context) error { - return c.compose.stopAndRemoveContainer(ctx, ctr, &service, timeout, false) - })) - continue - } - - mustRecreate, err := c.mustRecreate(service, ctr, recreate) - if err != nil { - return err - } - if mustRecreate { - err := c.stopDependentContainers(ctx, project, service) - if err != nil { - return err - } - - i, ctr := i, ctr - eg.Go(tracing.SpanWrapFuncForErrGroup(ctx, "container/recreate", tracing.ContainerOptions(ctr), func(ctx context.Context) error { - recreated, err := c.compose.recreateContainer(ctx, project, service, ctr, inherit, timeout) - updated[i] = recreated - return err - })) - continue - } - - // Enforce non-diverged containers are running - name := getContainerProgressName(ctr) - switch ctr.State { - case container.StateRunning: - c.compose.events.On(runningEvent(name)) - case container.StateCreated: - case container.StateRestarting: - case container.StateExited: - default: - ctr := ctr - eg.Go(tracing.EventWrapFuncForErrGroup(ctx, "service/start", tracing.ContainerOptions(ctr), func(ctx context.Context) error { - return c.compose.startContainer(ctx, ctr) - })) - } - updated[i] = ctr - } - - next := nextContainerNumber(containers) - for i := 0; i < expected-actual; i++ { - // Scale UP - number := next + i - name := getContainerName(project.Name, service, number) - eventOpts := tracing.SpanOptions{trace.WithAttributes(attribute.String("container.name", name))} - eg.Go(tracing.EventWrapFuncForErrGroup(ctx, "service/scale/up", eventOpts, func(ctx context.Context) error { - opts := createOptions{ - AutoRemove: false, - AttachStdin: false, - UseNetworkAliases: true, - Labels: mergeLabels(service.Labels, service.CustomLabels), - } - ctr, err := c.compose.createContainer(ctx, project, service, name, number, opts) - updated[actual+i] = ctr - return err - })) - continue - } - - err = eg.Wait() - c.setObservedState(service.Name, updated) - return err -} - -func (c *convergence) stopDependentContainers(ctx context.Context, project *types.Project, service types.ServiceConfig) error { - // Stop dependent containers, so they will be restarted after service is re-created - dependents := project.GetDependentsForService(service, func(dependency types.ServiceDependency) bool { - return dependency.Restart - }) - if len(dependents) == 0 { - return nil - } - err := c.compose.stop(ctx, project.Name, api.StopOptions{ - Services: dependents, - Project: project, - }, nil) - if err != nil { - return err - } - - for _, name := range dependents { - dependentStates := c.getObservedState(name) - for i, dependent := range dependentStates { - dependent.State = container.StateExited - dependentStates[i] = dependent - } - c.setObservedState(name, dependentStates) - } - return nil -} - -func getScale(config types.ServiceConfig) (int, error) { - scale := config.GetScale() - if scale > 1 && config.ContainerName != "" { - return 0, fmt.Errorf(doubledContainerNameWarning, - config.Name, - config.ContainerName) - } - return scale, nil -} - -// resolveServiceReferences replaces reference to another service with reference to an actual container -func (c *convergence) resolveServiceReferences(service *types.ServiceConfig) error { - err := c.resolveVolumeFrom(service) - if err != nil { - return err - } - - err = c.resolveSharedNamespaces(service) - if err != nil { - return err - } - return nil -} - -func (c *convergence) resolveVolumeFrom(service *types.ServiceConfig) error { - for i, vol := range service.VolumesFrom { - spec := strings.Split(vol, ":") - if len(spec) == 0 { - continue - } - if spec[0] == "container" { - service.VolumesFrom[i] = spec[1] - continue - } - name := spec[0] - dependencies := c.getObservedState(name) - if len(dependencies) == 0 { - return fmt.Errorf("cannot share volume with service %s: container missing", name) - } - service.VolumesFrom[i] = dependencies.sorted()[0].ID - } - return nil -} - -func (c *convergence) resolveSharedNamespaces(service *types.ServiceConfig) error { - str := service.NetworkMode - if name := getDependentServiceFromMode(str); name != "" { - dependencies := c.getObservedState(name) - if len(dependencies) == 0 { - return fmt.Errorf("cannot share network namespace with service %s: container missing", name) - } - service.NetworkMode = types.ContainerPrefix + dependencies.sorted()[0].ID - } - - str = service.Ipc - if name := getDependentServiceFromMode(str); name != "" { - dependencies := c.getObservedState(name) - if len(dependencies) == 0 { - return fmt.Errorf("cannot share IPC namespace with service %s: container missing", name) - } - service.Ipc = types.ContainerPrefix + dependencies.sorted()[0].ID - } - - str = service.Pid - if name := getDependentServiceFromMode(str); name != "" { - dependencies := c.getObservedState(name) - if len(dependencies) == 0 { - return fmt.Errorf("cannot share PID namespace with service %s: container missing", name) - } - service.Pid = types.ContainerPrefix + dependencies.sorted()[0].ID - } - - return nil -} - -func (c *convergence) mustRecreate(expected types.ServiceConfig, actual container.Summary, policy string) (bool, error) { - if policy == api.RecreateNever { - return false, nil - } - if policy == api.RecreateForce { - return true, nil - } - configHash, err := ServiceHash(expected) - if err != nil { - return false, err - } - configChanged := actual.Labels[api.ConfigHashLabel] != configHash - imageUpdated := actual.Labels[api.ImageDigestLabel] != expected.CustomLabels[api.ImageDigestLabel] - if configChanged || imageUpdated { - return true, nil - } - - if c.networks != nil && actual.State == "running" { - if checkExpectedNetworks(expected, actual, c.networks) { - return true, nil - } - } - - if c.volumes != nil { - if checkExpectedVolumes(expected, actual, c.volumes) { - return true, nil - } - } - - return false, nil -} - -func checkExpectedNetworks(expected types.ServiceConfig, actual container.Summary, networks map[string]string) bool { - // check the networks container is connected to are the expected ones - for net := range expected.Networks { - id := networks[net] - if id == "swarm" { - // corner-case : swarm overlay network isn't visible until a container is attached - continue - } - found := false - for _, settings := range actual.NetworkSettings.Networks { - if settings.NetworkID == id { - found = true - break - } - } - if !found { - // config is up-to-date but container is not connected to network - return true - } - } - return false -} - -func checkExpectedVolumes(expected types.ServiceConfig, actual container.Summary, volumes map[string]string) bool { - // check container's volume mounts and search for the expected ones - for _, vol := range expected.Volumes { - if vol.Type != string(mmount.TypeVolume) { - continue - } - if vol.Source == "" { - continue - } - id := volumes[vol.Source] - found := false - for _, mount := range actual.Mounts { - if mount.Type != mmount.TypeVolume { - continue - } - if mount.Name == id { - found = true - break - } - } - if !found { - // config is up-to-date but container doesn't have volume mounted - return true - } - } - return false -} - -func getContainerName(projectName string, service types.ServiceConfig, number int) string { - name := getDefaultContainerName(projectName, service.Name, strconv.Itoa(number)) - if service.ContainerName != "" { - name = service.ContainerName - } - return name -} - -func getDefaultContainerName(projectName, serviceName, index string) string { - return strings.Join([]string{projectName, serviceName, index}, api.Separator) -} - -func getContainerProgressName(ctr container.Summary) string { - return "Container " + getCanonicalContainerName(ctr) -} - -func containerEvents(containers Containers, eventFunc func(string) api.Resource) []api.Resource { - events := []api.Resource{} - for _, ctr := range containers { - events = append(events, eventFunc(getContainerProgressName(ctr))) - } - return events -} - -func containerReasonEvents(containers Containers, eventFunc func(string, string) api.Resource, reason string) []api.Resource { - events := []api.Resource{} - for _, ctr := range containers { - events = append(events, eventFunc(getContainerProgressName(ctr), reason)) - } - return events -} - -// ServiceConditionRunningOrHealthy is a service condition on status running or healthy -const ServiceConditionRunningOrHealthy = "running_or_healthy" - -//nolint:gocyclo -func (s *composeService) waitDependencies(ctx context.Context, project *types.Project, dependant string, dependencies types.DependsOnConfig, containers Containers, timeout time.Duration) error { - if timeout > 0 { - withTimeout, cancelFunc := context.WithTimeout(ctx, timeout) - defer cancelFunc() - ctx = withTimeout - } - eg, ctx := errgroup.WithContext(ctx) - for dep, config := range dependencies { - if shouldWait, err := shouldWaitForDependency(dep, config, project); err != nil { - return err - } else if !shouldWait { - continue - } - - waitingFor := containers.filter(isService(dep), isNotOneOff) - s.events.On(containerEvents(waitingFor, waiting)...) - if len(waitingFor) == 0 { - if config.Required { - return fmt.Errorf("%s is missing dependency %s", dependant, dep) - } - logrus.Warnf("%s is missing dependency %s", dependant, dep) - continue - } - - eg.Go(func() error { - ticker := time.NewTicker(500 * time.Millisecond) - defer ticker.Stop() - for { - select { - case <-ticker.C: - case <-ctx.Done(): - return nil - } - switch config.Condition { - case ServiceConditionRunningOrHealthy: - isHealthy, err := s.isServiceHealthy(ctx, waitingFor, true) - if err != nil { - if !config.Required { - s.events.On(containerReasonEvents(waitingFor, skippedEvent, - fmt.Sprintf("optional dependency %q is not running or is unhealthy", dep))...) - logrus.Warnf("optional dependency %q is not running or is unhealthy: %s", dep, err.Error()) - return nil - } - return err - } - if isHealthy { - s.events.On(containerEvents(waitingFor, healthy)...) - return nil - } - case types.ServiceConditionHealthy: - isHealthy, err := s.isServiceHealthy(ctx, waitingFor, false) - if err != nil { - if !config.Required { - s.events.On(containerReasonEvents(waitingFor, skippedEvent, - fmt.Sprintf("optional dependency %q failed to start", dep))...) - logrus.Warnf("optional dependency %q failed to start: %s", dep, err.Error()) - return nil - } - s.events.On(containerEvents(waitingFor, func(s string) api.Resource { - return errorEventf(s, "dependency %s failed to start", dep) - })...) - return fmt.Errorf("dependency failed to start: %w", err) - } - if isHealthy { - s.events.On(containerEvents(waitingFor, healthy)...) - return nil - } - case types.ServiceConditionCompletedSuccessfully: - isExited, code, err := s.isServiceCompleted(ctx, waitingFor) - if err != nil { - return err - } - if isExited { - if code == 0 { - s.events.On(containerEvents(waitingFor, exited)...) - return nil - } - - messageSuffix := fmt.Sprintf("%q didn't complete successfully: exit %d", dep, code) - if !config.Required { - // optional -> mark as skipped & don't propagate error - s.events.On(containerReasonEvents(waitingFor, skippedEvent, - fmt.Sprintf("optional dependency %s", messageSuffix))...) - logrus.Warnf("optional dependency %s", messageSuffix) - return nil - } - - msg := fmt.Sprintf("service %s", messageSuffix) - s.events.On(containerEvents(waitingFor, func(s string) api.Resource { - return errorEventf(s, "service %s", messageSuffix) - })...) - return errors.New(msg) - } - default: - logrus.Warnf("unsupported depends_on condition: %s", config.Condition) - return nil - } - } - }) - } - err := eg.Wait() - if errors.Is(err, context.DeadlineExceeded) { - return fmt.Errorf("timeout waiting for dependencies") - } - return err -} - -func shouldWaitForDependency(serviceName string, dependencyConfig types.ServiceDependency, project *types.Project) (bool, error) { - if dependencyConfig.Condition == types.ServiceConditionStarted { - // already managed by InDependencyOrder - return false, nil - } - if service, err := project.GetService(serviceName); err != nil { - for _, ds := range project.DisabledServices { - if ds.Name == serviceName { - // don't wait for disabled service (--no-deps) - return false, nil - } - } - return false, err - } else if service.GetScale() == 0 { - // don't wait for the dependency which configured to have 0 containers running - return false, nil - } else if service.Provider != nil { - // don't wait for provider services - return false, nil - } - return true, nil -} - -func nextContainerNumber(containers []container.Summary) int { - maxNumber := 0 - for _, c := range containers { - s, ok := c.Labels[api.ContainerNumberLabel] - if !ok { - logrus.Warnf("container %s is missing %s label", c.ID, api.ContainerNumberLabel) - } - n, err := strconv.Atoi(s) - if err != nil { - logrus.Warnf("container %s has invalid %s label: %s", c.ID, api.ContainerNumberLabel, s) - continue - } - if n > maxNumber { - maxNumber = n - } - } - return maxNumber + 1 -} - -func (s *composeService) createContainer(ctx context.Context, project *types.Project, service types.ServiceConfig, - name string, number int, opts createOptions, -) (ctr container.Summary, err error) { - eventName := "Container " + name - s.events.On(creatingEvent(eventName)) - ctr, err = s.createMobyContainer(ctx, project, service, name, number, nil, opts) - if err != nil { - if ctx.Err() == nil { - s.events.On(api.Resource{ - ID: eventName, - Status: api.Error, - Text: err.Error(), - }) - } - return ctr, err - } - s.events.On(createdEvent(eventName)) - return ctr, nil -} - -func (s *composeService) recreateContainer(ctx context.Context, project *types.Project, service types.ServiceConfig, - replaced container.Summary, inherit bool, timeout *time.Duration, -) (created container.Summary, err error) { - eventName := getContainerProgressName(replaced) - s.events.On(newEvent(eventName, api.Working, "Recreate")) - defer func() { - if err != nil && ctx.Err() == nil { - s.events.On(api.Resource{ - ID: eventName, - Status: api.Error, - Text: err.Error(), - }) - } - }() - - number, err := strconv.Atoi(replaced.Labels[api.ContainerNumberLabel]) - if err != nil { - return created, err - } - - var inherited *container.Summary - if inherit { - inherited = &replaced - } - - replacedContainerName := service.ContainerName - if replacedContainerName == "" { - replacedContainerName = service.Name + api.Separator + strconv.Itoa(number) - } - name := getContainerName(project.Name, service, number) - tmpName := fmt.Sprintf("%s_%s", replaced.ID[:12], name) - opts := createOptions{ - AutoRemove: false, - AttachStdin: false, - UseNetworkAliases: true, - Labels: mergeLabels(service.Labels, service.CustomLabels).Add(api.ContainerReplaceLabel, replacedContainerName), - } - created, err = s.createMobyContainer(ctx, project, service, tmpName, number, inherited, opts) - if err != nil { - return created, err - } - - timeoutInSecond := utils.DurationSecondToInt(timeout) - err = s.apiClient().ContainerStop(ctx, replaced.ID, container.StopOptions{Timeout: timeoutInSecond}) - if err != nil { - return created, err - } - - err = s.apiClient().ContainerRemove(ctx, replaced.ID, container.RemoveOptions{}) - if err != nil { - return created, err - } - - err = s.apiClient().ContainerRename(ctx, tmpName, name) - if err != nil { - return created, err - } - - s.events.On(newEvent(eventName, api.Done, "Recreated")) - return created, err -} - -// force sequential calls to ContainerStart to prevent race condition in engine assigning ports from ranges -var startMx sync.Mutex - -func (s *composeService) startContainer(ctx context.Context, ctr container.Summary) error { - s.events.On(newEvent(getContainerProgressName(ctr), api.Working, "Restart")) - startMx.Lock() - defer startMx.Unlock() - err := s.apiClient().ContainerStart(ctx, ctr.ID, container.StartOptions{}) - if err != nil { - return err - } - s.events.On(newEvent(getContainerProgressName(ctr), api.Done, "Restarted")) - return nil -} - -func (s *composeService) createMobyContainer(ctx context.Context, project *types.Project, service types.ServiceConfig, - name string, number int, inherit *container.Summary, opts createOptions, -) (container.Summary, error) { - var created container.Summary - cfgs, err := s.getCreateConfigs(ctx, project, service, number, inherit, opts) - if err != nil { - return created, err - } - platform := service.Platform - if platform == "" { - platform = project.Environment["DOCKER_DEFAULT_PLATFORM"] - } - var plat *specs.Platform - if platform != "" { - var p specs.Platform - p, err = platforms.Parse(platform) - if err != nil { - return created, err - } - plat = &p - } - - response, err := s.apiClient().ContainerCreate(ctx, cfgs.Container, cfgs.Host, cfgs.Network, plat, name) - if err != nil { - return created, err - } - for _, warning := range response.Warnings { - s.events.On(api.Resource{ - ID: service.Name, - Status: api.Warning, - Text: warning, - }) - } - inspectedContainer, err := s.apiClient().ContainerInspect(ctx, response.ID) - if err != nil { - return created, err - } - created = container.Summary{ - ID: inspectedContainer.ID, - Labels: inspectedContainer.Config.Labels, - Names: []string{inspectedContainer.Name}, - NetworkSettings: &container.NetworkSettingsSummary{ - Networks: inspectedContainer.NetworkSettings.Networks, - }, - } - - apiVersion, err := s.RuntimeVersion(ctx) - if err != nil { - return created, err - } - // Starting API version 1.44, the ContainerCreate API call takes multiple networks - // so we include all the configurations there and can skip the one-by-one calls here - if versions.LessThan(apiVersion, APIVersion144) { - // the highest-priority network is the primary and is included in the ContainerCreate API - // call via container.NetworkMode & network.NetworkingConfig - // any remaining networks are connected one-by-one here after creation (but before start) - serviceNetworks := service.NetworksByPriority() - for _, networkKey := range serviceNetworks { - mobyNetworkName := project.Networks[networkKey].Name - if string(cfgs.Host.NetworkMode) == mobyNetworkName { - // primary network already configured as part of ContainerCreate - continue - } - epSettings := createEndpointSettings(project, service, number, networkKey, cfgs.Links, opts.UseNetworkAliases) - if err := s.apiClient().NetworkConnect(ctx, mobyNetworkName, created.ID, epSettings); err != nil { - return created, err - } - } - } - return created, nil -} - -// getLinks mimics V1 compose/service.py::Service::_get_links() -func (s *composeService) getLinks(ctx context.Context, projectName string, service types.ServiceConfig, number int) ([]string, error) { - var links []string - format := func(k, v string) string { - return fmt.Sprintf("%s:%s", k, v) - } - getServiceContainers := func(serviceName string) (Containers, error) { - return s.getContainers(ctx, projectName, oneOffExclude, true, serviceName) - } - - for _, rawLink := range service.Links { - // linkName if informed like in: "serviceName[:linkName]" - linkServiceName, linkName, ok := strings.Cut(rawLink, ":") - if !ok { - linkName = linkServiceName - } - cnts, err := getServiceContainers(linkServiceName) - if err != nil { - return nil, err - } - for _, c := range cnts { - containerName := getCanonicalContainerName(c) - links = append(links, - format(containerName, linkName), - format(containerName, linkServiceName+api.Separator+strconv.Itoa(number)), - format(containerName, strings.Join([]string{projectName, linkServiceName, strconv.Itoa(number)}, api.Separator)), - ) - } - } - - if service.Labels[api.OneoffLabel] == "True" { - cnts, err := getServiceContainers(service.Name) - if err != nil { - return nil, err - } - for _, c := range cnts { - containerName := getCanonicalContainerName(c) - links = append(links, - format(containerName, service.Name), - format(containerName, strings.TrimPrefix(containerName, projectName+api.Separator)), - format(containerName, containerName), - ) - } - } - - for _, rawExtLink := range service.ExternalLinks { - externalLink, linkName, ok := strings.Cut(rawExtLink, ":") - if !ok { - linkName = externalLink - } - links = append(links, format(externalLink, linkName)) - } - return links, nil -} - -func (s *composeService) isServiceHealthy(ctx context.Context, containers Containers, fallbackRunning bool) (bool, error) { - for _, c := range containers { - ctr, err := s.apiClient().ContainerInspect(ctx, c.ID) - if err != nil { - return false, err - } - name := ctr.Name[1:] - - if ctr.State.Status == container.StateExited { - return false, fmt.Errorf("container %s exited (%d)", name, ctr.State.ExitCode) - } - - noHealthcheck := ctr.Config.Healthcheck == nil || (len(ctr.Config.Healthcheck.Test) > 0 && ctr.Config.Healthcheck.Test[0] == "NONE") - if noHealthcheck && fallbackRunning { - // Container does not define a health check, but we can fall back to "running" state - return ctr.State != nil && ctr.State.Status == container.StateRunning, nil - } - - if ctr.State == nil || ctr.State.Health == nil { - return false, fmt.Errorf("container %s has no healthcheck configured", name) - } - switch ctr.State.Health.Status { - case container.Healthy: - // Continue by checking the next container. - case container.Unhealthy: - return false, fmt.Errorf("container %s is unhealthy", name) - case container.Starting: - return false, nil - default: - return false, fmt.Errorf("container %s had unexpected health status %q", name, ctr.State.Health.Status) - } - } - return true, nil -} - -func (s *composeService) isServiceCompleted(ctx context.Context, containers Containers) (bool, int, error) { - for _, c := range containers { - ctr, err := s.apiClient().ContainerInspect(ctx, c.ID) - if err != nil { - return false, 0, err - } - if ctr.State != nil && ctr.State.Status == container.StateExited { - return true, ctr.State.ExitCode, nil - } - } - return false, 0, nil -} - -func (s *composeService) startService(ctx context.Context, - project *types.Project, service types.ServiceConfig, - containers Containers, listener api.ContainerEventListener, - timeout time.Duration, -) error { - if service.Deploy != nil && service.Deploy.Replicas != nil && *service.Deploy.Replicas == 0 { - return nil - } - - err := s.waitDependencies(ctx, project, service.Name, service.DependsOn, containers, timeout) - if err != nil { - return err - } - - if len(containers) == 0 { - if service.GetScale() == 0 { - return nil - } - return fmt.Errorf("service %q has no container to start", service.Name) - } - - for _, ctr := range containers.filter(isService(service.Name)) { - if ctr.State == container.StateRunning { - continue - } - - err = s.injectSecrets(ctx, project, service, ctr.ID) - if err != nil { - return err - } - - err = s.injectConfigs(ctx, project, service, ctr.ID) - if err != nil { - return err - } - - eventName := getContainerProgressName(ctr) - s.events.On(startingEvent(eventName)) - err = s.apiClient().ContainerStart(ctx, ctr.ID, container.StartOptions{}) - if err != nil { - return err - } - - for _, hook := range service.PostStart { - err = s.runHook(ctx, ctr, service, hook, listener) - if err != nil { - return err - } - } - - s.events.On(startedEvent(eventName)) - } - return nil -} - -func mergeLabels(ls ...types.Labels) types.Labels { - merged := types.Labels{} - for _, l := range ls { - maps.Copy(merged, l) - } - return merged -} diff --git a/pkg/compose/convergence_test.go b/pkg/compose/convergence_test.go deleted file mode 100644 index eb4c1adfbde..00000000000 --- a/pkg/compose/convergence_test.go +++ /dev/null @@ -1,568 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "fmt" - "strings" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/config/configfile" - moby "github.com/docker/docker/api/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/api/types/network" - "github.com/docker/go-connections/nat" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/mocks" -) - -func TestContainerName(t *testing.T) { - s := types.ServiceConfig{ - Name: "testservicename", - ContainerName: "testcontainername", - Scale: intPtr(1), - Deploy: &types.DeployConfig{}, - } - ret, err := getScale(s) - assert.NilError(t, err) - assert.Equal(t, ret, *s.Scale) - - s.Scale = intPtr(0) - ret, err = getScale(s) - assert.NilError(t, err) - assert.Equal(t, ret, *s.Scale) - - s.Scale = intPtr(2) - _, err = getScale(s) - assert.Error(t, err, fmt.Sprintf(doubledContainerNameWarning, s.Name, s.ContainerName)) -} - -func intPtr(i int) *int { - return &i -} - -func TestServiceLinks(t *testing.T) { - const dbContainerName = "/" + testProject + "-db-1" - const webContainerName = "/" + testProject + "-web-1" - s := types.ServiceConfig{ - Name: "web", - Scale: intPtr(1), - } - - containerListOptions := container.ListOptions{ - Filters: filters.NewArgs( - projectFilter(testProject), - serviceFilter("db"), - oneOffFilter(false), - hasConfigHashLabel(), - ), - All: true, - } - - t.Run("service links default", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - s.Links = []string{"db"} - - c := testContainer("db", dbContainerName, false) - apiClient.EXPECT().ContainerList(gomock.Any(), containerListOptions).Return([]container.Summary{c}, nil) - - links, err := tested.(*composeService).getLinks(t.Context(), testProject, s, 1) - assert.NilError(t, err) - - assert.Equal(t, len(links), 3) - assert.Equal(t, links[0], "testProject-db-1:db") - assert.Equal(t, links[1], "testProject-db-1:db-1") - assert.Equal(t, links[2], "testProject-db-1:testProject-db-1") - }) - - t.Run("service links", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - s.Links = []string{"db:db"} - - c := testContainer("db", dbContainerName, false) - - apiClient.EXPECT().ContainerList(gomock.Any(), containerListOptions).Return([]container.Summary{c}, nil) - links, err := tested.(*composeService).getLinks(t.Context(), testProject, s, 1) - assert.NilError(t, err) - - assert.Equal(t, len(links), 3) - assert.Equal(t, links[0], "testProject-db-1:db") - assert.Equal(t, links[1], "testProject-db-1:db-1") - assert.Equal(t, links[2], "testProject-db-1:testProject-db-1") - }) - - t.Run("service links name", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - s.Links = []string{"db:dbname"} - - c := testContainer("db", dbContainerName, false) - apiClient.EXPECT().ContainerList(gomock.Any(), containerListOptions).Return([]container.Summary{c}, nil) - - links, err := tested.(*composeService).getLinks(t.Context(), testProject, s, 1) - assert.NilError(t, err) - - assert.Equal(t, len(links), 3) - assert.Equal(t, links[0], "testProject-db-1:dbname") - assert.Equal(t, links[1], "testProject-db-1:db-1") - assert.Equal(t, links[2], "testProject-db-1:testProject-db-1") - }) - - t.Run("service links external links", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - s.Links = []string{"db:dbname"} - s.ExternalLinks = []string{"db1:db2"} - - c := testContainer("db", dbContainerName, false) - apiClient.EXPECT().ContainerList(gomock.Any(), containerListOptions).Return([]container.Summary{c}, nil) - - links, err := tested.(*composeService).getLinks(t.Context(), testProject, s, 1) - assert.NilError(t, err) - - assert.Equal(t, len(links), 4) - assert.Equal(t, links[0], "testProject-db-1:dbname") - assert.Equal(t, links[1], "testProject-db-1:db-1") - assert.Equal(t, links[2], "testProject-db-1:testProject-db-1") - - // ExternalLink - assert.Equal(t, links[3], "db1:db2") - }) - - t.Run("service links itself oneoff", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - s.Links = []string{} - s.ExternalLinks = []string{} - s.Labels = s.Labels.Add(api.OneoffLabel, "True") - - c := testContainer("web", webContainerName, true) - containerListOptionsOneOff := container.ListOptions{ - Filters: filters.NewArgs( - projectFilter(testProject), - serviceFilter("web"), - oneOffFilter(false), - hasConfigHashLabel(), - ), - All: true, - } - apiClient.EXPECT().ContainerList(gomock.Any(), containerListOptionsOneOff).Return([]container.Summary{c}, nil) - - links, err := tested.(*composeService).getLinks(t.Context(), testProject, s, 1) - assert.NilError(t, err) - - assert.Equal(t, len(links), 3) - assert.Equal(t, links[0], "testProject-web-1:web") - assert.Equal(t, links[1], "testProject-web-1:web-1") - assert.Equal(t, links[2], "testProject-web-1:testProject-web-1") - }) -} - -func TestWaitDependencies(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - t.Run("should skip dependencies with scale 0", func(t *testing.T) { - dbService := types.ServiceConfig{Name: "db", Scale: intPtr(0)} - redisService := types.ServiceConfig{Name: "redis", Scale: intPtr(0)} - project := types.Project{Name: strings.ToLower(testProject), Services: types.Services{ - "db": dbService, - "redis": redisService, - }} - dependencies := types.DependsOnConfig{ - "db": {Condition: ServiceConditionRunningOrHealthy}, - "redis": {Condition: ServiceConditionRunningOrHealthy}, - } - assert.NilError(t, tested.(*composeService).waitDependencies(t.Context(), &project, "", dependencies, nil, 0)) - }) - t.Run("should skip dependencies with condition service_started", func(t *testing.T) { - dbService := types.ServiceConfig{Name: "db", Scale: intPtr(1)} - redisService := types.ServiceConfig{Name: "redis", Scale: intPtr(1)} - project := types.Project{Name: strings.ToLower(testProject), Services: types.Services{ - "db": dbService, - "redis": redisService, - }} - dependencies := types.DependsOnConfig{ - "db": {Condition: types.ServiceConditionStarted, Required: true}, - "redis": {Condition: types.ServiceConditionStarted, Required: true}, - } - assert.NilError(t, tested.(*composeService).waitDependencies(t.Context(), &project, "", dependencies, nil, 0)) - }) -} - -func TestIsServiceHealthy(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - ctx := t.Context() - - t.Run("disabled healthcheck with fallback to running", func(t *testing.T) { - containerID := "test-container-id" - containers := Containers{ - {ID: containerID}, - } - - // Container with disabled healthcheck (Test: ["NONE"]) - apiClient.EXPECT().ContainerInspect(ctx, containerID).Return(container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: containerID, - Name: "test-container", - State: &container.State{Status: "running"}, - }, - Config: &container.Config{ - Healthcheck: &container.HealthConfig{ - Test: []string{"NONE"}, - }, - }, - }, nil) - - isHealthy, err := tested.(*composeService).isServiceHealthy(ctx, containers, true) - assert.NilError(t, err) - assert.Equal(t, true, isHealthy, "Container with disabled healthcheck should be considered healthy when running with fallbackRunning=true") - }) - - t.Run("disabled healthcheck without fallback", func(t *testing.T) { - containerID := "test-container-id" - containers := Containers{ - {ID: containerID}, - } - - // Container with disabled healthcheck (Test: ["NONE"]) but fallbackRunning=false - apiClient.EXPECT().ContainerInspect(ctx, containerID).Return(container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: containerID, - Name: "test-container", - State: &container.State{Status: "running"}, - }, - Config: &container.Config{ - Healthcheck: &container.HealthConfig{ - Test: []string{"NONE"}, - }, - }, - }, nil) - - _, err := tested.(*composeService).isServiceHealthy(ctx, containers, false) - assert.ErrorContains(t, err, "has no healthcheck configured") - }) - - t.Run("no healthcheck with fallback to running", func(t *testing.T) { - containerID := "test-container-id" - containers := Containers{ - {ID: containerID}, - } - - // Container with no healthcheck at all - apiClient.EXPECT().ContainerInspect(ctx, containerID).Return(container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: containerID, - Name: "test-container", - State: &container.State{Status: "running"}, - }, - Config: &container.Config{ - Healthcheck: nil, - }, - }, nil) - - isHealthy, err := tested.(*composeService).isServiceHealthy(ctx, containers, true) - assert.NilError(t, err) - assert.Equal(t, true, isHealthy, "Container with no healthcheck should be considered healthy when running with fallbackRunning=true") - }) - - t.Run("exited container with disabled healthcheck", func(t *testing.T) { - containerID := "test-container-id" - containers := Containers{ - {ID: containerID}, - } - - // Container with disabled healthcheck but exited - apiClient.EXPECT().ContainerInspect(ctx, containerID).Return(container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: containerID, - Name: "test-container", - State: &container.State{ - Status: "exited", - ExitCode: 1, - }, - }, - Config: &container.Config{ - Healthcheck: &container.HealthConfig{ - Test: []string{"NONE"}, - }, - }, - }, nil) - - _, err := tested.(*composeService).isServiceHealthy(ctx, containers, true) - assert.ErrorContains(t, err, "exited") - }) - - t.Run("healthy container with healthcheck", func(t *testing.T) { - containerID := "test-container-id" - containers := Containers{ - {ID: containerID}, - } - - // Container with actual healthcheck that is healthy - apiClient.EXPECT().ContainerInspect(ctx, containerID).Return(container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: containerID, - Name: "test-container", - State: &container.State{ - Status: "running", - Health: &container.Health{ - Status: container.Healthy, - }, - }, - }, - Config: &container.Config{ - Healthcheck: &container.HealthConfig{ - Test: []string{"CMD", "curl", "-f", "http://localhost"}, - }, - }, - }, nil) - - isHealthy, err := tested.(*composeService).isServiceHealthy(ctx, containers, false) - assert.NilError(t, err) - assert.Equal(t, true, isHealthy, "Container with healthy status should be healthy") - }) -} - -func TestCreateMobyContainer(t *testing.T) { - t.Run("connects container networks one by one if API <1.44", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - cli.EXPECT().ConfigFile().Return(&configfile.ConfigFile{}).AnyTimes() - apiClient.EXPECT().DaemonHost().Return("").AnyTimes() - apiClient.EXPECT().ImageInspect(gomock.Any(), gomock.Any()).Return(image.InspectResponse{}, nil).AnyTimes() - // force `RuntimeVersion` to fetch again - runtimeVersion = runtimeVersionCache{} - apiClient.EXPECT().ServerVersion(gomock.Any()).Return(moby.Version{ - APIVersion: "1.43", - }, nil).AnyTimes() - - service := types.ServiceConfig{ - Name: "test", - Networks: map[string]*types.ServiceNetworkConfig{ - "a": { - Priority: 10, - }, - "b": { - Priority: 100, - }, - }, - } - project := types.Project{ - Name: "bork", - Services: types.Services{ - "test": service, - }, - Networks: types.Networks{ - "a": types.NetworkConfig{ - Name: "a-moby-name", - }, - "b": types.NetworkConfig{ - Name: "b-moby-name", - }, - }, - } - - var falseBool bool - apiClient.EXPECT().ContainerCreate(gomock.Any(), gomock.Any(), gomock.Eq( - &container.HostConfig{ - PortBindings: nat.PortMap{}, - ExtraHosts: []string{}, - Tmpfs: map[string]string{}, - Resources: container.Resources{ - OomKillDisable: &falseBool, - }, - NetworkMode: "b-moby-name", - }), gomock.Eq( - &network.NetworkingConfig{ - EndpointsConfig: map[string]*network.EndpointSettings{ - "b-moby-name": { - IPAMConfig: &network.EndpointIPAMConfig{}, - Aliases: []string{"bork-test-0"}, - }, - }, - }), gomock.Any(), gomock.Any()).Times(1).Return( - container.CreateResponse{ - ID: "an-id", - }, nil) - - apiClient.EXPECT().ContainerInspect(gomock.Any(), gomock.Eq("an-id")).Times(1).Return( - container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: "an-id", - Name: "a-name", - }, - Config: &container.Config{}, - NetworkSettings: &container.NetworkSettings{}, - }, nil) - - apiClient.EXPECT().NetworkConnect(gomock.Any(), "a-moby-name", "an-id", gomock.Eq( - &network.EndpointSettings{ - IPAMConfig: &network.EndpointIPAMConfig{}, - Aliases: []string{"bork-test-0"}, - })) - - _, err = tested.(*composeService).createMobyContainer(t.Context(), &project, service, "test", 0, nil, createOptions{ - Labels: make(types.Labels), - }) - assert.NilError(t, err) - }) - - t.Run("includes all container networks in ContainerCreate call if API >=1.44", func(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - apiClient := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - cli.EXPECT().Client().Return(apiClient).AnyTimes() - cli.EXPECT().ConfigFile().Return(&configfile.ConfigFile{}).AnyTimes() - apiClient.EXPECT().DaemonHost().Return("").AnyTimes() - apiClient.EXPECT().ImageInspect(gomock.Any(), gomock.Any()).Return(image.InspectResponse{}, nil).AnyTimes() - // force `RuntimeVersion` to fetch fresh version - runtimeVersion = runtimeVersionCache{} - apiClient.EXPECT().ServerVersion(gomock.Any()).Return(moby.Version{ - APIVersion: APIVersion144, - }, nil).AnyTimes() - - service := types.ServiceConfig{ - Name: "test", - Networks: map[string]*types.ServiceNetworkConfig{ - "a": { - Priority: 10, - }, - "b": { - Priority: 100, - }, - }, - } - project := types.Project{ - Name: "bork", - Services: types.Services{ - "test": service, - }, - Networks: types.Networks{ - "a": types.NetworkConfig{ - Name: "a-moby-name", - }, - "b": types.NetworkConfig{ - Name: "b-moby-name", - }, - }, - } - - var falseBool bool - apiClient.EXPECT().ContainerCreate(gomock.Any(), gomock.Any(), gomock.Eq( - &container.HostConfig{ - PortBindings: nat.PortMap{}, - ExtraHosts: []string{}, - Tmpfs: map[string]string{}, - Resources: container.Resources{ - OomKillDisable: &falseBool, - }, - NetworkMode: "b-moby-name", - }), gomock.Eq( - &network.NetworkingConfig{ - EndpointsConfig: map[string]*network.EndpointSettings{ - "a-moby-name": { - IPAMConfig: &network.EndpointIPAMConfig{}, - Aliases: []string{"bork-test-0"}, - }, - "b-moby-name": { - IPAMConfig: &network.EndpointIPAMConfig{}, - Aliases: []string{"bork-test-0"}, - }, - }, - }), gomock.Any(), gomock.Any()).Times(1).Return( - container.CreateResponse{ - ID: "an-id", - }, nil) - - apiClient.EXPECT().ContainerInspect(gomock.Any(), gomock.Eq("an-id")).Times(1).Return( - container.InspectResponse{ - ContainerJSONBase: &container.ContainerJSONBase{ - ID: "an-id", - Name: "a-name", - }, - Config: &container.Config{}, - NetworkSettings: &container.NetworkSettings{}, - }, nil) - - _, err = tested.(*composeService).createMobyContainer(t.Context(), &project, service, "test", 0, nil, createOptions{ - Labels: make(types.Labels), - }) - assert.NilError(t, err) - }) -} diff --git a/pkg/compose/convert.go b/pkg/compose/convert.go deleted file mode 100644 index fd8fb7b3fb8..00000000000 --- a/pkg/compose/convert.go +++ /dev/null @@ -1,103 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "time" - - compose "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/versions" -) - -// ToMobyEnv convert into []string -func ToMobyEnv(environment compose.MappingWithEquals) []string { - var env []string - for k, v := range environment { - if v == nil { - env = append(env, k) - } else { - env = append(env, fmt.Sprintf("%s=%s", k, *v)) - } - } - return env -} - -// ToMobyHealthCheck convert into container.HealthConfig -func (s *composeService) ToMobyHealthCheck(ctx context.Context, check *compose.HealthCheckConfig) (*container.HealthConfig, error) { - if check == nil { - return nil, nil - } - var ( - interval time.Duration - timeout time.Duration - period time.Duration - retries int - ) - if check.Interval != nil { - interval = time.Duration(*check.Interval) - } - if check.Timeout != nil { - timeout = time.Duration(*check.Timeout) - } - if check.StartPeriod != nil { - period = time.Duration(*check.StartPeriod) - } - if check.Retries != nil { - retries = int(*check.Retries) - } - test := check.Test - if check.Disable { - test = []string{"NONE"} - } - var startInterval time.Duration - if check.StartInterval != nil { - version, err := s.RuntimeVersion(ctx) - if err != nil { - return nil, err - } - if versions.LessThan(version, APIVersion144) { - return nil, fmt.Errorf("can't set healthcheck.start_interval as feature require Docker Engine %s or later", DockerEngineV25) - } else { - startInterval = time.Duration(*check.StartInterval) - } - if check.StartPeriod == nil { - // see https://github.com/moby/moby/issues/48874 - return nil, errors.New("healthcheck.start_interval requires healthcheck.start_period to be set") - } - } - return &container.HealthConfig{ - Test: test, - Interval: interval, - Timeout: timeout, - StartPeriod: period, - StartInterval: startInterval, - Retries: retries, - }, nil -} - -// ToSeconds convert into seconds -func ToSeconds(d *compose.Duration) *int { - if d == nil { - return nil - } - s := int(time.Duration(*d).Seconds()) - return &s -} diff --git a/pkg/compose/cp.go b/pkg/compose/cp.go deleted file mode 100644 index f912c0827af..00000000000 --- a/pkg/compose/cp.go +++ /dev/null @@ -1,336 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "io" - "os" - "path/filepath" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/docker/docker/api/types/container" - "github.com/moby/go-archive" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -type copyDirection int - -const ( - fromService copyDirection = 1 << iota - toService - acrossServices = fromService | toService -) - -func (s *composeService) Copy(ctx context.Context, projectName string, options api.CopyOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.copy(ctx, projectName, options) - }, "copy", s.events) -} - -func (s *composeService) copy(ctx context.Context, projectName string, options api.CopyOptions) error { - projectName = strings.ToLower(projectName) - srcService, srcPath := splitCpArg(options.Source) - destService, dstPath := splitCpArg(options.Destination) - - var direction copyDirection - var serviceName string - var copyFunc func(ctx context.Context, containerID string, srcPath string, dstPath string, opts api.CopyOptions) error - if srcService != "" { - direction |= fromService - serviceName = srcService - copyFunc = s.copyFromContainer - } - if destService != "" { - direction |= toService - serviceName = destService - copyFunc = s.copyToContainer - } - if direction == acrossServices { - return errors.New("copying between services is not supported") - } - - if direction == 0 { - return errors.New("unknown copy direction") - } - - containers, err := s.listContainersTargetedForCopy(ctx, projectName, options, direction, serviceName) - if err != nil { - return err - } - - g := errgroup.Group{} - for _, cont := range containers { - ctr := cont - g.Go(func() error { - name := getCanonicalContainerName(ctr) - var msg string - if direction == fromService { - msg = fmt.Sprintf("%s:%s to %s", name, srcPath, dstPath) - } else { - msg = fmt.Sprintf("%s to %s:%s", srcPath, name, dstPath) - } - s.events.On(api.Resource{ - ID: name, - Text: api.StatusCopying, - Details: msg, - Status: api.Working, - }) - if err := copyFunc(ctx, ctr.ID, srcPath, dstPath, options); err != nil { - return err - } - s.events.On(api.Resource{ - ID: name, - Text: api.StatusCopied, - Details: msg, - Status: api.Done, - }) - return nil - }) - } - - return g.Wait() -} - -func (s *composeService) listContainersTargetedForCopy(ctx context.Context, projectName string, options api.CopyOptions, direction copyDirection, serviceName string) (Containers, error) { - var containers Containers - var err error - switch { - case options.Index > 0: - ctr, err := s.getSpecifiedContainer(ctx, projectName, oneOffExclude, true, serviceName, options.Index) - if err != nil { - return nil, err - } - return append(containers, ctr), nil - default: - withOneOff := oneOffExclude - if options.All { - withOneOff = oneOffInclude - } - containers, err = s.getContainers(ctx, projectName, withOneOff, true, serviceName) - if err != nil { - return nil, err - } - - if len(containers) < 1 { - return nil, fmt.Errorf("no container found for service %q", serviceName) - } - if direction == fromService { - return containers[:1], err - } - return containers, err - } -} - -func (s *composeService) copyToContainer(ctx context.Context, containerID string, srcPath string, dstPath string, opts api.CopyOptions) error { - var err error - if srcPath != "-" { - // Get an absolute source path. - srcPath, err = resolveLocalPath(srcPath) - if err != nil { - return err - } - } - - // Prepare destination copy info by stat-ing the container path. - dstInfo := archive.CopyInfo{Path: dstPath} - dstStat, err := s.apiClient().ContainerStatPath(ctx, containerID, dstPath) - - // If the destination is a symbolic link, we should evaluate it. - if err == nil && dstStat.Mode&os.ModeSymlink != 0 { - linkTarget := dstStat.LinkTarget - if !isAbs(linkTarget) { - // Join with the parent directory. - dstParent, _ := archive.SplitPathDirEntry(dstPath) - linkTarget = filepath.Join(dstParent, linkTarget) - } - - dstInfo.Path = linkTarget - dstStat, err = s.apiClient().ContainerStatPath(ctx, containerID, linkTarget) - } - - // Validate the destination path - if err := command.ValidateOutputPathFileMode(dstStat.Mode); err != nil { - return fmt.Errorf(`destination "%s:%s" must be a directory or a regular file: %w`, containerID, dstPath, err) - } - - // Ignore any error and assume that the parent directory of the destination - // path exists, in which case the copy may still succeed. If there is any - // type of conflict (e.g., non-directory overwriting an existing directory - // or vice versa) the extraction will fail. If the destination simply did - // not exist, but the parent directory does, the extraction will still - // succeed. - if err == nil { - dstInfo.Exists, dstInfo.IsDir = true, dstStat.Mode.IsDir() - } - - var ( - content io.Reader - resolvedDstPath string - ) - - if srcPath == "-" { - content = s.stdin() - resolvedDstPath = dstInfo.Path - if !dstInfo.IsDir { - return fmt.Errorf("destination \"%s:%s\" must be a directory", containerID, dstPath) - } - } else { - // Prepare source copy info. - srcInfo, err := archive.CopyInfoSourcePath(srcPath, opts.FollowLink) - if err != nil { - return err - } - - srcArchive, err := archive.TarResource(srcInfo) - if err != nil { - return err - } - defer srcArchive.Close() //nolint:errcheck - - // With the stat info about the local source as well as the - // destination, we have enough information to know whether we need to - // alter the archive that we upload so that when the server extracts - // it to the specified directory in the container we get the desired - // copy behavior. - - // See comments in the implementation of `archive.PrepareArchiveCopy` - // for exactly what goes into deciding how and whether the source - // archive needs to be altered for the correct copy behavior when it is - // extracted. This function also infers from the source and destination - // info which directory to extract to, which may be the parent of the - // destination that the user specified. - // Don't create the archive if running in Dry Run mode - if !s.dryRun { - dstDir, preparedArchive, err := archive.PrepareArchiveCopy(srcArchive, srcInfo, dstInfo) - if err != nil { - return err - } - defer preparedArchive.Close() //nolint:errcheck - - resolvedDstPath = dstDir - content = preparedArchive - } - } - - options := container.CopyToContainerOptions{ - AllowOverwriteDirWithFile: false, - CopyUIDGID: opts.CopyUIDGID, - } - return s.apiClient().CopyToContainer(ctx, containerID, resolvedDstPath, content, options) -} - -func (s *composeService) copyFromContainer(ctx context.Context, containerID, srcPath, dstPath string, opts api.CopyOptions) error { - var err error - if dstPath != "-" { - // Get an absolute destination path. - dstPath, err = resolveLocalPath(dstPath) - if err != nil { - return err - } - } - - if err := command.ValidateOutputPath(dstPath); err != nil { - return err - } - - // if client requests to follow symbol link, then must decide target file to be copied - var rebaseName string - if opts.FollowLink { - srcStat, err := s.apiClient().ContainerStatPath(ctx, containerID, srcPath) - - // If the destination is a symbolic link, we should follow it. - if err == nil && srcStat.Mode&os.ModeSymlink != 0 { - linkTarget := srcStat.LinkTarget - if !isAbs(linkTarget) { - // Join with the parent directory. - srcParent, _ := archive.SplitPathDirEntry(srcPath) - linkTarget = filepath.Join(srcParent, linkTarget) - } - - linkTarget, rebaseName = archive.GetRebaseName(srcPath, linkTarget) - srcPath = linkTarget - } - } - - content, stat, err := s.apiClient().CopyFromContainer(ctx, containerID, srcPath) - if err != nil { - return err - } - defer content.Close() //nolint:errcheck - - if dstPath == "-" { - _, err = io.Copy(s.stdout(), content) - return err - } - - srcInfo := archive.CopyInfo{ - Path: srcPath, - Exists: true, - IsDir: stat.Mode.IsDir(), - RebaseName: rebaseName, - } - - preArchive := content - if srcInfo.RebaseName != "" { - _, srcBase := archive.SplitPathDirEntry(srcInfo.Path) - preArchive = archive.RebaseArchiveEntries(content, srcBase, srcInfo.RebaseName) - } - - return archive.CopyTo(preArchive, srcInfo, dstPath) -} - -// IsAbs is a platform-agnostic wrapper for filepath.IsAbs. -// -// On Windows, golang filepath.IsAbs does not consider a path \windows\system32 -// as absolute as it doesn't start with a drive-letter/colon combination. However, -// in docker we need to verify things such as WORKDIR /windows/system32 in -// a Dockerfile (which gets translated to \windows\system32 when being processed -// by the daemon). This SHOULD be treated as absolute from a docker processing -// perspective. -func isAbs(path string) bool { - return filepath.IsAbs(path) || strings.HasPrefix(path, string(os.PathSeparator)) -} - -func splitCpArg(arg string) (ctr, path string) { - if isAbs(arg) { - // Explicit local absolute path, e.g., `C:\foo` or `/foo`. - return "", arg - } - - ctr, path, ok := strings.Cut(arg, ":") - - if !ok || strings.HasPrefix(ctr, ".") { - // Either there's no `:` in the arg - // OR it's an explicit local relative path like `./file:name.txt`. - return "", arg - } - - return ctr, path -} - -func resolveLocalPath(localPath string) (absPath string, err error) { - if absPath, err = filepath.Abs(localPath); err != nil { - return absPath, err - } - return archive.PreserveTrailingDotOrSeparator(absPath, localPath), nil -} diff --git a/pkg/compose/create.go b/pkg/compose/create.go deleted file mode 100644 index f01c3e9cec4..00000000000 --- a/pkg/compose/create.go +++ /dev/null @@ -1,1649 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "context" - "encoding/json" - "errors" - "fmt" - "os" - "path/filepath" - "slices" - "strconv" - "strings" - - "github.com/compose-spec/compose-go/v2/paths" - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/docker/docker/api/types/blkiodev" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/mount" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/api/types/versions" - volumetypes "github.com/docker/docker/api/types/volume" - "github.com/docker/go-connections/nat" - "github.com/sirupsen/logrus" - cdi "tags.cncf.io/container-device-interface/pkg/parser" - - "github.com/docker/compose/v5/pkg/api" -) - -type createOptions struct { - AutoRemove bool - AttachStdin bool - UseNetworkAliases bool - Labels types.Labels -} - -type createConfigs struct { - Container *container.Config - Host *container.HostConfig - Network *network.NetworkingConfig - Links []string -} - -func (s *composeService) Create(ctx context.Context, project *types.Project, createOpts api.CreateOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.create(ctx, project, createOpts) - }, "create", s.events) -} - -func (s *composeService) create(ctx context.Context, project *types.Project, options api.CreateOptions) error { - if len(options.Services) == 0 { - options.Services = project.ServiceNames() - } - - err := project.CheckContainerNameUnicity() - if err != nil { - return err - } - - err = s.ensureImagesExists(ctx, project, options.Build, options.QuietPull) - if err != nil { - return err - } - - err = s.ensureModels(ctx, project, options.QuietPull) - if err != nil { - return err - } - - prepareNetworks(project) - - networks, err := s.ensureNetworks(ctx, project) - if err != nil { - return err - } - - volumes, err := s.ensureProjectVolumes(ctx, project) - if err != nil { - return err - } - - var observedState Containers - observedState, err = s.getContainers(ctx, project.Name, oneOffInclude, true) - if err != nil { - return err - } - orphans := observedState.filter(isOrphaned(project)) - if len(orphans) > 0 && !options.IgnoreOrphans { - if options.RemoveOrphans { - err := s.removeContainers(ctx, orphans, nil, nil, false) - if err != nil { - return err - } - } else { - logrus.Warnf("Found orphan containers (%s) for this project. If "+ - "you removed or renamed this service in your compose "+ - "file, you can run this command with the "+ - "--remove-orphans flag to clean it up.", orphans.names()) - } - } - - // Temporary implementation of use_api_socket until we get actual support inside docker engine - project, err = s.useAPISocket(project) - if err != nil { - return err - } - - return newConvergence(options.Services, observedState, networks, volumes, s).apply(ctx, project, options) -} - -func prepareNetworks(project *types.Project) { - for k, nw := range project.Networks { - nw.CustomLabels = nw.CustomLabels. - Add(api.NetworkLabel, k). - Add(api.ProjectLabel, project.Name). - Add(api.VersionLabel, api.ComposeVersion) - project.Networks[k] = nw - } -} - -func (s *composeService) ensureNetworks(ctx context.Context, project *types.Project) (map[string]string, error) { - networks := map[string]string{} - for name, nw := range project.Networks { - id, err := s.ensureNetwork(ctx, project, name, &nw) - if err != nil { - return nil, err - } - networks[name] = id - project.Networks[name] = nw - } - return networks, nil -} - -func (s *composeService) ensureProjectVolumes(ctx context.Context, project *types.Project) (map[string]string, error) { - ids := map[string]string{} - for k, volume := range project.Volumes { - volume.CustomLabels = volume.CustomLabels.Add(api.VolumeLabel, k) - volume.CustomLabels = volume.CustomLabels.Add(api.ProjectLabel, project.Name) - volume.CustomLabels = volume.CustomLabels.Add(api.VersionLabel, api.ComposeVersion) - id, err := s.ensureVolume(ctx, k, volume, project) - if err != nil { - return nil, err - } - ids[k] = id - } - - return ids, nil -} - -//nolint:gocyclo -func (s *composeService) getCreateConfigs(ctx context.Context, - p *types.Project, - service types.ServiceConfig, - number int, - inherit *container.Summary, - opts createOptions, -) (createConfigs, error) { - labels, err := s.prepareLabels(opts.Labels, service, number) - if err != nil { - return createConfigs{}, err - } - - var runCmd, entrypoint []string - if service.Command != nil { - runCmd = service.Command - } - if service.Entrypoint != nil { - entrypoint = service.Entrypoint - } - - var ( - tty = service.Tty - stdinOpen = service.StdinOpen - ) - - proxyConfig := types.MappingWithEquals(s.configFile().ParseProxyConfig(s.apiClient().DaemonHost(), nil)) - env := proxyConfig.OverrideBy(service.Environment) - - var mainNwName string - var mainNw *types.ServiceNetworkConfig - if len(service.Networks) > 0 { - mainNwName = service.NetworksByPriority()[0] - mainNw = service.Networks[mainNwName] - } - - macAddress, err := s.prepareContainerMACAddress(ctx, service, mainNw, mainNwName) - if err != nil { - return createConfigs{}, err - } - - healthcheck, err := s.ToMobyHealthCheck(ctx, service.HealthCheck) - if err != nil { - return createConfigs{}, err - } - - exposed, err := buildContainerPorts(service) - if err != nil { - return createConfigs{}, err - } - - containerConfig := container.Config{ - Hostname: service.Hostname, - Domainname: service.DomainName, - User: service.User, - ExposedPorts: exposed, - Tty: tty, - OpenStdin: stdinOpen, - StdinOnce: opts.AttachStdin && stdinOpen, - AttachStdin: opts.AttachStdin, - AttachStderr: true, - AttachStdout: true, - Cmd: runCmd, - Image: api.GetImageNameOrDefault(service, p.Name), - WorkingDir: service.WorkingDir, - Entrypoint: entrypoint, - NetworkDisabled: service.NetworkMode == "disabled", - MacAddress: macAddress, // Field is deprecated since API v1.44, but kept for compatibility with older API versions. - Labels: labels, - StopSignal: service.StopSignal, - Env: ToMobyEnv(env), - Healthcheck: healthcheck, - StopTimeout: ToSeconds(service.StopGracePeriod), - } // VOLUMES/MOUNTS/FILESYSTEMS - tmpfs := map[string]string{} - for _, t := range service.Tmpfs { - k, v, _ := strings.Cut(t, ":") - tmpfs[k] = v - } - binds, mounts, err := s.buildContainerVolumes(ctx, *p, service, inherit) - if err != nil { - return createConfigs{}, err - } - - // NETWORKING - links, err := s.getLinks(ctx, p.Name, service, number) - if err != nil { - return createConfigs{}, err - } - apiVersion, err := s.RuntimeVersion(ctx) - if err != nil { - return createConfigs{}, err - } - networkMode, networkingConfig, err := defaultNetworkSettings(p, service, number, links, opts.UseNetworkAliases, apiVersion) - if err != nil { - return createConfigs{}, err - } - portBindings := buildContainerPortBindingOptions(service) - - // MISC - resources := getDeployResources(service) - var logConfig container.LogConfig - if service.Logging != nil { - logConfig = container.LogConfig{ - Type: service.Logging.Driver, - Config: service.Logging.Options, - } - } - securityOpts, unconfined, err := parseSecurityOpts(p, service.SecurityOpt) - if err != nil { - return createConfigs{}, err - } - - hostConfig := container.HostConfig{ - AutoRemove: opts.AutoRemove, - Annotations: service.Annotations, - Binds: binds, - Mounts: mounts, - CapAdd: service.CapAdd, - CapDrop: service.CapDrop, - NetworkMode: networkMode, - Init: service.Init, - IpcMode: container.IpcMode(service.Ipc), - CgroupnsMode: container.CgroupnsMode(service.Cgroup), - ReadonlyRootfs: service.ReadOnly, - RestartPolicy: getRestartPolicy(service), - ShmSize: int64(service.ShmSize), - Sysctls: service.Sysctls, - PortBindings: portBindings, - Resources: resources, - VolumeDriver: service.VolumeDriver, - VolumesFrom: service.VolumesFrom, - DNS: service.DNS, - DNSSearch: service.DNSSearch, - DNSOptions: service.DNSOpts, - ExtraHosts: service.ExtraHosts.AsList(":"), - SecurityOpt: securityOpts, - StorageOpt: service.StorageOpt, - UsernsMode: container.UsernsMode(service.UserNSMode), - UTSMode: container.UTSMode(service.Uts), - Privileged: service.Privileged, - PidMode: container.PidMode(service.Pid), - Tmpfs: tmpfs, - Isolation: container.Isolation(service.Isolation), - Runtime: service.Runtime, - LogConfig: logConfig, - GroupAdd: service.GroupAdd, - Links: links, - OomScoreAdj: int(service.OomScoreAdj), - } - - if unconfined { - hostConfig.MaskedPaths = []string{} - hostConfig.ReadonlyPaths = []string{} - } - - cfgs := createConfigs{ - Container: &containerConfig, - Host: &hostConfig, - Network: networkingConfig, - Links: links, - } - return cfgs, nil -} - -// prepareContainerMACAddress handles the service-level mac_address field and the newer mac_address field added to service -// network config. This newer field is only compatible with the Engine API v1.44 (and onwards), and this API version -// also deprecates the container-wide mac_address field. Thus, this method will validate service config and mutate the -// passed mainNw to provide backward-compatibility whenever possible. -// -// It returns the container-wide MAC address, but this value will be kept empty for newer API versions. -func (s *composeService) prepareContainerMACAddress(ctx context.Context, service types.ServiceConfig, mainNw *types.ServiceNetworkConfig, nwName string) (string, error) { - version, err := s.RuntimeVersion(ctx) - if err != nil { - return "", err - } - - // Engine API 1.44 added support for endpoint-specific MAC address and now returns a warning when a MAC address is - // set in container.Config. Thus, we have to jump through a number of hoops: - // - // 1. Top-level mac_address and main endpoint's MAC address should be the same ; - // 2. If supported by the API, top-level mac_address should be migrated to the main endpoint and container.Config - // should be kept empty ; - // 3. Otherwise, the endpoint mac_address should be set in container.Config and no other endpoint-specific - // mac_address can be specified. If that's the case, use top-level mac_address ; - // - // After that, if an endpoint mac_address is set, it's either user-defined or migrated by the code below, so - // there's no need to check for API version in defaultNetworkSettings. - macAddress := service.MacAddress - if macAddress != "" && mainNw != nil && mainNw.MacAddress != "" && mainNw.MacAddress != macAddress { - return "", fmt.Errorf("the service-level mac_address should have the same value as network %s", nwName) - } - if versions.GreaterThanOrEqualTo(version, APIVersion144) { - if mainNw != nil && mainNw.MacAddress == "" { - mainNw.MacAddress = macAddress - } - macAddress = "" - } else if len(service.Networks) > 0 { - var withMacAddress []string - for nwName, nw := range service.Networks { - if nw != nil && nw.MacAddress != "" { - withMacAddress = append(withMacAddress, nwName) - } - } - - if len(withMacAddress) > 1 { - return "", fmt.Errorf("a MAC address is specified for multiple networks (%s), but this feature requires Docker Engine %s or later", strings.Join(withMacAddress, ", "), DockerEngineV25) - } - - if mainNw != nil && mainNw.MacAddress != "" { - macAddress = mainNw.MacAddress - } - } - - return macAddress, nil -} - -func getAliases(project *types.Project, service types.ServiceConfig, serviceIndex int, cfg *types.ServiceNetworkConfig, useNetworkAliases bool) []string { - aliases := []string{getContainerName(project.Name, service, serviceIndex)} - if useNetworkAliases { - aliases = append(aliases, service.Name) - if cfg != nil { - aliases = append(aliases, cfg.Aliases...) - } - } - return aliases -} - -func createEndpointSettings(p *types.Project, service types.ServiceConfig, serviceIndex int, networkKey string, links []string, useNetworkAliases bool) *network.EndpointSettings { - const ifname = "com.docker.network.endpoint.ifname" - - config := service.Networks[networkKey] - var ipam *network.EndpointIPAMConfig - var ( - ipv4Address string - ipv6Address string - macAddress string - driverOpts types.Options - gwPriority int - ) - if config != nil { - ipv4Address = config.Ipv4Address - ipv6Address = config.Ipv6Address - ipam = &network.EndpointIPAMConfig{ - IPv4Address: ipv4Address, - IPv6Address: ipv6Address, - LinkLocalIPs: config.LinkLocalIPs, - } - macAddress = config.MacAddress - driverOpts = config.DriverOpts - if config.InterfaceName != "" { - if driverOpts == nil { - driverOpts = map[string]string{} - } - if name, ok := driverOpts[ifname]; ok && name != config.InterfaceName { - logrus.Warnf("ignoring services.%s.networks.%s.interface_name as %s driver_opts is already declared", service.Name, networkKey, ifname) - } - driverOpts[ifname] = config.InterfaceName - } - gwPriority = config.GatewayPriority - } - return &network.EndpointSettings{ - Aliases: getAliases(p, service, serviceIndex, config, useNetworkAliases), - Links: links, - IPAddress: ipv4Address, - IPv6Gateway: ipv6Address, - IPAMConfig: ipam, - MacAddress: macAddress, - DriverOpts: driverOpts, - GwPriority: gwPriority, - } -} - -// copy/pasted from https://github.com/docker/cli/blob/9de1b162f/cli/command/container/opts.go#L673-L697 + RelativePath -// TODO find so way to share this code with docker/cli -func parseSecurityOpts(p *types.Project, securityOpts []string) ([]string, bool, error) { - var ( - unconfined bool - parsed []string - ) - for _, opt := range securityOpts { - if opt == "systempaths=unconfined" { - unconfined = true - continue - } - con := strings.SplitN(opt, "=", 2) - if len(con) == 1 && con[0] != "no-new-privileges" { - if strings.Contains(opt, ":") { - con = strings.SplitN(opt, ":", 2) - } else { - return securityOpts, false, fmt.Errorf("invalid security-opt: %q", opt) - } - } - if con[0] == "seccomp" && con[1] != "unconfined" && con[1] != "builtin" { - f, err := os.ReadFile(p.RelativePath(con[1])) - if err != nil { - return securityOpts, false, fmt.Errorf("opening seccomp profile (%s) failed: %w", con[1], err) - } - b := bytes.NewBuffer(nil) - if err := json.Compact(b, f); err != nil { - return securityOpts, false, fmt.Errorf("compacting json for seccomp profile (%s) failed: %w", con[1], err) - } - parsed = append(parsed, fmt.Sprintf("seccomp=%s", b.Bytes())) - } else { - parsed = append(parsed, opt) - } - } - - return parsed, unconfined, nil -} - -func (s *composeService) prepareLabels(labels types.Labels, service types.ServiceConfig, number int) (map[string]string, error) { - hash, err := ServiceHash(service) - if err != nil { - return nil, err - } - labels[api.ConfigHashLabel] = hash - - if number > 0 { - // One-off containers are not indexed - labels[api.ContainerNumberLabel] = strconv.Itoa(number) - } - - var dependencies []string - for s, d := range service.DependsOn { - dependencies = append(dependencies, fmt.Sprintf("%s:%s:%t", s, d.Condition, d.Restart)) - } - labels[api.DependenciesLabel] = strings.Join(dependencies, ",") - return labels, nil -} - -// defaultNetworkSettings determines the container.NetworkMode and corresponding network.NetworkingConfig (nil if not applicable). -func defaultNetworkSettings(project *types.Project, - service types.ServiceConfig, serviceIndex int, - links []string, useNetworkAliases bool, - version string, -) (container.NetworkMode, *network.NetworkingConfig, error) { - if service.NetworkMode != "" { - return container.NetworkMode(service.NetworkMode), nil, nil - } - - if len(project.Networks) == 0 { - return "none", nil, nil - } - - var primaryNetworkKey string - if len(service.Networks) > 0 { - primaryNetworkKey = service.NetworksByPriority()[0] - } else { - primaryNetworkKey = "default" - } - primaryNetworkMobyNetworkName := project.Networks[primaryNetworkKey].Name - primaryNetworkEndpoint := createEndpointSettings(project, service, serviceIndex, primaryNetworkKey, links, useNetworkAliases) - endpointsConfig := map[string]*network.EndpointSettings{} - - // Starting from API version 1.44, the Engine will take several EndpointsConfigs - // so we can pass all the extra networks we want the container to be connected to - // in the network configuration instead of connecting the container to each extra - // network individually after creation. - if versions.GreaterThanOrEqualTo(version, APIVersion144) { - if len(service.Networks) > 1 { - serviceNetworks := service.NetworksByPriority() - for _, networkKey := range serviceNetworks[1:] { - mobyNetworkName := project.Networks[networkKey].Name - epSettings := createEndpointSettings(project, service, serviceIndex, networkKey, links, useNetworkAliases) - endpointsConfig[mobyNetworkName] = epSettings - } - } - if primaryNetworkEndpoint.MacAddress == "" { - primaryNetworkEndpoint.MacAddress = service.MacAddress - } - } - - if versions.LessThan(version, APIVersion149) { - for _, config := range service.Networks { - if config != nil && config.InterfaceName != "" { - return "", nil, fmt.Errorf("interface_name requires Docker Engine %s or later", DockerEngineV28_1) - } - } - } - - endpointsConfig[primaryNetworkMobyNetworkName] = primaryNetworkEndpoint - networkConfig := &network.NetworkingConfig{ - EndpointsConfig: endpointsConfig, - } - - // From the Engine API docs: - // > Supported standard values are: bridge, host, none, and container:<name|id>. - // > Any other value is taken as a custom network's name to which this container should connect to. - return container.NetworkMode(primaryNetworkMobyNetworkName), networkConfig, nil -} - -func getRestartPolicy(service types.ServiceConfig) container.RestartPolicy { - var restart container.RestartPolicy - if service.Restart != "" { - name, num, ok := strings.Cut(service.Restart, ":") - var attempts int - if ok { - attempts, _ = strconv.Atoi(num) - } - restart = container.RestartPolicy{ - Name: mapRestartPolicyCondition(name), - MaximumRetryCount: attempts, - } - } - if service.Deploy != nil && service.Deploy.RestartPolicy != nil { - policy := *service.Deploy.RestartPolicy - var attempts int - if policy.MaxAttempts != nil { - attempts = int(*policy.MaxAttempts) - } - restart = container.RestartPolicy{ - Name: mapRestartPolicyCondition(policy.Condition), - MaximumRetryCount: attempts, - } - } - return restart -} - -func mapRestartPolicyCondition(condition string) container.RestartPolicyMode { - // map definitions of deploy.restart_policy to engine definitions - switch condition { - case "none", "no": - return container.RestartPolicyDisabled - case "on-failure": - return container.RestartPolicyOnFailure - case "unless-stopped": - return container.RestartPolicyUnlessStopped - case "any", "always": - return container.RestartPolicyAlways - default: - return container.RestartPolicyMode(condition) - } -} - -func getDeployResources(s types.ServiceConfig) container.Resources { - var swappiness *int64 - if s.MemSwappiness != 0 { - val := int64(s.MemSwappiness) - swappiness = &val - } - resources := container.Resources{ - CgroupParent: s.CgroupParent, - Memory: int64(s.MemLimit), - MemorySwap: int64(s.MemSwapLimit), - MemorySwappiness: swappiness, - MemoryReservation: int64(s.MemReservation), - OomKillDisable: &s.OomKillDisable, - CPUCount: s.CPUCount, - CPUPeriod: s.CPUPeriod, - CPUQuota: s.CPUQuota, - CPURealtimePeriod: s.CPURTPeriod, - CPURealtimeRuntime: s.CPURTRuntime, - CPUShares: s.CPUShares, - NanoCPUs: int64(s.CPUS * 1e9), - CPUPercent: int64(s.CPUPercent * 100), - CpusetCpus: s.CPUSet, - DeviceCgroupRules: s.DeviceCgroupRules, - } - - if s.PidsLimit != 0 { - resources.PidsLimit = &s.PidsLimit - } - - setBlkio(s.BlkioConfig, &resources) - - if s.Deploy != nil { - setLimits(s.Deploy.Resources.Limits, &resources) - setReservations(s.Deploy.Resources.Reservations, &resources) - } - - var cdiDeviceNames []string - for _, device := range s.Devices { - - if device.Source == device.Target && cdi.IsQualifiedName(device.Source) { - cdiDeviceNames = append(cdiDeviceNames, device.Source) - continue - } - - resources.Devices = append(resources.Devices, container.DeviceMapping{ - PathOnHost: device.Source, - PathInContainer: device.Target, - CgroupPermissions: device.Permissions, - }) - } - - if len(cdiDeviceNames) > 0 { - resources.DeviceRequests = append(resources.DeviceRequests, container.DeviceRequest{ - Driver: "cdi", - DeviceIDs: cdiDeviceNames, - }) - } - - for _, gpus := range s.Gpus { - resources.DeviceRequests = append(resources.DeviceRequests, container.DeviceRequest{ - Driver: gpus.Driver, - Count: int(gpus.Count), - DeviceIDs: gpus.IDs, - Capabilities: [][]string{append(gpus.Capabilities, "gpu")}, - Options: gpus.Options, - }) - } - - ulimits := toUlimits(s.Ulimits) - resources.Ulimits = ulimits - return resources -} - -func toUlimits(m map[string]*types.UlimitsConfig) []*container.Ulimit { - var ulimits []*container.Ulimit - for name, u := range m { - soft := u.Single - if u.Soft != 0 { - soft = u.Soft - } - hard := u.Single - if u.Hard != 0 { - hard = u.Hard - } - ulimits = append(ulimits, &container.Ulimit{ - Name: name, - Hard: int64(hard), - Soft: int64(soft), - }) - } - return ulimits -} - -func setReservations(reservations *types.Resource, resources *container.Resources) { - if reservations == nil { - return - } - // Cpu reservation is a swarm option and PIDs is only a limit - // So we only need to map memory reservation and devices - if reservations.MemoryBytes != 0 { - resources.MemoryReservation = int64(reservations.MemoryBytes) - } - - for _, device := range reservations.Devices { - resources.DeviceRequests = append(resources.DeviceRequests, container.DeviceRequest{ - Capabilities: [][]string{device.Capabilities}, - Count: int(device.Count), - DeviceIDs: device.IDs, - Driver: device.Driver, - Options: device.Options, - }) - } -} - -func setLimits(limits *types.Resource, resources *container.Resources) { - if limits == nil { - return - } - if limits.MemoryBytes != 0 { - resources.Memory = int64(limits.MemoryBytes) - } - if limits.NanoCPUs != 0 { - resources.NanoCPUs = int64(limits.NanoCPUs * 1e9) - } - if limits.Pids > 0 { - resources.PidsLimit = &limits.Pids - } -} - -func setBlkio(blkio *types.BlkioConfig, resources *container.Resources) { - if blkio == nil { - return - } - resources.BlkioWeight = blkio.Weight - for _, b := range blkio.WeightDevice { - resources.BlkioWeightDevice = append(resources.BlkioWeightDevice, &blkiodev.WeightDevice{ - Path: b.Path, - Weight: b.Weight, - }) - } - for _, b := range blkio.DeviceReadBps { - resources.BlkioDeviceReadBps = append(resources.BlkioDeviceReadBps, &blkiodev.ThrottleDevice{ - Path: b.Path, - Rate: uint64(b.Rate), - }) - } - for _, b := range blkio.DeviceReadIOps { - resources.BlkioDeviceReadIOps = append(resources.BlkioDeviceReadIOps, &blkiodev.ThrottleDevice{ - Path: b.Path, - Rate: uint64(b.Rate), - }) - } - for _, b := range blkio.DeviceWriteBps { - resources.BlkioDeviceWriteBps = append(resources.BlkioDeviceWriteBps, &blkiodev.ThrottleDevice{ - Path: b.Path, - Rate: uint64(b.Rate), - }) - } - for _, b := range blkio.DeviceWriteIOps { - resources.BlkioDeviceWriteIOps = append(resources.BlkioDeviceWriteIOps, &blkiodev.ThrottleDevice{ - Path: b.Path, - Rate: uint64(b.Rate), - }) - } -} - -func buildContainerPorts(s types.ServiceConfig) (nat.PortSet, error) { - ports := nat.PortSet{} - for _, s := range s.Expose { - proto, port := nat.SplitProtoPort(s) - start, end, err := nat.ParsePortRange(port) - if err != nil { - return nil, err - } - for i := start; i <= end; i++ { - p := nat.Port(fmt.Sprintf("%d/%s", i, proto)) - ports[p] = struct{}{} - } - } - for _, p := range s.Ports { - p := nat.Port(fmt.Sprintf("%d/%s", p.Target, p.Protocol)) - ports[p] = struct{}{} - } - return ports, nil -} - -func buildContainerPortBindingOptions(s types.ServiceConfig) nat.PortMap { - bindings := nat.PortMap{} - for _, port := range s.Ports { - p := nat.Port(fmt.Sprintf("%d/%s", port.Target, port.Protocol)) - binding := nat.PortBinding{ - HostIP: port.HostIP, - HostPort: port.Published, - } - bindings[p] = append(bindings[p], binding) - } - return bindings -} - -func getDependentServiceFromMode(mode string) string { - if strings.HasPrefix( - mode, - types.NetworkModeServicePrefix, - ) { - return mode[len(types.NetworkModeServicePrefix):] - } - return "" -} - -func (s *composeService) buildContainerVolumes( - ctx context.Context, - p types.Project, - service types.ServiceConfig, - inherit *container.Summary, -) ([]string, []mount.Mount, error) { - var mounts []mount.Mount - var binds []string - - mountOptions, err := s.buildContainerMountOptions(ctx, p, service, inherit) - if err != nil { - return nil, nil, err - } - - for _, m := range mountOptions { - switch m.Type { - case mount.TypeBind: - // `Mount` is preferred but does not offer option to created host path if missing - // so `Bind` API is used here with raw volume string - // see https://github.com/moby/moby/issues/43483 - v := findVolumeByTarget(service.Volumes, m.Target) - if v != nil { - if v.Type != types.VolumeTypeBind { - v.Source = m.Source - } - if !bindRequiresMountAPI(v.Bind) { - source := m.Source - if vol := findVolumeByName(p.Volumes, m.Source); vol != nil { - source = m.Source - } - binds = append(binds, toBindString(source, v)) - continue - } - } - case mount.TypeVolume: - v := findVolumeByTarget(service.Volumes, m.Target) - vol := findVolumeByName(p.Volumes, m.Source) - if v != nil && vol != nil { - // Prefer the bind API if no advanced option is used, to preserve backward compatibility - if !volumeRequiresMountAPI(v.Volume) { - binds = append(binds, toBindString(vol.Name, v)) - continue - } - } - case mount.TypeImage: - version, err := s.RuntimeVersion(ctx) - if err != nil { - return nil, nil, err - } - if versions.LessThan(version, APIVersion148) { - return nil, nil, fmt.Errorf("volume with type=image require Docker Engine %s or later", DockerEngineV28) - } - } - mounts = append(mounts, m) - } - return binds, mounts, nil -} - -func toBindString(name string, v *types.ServiceVolumeConfig) string { - access := "rw" - if v.ReadOnly { - access = "ro" - } - options := []string{access} - if v.Bind != nil && v.Bind.SELinux != "" { - options = append(options, v.Bind.SELinux) - } - if v.Bind != nil && v.Bind.Propagation != "" { - options = append(options, v.Bind.Propagation) - } - if v.Volume != nil && v.Volume.NoCopy { - options = append(options, "nocopy") - } - return fmt.Sprintf("%s:%s:%s", name, v.Target, strings.Join(options, ",")) -} - -func findVolumeByName(volumes types.Volumes, name string) *types.VolumeConfig { - for _, vol := range volumes { - if vol.Name == name { - return &vol - } - } - return nil -} - -func findVolumeByTarget(volumes []types.ServiceVolumeConfig, target string) *types.ServiceVolumeConfig { - for _, v := range volumes { - if v.Target == target { - return &v - } - } - return nil -} - -// bindRequiresMountAPI check if Bind declaration can be implemented by the plain old Bind API or uses any of the advanced -// options which require use of Mount API -func bindRequiresMountAPI(bind *types.ServiceVolumeBind) bool { - switch { - case bind == nil: - return false - case !bool(bind.CreateHostPath): - return true - case bind.Propagation != "": - return true - case bind.Recursive != "": - return true - default: - return false - } -} - -// volumeRequiresMountAPI check if Volume declaration can be implemented by the plain old Bind API or uses any of the advanced -// options which require use of Mount API -func volumeRequiresMountAPI(vol *types.ServiceVolumeVolume) bool { - switch { - case vol == nil: - return false - case len(vol.Labels) > 0: - return true - case vol.Subpath != "": - return true - case vol.NoCopy: - return true - default: - return false - } -} - -func (s *composeService) buildContainerMountOptions(ctx context.Context, p types.Project, service types.ServiceConfig, inherit *container.Summary) ([]mount.Mount, error) { - mounts := map[string]mount.Mount{} - if inherit != nil { - for _, m := range inherit.Mounts { - if m.Type == "tmpfs" { - continue - } - src := m.Source - if m.Type == "volume" { - src = m.Name - } - - img, err := s.apiClient().ImageInspect(ctx, api.GetImageNameOrDefault(service, p.Name)) - if err != nil { - return nil, err - } - - if img.Config != nil { - if _, ok := img.Config.Volumes[m.Destination]; ok { - // inherit previous container's anonymous volume - mounts[m.Destination] = mount.Mount{ - Type: m.Type, - Source: src, - Target: m.Destination, - ReadOnly: !m.RW, - } - } - } - volumes := []types.ServiceVolumeConfig{} - for _, v := range service.Volumes { - if v.Target != m.Destination || v.Source != "" { - volumes = append(volumes, v) - continue - } - // inherit previous container's anonymous volume - mounts[m.Destination] = mount.Mount{ - Type: m.Type, - Source: src, - Target: m.Destination, - ReadOnly: !m.RW, - } - } - service.Volumes = volumes - } - } - - mounts, err := fillBindMounts(p, service, mounts) - if err != nil { - return nil, err - } - - values := make([]mount.Mount, 0, len(mounts)) - for _, v := range mounts { - values = append(values, v) - } - return values, nil -} - -func fillBindMounts(p types.Project, s types.ServiceConfig, m map[string]mount.Mount) (map[string]mount.Mount, error) { - for _, v := range s.Volumes { - bindMount, err := buildMount(p, v) - if err != nil { - return nil, err - } - m[bindMount.Target] = bindMount - } - - secrets, err := buildContainerSecretMounts(p, s) - if err != nil { - return nil, err - } - for _, s := range secrets { - if _, found := m[s.Target]; found { - continue - } - m[s.Target] = s - } - - configs, err := buildContainerConfigMounts(p, s) - if err != nil { - return nil, err - } - for _, c := range configs { - if _, found := m[c.Target]; found { - continue - } - m[c.Target] = c - } - return m, nil -} - -func buildContainerConfigMounts(p types.Project, s types.ServiceConfig) ([]mount.Mount, error) { - mounts := map[string]mount.Mount{} - - configsBaseDir := "/" - for _, config := range s.Configs { - target := config.Target - if config.Target == "" { - target = configsBaseDir + config.Source - } else if !isAbsTarget(config.Target) { - target = configsBaseDir + config.Target - } - - definedConfig := p.Configs[config.Source] - if definedConfig.External { - return nil, fmt.Errorf("unsupported external config %s", definedConfig.Name) - } - - if definedConfig.Driver != "" { - return nil, errors.New("Docker Compose does not support configs.*.driver") //nolint:staticcheck - } - if definedConfig.TemplateDriver != "" { - return nil, errors.New("Docker Compose does not support configs.*.template_driver") //nolint:staticcheck - } - - if definedConfig.Environment != "" || definedConfig.Content != "" { - continue - } - - if config.UID != "" || config.GID != "" || config.Mode != nil { - logrus.Warn("config `uid`, `gid` and `mode` are not supported, they will be ignored") - } - - bindMount, err := buildMount(p, types.ServiceVolumeConfig{ - Type: types.VolumeTypeBind, - Source: definedConfig.File, - Target: target, - ReadOnly: true, - }) - if err != nil { - return nil, err - } - mounts[target] = bindMount - } - values := make([]mount.Mount, 0, len(mounts)) - for _, v := range mounts { - values = append(values, v) - } - return values, nil -} - -func buildContainerSecretMounts(p types.Project, s types.ServiceConfig) ([]mount.Mount, error) { - mounts := map[string]mount.Mount{} - - secretsDir := "/run/secrets/" - for _, secret := range s.Secrets { - target := secret.Target - if secret.Target == "" { - target = secretsDir + secret.Source - } else if !isAbsTarget(secret.Target) { - target = secretsDir + secret.Target - } - - definedSecret := p.Secrets[secret.Source] - if definedSecret.External { - return nil, fmt.Errorf("unsupported external secret %s", definedSecret.Name) - } - - if definedSecret.Driver != "" { - return nil, errors.New("Docker Compose does not support secrets.*.driver") //nolint:staticcheck - } - if definedSecret.TemplateDriver != "" { - return nil, errors.New("Docker Compose does not support secrets.*.template_driver") //nolint:staticcheck - } - - if definedSecret.Environment != "" { - continue - } - - if secret.UID != "" || secret.GID != "" || secret.Mode != nil { - logrus.Warn("secrets `uid`, `gid` and `mode` are not supported, they will be ignored") - } - - if _, err := os.Stat(definedSecret.File); os.IsNotExist(err) { - logrus.Warnf("secret file %s does not exist", definedSecret.Name) - } - - mnt, err := buildMount(p, types.ServiceVolumeConfig{ - Type: types.VolumeTypeBind, - Source: definedSecret.File, - Target: target, - ReadOnly: true, - Bind: &types.ServiceVolumeBind{ - CreateHostPath: false, - }, - }) - if err != nil { - return nil, err - } - mounts[target] = mnt - } - values := make([]mount.Mount, 0, len(mounts)) - for _, v := range mounts { - values = append(values, v) - } - return values, nil -} - -func isAbsTarget(p string) bool { - return isUnixAbs(p) || isWindowsAbs(p) -} - -func isUnixAbs(p string) bool { - return strings.HasPrefix(p, "/") -} - -func isWindowsAbs(p string) bool { - return paths.IsWindowsAbs(p) -} - -func buildMount(project types.Project, volume types.ServiceVolumeConfig) (mount.Mount, error) { - source := volume.Source - switch volume.Type { - case types.VolumeTypeBind: - if !filepath.IsAbs(source) && !isUnixAbs(source) && !isWindowsAbs(source) { - // volume source has already been prefixed with workdir if required, by compose-go project loader - var err error - source, err = filepath.Abs(source) - if err != nil { - return mount.Mount{}, err - } - } - case types.VolumeTypeVolume: - if volume.Source != "" { - pVolume, ok := project.Volumes[volume.Source] - if ok { - source = pVolume.Name - } - } - } - - bind, vol, tmpfs, img := buildMountOptions(volume) - - if bind != nil { - volume.Type = types.VolumeTypeBind - } - - return mount.Mount{ - Type: mount.Type(volume.Type), - Source: source, - Target: volume.Target, - ReadOnly: volume.ReadOnly, - Consistency: mount.Consistency(volume.Consistency), - BindOptions: bind, - VolumeOptions: vol, - TmpfsOptions: tmpfs, - ImageOptions: img, - }, nil -} - -func buildMountOptions(volume types.ServiceVolumeConfig) (*mount.BindOptions, *mount.VolumeOptions, *mount.TmpfsOptions, *mount.ImageOptions) { - if volume.Type != types.VolumeTypeBind && volume.Bind != nil { - logrus.Warnf("mount of type `%s` should not define `bind` option", volume.Type) - } - if volume.Type != types.VolumeTypeVolume && volume.Volume != nil { - logrus.Warnf("mount of type `%s` should not define `volume` option", volume.Type) - } - if volume.Type != types.VolumeTypeTmpfs && volume.Tmpfs != nil { - logrus.Warnf("mount of type `%s` should not define `tmpfs` option", volume.Type) - } - if volume.Type != types.VolumeTypeImage && volume.Image != nil { - logrus.Warnf("mount of type `%s` should not define `image` option", volume.Type) - } - - switch volume.Type { - case "bind": - return buildBindOption(volume.Bind), nil, nil, nil - case "volume": - return nil, buildVolumeOptions(volume.Volume), nil, nil - case "tmpfs": - return nil, nil, buildTmpfsOptions(volume.Tmpfs), nil - case "image": - return nil, nil, nil, buildImageOptions(volume.Image) - } - return nil, nil, nil, nil -} - -func buildBindOption(bind *types.ServiceVolumeBind) *mount.BindOptions { - if bind == nil { - return nil - } - opts := &mount.BindOptions{ - Propagation: mount.Propagation(bind.Propagation), - CreateMountpoint: bool(bind.CreateHostPath), - } - switch bind.Recursive { - case "disabled": - opts.NonRecursive = true - case "writable": - opts.ReadOnlyNonRecursive = true - case "readonly": - opts.ReadOnlyForceRecursive = true - } - return opts -} - -func buildVolumeOptions(vol *types.ServiceVolumeVolume) *mount.VolumeOptions { - if vol == nil { - return nil - } - return &mount.VolumeOptions{ - NoCopy: vol.NoCopy, - Subpath: vol.Subpath, - Labels: vol.Labels, - // DriverConfig: , // FIXME missing from model ? - } -} - -func buildTmpfsOptions(tmpfs *types.ServiceVolumeTmpfs) *mount.TmpfsOptions { - if tmpfs == nil { - return nil - } - return &mount.TmpfsOptions{ - SizeBytes: int64(tmpfs.Size), - Mode: os.FileMode(tmpfs.Mode), - } -} - -func buildImageOptions(image *types.ServiceVolumeImage) *mount.ImageOptions { - if image == nil { - return nil - } - return &mount.ImageOptions{ - Subpath: image.SubPath, - } -} - -func (s *composeService) ensureNetwork(ctx context.Context, project *types.Project, name string, n *types.NetworkConfig) (string, error) { - if n.External { - return s.resolveExternalNetwork(ctx, n) - } - - id, err := s.resolveOrCreateNetwork(ctx, project, name, n) - if errdefs.IsConflict(err) { - // Maybe another execution of `docker compose up|run` created same network - // let's retry once - return s.resolveOrCreateNetwork(ctx, project, name, n) - } - return id, err -} - -func (s *composeService) resolveOrCreateNetwork(ctx context.Context, project *types.Project, name string, n *types.NetworkConfig) (string, error) { //nolint:gocyclo - // This is containers that could be left after a diverged network was removed - var dangledContainers Containers - - // First, try to find a unique network matching by name or ID - inspect, err := s.apiClient().NetworkInspect(ctx, n.Name, network.InspectOptions{}) - if err == nil { - // NetworkInspect will match on ID prefix, so double check we get the expected one - // as looking for network named `db` we could erroneously match network ID `db9086999caf` - if inspect.Name == n.Name || inspect.ID == n.Name { - p, ok := inspect.Labels[api.ProjectLabel] - if !ok { - logrus.Warnf("a network with name %s exists but was not created by compose.\n"+ - "Set `external: true` to use an existing network", n.Name) - } else if p != project.Name { - logrus.Warnf("a network with name %s exists but was not created for project %q.\n"+ - "Set `external: true` to use an existing network", n.Name, project.Name) - } - if inspect.Labels[api.NetworkLabel] != name { - return "", fmt.Errorf( - "network %s was found but has incorrect label %s set to %q (expected: %q)", - n.Name, - api.NetworkLabel, - inspect.Labels[api.NetworkLabel], - name, - ) - } - - hash := inspect.Labels[api.ConfigHashLabel] - expected, err := NetworkHash(n) - if err != nil { - return "", err - } - if hash == "" || hash == expected { - return inspect.ID, nil - } - - dangledContainers, err = s.removeDivergedNetwork(ctx, project, name, n) - if err != nil { - return "", err - } - } - } - // ignore other errors. Typically, an ambiguous request by name results in some generic `invalidParameter` error - - // Either not found, or name is ambiguous - use NetworkList to list by name - networks, err := s.apiClient().NetworkList(ctx, network.ListOptions{ - Filters: filters.NewArgs(filters.Arg("name", n.Name)), - }) - if err != nil { - return "", err - } - - // NetworkList Matches all or part of a network name, so we have to filter for a strict match - networks = slices.DeleteFunc(networks, func(net network.Summary) bool { - return net.Name != n.Name - }) - - for _, net := range networks { - if net.Labels[api.ProjectLabel] == project.Name && - net.Labels[api.NetworkLabel] == name { - return net.ID, nil - } - } - - // we could have set NetworkList with a projectFilter and networkFilter but not doing so allows to catch this - // scenario were a network with same name exists but doesn't have label, and use of `CheckDuplicate: true` - // prevents to create another one. - if len(networks) > 0 { - logrus.Warnf("a network with name %s exists but was not created by compose.\n"+ - "Set `external: true` to use an existing network", n.Name) - return networks[0].ID, nil - } - - var ipam *network.IPAM - if n.Ipam.Config != nil { - var config []network.IPAMConfig - for _, pool := range n.Ipam.Config { - config = append(config, network.IPAMConfig{ - Subnet: pool.Subnet, - IPRange: pool.IPRange, - Gateway: pool.Gateway, - AuxAddress: pool.AuxiliaryAddresses, - }) - } - ipam = &network.IPAM{ - Driver: n.Ipam.Driver, - Config: config, - } - } - hash, err := NetworkHash(n) - if err != nil { - return "", err - } - n.CustomLabels = n.CustomLabels.Add(api.ConfigHashLabel, hash) - createOpts := network.CreateOptions{ - Labels: mergeLabels(n.Labels, n.CustomLabels), - Driver: n.Driver, - Options: n.DriverOpts, - Internal: n.Internal, - Attachable: n.Attachable, - IPAM: ipam, - EnableIPv6: n.EnableIPv6, - EnableIPv4: n.EnableIPv4, - } - - if n.Ipam.Driver != "" || len(n.Ipam.Config) > 0 { - createOpts.IPAM = &network.IPAM{} - } - - if n.Ipam.Driver != "" { - createOpts.IPAM.Driver = n.Ipam.Driver - } - - for _, ipamConfig := range n.Ipam.Config { - config := network.IPAMConfig{ - Subnet: ipamConfig.Subnet, - IPRange: ipamConfig.IPRange, - Gateway: ipamConfig.Gateway, - AuxAddress: ipamConfig.AuxiliaryAddresses, - } - createOpts.IPAM.Config = append(createOpts.IPAM.Config, config) - } - - networkEventName := fmt.Sprintf("Network %s", n.Name) - s.events.On(creatingEvent(networkEventName)) - - resp, err := s.apiClient().NetworkCreate(ctx, n.Name, createOpts) - if err != nil { - s.events.On(errorEvent(networkEventName, err.Error())) - return "", fmt.Errorf("failed to create network %s: %w", n.Name, err) - } - s.events.On(createdEvent(networkEventName)) - - err = s.connectNetwork(ctx, n.Name, dangledContainers, nil) - if err != nil { - return "", err - } - - return resp.ID, nil -} - -func (s *composeService) removeDivergedNetwork(ctx context.Context, project *types.Project, name string, n *types.NetworkConfig) (Containers, error) { - // Remove services attached to this network to force recreation - var services []string - for _, service := range project.Services.Filter(func(config types.ServiceConfig) bool { - _, ok := config.Networks[name] - return ok - }) { - services = append(services, service.Name) - } - - // Stop containers so we can remove network - // They will be restarted (actually: recreated) with the updated network - err := s.stop(ctx, project.Name, api.StopOptions{ - Services: services, - Project: project, - }, nil) - if err != nil { - return nil, err - } - - containers, err := s.getContainers(ctx, project.Name, oneOffExclude, true, services...) - if err != nil { - return nil, err - } - - err = s.disconnectNetwork(ctx, n.Name, containers) - if err != nil { - return nil, err - } - - err = s.apiClient().NetworkRemove(ctx, n.Name) - eventName := fmt.Sprintf("Network %s", n.Name) - s.events.On(removedEvent(eventName)) - return containers, err -} - -func (s *composeService) disconnectNetwork( - ctx context.Context, - nwName string, - containers Containers, -) error { - for _, c := range containers { - err := s.apiClient().NetworkDisconnect(ctx, nwName, c.ID, true) - if err != nil { - return err - } - } - - return nil -} - -func (s *composeService) connectNetwork( - ctx context.Context, - nwName string, - containers Containers, - config *network.EndpointSettings, -) error { - for _, c := range containers { - err := s.apiClient().NetworkConnect(ctx, nwName, c.ID, config) - if err != nil { - return err - } - } - - return nil -} - -func (s *composeService) resolveExternalNetwork(ctx context.Context, n *types.NetworkConfig) (string, error) { - // NetworkInspect will match on ID prefix, so NetworkList with a name - // filter is used to look for an exact match to prevent e.g. a network - // named `db` from getting erroneously matched to a network with an ID - // like `db9086999caf` - networks, err := s.apiClient().NetworkList(ctx, network.ListOptions{ - Filters: filters.NewArgs(filters.Arg("name", n.Name)), - }) - if err != nil { - return "", err - } - - if len(networks) == 0 { - // in this instance, n.Name is really an ID - sn, err := s.apiClient().NetworkInspect(ctx, n.Name, network.InspectOptions{}) - if err == nil { - networks = append(networks, sn) - } else if !errdefs.IsNotFound(err) { - return "", err - } - - } - - // NetworkList API doesn't return the exact name match, so we can retrieve more than one network with a request - networks = slices.DeleteFunc(networks, func(net network.Inspect) bool { - // this function is called during the rebuild stage of `compose watch`. - // we still require just one network back, but we need to run the search on the ID - return net.Name != n.Name && net.ID != n.Name - }) - - switch len(networks) { - case 1: - return networks[0].ID, nil - case 0: - enabled, err := s.isSwarmEnabled(ctx) - if err != nil { - return "", err - } - if enabled { - // Swarm nodes do not register overlay networks that were - // created on a different node unless they're in use. - // So we can't preemptively check network exists, but - // networkAttach will later fail anyway if network actually doesn't exist - return "swarm", nil - } - return "", fmt.Errorf("network %s declared as external, but could not be found", n.Name) - default: - return "", fmt.Errorf("multiple networks with name %q were found. Use network ID as `name` to avoid ambiguity", n.Name) - } -} - -func (s *composeService) ensureVolume(ctx context.Context, name string, volume types.VolumeConfig, project *types.Project) (string, error) { - inspected, err := s.apiClient().VolumeInspect(ctx, volume.Name) - if err != nil { - if !errdefs.IsNotFound(err) { - return "", err - } - if volume.External { - return "", fmt.Errorf("external volume %q not found", volume.Name) - } - err = s.createVolume(ctx, volume) - return volume.Name, err - } - - if volume.External { - return volume.Name, nil - } - - // Volume exists with name, but let's double-check this is the expected one - p, ok := inspected.Labels[api.ProjectLabel] - if !ok { - logrus.Warnf("volume %q already exists but was not created by Docker Compose. Use `external: true` to use an existing volume", volume.Name) - } - if ok && p != project.Name { - logrus.Warnf("volume %q already exists but was created for project %q (expected %q). Use `external: true` to use an existing volume", volume.Name, p, project.Name) - } - - expected, err := VolumeHash(volume) - if err != nil { - return "", err - } - actual, ok := inspected.Labels[api.ConfigHashLabel] - if ok && actual != expected { - msg := fmt.Sprintf("Volume %q exists but doesn't match configuration in compose file. Recreate (data will be lost)?", volume.Name) - confirm, err := s.prompt(msg, false) - if err != nil { - return "", err - } - if confirm { - err = s.removeDivergedVolume(ctx, name, volume, project) - if err != nil { - return "", err - } - return volume.Name, s.createVolume(ctx, volume) - } - } - return inspected.Name, nil -} - -func (s *composeService) removeDivergedVolume(ctx context.Context, name string, volume types.VolumeConfig, project *types.Project) error { - // Remove services mounting divergent volume - var services []string - for _, service := range project.Services.Filter(func(config types.ServiceConfig) bool { - for _, cfg := range config.Volumes { - if cfg.Source == name { - return true - } - } - return false - }) { - services = append(services, service.Name) - } - - err := s.stop(ctx, project.Name, api.StopOptions{ - Services: services, - Project: project, - }, nil) - if err != nil { - return err - } - - containers, err := s.getContainers(ctx, project.Name, oneOffExclude, true, services...) - if err != nil { - return err - } - - // FIXME (ndeloof) we have to remove container so we can recreate volume - // but doing so we can't inherit anonymous volumes from previous instance - err = s.remove(ctx, containers, api.RemoveOptions{ - Services: services, - Project: project, - }) - if err != nil { - return err - } - - return s.apiClient().VolumeRemove(ctx, volume.Name, true) -} - -func (s *composeService) createVolume(ctx context.Context, volume types.VolumeConfig) error { - eventName := fmt.Sprintf("Volume %s", volume.Name) - s.events.On(creatingEvent(eventName)) - hash, err := VolumeHash(volume) - if err != nil { - return err - } - volume.CustomLabels.Add(api.ConfigHashLabel, hash) - _, err = s.apiClient().VolumeCreate(ctx, volumetypes.CreateOptions{ - Labels: mergeLabels(volume.Labels, volume.CustomLabels), - Name: volume.Name, - Driver: volume.Driver, - DriverOpts: volume.DriverOpts, - }) - if err != nil { - s.events.On(errorEvent(eventName, err.Error())) - return err - } - s.events.On(createdEvent(eventName)) - return nil -} diff --git a/pkg/compose/create_test.go b/pkg/compose/create_test.go deleted file mode 100644 index 41cd3bcb8ac..00000000000 --- a/pkg/compose/create_test.go +++ /dev/null @@ -1,456 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "os" - "path/filepath" - "sort" - "testing" - - composeloader "github.com/compose-spec/compose-go/v2/loader" - composetypes "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/image" - mountTypes "github.com/docker/docker/api/types/mount" - "github.com/docker/docker/api/types/network" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - "gotest.tools/v3/assert/cmp" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestBuildBindMount(t *testing.T) { - project := composetypes.Project{} - volume := composetypes.ServiceVolumeConfig{ - Type: composetypes.VolumeTypeBind, - Source: "", - Target: "/data", - } - mount, err := buildMount(project, volume) - assert.NilError(t, err) - assert.Assert(t, filepath.IsAbs(mount.Source)) - _, err = os.Stat(mount.Source) - assert.NilError(t, err) - assert.Equal(t, mount.Type, mountTypes.TypeBind) -} - -func TestBuildNamedPipeMount(t *testing.T) { - project := composetypes.Project{} - volume := composetypes.ServiceVolumeConfig{ - Type: composetypes.VolumeTypeNamedPipe, - Source: "\\\\.\\pipe\\docker_engine_windows", - Target: "\\\\.\\pipe\\docker_engine", - } - mount, err := buildMount(project, volume) - assert.NilError(t, err) - assert.Equal(t, mount.Type, mountTypes.TypeNamedPipe) -} - -func TestBuildVolumeMount(t *testing.T) { - project := composetypes.Project{ - Name: "myProject", - Volumes: composetypes.Volumes(map[string]composetypes.VolumeConfig{ - "myVolume": { - Name: "myProject_myVolume", - }, - }), - } - volume := composetypes.ServiceVolumeConfig{ - Type: composetypes.VolumeTypeVolume, - Source: "myVolume", - Target: "/data", - } - mount, err := buildMount(project, volume) - assert.NilError(t, err) - assert.Equal(t, mount.Source, "myProject_myVolume") - assert.Equal(t, mount.Type, mountTypes.TypeVolume) -} - -func TestServiceImageName(t *testing.T) { - assert.Equal(t, api.GetImageNameOrDefault(composetypes.ServiceConfig{Image: "myImage"}, "myProject"), "myImage") - assert.Equal(t, api.GetImageNameOrDefault(composetypes.ServiceConfig{Name: "aService"}, "myProject"), "myProject-aService") -} - -func TestPrepareNetworkLabels(t *testing.T) { - project := composetypes.Project{ - Name: "myProject", - Networks: composetypes.Networks(map[string]composetypes.NetworkConfig{"skynet": {}}), - } - prepareNetworks(&project) - assert.DeepEqual(t, project.Networks["skynet"].CustomLabels, composetypes.Labels(map[string]string{ - "com.docker.compose.network": "skynet", - "com.docker.compose.project": "myProject", - "com.docker.compose.version": api.ComposeVersion, - })) -} - -func TestBuildContainerMountOptions(t *testing.T) { - project := composetypes.Project{ - Name: "myProject", - Services: composetypes.Services{ - "myService": { - Name: "myService", - Volumes: []composetypes.ServiceVolumeConfig{ - { - Type: composetypes.VolumeTypeVolume, - Target: "/var/myvolume1", - }, - { - Type: composetypes.VolumeTypeVolume, - Target: "/var/myvolume2", - }, - { - Type: composetypes.VolumeTypeVolume, - Source: "myVolume3", - Target: "/var/myvolume3", - Volume: &composetypes.ServiceVolumeVolume{ - Subpath: "etc", - }, - }, - { - Type: composetypes.VolumeTypeNamedPipe, - Source: "\\\\.\\pipe\\docker_engine_windows", - Target: "\\\\.\\pipe\\docker_engine", - }, - }, - }, - }, - Volumes: composetypes.Volumes(map[string]composetypes.VolumeConfig{ - "myVolume1": { - Name: "myProject_myVolume1", - }, - "myVolume2": { - Name: "myProject_myVolume2", - }, - }), - } - - inherit := &container.Summary{ - Mounts: []container.MountPoint{ - { - Type: composetypes.VolumeTypeVolume, - Destination: "/var/myvolume1", - }, - { - Type: composetypes.VolumeTypeVolume, - Destination: "/var/myvolume2", - }, - }, - } - - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - mock, cli := prepareMocks(mockCtrl) - s := composeService{ - dockerCli: cli, - } - mock.EXPECT().ImageInspect(gomock.Any(), "myProject-myService").AnyTimes().Return(image.InspectResponse{}, nil) - - mounts, err := s.buildContainerMountOptions(t.Context(), project, project.Services["myService"], inherit) - sort.Slice(mounts, func(i, j int) bool { - return mounts[i].Target < mounts[j].Target - }) - assert.NilError(t, err) - assert.Assert(t, len(mounts) == 4) - assert.Equal(t, mounts[0].Target, "/var/myvolume1") - assert.Equal(t, mounts[1].Target, "/var/myvolume2") - assert.Equal(t, mounts[2].Target, "/var/myvolume3") - assert.Equal(t, mounts[2].VolumeOptions.Subpath, "etc") - assert.Equal(t, mounts[3].Target, "\\\\.\\pipe\\docker_engine") - - mounts, err = s.buildContainerMountOptions(t.Context(), project, project.Services["myService"], inherit) - sort.Slice(mounts, func(i, j int) bool { - return mounts[i].Target < mounts[j].Target - }) - assert.NilError(t, err) - assert.Assert(t, len(mounts) == 4) - assert.Equal(t, mounts[0].Target, "/var/myvolume1") - assert.Equal(t, mounts[1].Target, "/var/myvolume2") - assert.Equal(t, mounts[2].Target, "/var/myvolume3") - assert.Equal(t, mounts[2].VolumeOptions.Subpath, "etc") - assert.Equal(t, mounts[3].Target, "\\\\.\\pipe\\docker_engine") -} - -func TestDefaultNetworkSettings(t *testing.T) { - t.Run("returns the network with the highest priority when service has multiple networks", func(t *testing.T) { - service := composetypes.ServiceConfig{ - Name: "myService", - Networks: map[string]*composetypes.ServiceNetworkConfig{ - "myNetwork1": { - Priority: 10, - }, - "myNetwork2": { - Priority: 1000, - }, - }, - } - project := composetypes.Project{ - Name: "myProject", - Services: composetypes.Services{ - "myService": service, - }, - Networks: composetypes.Networks(map[string]composetypes.NetworkConfig{ - "myNetwork1": { - Name: "myProject_myNetwork1", - }, - "myNetwork2": { - Name: "myProject_myNetwork2", - }, - }), - } - - networkMode, networkConfig, err := defaultNetworkSettings(&project, service, 1, nil, true, "1.43") - assert.NilError(t, err) - assert.Equal(t, string(networkMode), "myProject_myNetwork2") - assert.Check(t, cmp.Len(networkConfig.EndpointsConfig, 1)) - assert.Check(t, cmp.Contains(networkConfig.EndpointsConfig, "myProject_myNetwork2")) - }) - - t.Run("returns default network when service has no networks", func(t *testing.T) { - service := composetypes.ServiceConfig{ - Name: "myService", - } - project := composetypes.Project{ - Name: "myProject", - Services: composetypes.Services{ - "myService": service, - }, - Networks: composetypes.Networks(map[string]composetypes.NetworkConfig{ - "myNetwork1": { - Name: "myProject_myNetwork1", - }, - "myNetwork2": { - Name: "myProject_myNetwork2", - }, - "default": { - Name: "myProject_default", - }, - }), - } - - networkMode, networkConfig, err := defaultNetworkSettings(&project, service, 1, nil, true, "1.43") - assert.NilError(t, err) - assert.Equal(t, string(networkMode), "myProject_default") - assert.Check(t, cmp.Len(networkConfig.EndpointsConfig, 1)) - assert.Check(t, cmp.Contains(networkConfig.EndpointsConfig, "myProject_default")) - }) - - t.Run("returns none if project has no networks", func(t *testing.T) { - service := composetypes.ServiceConfig{ - Name: "myService", - } - project := composetypes.Project{ - Name: "myProject", - Services: composetypes.Services{ - "myService": service, - }, - } - - networkMode, networkConfig, err := defaultNetworkSettings(&project, service, 1, nil, true, "1.43") - assert.NilError(t, err) - assert.Equal(t, string(networkMode), "none") - assert.Check(t, cmp.Nil(networkConfig)) - }) - - t.Run("returns defined network mode if explicitly set", func(t *testing.T) { - service := composetypes.ServiceConfig{ - Name: "myService", - NetworkMode: "host", - } - project := composetypes.Project{ - Name: "myProject", - Services: composetypes.Services{"myService": service}, - Networks: composetypes.Networks(map[string]composetypes.NetworkConfig{ - "default": { - Name: "myProject_default", - }, - }), - } - - networkMode, networkConfig, err := defaultNetworkSettings(&project, service, 1, nil, true, "1.43") - assert.NilError(t, err) - assert.Equal(t, string(networkMode), "host") - assert.Check(t, cmp.Nil(networkConfig)) - }) -} - -func TestCreateEndpointSettings(t *testing.T) { - eps := createEndpointSettings(&composetypes.Project{ - Name: "projName", - }, composetypes.ServiceConfig{ - Name: "serviceName", - ContainerName: "containerName", - Networks: map[string]*composetypes.ServiceNetworkConfig{ - "netName": { - Priority: 100, - Aliases: []string{"alias1", "alias2"}, - Ipv4Address: "10.16.17.18", - Ipv6Address: "fdb4:7a7f:373a:3f0c::42", - LinkLocalIPs: []string{"169.254.10.20"}, - MacAddress: "10:00:00:00:01", - DriverOpts: composetypes.Options{ - "driverOpt1": "optval1", - "driverOpt2": "optval2", - }, - }, - }, - }, 0, "netName", []string{"link1", "link2"}, true) - assert.Check(t, cmp.DeepEqual(eps, &network.EndpointSettings{ - IPAMConfig: &network.EndpointIPAMConfig{ - IPv4Address: "10.16.17.18", - IPv6Address: "fdb4:7a7f:373a:3f0c::42", - LinkLocalIPs: []string{"169.254.10.20"}, - }, - Links: []string{"link1", "link2"}, - Aliases: []string{"containerName", "serviceName", "alias1", "alias2"}, - MacAddress: "10:00:00:00:01", - DriverOpts: map[string]string{ - "driverOpt1": "optval1", - "driverOpt2": "optval2", - }, - - // FIXME(robmry) - IPAddress and IPv6Gateway are "operational data" fields... - // - The IPv6 address here is the container's address, not the gateway. - // - Both fields will be cleared by the daemon, but they could be removed from - // the request. - IPAddress: "10.16.17.18", - IPv6Gateway: "fdb4:7a7f:373a:3f0c::42", - })) -} - -func Test_buildContainerVolumes(t *testing.T) { - pwd, err := os.Getwd() - assert.NilError(t, err) - - tests := []struct { - name string - yaml string - binds []string - mounts []mountTypes.Mount - }{ - { - name: "bind mount local path", - yaml: ` -services: - test: - volumes: - - ./data:/data -`, - binds: []string{filepath.Join(pwd, "data") + ":/data:rw"}, - mounts: nil, - }, - { - name: "bind mount, not create host path", - yaml: ` -services: - test: - volumes: - - type: bind - source: ./data - target: /data - bind: - create_host_path: false -`, - binds: nil, - mounts: []mountTypes.Mount{ - { - Type: "bind", - Source: filepath.Join(pwd, "data"), - Target: "/data", - BindOptions: &mountTypes.BindOptions{CreateMountpoint: false}, - }, - }, - }, - { - name: "mount volume", - yaml: ` -services: - test: - volumes: - - data:/data -volumes: - data: - name: my_volume -`, - binds: []string{"my_volume:/data:rw"}, - mounts: nil, - }, - { - name: "mount volume, readonly", - yaml: ` -services: - test: - volumes: - - data:/data:ro -volumes: - data: - name: my_volume -`, - binds: []string{"my_volume:/data:ro"}, - mounts: nil, - }, - { - name: "mount volume subpath", - yaml: ` -services: - test: - volumes: - - type: volume - source: data - target: /data - volume: - subpath: test/ -volumes: - data: - name: my_volume -`, - binds: nil, - mounts: []mountTypes.Mount{ - { - Type: "volume", - Source: "my_volume", - Target: "/data", - VolumeOptions: &mountTypes.VolumeOptions{Subpath: "test/"}, - }, - }, - }, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - p, err := composeloader.LoadWithContext(t.Context(), composetypes.ConfigDetails{ - ConfigFiles: []composetypes.ConfigFile{ - { - Filename: "test", - Content: []byte(tt.yaml), - }, - }, - }, func(options *composeloader.Options) { - options.SkipValidation = true - options.SkipConsistencyCheck = true - }) - assert.NilError(t, err) - s := &composeService{} - binds, mounts, err := s.buildContainerVolumes(t.Context(), *p, p.Services["test"], nil) - assert.NilError(t, err) - assert.DeepEqual(t, tt.binds, binds) - assert.DeepEqual(t, tt.mounts, mounts) - }) - } -} diff --git a/pkg/compose/dependencies.go b/pkg/compose/dependencies.go deleted file mode 100644 index c448fd5a176..00000000000 --- a/pkg/compose/dependencies.go +++ /dev/null @@ -1,478 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "slices" - "strings" - "sync" - - "github.com/compose-spec/compose-go/v2/types" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -// ServiceStatus indicates the status of a service -type ServiceStatus int - -// Services status flags -const ( - ServiceStopped ServiceStatus = iota - ServiceStarted -) - -type graphTraversal struct { - mu sync.Mutex - seen map[string]struct{} - ignored map[string]struct{} - - extremityNodesFn func(*Graph) []*Vertex // leaves or roots - adjacentNodesFn func(*Vertex) []*Vertex // getParents or getChildren - filterAdjacentByStatusFn func(*Graph, string, ServiceStatus) []*Vertex // filterChildren or filterParents - targetServiceStatus ServiceStatus - adjacentServiceStatusToSkip ServiceStatus - - visitorFn func(context.Context, string) error - maxConcurrency int -} - -func upDirectionTraversal(visitorFn func(context.Context, string) error) *graphTraversal { - return &graphTraversal{ - extremityNodesFn: leaves, - adjacentNodesFn: getParents, - filterAdjacentByStatusFn: filterChildren, - adjacentServiceStatusToSkip: ServiceStopped, - targetServiceStatus: ServiceStarted, - visitorFn: visitorFn, - } -} - -func downDirectionTraversal(visitorFn func(context.Context, string) error) *graphTraversal { - return &graphTraversal{ - extremityNodesFn: roots, - adjacentNodesFn: getChildren, - filterAdjacentByStatusFn: filterParents, - adjacentServiceStatusToSkip: ServiceStarted, - targetServiceStatus: ServiceStopped, - visitorFn: visitorFn, - } -} - -// InDependencyOrder applies the function to the services of the project taking in account the dependency order -func InDependencyOrder(ctx context.Context, project *types.Project, fn func(context.Context, string) error, options ...func(*graphTraversal)) error { - graph, err := NewGraph(project, ServiceStopped) - if err != nil { - return err - } - t := upDirectionTraversal(fn) - for _, option := range options { - option(t) - } - return t.visit(ctx, graph) -} - -// InReverseDependencyOrder applies the function to the services of the project in reverse order of dependencies -func InReverseDependencyOrder(ctx context.Context, project *types.Project, fn func(context.Context, string) error, options ...func(*graphTraversal)) error { - graph, err := NewGraph(project, ServiceStarted) - if err != nil { - return err - } - t := downDirectionTraversal(fn) - for _, option := range options { - option(t) - } - return t.visit(ctx, graph) -} - -func WithRootNodesAndDown(nodes []string) func(*graphTraversal) { - return func(t *graphTraversal) { - if len(nodes) == 0 { - return - } - originalFn := t.extremityNodesFn - t.extremityNodesFn = func(graph *Graph) []*Vertex { - var want []string - for _, node := range nodes { - vertex := graph.Vertices[node] - want = append(want, vertex.Service) - for _, v := range getAncestors(vertex) { - want = append(want, v.Service) - } - } - - t.ignored = map[string]struct{}{} - for k := range graph.Vertices { - if !slices.Contains(want, k) { - t.ignored[k] = struct{}{} - } - } - - return originalFn(graph) - } - } -} - -func (t *graphTraversal) visit(ctx context.Context, g *Graph) error { - expect := len(g.Vertices) - if expect == 0 { - return nil - } - - eg, ctx := errgroup.WithContext(ctx) - if t.maxConcurrency > 0 { - eg.SetLimit(t.maxConcurrency + 1) - } - nodeCh := make(chan *Vertex, expect) - defer close(nodeCh) - // nodeCh need to allow n=expect writers while reader goroutine could have returner after ctx.Done - eg.Go(func() error { - for { - select { - case <-ctx.Done(): - return nil - case node := <-nodeCh: - expect-- - if expect == 0 { - return nil - } - t.run(ctx, g, eg, t.adjacentNodesFn(node), nodeCh) - } - } - }) - - nodes := t.extremityNodesFn(g) - t.run(ctx, g, eg, nodes, nodeCh) - - return eg.Wait() -} - -// Note: this could be `graph.walk` or whatever -func (t *graphTraversal) run(ctx context.Context, graph *Graph, eg *errgroup.Group, nodes []*Vertex, nodeCh chan *Vertex) { - for _, node := range nodes { - // Don't start this service yet if all of its children have - // not been started yet. - if len(t.filterAdjacentByStatusFn(graph, node.Key, t.adjacentServiceStatusToSkip)) != 0 { - continue - } - - if !t.consume(node.Key) { - // another worker already visited this node - continue - } - - eg.Go(func() error { - var err error - if _, ignore := t.ignored[node.Service]; !ignore { - err = t.visitorFn(ctx, node.Service) - } - if err == nil { - graph.UpdateStatus(node.Key, t.targetServiceStatus) - } - nodeCh <- node - return err - }) - } -} - -func (t *graphTraversal) consume(nodeKey string) bool { - t.mu.Lock() - defer t.mu.Unlock() - if t.seen == nil { - t.seen = make(map[string]struct{}) - } - if _, ok := t.seen[nodeKey]; ok { - return false - } - t.seen[nodeKey] = struct{}{} - return true -} - -// Graph represents project as service dependencies -type Graph struct { - Vertices map[string]*Vertex - lock sync.RWMutex -} - -// Vertex represents a service in the dependencies structure -type Vertex struct { - Key string - Service string - Status ServiceStatus - Children map[string]*Vertex - Parents map[string]*Vertex -} - -func getParents(v *Vertex) []*Vertex { - return v.GetParents() -} - -// GetParents returns a slice with the parent vertices of the Vertex -func (v *Vertex) GetParents() []*Vertex { - var res []*Vertex - for _, p := range v.Parents { - res = append(res, p) - } - return res -} - -func getChildren(v *Vertex) []*Vertex { - return v.GetChildren() -} - -// getAncestors return all descendents for a vertex, might contain duplicates -func getAncestors(v *Vertex) []*Vertex { - var descendents []*Vertex - for _, parent := range v.GetParents() { - descendents = append(descendents, parent) - descendents = append(descendents, getAncestors(parent)...) - } - return descendents -} - -// GetChildren returns a slice with the child vertices of the Vertex -func (v *Vertex) GetChildren() []*Vertex { - var res []*Vertex - for _, p := range v.Children { - res = append(res, p) - } - return res -} - -// NewGraph returns the dependency graph of the services -func NewGraph(project *types.Project, initialStatus ServiceStatus) (*Graph, error) { - graph := &Graph{ - lock: sync.RWMutex{}, - Vertices: map[string]*Vertex{}, - } - - for _, s := range project.Services { - graph.AddVertex(s.Name, s.Name, initialStatus) - } - - for index, s := range project.Services { - for _, name := range s.GetDependencies() { - err := graph.AddEdge(s.Name, name) - if err != nil { - if !s.DependsOn[name].Required { - delete(s.DependsOn, name) - project.Services[index] = s - continue - } - if api.IsNotFoundError(err) { - ds, err := project.GetDisabledService(name) - if err == nil { - return nil, fmt.Errorf("service %s is required by %s but is disabled. Can be enabled by profiles %s", name, s.Name, ds.Profiles) - } - } - return nil, err - } - } - } - - if b, err := graph.HasCycles(); b { - return nil, err - } - - return graph, nil -} - -// NewVertex is the constructor function for the Vertex -func NewVertex(key string, service string, initialStatus ServiceStatus) *Vertex { - return &Vertex{ - Key: key, - Service: service, - Status: initialStatus, - Parents: map[string]*Vertex{}, - Children: map[string]*Vertex{}, - } -} - -// AddVertex adds a vertex to the Graph -func (g *Graph) AddVertex(key string, service string, initialStatus ServiceStatus) { - g.lock.Lock() - defer g.lock.Unlock() - - v := NewVertex(key, service, initialStatus) - g.Vertices[key] = v -} - -// AddEdge adds a relationship of dependency between vertices `source` and `destination` -func (g *Graph) AddEdge(source string, destination string) error { - g.lock.Lock() - defer g.lock.Unlock() - - sourceVertex := g.Vertices[source] - destinationVertex := g.Vertices[destination] - - if sourceVertex == nil { - return fmt.Errorf("could not find %s: %w", source, api.ErrNotFound) - } - if destinationVertex == nil { - return fmt.Errorf("could not find %s: %w", destination, api.ErrNotFound) - } - - // If they are already connected - if _, ok := sourceVertex.Children[destination]; ok { - return nil - } - - sourceVertex.Children[destination] = destinationVertex - destinationVertex.Parents[source] = sourceVertex - - return nil -} - -func leaves(g *Graph) []*Vertex { - return g.Leaves() -} - -// Leaves returns the slice of leaves of the graph -func (g *Graph) Leaves() []*Vertex { - g.lock.Lock() - defer g.lock.Unlock() - - var res []*Vertex - for _, v := range g.Vertices { - if len(v.Children) == 0 { - res = append(res, v) - } - } - - return res -} - -func roots(g *Graph) []*Vertex { - return g.Roots() -} - -// Roots returns the slice of "Roots" of the graph -func (g *Graph) Roots() []*Vertex { - g.lock.Lock() - defer g.lock.Unlock() - - var res []*Vertex - for _, v := range g.Vertices { - if len(v.Parents) == 0 { - res = append(res, v) - } - } - return res -} - -// UpdateStatus updates the status of a certain vertex -func (g *Graph) UpdateStatus(key string, status ServiceStatus) { - g.lock.Lock() - defer g.lock.Unlock() - g.Vertices[key].Status = status -} - -func filterChildren(g *Graph, k string, s ServiceStatus) []*Vertex { - return g.FilterChildren(k, s) -} - -// FilterChildren returns children of a certain vertex that are in a certain status -func (g *Graph) FilterChildren(key string, status ServiceStatus) []*Vertex { - g.lock.Lock() - defer g.lock.Unlock() - - var res []*Vertex - vertex := g.Vertices[key] - - for _, child := range vertex.Children { - if child.Status == status { - res = append(res, child) - } - } - - return res -} - -func filterParents(g *Graph, k string, s ServiceStatus) []*Vertex { - return g.FilterParents(k, s) -} - -// FilterParents returns the parents of a certain vertex that are in a certain status -func (g *Graph) FilterParents(key string, status ServiceStatus) []*Vertex { - g.lock.Lock() - defer g.lock.Unlock() - - var res []*Vertex - vertex := g.Vertices[key] - - for _, parent := range vertex.Parents { - if parent.Status == status { - res = append(res, parent) - } - } - - return res -} - -// HasCycles detects cycles in the graph -func (g *Graph) HasCycles() (bool, error) { - discovered := []string{} - finished := []string{} - - for _, vertex := range g.Vertices { - path := []string{ - vertex.Key, - } - if !slices.Contains(discovered, vertex.Key) && !slices.Contains(finished, vertex.Key) { - var err error - discovered, finished, err = g.visit(vertex.Key, path, discovered, finished) - if err != nil { - return true, err - } - } - } - - return false, nil -} - -func (g *Graph) visit(key string, path []string, discovered []string, finished []string) ([]string, []string, error) { - discovered = append(discovered, key) - - for _, v := range g.Vertices[key].Children { - path := append(path, v.Key) - if slices.Contains(discovered, v.Key) { - return nil, nil, fmt.Errorf("cycle found: %s", strings.Join(path, " -> ")) - } - - if !slices.Contains(finished, v.Key) { - if _, _, err := g.visit(v.Key, path, discovered, finished); err != nil { - return nil, nil, err - } - } - } - - discovered = remove(discovered, key) - finished = append(finished, key) - return discovered, finished, nil -} - -func remove(slice []string, item string) []string { - var s []string - for _, i := range slice { - if i != item { - s = append(s, i) - } - } - return s -} diff --git a/pkg/compose/dependencies_test.go b/pkg/compose/dependencies_test.go deleted file mode 100644 index b22b68b9908..00000000000 --- a/pkg/compose/dependencies_test.go +++ /dev/null @@ -1,429 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "sort" - "sync" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - testify "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/utils" -) - -func createTestProject() *types.Project { - return &types.Project{ - Services: types.Services{ - "test1": { - Name: "test1", - DependsOn: map[string]types.ServiceDependency{ - "test2": {}, - }, - }, - "test2": { - Name: "test2", - DependsOn: map[string]types.ServiceDependency{ - "test3": {}, - }, - }, - "test3": { - Name: "test3", - }, - }, - } -} - -func TestTraversalWithMultipleParents(t *testing.T) { - dependent := types.ServiceConfig{ - Name: "dependent", - DependsOn: make(types.DependsOnConfig), - } - - project := types.Project{ - Services: types.Services{"dependent": dependent}, - } - - for i := 1; i <= 100; i++ { - name := fmt.Sprintf("svc_%d", i) - dependent.DependsOn[name] = types.ServiceDependency{} - - svc := types.ServiceConfig{Name: name} - project.Services[name] = svc - } - - svc := make(chan string, 10) - seen := make(map[string]int) - done := make(chan struct{}) - go func() { - for service := range svc { - seen[service]++ - } - done <- struct{}{} - }() - - err := InDependencyOrder(t.Context(), &project, func(ctx context.Context, service string) error { - svc <- service - return nil - }) - require.NoError(t, err, "Error during iteration") - close(svc) - <-done - - testify.Len(t, seen, 101) - for svc, count := range seen { - assert.Equal(t, 1, count, "Service: %s", svc) - } -} - -func TestInDependencyUpCommandOrder(t *testing.T) { - var order []string - err := InDependencyOrder(t.Context(), createTestProject(), func(ctx context.Context, service string) error { - order = append(order, service) - return nil - }) - require.NoError(t, err, "Error during iteration") - require.Equal(t, []string{"test3", "test2", "test1"}, order) -} - -func TestInDependencyReverseDownCommandOrder(t *testing.T) { - var order []string - err := InReverseDependencyOrder(t.Context(), createTestProject(), func(ctx context.Context, service string) error { - order = append(order, service) - return nil - }) - require.NoError(t, err, "Error during iteration") - require.Equal(t, []string{"test1", "test2", "test3"}, order) -} - -func TestBuildGraph(t *testing.T) { - testCases := []struct { - desc string - services types.Services - expectedVertices map[string]*Vertex - }{ - { - desc: "builds graph with single service", - services: types.Services{ - "test": { - Name: "test", - DependsOn: types.DependsOnConfig{}, - }, - }, - expectedVertices: map[string]*Vertex{ - "test": { - Key: "test", - Service: "test", - Status: ServiceStopped, - Children: map[string]*Vertex{}, - Parents: map[string]*Vertex{}, - }, - }, - }, - { - desc: "builds graph with two separate services", - services: types.Services{ - "test": { - Name: "test", - DependsOn: types.DependsOnConfig{}, - }, - "another": { - Name: "another", - DependsOn: types.DependsOnConfig{}, - }, - }, - expectedVertices: map[string]*Vertex{ - "test": { - Key: "test", - Service: "test", - Status: ServiceStopped, - Children: map[string]*Vertex{}, - Parents: map[string]*Vertex{}, - }, - "another": { - Key: "another", - Service: "another", - Status: ServiceStopped, - Children: map[string]*Vertex{}, - Parents: map[string]*Vertex{}, - }, - }, - }, - { - desc: "builds graph with a service and a dependency", - services: types.Services{ - "test": { - Name: "test", - DependsOn: types.DependsOnConfig{ - "another": types.ServiceDependency{}, - }, - }, - "another": { - Name: "another", - DependsOn: types.DependsOnConfig{}, - }, - }, - expectedVertices: map[string]*Vertex{ - "test": { - Key: "test", - Service: "test", - Status: ServiceStopped, - Children: map[string]*Vertex{ - "another": {}, - }, - Parents: map[string]*Vertex{}, - }, - "another": { - Key: "another", - Service: "another", - Status: ServiceStopped, - Children: map[string]*Vertex{}, - Parents: map[string]*Vertex{ - "test": {}, - }, - }, - }, - }, - { - desc: "builds graph with multiple dependency levels", - services: types.Services{ - "test": { - Name: "test", - DependsOn: types.DependsOnConfig{ - "another": types.ServiceDependency{}, - }, - }, - "another": { - Name: "another", - DependsOn: types.DependsOnConfig{ - "another_dep": types.ServiceDependency{}, - }, - }, - "another_dep": { - Name: "another_dep", - DependsOn: types.DependsOnConfig{}, - }, - }, - expectedVertices: map[string]*Vertex{ - "test": { - Key: "test", - Service: "test", - Status: ServiceStopped, - Children: map[string]*Vertex{ - "another": {}, - }, - Parents: map[string]*Vertex{}, - }, - "another": { - Key: "another", - Service: "another", - Status: ServiceStopped, - Children: map[string]*Vertex{ - "another_dep": {}, - }, - Parents: map[string]*Vertex{ - "test": {}, - }, - }, - "another_dep": { - Key: "another_dep", - Service: "another_dep", - Status: ServiceStopped, - Children: map[string]*Vertex{}, - Parents: map[string]*Vertex{ - "another": {}, - }, - }, - }, - }, - } - for _, tC := range testCases { - t.Run(tC.desc, func(t *testing.T) { - project := types.Project{ - Services: tC.services, - } - - graph, err := NewGraph(&project, ServiceStopped) - assert.NilError(t, err, fmt.Sprintf("failed to build graph for: %s", tC.desc)) - - for k, vertex := range graph.Vertices { - expected, ok := tC.expectedVertices[k] - assert.Equal(t, true, ok) - assert.Equal(t, true, isVertexEqual(*expected, *vertex)) - } - }) - } -} - -func TestBuildGraphDependsOn(t *testing.T) { - testCases := []struct { - desc string - services types.Services - expectedVertices map[string]*Vertex - }{ - { - desc: "service depends on init container which is already removed", - services: types.Services{ - "test": { - Name: "test", - DependsOn: types.DependsOnConfig{ - "test-removed-init-container": types.ServiceDependency{ - Condition: "service_completed_successfully", - Restart: false, - Extensions: types.Extensions(nil), - Required: false, - }, - }, - }, - }, - expectedVertices: map[string]*Vertex{ - "test": { - Key: "test", - Service: "test", - Status: ServiceStopped, - Children: map[string]*Vertex{}, - Parents: map[string]*Vertex{}, - }, - }, - }, - } - for _, tC := range testCases { - t.Run(tC.desc, func(t *testing.T) { - project := types.Project{ - Services: tC.services, - } - - graph, err := NewGraph(&project, ServiceStopped) - assert.NilError(t, err, fmt.Sprintf("failed to build graph for: %s", tC.desc)) - - for k, vertex := range graph.Vertices { - expected, ok := tC.expectedVertices[k] - assert.Equal(t, true, ok) - assert.Equal(t, true, isVertexEqual(*expected, *vertex)) - } - }) - } -} - -func isVertexEqual(a, b Vertex) bool { - childrenEquality := true - for c := range a.Children { - if _, ok := b.Children[c]; !ok { - childrenEquality = false - } - } - parentEquality := true - for p := range a.Parents { - if _, ok := b.Parents[p]; !ok { - parentEquality = false - } - } - return a.Key == b.Key && - a.Service == b.Service && - childrenEquality && - parentEquality -} - -func TestWith_RootNodesAndUp(t *testing.T) { - graph := &Graph{ - lock: sync.RWMutex{}, - Vertices: map[string]*Vertex{}, - } - - /** graph topology: - A B - / \ / \ - G C E - \ / - D - | - F - */ - - graph.AddVertex("A", "A", 0) - graph.AddVertex("B", "B", 0) - graph.AddVertex("C", "C", 0) - graph.AddVertex("D", "D", 0) - graph.AddVertex("E", "E", 0) - graph.AddVertex("F", "F", 0) - graph.AddVertex("G", "G", 0) - - _ = graph.AddEdge("C", "A") - _ = graph.AddEdge("C", "B") - _ = graph.AddEdge("E", "B") - _ = graph.AddEdge("D", "C") - _ = graph.AddEdge("D", "E") - _ = graph.AddEdge("F", "D") - _ = graph.AddEdge("G", "A") - - tests := []struct { - name string - nodes []string - want []string - }{ - { - name: "whole graph", - nodes: []string{"A", "B"}, - want: []string{"A", "B", "C", "D", "E", "F", "G"}, - }, - { - name: "only leaves", - nodes: []string{"F", "G"}, - want: []string{"F", "G"}, - }, - { - name: "simple dependent", - nodes: []string{"D"}, - want: []string{"D", "F"}, - }, - { - name: "diamond dependents", - nodes: []string{"B"}, - want: []string{"B", "C", "D", "E", "F"}, - }, - { - name: "partial graph", - nodes: []string{"A"}, - want: []string{"A", "C", "D", "F", "G"}, - }, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - mx := sync.Mutex{} - expected := utils.Set[string]{} - expected.AddAll("C", "G", "D", "F") - var visited []string - - gt := downDirectionTraversal(func(ctx context.Context, s string) error { - mx.Lock() - defer mx.Unlock() - visited = append(visited, s) - return nil - }) - WithRootNodesAndDown(tt.nodes)(gt) - err := gt.visit(t.Context(), graph) - assert.NilError(t, err) - sort.Strings(visited) - assert.DeepEqual(t, tt.want, visited) - }) - } -} diff --git a/pkg/compose/desktop.go b/pkg/compose/desktop.go deleted file mode 100644 index 9b985407b23..00000000000 --- a/pkg/compose/desktop.go +++ /dev/null @@ -1,41 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strings" -) - -// engineLabelDesktopAddress is used to detect that Compose is running with a -// Docker Desktop context. When this label is present, the value is an endpoint -// address for an in-memory socket (AF_UNIX or named pipe). -const engineLabelDesktopAddress = "com.docker.desktop.address" - -func (s *composeService) isDesktopIntegrationActive(ctx context.Context) (bool, error) { - info, err := s.apiClient().Info(ctx) - if err != nil { - return false, err - } - for _, l := range info.Labels { - k, _, ok := strings.Cut(l, "=") - if ok && k == engineLabelDesktopAddress { - return true, nil - } - } - return false, nil -} diff --git a/pkg/compose/docker_cli_providers.go b/pkg/compose/docker_cli_providers.go deleted file mode 100644 index 207fa3e37a7..00000000000 --- a/pkg/compose/docker_cli_providers.go +++ /dev/null @@ -1,38 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "github.com/docker/cli/cli/command" -) - -// dockerCliContextInfo implements api.ContextInfo using Docker CLI -type dockerCliContextInfo struct { - cli command.Cli -} - -func (c *dockerCliContextInfo) CurrentContext() string { - return c.cli.CurrentContext() -} - -func (c *dockerCliContextInfo) ServerOSType() string { - return c.cli.ServerInfo().OSType -} - -func (c *dockerCliContextInfo) BuildKitEnabled() (bool, error) { - return c.cli.BuildKitEnabled() -} diff --git a/pkg/compose/down.go b/pkg/compose/down.go deleted file mode 100644 index 35eec82ebe3..00000000000 --- a/pkg/compose/down.go +++ /dev/null @@ -1,402 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "strings" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - containerType "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - imageapi "github.com/docker/docker/api/types/image" - "github.com/docker/docker/api/types/network" - "github.com/sirupsen/logrus" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -type downOp func() error - -func (s *composeService) Down(ctx context.Context, projectName string, options api.DownOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.down(ctx, strings.ToLower(projectName), options) - }, "down", s.events) -} - -func (s *composeService) down(ctx context.Context, projectName string, options api.DownOptions) error { //nolint:gocyclo - resourceToRemove := false - - include := oneOffExclude - if options.RemoveOrphans { - include = oneOffInclude - } - containers, err := s.getContainers(ctx, projectName, include, true) - if err != nil { - return err - } - - project := options.Project - if project == nil { - project, err = s.getProjectWithResources(ctx, containers, projectName) - if err != nil { - return err - } - } - - // Check requested services exists in model - services, err := checkSelectedServices(options, project) - if err != nil { - return err - } - - if len(options.Services) > 0 && len(services) == 0 { - logrus.Infof("Any of the services %v not running in project %q", options.Services, projectName) - return nil - } - - options.Services = services - - if len(containers) > 0 { - resourceToRemove = true - } - - err = InReverseDependencyOrder(ctx, project, func(c context.Context, service string) error { - serv := project.Services[service] - if serv.Provider != nil { - return s.runPlugin(ctx, project, serv, "down") - } - serviceContainers := containers.filter(isService(service)) - err := s.removeContainers(ctx, serviceContainers, &serv, options.Timeout, options.Volumes) - return err - }, WithRootNodesAndDown(options.Services)) - if err != nil { - return err - } - - orphans := containers.filter(isOrphaned(project)) - if options.RemoveOrphans && len(orphans) > 0 { - err := s.removeContainers(ctx, orphans, nil, options.Timeout, false) - if err != nil { - return err - } - } - - ops := s.ensureNetworksDown(ctx, project) - - if options.Images != "" { - imgOps, err := s.ensureImagesDown(ctx, project, options) - if err != nil { - return err - } - ops = append(ops, imgOps...) - } - - if options.Volumes { - ops = append(ops, s.ensureVolumesDown(ctx, project)...) - } - - if !resourceToRemove && len(ops) == 0 { - logrus.Warnf("Warning: No resource found to remove for project %q.", projectName) - } - - eg, ctx := errgroup.WithContext(ctx) - for _, op := range ops { - eg.Go(op) - } - return eg.Wait() -} - -func checkSelectedServices(options api.DownOptions, project *types.Project) ([]string, error) { - var services []string - for _, service := range options.Services { - _, err := project.GetService(service) - if err != nil { - if options.Project != nil { - // ran with an explicit compose.yaml file, so we should not ignore - return nil, err - } - // ran without an explicit compose.yaml file, so can't distinguish typo vs container already removed - } else { - services = append(services, service) - } - } - return services, nil -} - -func (s *composeService) ensureVolumesDown(ctx context.Context, project *types.Project) []downOp { - var ops []downOp - for _, vol := range project.Volumes { - if vol.External { - continue - } - volumeName := vol.Name - ops = append(ops, func() error { - return s.removeVolume(ctx, volumeName) - }) - } - - return ops -} - -func (s *composeService) ensureImagesDown(ctx context.Context, project *types.Project, options api.DownOptions) ([]downOp, error) { - imagePruner := NewImagePruner(s.apiClient(), project) - pruneOpts := ImagePruneOptions{ - Mode: ImagePruneMode(options.Images), - RemoveOrphans: options.RemoveOrphans, - } - images, err := imagePruner.ImagesToPrune(ctx, pruneOpts) - if err != nil { - return nil, err - } - - var ops []downOp - for i := range images { - img := images[i] - ops = append(ops, func() error { - return s.removeImage(ctx, img) - }) - } - return ops, nil -} - -func (s *composeService) ensureNetworksDown(ctx context.Context, project *types.Project) []downOp { - var ops []downOp - for key, n := range project.Networks { - if n.External { - continue - } - // loop capture variable for op closure - networkKey := key - idOrName := n.Name - ops = append(ops, func() error { - return s.removeNetwork(ctx, networkKey, project.Name, idOrName) - }) - } - return ops -} - -func (s *composeService) removeNetwork(ctx context.Context, composeNetworkName string, projectName string, name string) error { - networks, err := s.apiClient().NetworkList(ctx, network.ListOptions{ - Filters: filters.NewArgs( - projectFilter(projectName), - networkFilter(composeNetworkName)), - }) - if err != nil { - return fmt.Errorf("failed to list networks: %w", err) - } - - if len(networks) == 0 { - return nil - } - - eventName := fmt.Sprintf("Network %s", name) - s.events.On(removingEvent(eventName)) - - var found int - for _, net := range networks { - if net.Name != name { - continue - } - nw, err := s.apiClient().NetworkInspect(ctx, net.ID, network.InspectOptions{}) - if errdefs.IsNotFound(err) { - s.events.On(newEvent(eventName, api.Warning, "No resource found to remove")) - return nil - } - if err != nil { - return err - } - if len(nw.Containers) > 0 { - s.events.On(newEvent(eventName, api.Warning, "Resource is still in use")) - found++ - continue - } - - if err := s.apiClient().NetworkRemove(ctx, net.ID); err != nil { - if errdefs.IsNotFound(err) { - continue - } - s.events.On(errorEvent(eventName, err.Error())) - return fmt.Errorf("failed to remove network %s: %w", name, err) - } - s.events.On(removedEvent(eventName)) - found++ - } - - if found == 0 { - // in practice, it's extremely unlikely for this to ever occur, as it'd - // mean the network was present when we queried at the start of this - // method but was then deleted by something else in the interim - s.events.On(newEvent(eventName, api.Warning, "No resource found to remove")) - return nil - } - return nil -} - -func (s *composeService) removeImage(ctx context.Context, image string) error { - id := fmt.Sprintf("Image %s", image) - s.events.On(newEvent(id, api.Working, "Removing")) - _, err := s.apiClient().ImageRemove(ctx, image, imageapi.RemoveOptions{}) - if err == nil { - s.events.On(newEvent(id, api.Done, "Removed")) - return nil - } - if errdefs.IsConflict(err) { - s.events.On(newEvent(id, api.Warning, "Resource is still in use")) - return nil - } - if errdefs.IsNotFound(err) { - s.events.On(newEvent(id, api.Done, "Warning: No resource found to remove")) - return nil - } - return err -} - -func (s *composeService) removeVolume(ctx context.Context, id string) error { - resource := fmt.Sprintf("Volume %s", id) - - _, err := s.apiClient().VolumeInspect(ctx, id) - if errdefs.IsNotFound(err) { - // Already gone - return nil - } - - s.events.On(newEvent(resource, api.Working, "Removing")) - err = s.apiClient().VolumeRemove(ctx, id, true) - if err == nil { - s.events.On(newEvent(resource, api.Done, "Removed")) - return nil - } - if errdefs.IsConflict(err) { - s.events.On(newEvent(resource, api.Warning, "Resource is still in use")) - return nil - } - if errdefs.IsNotFound(err) { - s.events.On(newEvent(resource, api.Done, "Warning: No resource found to remove")) - return nil - } - return err -} - -func (s *composeService) stopContainer(ctx context.Context, service *types.ServiceConfig, ctr containerType.Summary, timeout *time.Duration, listener api.ContainerEventListener) error { - eventName := getContainerProgressName(ctr) - s.events.On(stoppingEvent(eventName)) - - if service != nil { - for _, hook := range service.PreStop { - err := s.runHook(ctx, ctr, *service, hook, listener) - if err != nil { - // Ignore errors indicating that some containers were already stopped or removed. - if errdefs.IsNotFound(err) || errdefs.IsConflict(err) { - return nil - } - return err - } - } - } - - timeoutInSecond := utils.DurationSecondToInt(timeout) - err := s.apiClient().ContainerStop(ctx, ctr.ID, containerType.StopOptions{Timeout: timeoutInSecond}) - if err != nil { - s.events.On(errorEvent(eventName, "Error while Stopping")) - return err - } - s.events.On(stoppedEvent(eventName)) - return nil -} - -func (s *composeService) stopContainers(ctx context.Context, serv *types.ServiceConfig, containers []containerType.Summary, timeout *time.Duration, listener api.ContainerEventListener) error { - eg, ctx := errgroup.WithContext(ctx) - for _, ctr := range containers { - eg.Go(func() error { - return s.stopContainer(ctx, serv, ctr, timeout, listener) - }) - } - return eg.Wait() -} - -func (s *composeService) removeContainers(ctx context.Context, containers []containerType.Summary, service *types.ServiceConfig, timeout *time.Duration, volumes bool) error { - eg, ctx := errgroup.WithContext(ctx) - for _, ctr := range containers { - eg.Go(func() error { - return s.stopAndRemoveContainer(ctx, ctr, service, timeout, volumes) - }) - } - return eg.Wait() -} - -func (s *composeService) stopAndRemoveContainer(ctx context.Context, ctr containerType.Summary, service *types.ServiceConfig, timeout *time.Duration, volumes bool) error { - eventName := getContainerProgressName(ctr) - err := s.stopContainer(ctx, service, ctr, timeout, nil) - if errdefs.IsNotFound(err) { - s.events.On(removedEvent(eventName)) - return nil - } - if err != nil { - return err - } - s.events.On(removingEvent(eventName)) - err = s.apiClient().ContainerRemove(ctx, ctr.ID, containerType.RemoveOptions{ - Force: true, - RemoveVolumes: volumes, - }) - if err != nil && !errdefs.IsNotFound(err) && !errdefs.IsConflict(err) { - s.events.On(errorEvent(eventName, "Error while Removing")) - return err - } - s.events.On(removedEvent(eventName)) - return nil -} - -func (s *composeService) getProjectWithResources(ctx context.Context, containers Containers, projectName string) (*types.Project, error) { - containers = containers.filter(isNotOneOff) - p, err := s.projectFromName(containers, projectName) - if err != nil && !api.IsNotFoundError(err) { - return nil, err - } - project, err := p.WithServicesTransform(func(name string, service types.ServiceConfig) (types.ServiceConfig, error) { - for k := range service.DependsOn { - if dependency, ok := service.DependsOn[k]; ok { - dependency.Required = false - service.DependsOn[k] = dependency - } - } - return service, nil - }) - if err != nil { - return nil, err - } - - volumes, err := s.actualVolumes(ctx, projectName) - if err != nil { - return nil, err - } - project.Volumes = volumes - - networks, err := s.actualNetworks(ctx, projectName) - if err != nil { - return nil, err - } - project.Networks = networks - - return project, nil -} diff --git a/pkg/compose/down_test.go b/pkg/compose/down_test.go deleted file mode 100644 index 0b5852a2fa3..00000000000 --- a/pkg/compose/down_test.go +++ /dev/null @@ -1,419 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "fmt" - "os" - "strings" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/docker/cli/cli/streams" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/api/types/volume" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - compose "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/mocks" -) - -func TestDown(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)).Return( - []container.Summary{ - testContainer("service1", "123", false), - testContainer("service2", "456", false), - testContainer("service2", "789", false), - testContainer("service_orphan", "321", true), - }, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - - // network names are not guaranteed to be unique, ensure Compose handles - // cleanup properly if duplicates are inadvertently created - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{ - {ID: "abc123", Name: "myProject_default", Labels: map[string]string{compose.NetworkLabel: "default"}}, - {ID: "def456", Name: "myProject_default", Labels: map[string]string{compose.NetworkLabel: "default"}}, - }, nil) - - stopOptions := container.StopOptions{} - api.EXPECT().ContainerStop(gomock.Any(), "123", stopOptions).Return(nil) - api.EXPECT().ContainerStop(gomock.Any(), "456", stopOptions).Return(nil) - api.EXPECT().ContainerStop(gomock.Any(), "789", stopOptions).Return(nil) - - api.EXPECT().ContainerRemove(gomock.Any(), "123", container.RemoveOptions{Force: true}).Return(nil) - api.EXPECT().ContainerRemove(gomock.Any(), "456", container.RemoveOptions{Force: true}).Return(nil) - api.EXPECT().ContainerRemove(gomock.Any(), "789", container.RemoveOptions{Force: true}).Return(nil) - - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{ - Filters: filters.NewArgs( - projectFilter(strings.ToLower(testProject)), - networkFilter("default")), - }).Return([]network.Summary{ - {ID: "abc123", Name: "myProject_default"}, - {ID: "def456", Name: "myProject_default"}, - }, nil) - api.EXPECT().NetworkInspect(gomock.Any(), "abc123", gomock.Any()).Return(network.Inspect{ID: "abc123"}, nil) - api.EXPECT().NetworkInspect(gomock.Any(), "def456", gomock.Any()).Return(network.Inspect{ID: "def456"}, nil) - api.EXPECT().NetworkRemove(gomock.Any(), "abc123").Return(nil) - api.EXPECT().NetworkRemove(gomock.Any(), "def456").Return(nil) - - err = tested.Down(t.Context(), strings.ToLower(testProject), compose.DownOptions{}) - assert.NilError(t, err) -} - -func TestDownWithGivenServices(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)).Return( - []container.Summary{ - testContainer("service1", "123", false), - testContainer("service2", "456", false), - testContainer("service2", "789", false), - testContainer("service_orphan", "321", true), - }, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - - // network names are not guaranteed to be unique, ensure Compose handles - // cleanup properly if duplicates are inadvertently created - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{ - {ID: "abc123", Name: "myProject_default", Labels: map[string]string{compose.NetworkLabel: "default"}}, - {ID: "def456", Name: "myProject_default", Labels: map[string]string{compose.NetworkLabel: "default"}}, - }, nil) - - stopOptions := container.StopOptions{} - api.EXPECT().ContainerStop(gomock.Any(), "123", stopOptions).Return(nil) - - api.EXPECT().ContainerRemove(gomock.Any(), "123", container.RemoveOptions{Force: true}).Return(nil) - - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{ - Filters: filters.NewArgs( - projectFilter(strings.ToLower(testProject)), - networkFilter("default")), - }).Return([]network.Summary{ - {ID: "abc123", Name: "myProject_default"}, - }, nil) - api.EXPECT().NetworkInspect(gomock.Any(), "abc123", gomock.Any()).Return(network.Inspect{ID: "abc123"}, nil) - api.EXPECT().NetworkRemove(gomock.Any(), "abc123").Return(nil) - - err = tested.Down(t.Context(), strings.ToLower(testProject), compose.DownOptions{ - Services: []string{"service1", "not-running-service"}, - }) - assert.NilError(t, err) -} - -func TestDownWithSpecifiedServiceButTheServicesAreNotRunning(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)).Return( - []container.Summary{ - testContainer("service1", "123", false), - testContainer("service2", "456", false), - testContainer("service2", "789", false), - testContainer("service_orphan", "321", true), - }, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - - // network names are not guaranteed to be unique, ensure Compose handles - // cleanup properly if duplicates are inadvertently created - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{ - {ID: "abc123", Name: "myProject_default", Labels: map[string]string{compose.NetworkLabel: "default"}}, - {ID: "def456", Name: "myProject_default", Labels: map[string]string{compose.NetworkLabel: "default"}}, - }, nil) - - err = tested.Down(t.Context(), strings.ToLower(testProject), compose.DownOptions{ - Services: []string{"not-running-service1", "not-running-service2"}, - }) - assert.NilError(t, err) -} - -func TestDownRemoveOrphans(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(true)).Return( - []container.Summary{ - testContainer("service1", "123", false), - testContainer("service2", "789", false), - testContainer("service_orphan", "321", true), - }, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{ - { - Name: "myProject_default", - Labels: map[string]string{compose.NetworkLabel: "default"}, - }, - }, nil) - - stopOptions := container.StopOptions{} - api.EXPECT().ContainerStop(gomock.Any(), "123", stopOptions).Return(nil) - api.EXPECT().ContainerStop(gomock.Any(), "789", stopOptions).Return(nil) - api.EXPECT().ContainerStop(gomock.Any(), "321", stopOptions).Return(nil) - - api.EXPECT().ContainerRemove(gomock.Any(), "123", container.RemoveOptions{Force: true}).Return(nil) - api.EXPECT().ContainerRemove(gomock.Any(), "789", container.RemoveOptions{Force: true}).Return(nil) - api.EXPECT().ContainerRemove(gomock.Any(), "321", container.RemoveOptions{Force: true}).Return(nil) - - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{ - Filters: filters.NewArgs( - networkFilter("default"), - projectFilter(strings.ToLower(testProject)), - ), - }).Return([]network.Summary{{ID: "abc123", Name: "myProject_default"}}, nil) - api.EXPECT().NetworkInspect(gomock.Any(), "abc123", gomock.Any()).Return(network.Inspect{ID: "abc123"}, nil) - api.EXPECT().NetworkRemove(gomock.Any(), "abc123").Return(nil) - - err = tested.Down(t.Context(), strings.ToLower(testProject), compose.DownOptions{RemoveOrphans: true}) - assert.NilError(t, err) -} - -func TestDownRemoveVolumes(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)).Return( - []container.Summary{testContainer("service1", "123", false)}, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{ - Volumes: []*volume.Volume{{Name: "myProject_volume"}}, - }, nil) - api.EXPECT().VolumeInspect(gomock.Any(), "myProject_volume"). - Return(volume.Volume{}, nil) - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return(nil, nil) - - api.EXPECT().ContainerStop(gomock.Any(), "123", container.StopOptions{}).Return(nil) - api.EXPECT().ContainerRemove(gomock.Any(), "123", container.RemoveOptions{Force: true, RemoveVolumes: true}).Return(nil) - - api.EXPECT().VolumeRemove(gomock.Any(), "myProject_volume", true).Return(nil) - - err = tested.Down(t.Context(), strings.ToLower(testProject), compose.DownOptions{Volumes: true}) - assert.NilError(t, err) -} - -func TestDownRemoveImages(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - opts := compose.DownOptions{ - Project: &types.Project{ - Name: strings.ToLower(testProject), - Services: types.Services{ - "local-anonymous": {Name: "local-anonymous"}, - "local-named": {Name: "local-named", Image: "local-named-image"}, - "remote": {Name: "remote", Image: "remote-image"}, - "remote-tagged": {Name: "remote-tagged", Image: "registry.example.com/remote-image-tagged:v1.0"}, - "no-images-anonymous": {Name: "no-images-anonymous"}, - "no-images-named": {Name: "no-images-named", Image: "missing-named-image"}, - }, - }, - } - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)). - Return([]container.Summary{ - testContainer("service1", "123", false), - }, nil). - AnyTimes() - - api.EXPECT().ImageList(gomock.Any(), image.ListOptions{ - Filters: filters.NewArgs( - projectFilter(strings.ToLower(testProject)), - filters.Arg("dangling", "false"), - ), - }).Return([]image.Summary{ - { - Labels: types.Labels{compose.ServiceLabel: "local-anonymous"}, - RepoTags: []string{"testproject-local-anonymous:latest"}, - }, - { - Labels: types.Labels{compose.ServiceLabel: "local-named"}, - RepoTags: []string{"local-named-image:latest"}, - }, - }, nil).AnyTimes() - - imagesToBeInspected := map[string]bool{ - "testproject-local-anonymous": true, - "local-named-image": true, - "remote-image": true, - "testproject-no-images-anonymous": false, - "missing-named-image": false, - } - for img, exists := range imagesToBeInspected { - var resp image.InspectResponse - var err error - if exists { - resp.RepoTags = []string{img} - } else { - err = errdefs.ErrNotFound.WithMessage(fmt.Sprintf("test specified that image %q should not exist", img)) - } - - api.EXPECT().ImageInspect(gomock.Any(), img). - Return(resp, err). - AnyTimes() - } - - api.EXPECT().ImageInspect(gomock.Any(), "registry.example.com/remote-image-tagged:v1.0"). - Return(image.InspectResponse{RepoTags: []string{"registry.example.com/remote-image-tagged:v1.0"}}, nil). - AnyTimes() - - localImagesToBeRemoved := []string{ - "testproject-local-anonymous:latest", - "local-named-image:latest", - } - for _, img := range localImagesToBeRemoved { - // test calls down --rmi=local then down --rmi=all, so local images - // get "removed" 2x, while other images are only 1x - api.EXPECT().ImageRemove(gomock.Any(), img, image.RemoveOptions{}). - Return(nil, nil). - Times(2) - } - - t.Log("-> docker compose down --rmi=local") - opts.Images = "local" - err = tested.Down(t.Context(), strings.ToLower(testProject), opts) - assert.NilError(t, err) - - otherImagesToBeRemoved := []string{ - "remote-image:latest", - "registry.example.com/remote-image-tagged:v1.0", - } - for _, img := range otherImagesToBeRemoved { - api.EXPECT().ImageRemove(gomock.Any(), img, image.RemoveOptions{}). - Return(nil, nil). - Times(1) - } - - t.Log("-> docker compose down --rmi=all") - opts.Images = "all" - err = tested.Down(t.Context(), strings.ToLower(testProject), opts) - assert.NilError(t, err) -} - -func TestDownRemoveImages_NoLabel(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - ctr := testContainer("service1", "123", false) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)).Return( - []container.Summary{ctr}, nil) - - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{ - Volumes: []*volume.Volume{{Name: "myProject_volume"}}, - }, nil) - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return(nil, nil) - - // ImageList returns no images for the project since they were unlabeled - // (created by an older version of Compose) - api.EXPECT().ImageList(gomock.Any(), image.ListOptions{ - Filters: filters.NewArgs( - projectFilter(strings.ToLower(testProject)), - filters.Arg("dangling", "false"), - ), - }).Return(nil, nil) - - api.EXPECT().ImageInspect(gomock.Any(), "testproject-service1"). - Return(image.InspectResponse{}, nil) - - api.EXPECT().ContainerStop(gomock.Any(), "123", container.StopOptions{}).Return(nil) - api.EXPECT().ContainerRemove(gomock.Any(), "123", container.RemoveOptions{Force: true}).Return(nil) - - api.EXPECT().ImageRemove(gomock.Any(), "testproject-service1:latest", image.RemoveOptions{}).Return(nil, nil) - - err = tested.Down(t.Context(), strings.ToLower(testProject), compose.DownOptions{Images: "local"}) - assert.NilError(t, err) -} - -func prepareMocks(mockCtrl *gomock.Controller) (*mocks.MockAPIClient, *mocks.MockCli) { - api := mocks.NewMockAPIClient(mockCtrl) - cli := mocks.NewMockCli(mockCtrl) - cli.EXPECT().Client().Return(api).AnyTimes() - cli.EXPECT().Err().Return(streams.NewOut(os.Stderr)).AnyTimes() - cli.EXPECT().Out().Return(streams.NewOut(os.Stdout)).AnyTimes() - return api, cli -} diff --git a/pkg/compose/envresolver.go b/pkg/compose/envresolver.go deleted file mode 100644 index a86d9351919..00000000000 --- a/pkg/compose/envresolver.go +++ /dev/null @@ -1,66 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "runtime" - "strings" -) - -// isCaseInsensitiveEnvVars is true on platforms where environment variable names are treated case-insensitively. -var isCaseInsensitiveEnvVars = (runtime.GOOS == "windows") - -// envResolver returns resolver for environment variables suitable for the current platform. -// Expected to be used with `MappingWithEquals.Resolve`. -// Updates in `environment` may not be reflected. -func envResolver(environment map[string]string) func(string) (string, bool) { - return envResolverWithCase(environment, isCaseInsensitiveEnvVars) -} - -// envResolverWithCase returns resolver for environment variables with the specified case-sensitive condition. -// Expected to be used with `MappingWithEquals.Resolve`. -// Updates in `environment` may not be reflected. -func envResolverWithCase(environment map[string]string, caseInsensitive bool) func(string) (string, bool) { - if environment == nil { - return func(s string) (string, bool) { - return "", false - } - } - if !caseInsensitive { - return func(s string) (string, bool) { - v, ok := environment[s] - return v, ok - } - } - // variable names must be treated case-insensitively. - // Resolves in this way: - // * Return the value if its name matches with the passed name case-sensitively. - // * Otherwise, return the value if its lower-cased name matches lower-cased passed name. - // * The value is indefinite if multiple variable matches. - loweredEnvironment := make(map[string]string, len(environment)) - for k, v := range environment { - loweredEnvironment[strings.ToLower(k)] = v - } - return func(s string) (string, bool) { - v, ok := environment[s] - if ok { - return v, ok - } - v, ok = loweredEnvironment[strings.ToLower(s)] - return v, ok - } -} diff --git a/pkg/compose/envresolver_test.go b/pkg/compose/envresolver_test.go deleted file mode 100644 index fca5f719186..00000000000 --- a/pkg/compose/envresolver_test.go +++ /dev/null @@ -1,115 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "gotest.tools/v3/assert" -) - -func Test_EnvResolverWithCase(t *testing.T) { - tests := []struct { - name string - environment map[string]string - caseInsensitive bool - search string - expectedValue string - expectedOk bool - }{ - { - name: "case sensitive/case match", - environment: map[string]string{ - "Env1": "Value1", - "Env2": "Value2", - }, - caseInsensitive: false, - search: "Env1", - expectedValue: "Value1", - expectedOk: true, - }, - { - name: "case sensitive/case unmatch", - environment: map[string]string{ - "Env1": "Value1", - "Env2": "Value2", - }, - caseInsensitive: false, - search: "ENV1", - expectedValue: "", - expectedOk: false, - }, - { - name: "case sensitive/nil environment", - environment: nil, - caseInsensitive: false, - search: "Env1", - expectedValue: "", - expectedOk: false, - }, - { - name: "case insensitive/case match", - environment: map[string]string{ - "Env1": "Value1", - "Env2": "Value2", - }, - caseInsensitive: true, - search: "Env1", - expectedValue: "Value1", - expectedOk: true, - }, - { - name: "case insensitive/case unmatch", - environment: map[string]string{ - "Env1": "Value1", - "Env2": "Value2", - }, - caseInsensitive: true, - search: "ENV1", - expectedValue: "Value1", - expectedOk: true, - }, - { - name: "case insensitive/unmatch", - environment: map[string]string{ - "Env1": "Value1", - "Env2": "Value2", - }, - caseInsensitive: true, - search: "Env3", - expectedValue: "", - expectedOk: false, - }, - { - name: "case insensitive/nil environment", - environment: nil, - caseInsensitive: true, - search: "Env1", - expectedValue: "", - expectedOk: false, - }, - } - - for _, test := range tests { - t.Run(test.name, func(t *testing.T) { - f := envResolverWithCase(test.environment, test.caseInsensitive) - v, ok := f(test.search) - assert.Equal(t, v, test.expectedValue) - assert.Equal(t, ok, test.expectedOk) - }) - } -} diff --git a/pkg/compose/events.go b/pkg/compose/events.go deleted file mode 100644 index af75bd7c530..00000000000 --- a/pkg/compose/events.go +++ /dev/null @@ -1,83 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "slices" - "strings" - "time" - - "github.com/docker/docker/api/types/events" - "github.com/docker/docker/api/types/filters" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Events(ctx context.Context, projectName string, options api.EventsOptions) error { - projectName = strings.ToLower(projectName) - evts, errors := s.apiClient().Events(ctx, events.ListOptions{ - Filters: filters.NewArgs(projectFilter(projectName)), - Since: options.Since, - Until: options.Until, - }) - for { - select { - case event := <-evts: - // TODO: support other event types - if event.Type != "container" { - continue - } - - oneOff := event.Actor.Attributes[api.OneoffLabel] - if oneOff == "True" { - // ignore - continue - } - service := event.Actor.Attributes[api.ServiceLabel] - if len(options.Services) > 0 && !slices.Contains(options.Services, service) { - continue - } - - attributes := map[string]string{} - for k, v := range event.Actor.Attributes { - if strings.HasPrefix(k, "com.docker.compose.") { - continue - } - attributes[k] = v - } - - timestamp := time.Unix(event.Time, 0) - if event.TimeNano != 0 { - timestamp = time.Unix(0, event.TimeNano) - } - err := options.Consumer(api.Event{ - Timestamp: timestamp, - Service: service, - Container: event.Actor.ID, - Status: string(event.Action), - Attributes: attributes, - }) - if err != nil { - return err - } - - case err := <-errors: - return err - } - } -} diff --git a/pkg/compose/exec.go b/pkg/compose/exec.go deleted file mode 100644 index 311aebaa2c9..00000000000 --- a/pkg/compose/exec.go +++ /dev/null @@ -1,63 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "strings" - - "github.com/docker/cli/cli" - "github.com/docker/cli/cli/command/container" - containerType "github.com/docker/docker/api/types/container" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Exec(ctx context.Context, projectName string, options api.RunOptions) (int, error) { - projectName = strings.ToLower(projectName) - target, err := s.getExecTarget(ctx, projectName, options) - if err != nil { - return 0, err - } - - exec := container.NewExecOptions() - exec.Interactive = options.Interactive - exec.TTY = options.Tty - exec.Detach = options.Detach - exec.User = options.User - exec.Privileged = options.Privileged - exec.Workdir = options.WorkingDir - exec.Command = options.Command - for _, v := range options.Environment { - err := exec.Env.Set(v) - if err != nil { - return 0, err - } - } - - err = container.RunExec(ctx, s.dockerCli, target.ID, exec) - var sterr cli.StatusError - if errors.As(err, &sterr) { - return sterr.StatusCode, err - } - return 0, err -} - -func (s *composeService) getExecTarget(ctx context.Context, projectName string, opts api.RunOptions) (containerType.Summary, error) { - return s.getSpecifiedContainer(ctx, projectName, oneOffInclude, false, opts.Service, opts.Index) -} diff --git a/pkg/compose/export.go b/pkg/compose/export.go deleted file mode 100644 index 5de9d837be9..00000000000 --- a/pkg/compose/export.go +++ /dev/null @@ -1,94 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - "strings" - - "github.com/docker/cli/cli/command" - "github.com/moby/sys/atomicwriter" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Export(ctx context.Context, projectName string, options api.ExportOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.export(ctx, projectName, options) - }, "export", s.events) -} - -func (s *composeService) export(ctx context.Context, projectName string, options api.ExportOptions) error { - projectName = strings.ToLower(projectName) - - container, err := s.getSpecifiedContainer(ctx, projectName, oneOffInclude, false, options.Service, options.Index) - if err != nil { - return err - } - - if options.Output == "" { - if s.stdout().IsTerminal() { - return fmt.Errorf("output option is required when exporting to terminal") - } - } else if err := command.ValidateOutputPath(options.Output); err != nil { - return fmt.Errorf("failed to export container: %w", err) - } - - name := getCanonicalContainerName(container) - s.events.On(api.Resource{ - ID: name, - Text: api.StatusExporting, - Status: api.Working, - }) - - responseBody, err := s.apiClient().ContainerExport(ctx, container.ID) - if err != nil { - return err - } - - defer func() { - if err := responseBody.Close(); err != nil { - s.events.On(errorEventf(name, "Failed to close response body: %s", err.Error())) - } - }() - - if !s.dryRun { - if options.Output == "" { - _, err := io.Copy(s.stdout(), responseBody) - return err - } else { - writer, err := atomicwriter.New(options.Output, 0o600) - if err != nil { - return err - } - defer func() { _ = writer.Close() }() - - _, err = io.Copy(writer, responseBody) - return err - } - } - - s.events.On(api.Resource{ - ID: name, - Text: api.StatusExported, - Status: api.Done, - }) - - return nil -} diff --git a/pkg/compose/filters.go b/pkg/compose/filters.go deleted file mode 100644 index f3038ab1444..00000000000 --- a/pkg/compose/filters.go +++ /dev/null @@ -1,57 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "fmt" - - "github.com/docker/docker/api/types/filters" - - "github.com/docker/compose/v5/pkg/api" -) - -func projectFilter(projectName string) filters.KeyValuePair { - return filters.Arg("label", fmt.Sprintf("%s=%s", api.ProjectLabel, projectName)) -} - -func serviceFilter(serviceName string) filters.KeyValuePair { - return filters.Arg("label", fmt.Sprintf("%s=%s", api.ServiceLabel, serviceName)) -} - -func networkFilter(name string) filters.KeyValuePair { - return filters.Arg("label", fmt.Sprintf("%s=%s", api.NetworkLabel, name)) -} - -func oneOffFilter(b bool) filters.KeyValuePair { - v := "False" - if b { - v = "True" - } - return filters.Arg("label", fmt.Sprintf("%s=%s", api.OneoffLabel, v)) -} - -func containerNumberFilter(index int) filters.KeyValuePair { - return filters.Arg("label", fmt.Sprintf("%s=%d", api.ContainerNumberLabel, index)) -} - -func hasProjectLabelFilter() filters.KeyValuePair { - return filters.Arg("label", api.ProjectLabel) -} - -func hasConfigHashLabel() filters.KeyValuePair { - return filters.Arg("label", api.ConfigHashLabel) -} diff --git a/pkg/compose/generate.go b/pkg/compose/generate.go deleted file mode 100644 index a23bfe8f217..00000000000 --- a/pkg/compose/generate.go +++ /dev/null @@ -1,248 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "maps" - "slices" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/mount" - "github.com/docker/docker/api/types/network" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Generate(ctx context.Context, options api.GenerateOptions) (*types.Project, error) { - filtersListNames := filters.NewArgs() - filtersListIDs := filters.NewArgs() - for _, containerName := range options.Containers { - filtersListNames.Add("name", containerName) - filtersListIDs.Add("id", containerName) - } - containers, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - Filters: filtersListNames, - All: true, - }) - if err != nil { - return nil, err - } - - containersByIds, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - Filters: filtersListIDs, - All: true, - }) - if err != nil { - return nil, err - } - - for _, ctr := range containersByIds { - if !slices.ContainsFunc(containers, func(summary container.Summary) bool { - return summary.ID == ctr.ID - }) { - containers = append(containers, ctr) - } - } - - if len(containers) == 0 { - return nil, fmt.Errorf("no container(s) found with the following name(s): %s", strings.Join(options.Containers, ",")) - } - - return s.createProjectFromContainers(containers, options.ProjectName) -} - -func (s *composeService) createProjectFromContainers(containers []container.Summary, projectName string) (*types.Project, error) { - project := &types.Project{} - services := types.Services{} - networks := types.Networks{} - volumes := types.Volumes{} - secrets := types.Secrets{} - - if projectName != "" { - project.Name = projectName - } - - for _, c := range containers { - // if the container is from a previous Compose application, use the existing service name - serviceLabel, ok := c.Labels[api.ServiceLabel] - if !ok { - serviceLabel = getCanonicalContainerName(c) - } - service, ok := services[serviceLabel] - if !ok { - service = types.ServiceConfig{ - Name: serviceLabel, - Image: c.Image, - Labels: c.Labels, - } - } - service.Scale = increment(service.Scale) - - inspect, err := s.apiClient().ContainerInspect(context.Background(), c.ID) - if err != nil { - services[serviceLabel] = service - continue - } - s.extractComposeConfiguration(&service, inspect, volumes, secrets, networks) - service.Labels = cleanDockerPreviousLabels(service.Labels) - services[serviceLabel] = service - } - - project.Services = services - project.Networks = networks - project.Volumes = volumes - project.Secrets = secrets - return project, nil -} - -func (s *composeService) extractComposeConfiguration(service *types.ServiceConfig, inspect container.InspectResponse, volumes types.Volumes, secrets types.Secrets, networks types.Networks) { - service.Environment = types.NewMappingWithEquals(inspect.Config.Env) - if inspect.Config.Healthcheck != nil { - healthConfig := inspect.Config.Healthcheck - service.HealthCheck = s.toComposeHealthCheck(healthConfig) - } - if len(inspect.Mounts) > 0 { - detectedVolumes, volumeConfigs, detectedSecrets, secretsConfigs := s.toComposeVolumes(inspect.Mounts) - service.Volumes = append(service.Volumes, volumeConfigs...) - service.Secrets = append(service.Secrets, secretsConfigs...) - maps.Copy(volumes, detectedVolumes) - maps.Copy(secrets, detectedSecrets) - } - if len(inspect.NetworkSettings.Networks) > 0 { - detectedNetworks, networkConfigs := s.toComposeNetwork(inspect.NetworkSettings.Networks) - service.Networks = networkConfigs - maps.Copy(networks, detectedNetworks) - } - if len(inspect.HostConfig.PortBindings) > 0 { - for key, portBindings := range inspect.HostConfig.PortBindings { - for _, portBinding := range portBindings { - service.Ports = append(service.Ports, types.ServicePortConfig{ - Target: uint32(key.Int()), - Published: portBinding.HostPort, - Protocol: key.Proto(), - HostIP: portBinding.HostIP, - }) - } - } - } -} - -func (s *composeService) toComposeHealthCheck(healthConfig *container.HealthConfig) *types.HealthCheckConfig { - var healthCheck types.HealthCheckConfig - healthCheck.Test = healthConfig.Test - if healthConfig.Timeout != 0 { - timeout := types.Duration(healthConfig.Timeout) - healthCheck.Timeout = &timeout - } - if healthConfig.Interval != 0 { - interval := types.Duration(healthConfig.Interval) - healthCheck.Interval = &interval - } - if healthConfig.StartPeriod != 0 { - startPeriod := types.Duration(healthConfig.StartPeriod) - healthCheck.StartPeriod = &startPeriod - } - if healthConfig.StartInterval != 0 { - startInterval := types.Duration(healthConfig.StartInterval) - healthCheck.StartInterval = &startInterval - } - if healthConfig.Retries != 0 { - retries := uint64(healthConfig.Retries) - healthCheck.Retries = &retries - } - return &healthCheck -} - -func (s *composeService) toComposeVolumes(volumes []container.MountPoint) (map[string]types.VolumeConfig, - []types.ServiceVolumeConfig, map[string]types.SecretConfig, []types.ServiceSecretConfig, -) { - volumeConfigs := make(map[string]types.VolumeConfig) - secretConfigs := make(map[string]types.SecretConfig) - var serviceVolumeConfigs []types.ServiceVolumeConfig - var serviceSecretConfigs []types.ServiceSecretConfig - - for _, volume := range volumes { - serviceVC := types.ServiceVolumeConfig{ - Type: string(volume.Type), - Source: volume.Source, - Target: volume.Destination, - ReadOnly: !volume.RW, - } - switch volume.Type { - case mount.TypeVolume: - serviceVC.Source = volume.Name - vol := types.VolumeConfig{} - if volume.Driver != "local" { - vol.Driver = volume.Driver - vol.Name = volume.Name - } - volumeConfigs[volume.Name] = vol - serviceVolumeConfigs = append(serviceVolumeConfigs, serviceVC) - case mount.TypeBind: - if strings.HasPrefix(volume.Destination, "/run/secrets") { - destination := strings.Split(volume.Destination, "/") - secret := types.SecretConfig{ - Name: destination[len(destination)-1], - File: strings.TrimPrefix(volume.Source, "/host_mnt"), - } - secretConfigs[secret.Name] = secret - serviceSecretConfigs = append(serviceSecretConfigs, types.ServiceSecretConfig{ - Source: secret.Name, - Target: volume.Destination, - }) - } else { - serviceVolumeConfigs = append(serviceVolumeConfigs, serviceVC) - } - } - } - return volumeConfigs, serviceVolumeConfigs, secretConfigs, serviceSecretConfigs -} - -func (s *composeService) toComposeNetwork(networks map[string]*network.EndpointSettings) (map[string]types.NetworkConfig, map[string]*types.ServiceNetworkConfig) { - networkConfigs := make(map[string]types.NetworkConfig) - serviceNetworkConfigs := make(map[string]*types.ServiceNetworkConfig) - - for name, net := range networks { - inspect, err := s.apiClient().NetworkInspect(context.Background(), name, network.InspectOptions{}) - if err != nil { - networkConfigs[name] = types.NetworkConfig{} - } else { - networkConfigs[name] = types.NetworkConfig{ - Internal: inspect.Internal, - } - } - serviceNetworkConfigs[name] = &types.ServiceNetworkConfig{ - Aliases: net.Aliases, - } - } - return networkConfigs, serviceNetworkConfigs -} - -func cleanDockerPreviousLabels(labels types.Labels) types.Labels { - cleanedLabels := types.Labels{} - for key, value := range labels { - if !strings.HasPrefix(key, "com.docker.compose.") && !strings.HasPrefix(key, "desktop.docker.io") { - cleanedLabels[key] = value - } - } - return cleanedLabels -} diff --git a/pkg/compose/hash.go b/pkg/compose/hash.go deleted file mode 100644 index 24c6cce2e88..00000000000 --- a/pkg/compose/hash.go +++ /dev/null @@ -1,64 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "encoding/json" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/opencontainers/go-digest" -) - -// ServiceHash computes the configuration hash for a service. -func ServiceHash(o types.ServiceConfig) (string, error) { - // remove the Build config when generating the service hash - o.Build = nil - o.PullPolicy = "" - o.Scale = nil - if o.Deploy != nil { - o.Deploy.Replicas = nil - } - o.DependsOn = nil - o.Profiles = nil - - bytes, err := json.Marshal(o) - if err != nil { - return "", err - } - return digest.SHA256.FromBytes(bytes).Encoded(), nil -} - -// NetworkHash computes the configuration hash for a network. -func NetworkHash(o *types.NetworkConfig) (string, error) { - bytes, err := json.Marshal(o) - if err != nil { - return "", err - } - return digest.SHA256.FromBytes(bytes).Encoded(), nil -} - -// VolumeHash computes the configuration hash for a volume. -func VolumeHash(o types.VolumeConfig) (string, error) { - if o.Driver == "" { // (TODO: jhrotko) This probably should be fixed in compose-go - o.Driver = "local" - } - bytes, err := json.Marshal(o) - if err != nil { - return "", err - } - return digest.SHA256.FromBytes(bytes).Encoded(), nil -} diff --git a/pkg/compose/hash_test.go b/pkg/compose/hash_test.go deleted file mode 100644 index 73b7f387735..00000000000 --- a/pkg/compose/hash_test.go +++ /dev/null @@ -1,43 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "gotest.tools/v3/assert" -) - -func TestServiceHash(t *testing.T) { - hash1, err := ServiceHash(serviceConfig(1)) - assert.NilError(t, err) - hash2, err := ServiceHash(serviceConfig(2)) - assert.NilError(t, err) - assert.Equal(t, hash1, hash2) -} - -func serviceConfig(replicas int) types.ServiceConfig { - return types.ServiceConfig{ - Scale: &replicas, - Deploy: &types.DeployConfig{ - Replicas: &replicas, - }, - Name: "foo", - Image: "bar", - } -} diff --git a/pkg/compose/hook.go b/pkg/compose/hook.go deleted file mode 100644 index 45357e388e8..00000000000 --- a/pkg/compose/hook.go +++ /dev/null @@ -1,121 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "io" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/pkg/stdcopy" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -func (s composeService) runHook(ctx context.Context, ctr container.Summary, service types.ServiceConfig, hook types.ServiceHook, listener api.ContainerEventListener) error { - wOut := utils.GetWriter(func(line string) { - listener(api.ContainerEvent{ - Type: api.HookEventLog, - Source: getContainerNameWithoutProject(ctr) + " ->", - ID: ctr.ID, - Service: service.Name, - Line: line, - }) - }) - defer wOut.Close() //nolint:errcheck - - detached := listener == nil - exec, err := s.apiClient().ContainerExecCreate(ctx, ctr.ID, container.ExecOptions{ - User: hook.User, - Privileged: hook.Privileged, - Env: ToMobyEnv(hook.Environment), - WorkingDir: hook.WorkingDir, - Cmd: hook.Command, - AttachStdout: !detached, - AttachStderr: !detached, - }) - if err != nil { - return err - } - - if detached { - return s.runWaitExec(ctx, exec.ID, service, listener) - } - - height, width := s.stdout().GetTtySize() - consoleSize := &[2]uint{height, width} - attach, err := s.apiClient().ContainerExecAttach(ctx, exec.ID, container.ExecAttachOptions{ - Tty: service.Tty, - ConsoleSize: consoleSize, - }) - if err != nil { - return err - } - defer attach.Close() - - if service.Tty { - _, err = io.Copy(wOut, attach.Reader) - } else { - _, err = stdcopy.StdCopy(wOut, wOut, attach.Reader) - } - if err != nil { - return err - } - - inspected, err := s.apiClient().ContainerExecInspect(ctx, exec.ID) - if err != nil { - return err - } - if inspected.ExitCode != 0 { - return fmt.Errorf("%s hook exited with status %d", service.Name, inspected.ExitCode) - } - return nil -} - -func (s composeService) runWaitExec(ctx context.Context, execID string, service types.ServiceConfig, listener api.ContainerEventListener) error { - err := s.apiClient().ContainerExecStart(ctx, execID, container.ExecStartOptions{ - Detach: listener == nil, - Tty: service.Tty, - }) - if err != nil { - return nil - } - - // We miss a ContainerExecWait API - tick := time.NewTicker(100 * time.Millisecond) - for { - select { - case <-ctx.Done(): - return nil - case <-tick.C: - inspect, err := s.apiClient().ContainerExecInspect(ctx, execID) - if err != nil { - return nil - } - if !inspect.Running { - if inspect.ExitCode != 0 { - return fmt.Errorf("%s hook exited with status %d", service.Name, inspect.ExitCode) - } - return nil - } - } - } -} diff --git a/pkg/compose/image_pruner.go b/pkg/compose/image_pruner.go deleted file mode 100644 index 6e09d901442..00000000000 --- a/pkg/compose/image_pruner.go +++ /dev/null @@ -1,247 +0,0 @@ -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "sort" - "sync" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/distribution/reference" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/client" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -// ImagePruneMode controls how aggressively images associated with the project -// are removed from the engine. -type ImagePruneMode string - -const ( - // ImagePruneNone indicates that no project images should be removed. - ImagePruneNone ImagePruneMode = "" - // ImagePruneLocal indicates that only images built locally by Compose - // should be removed. - ImagePruneLocal ImagePruneMode = "local" - // ImagePruneAll indicates that all project-associated images, including - // remote images should be removed. - ImagePruneAll ImagePruneMode = "all" -) - -// ImagePruneOptions controls the behavior of image pruning. -type ImagePruneOptions struct { - Mode ImagePruneMode - - // RemoveOrphans will result in the removal of images that were built for - // the project regardless of whether they are for a known service if true. - RemoveOrphans bool -} - -// ImagePruner handles image removal during Compose `down` operations. -type ImagePruner struct { - client client.ImageAPIClient - project *types.Project -} - -// NewImagePruner creates an ImagePruner object for a project. -func NewImagePruner(imageClient client.ImageAPIClient, project *types.Project) *ImagePruner { - return &ImagePruner{ - client: imageClient, - project: project, - } -} - -// ImagesToPrune returns the set of images that should be removed. -func (p *ImagePruner) ImagesToPrune(ctx context.Context, opts ImagePruneOptions) ([]string, error) { - if opts.Mode == ImagePruneNone { - return nil, nil - } else if opts.Mode != ImagePruneLocal && opts.Mode != ImagePruneAll { - return nil, fmt.Errorf("unsupported image prune mode: %s", opts.Mode) - } - var images []string - - if opts.Mode == ImagePruneAll { - namedImages, err := p.namedImages(ctx) - if err != nil { - return nil, err - } - images = append(images, namedImages...) - } - - projectImages, err := p.labeledLocalImages(ctx) - if err != nil { - return nil, err - } - for _, img := range projectImages { - if len(img.RepoTags) == 0 { - // currently, we're only pruning the tagged references, but - // if we start removing the dangling images and grouping by - // service, we can remove this (and should rely on `Image::ID`) - continue - } - - var shouldPrune bool - if opts.RemoveOrphans { - // indiscriminately prune all project images even if they're not - // referenced by the current Compose state (e.g. the service was - // removed from YAML) - shouldPrune = true - } else { - // only prune the image if it belongs to a known service for the project. - if _, err := p.project.GetService(img.Labels[api.ServiceLabel]); err == nil { - shouldPrune = true - } - } - - if shouldPrune { - images = append(images, img.RepoTags[0]) - } - } - - fallbackImages, err := p.unlabeledLocalImages(ctx) - if err != nil { - return nil, err - } - images = append(images, fallbackImages...) - - images = normalizeAndDedupeImages(images) - return images, nil -} - -// namedImages are those that are explicitly named in the service config. -// -// These could be registry-only images (no local build), hybrid (support build -// as a fallback if cannot pull), or local-only (image does not exist in a -// registry). -func (p *ImagePruner) namedImages(ctx context.Context) ([]string, error) { - var images []string - for _, service := range p.project.Services { - if service.Image == "" { - continue - } - images = append(images, service.Image) - } - return p.filterImagesByExistence(ctx, images) -} - -// labeledLocalImages are images that were locally-built by a current version of -// Compose (it did not always label built images). -// -// The image name could either have been defined by the user or implicitly -// created from the project + service name. -func (p *ImagePruner) labeledLocalImages(ctx context.Context) ([]image.Summary, error) { - imageListOpts := image.ListOptions{ - Filters: filters.NewArgs( - projectFilter(p.project.Name), - // TODO(milas): we should really clean up the dangling images as - // well (historically we have NOT); need to refactor this to handle - // it gracefully without producing confusing CLI output, i.e. we - // do not want to print out a bunch of untagged/dangling image IDs, - // they should be grouped into a logical operation for the relevant - // service - filters.Arg("dangling", "false"), - ), - } - projectImages, err := p.client.ImageList(ctx, imageListOpts) - if err != nil { - return nil, err - } - return projectImages, nil -} - -// unlabeledLocalImages are images that match the implicit naming convention -// for locally-built images but did not get labeled, presumably because they -// were produced by an older version of Compose. -// -// This is transitional to ensure `down` continues to work as expected on -// projects built/launched by previous versions of Compose. It can safely -// be removed after some time. -func (p *ImagePruner) unlabeledLocalImages(ctx context.Context) ([]string, error) { - var images []string - for _, service := range p.project.Services { - if service.Image != "" { - continue - } - img := api.GetImageNameOrDefault(service, p.project.Name) - images = append(images, img) - } - return p.filterImagesByExistence(ctx, images) -} - -// filterImagesByExistence returns the subset of images that exist in the -// engine store. -// -// NOTE: Any transient errors communicating with the API will result in an -// image being returned as "existing", as this method is exclusively used to -// find images to remove, so the worst case of being conservative here is an -// attempt to remove an image that doesn't exist, which will cause a warning -// but is otherwise harmless. -func (p *ImagePruner) filterImagesByExistence(ctx context.Context, imageNames []string) ([]string, error) { - var mu sync.Mutex - var ret []string - - eg, ctx := errgroup.WithContext(ctx) - for _, img := range imageNames { - eg.Go(func() error { - _, err := p.client.ImageInspect(ctx, img) - if errdefs.IsNotFound(err) { - // err on the side of caution: only skip if we successfully - // queried the API and got back a definitive "not exists" - return nil - } - mu.Lock() - defer mu.Unlock() - ret = append(ret, img) - return nil - }) - } - - if err := eg.Wait(); err != nil { - return nil, err - } - - return ret, nil -} - -// normalizeAndDedupeImages returns the unique set of images after normalization. -func normalizeAndDedupeImages(images []string) []string { - seen := make(map[string]struct{}, len(images)) - for _, img := range images { - // since some references come from user input (service.image) and some - // come from the engine API, we standardize them, opting for the - // familiar name format since they'll also be displayed in the CLI - ref, err := reference.ParseNormalizedNamed(img) - if err == nil { - ref = reference.TagNameOnly(ref) - img = reference.FamiliarString(ref) - } - seen[img] = struct{}{} - } - ret := make([]string, 0, len(seen)) - for v := range seen { - ret = append(ret, v) - } - // ensure a deterministic return result - the actual ordering is not useful - sort.Strings(ret) - return ret -} diff --git a/pkg/compose/images.go b/pkg/compose/images.go deleted file mode 100644 index e322920e59c..00000000000 --- a/pkg/compose/images.go +++ /dev/null @@ -1,163 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "slices" - "strings" - "sync" - "time" - - "github.com/containerd/errdefs" - "github.com/containerd/platforms" - "github.com/distribution/reference" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/versions" - "github.com/docker/docker/client" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Images(ctx context.Context, projectName string, options api.ImagesOptions) (map[string]api.ImageSummary, error) { - projectName = strings.ToLower(projectName) - allContainers, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - All: true, - Filters: filters.NewArgs(projectFilter(projectName)), - }) - if err != nil { - return nil, err - } - var containers []container.Summary - if len(options.Services) > 0 { - // filter service containers - for _, c := range allContainers { - if slices.Contains(options.Services, c.Labels[api.ServiceLabel]) { - containers = append(containers, c) - } - } - } else { - containers = allContainers - } - - version, err := s.RuntimeVersion(ctx) - if err != nil { - return nil, err - } - withPlatform := versions.GreaterThanOrEqualTo(version, APIVersion149) - - summary := map[string]api.ImageSummary{} - var mux sync.Mutex - eg, ctx := errgroup.WithContext(ctx) - for _, c := range containers { - eg.Go(func() error { - image, err := s.apiClient().ImageInspect(ctx, c.Image) - if err != nil { - return err - } - id := image.ID // platform-specific image ID can't be combined with image tag, see https://github.com/moby/moby/issues/49995 - - if withPlatform && c.ImageManifestDescriptor != nil && c.ImageManifestDescriptor.Platform != nil { - image, err = s.apiClient().ImageInspect(ctx, c.Image, client.ImageInspectWithPlatform(c.ImageManifestDescriptor.Platform)) - if err != nil { - return err - } - } - - var repository, tag string - ref, err := reference.ParseDockerRef(c.Image) - if err == nil { - // ParseDockerRef will reject a local image ID - repository = reference.FamiliarName(ref) - if tagged, ok := ref.(reference.Tagged); ok { - tag = tagged.Tag() - } - } - - var created *time.Time - if image.Created != "" { - t, err := time.Parse(time.RFC3339Nano, image.Created) - if err != nil { - return err - } - created = &t - } - - mux.Lock() - defer mux.Unlock() - summary[getCanonicalContainerName(c)] = api.ImageSummary{ - ID: id, - Repository: repository, - Tag: tag, - Platform: platforms.Platform{ - Architecture: image.Architecture, - OS: image.Os, - OSVersion: image.OsVersion, - Variant: image.Variant, - }, - Size: image.Size, - Created: created, - LastTagTime: image.Metadata.LastTagTime, - } - return nil - }) - } - - err = eg.Wait() - return summary, err -} - -func (s *composeService) getImageSummaries(ctx context.Context, repoTags []string) (map[string]api.ImageSummary, error) { - summary := map[string]api.ImageSummary{} - l := sync.Mutex{} - eg, ctx := errgroup.WithContext(ctx) - for _, repoTag := range repoTags { - eg.Go(func() error { - inspect, err := s.apiClient().ImageInspect(ctx, repoTag) - if err != nil { - if errdefs.IsNotFound(err) { - return nil - } - return fmt.Errorf("unable to get image '%s': %w", repoTag, err) - } - tag := "" - repository := "" - ref, err := reference.ParseDockerRef(repoTag) - if err == nil { - // ParseDockerRef will reject a local image ID - repository = reference.FamiliarName(ref) - if tagged, ok := ref.(reference.Tagged); ok { - tag = tagged.Tag() - } - } - l.Lock() - summary[repoTag] = api.ImageSummary{ - ID: inspect.ID, - Repository: repository, - Tag: tag, - Size: inspect.Size, - LastTagTime: inspect.Metadata.LastTagTime, - } - l.Unlock() - return nil - }) - } - return summary, eg.Wait() -} diff --git a/pkg/compose/images_test.go b/pkg/compose/images_test.go deleted file mode 100644 index b6eb9cce138..00000000000 --- a/pkg/compose/images_test.go +++ /dev/null @@ -1,108 +0,0 @@ -/* - Copyright 2024 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "strings" - "testing" - "time" - - "github.com/docker/docker/api/types" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - compose "github.com/docker/compose/v5/pkg/api" -) - -func TestImages(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - args := filters.NewArgs(projectFilter(strings.ToLower(testProject))) - listOpts := container.ListOptions{All: true, Filters: args} - api.EXPECT().ServerVersion(gomock.Any()).Return(types.Version{APIVersion: "1.96"}, nil).AnyTimes() - timeStr1 := "2025-06-06T06:06:06.000000000Z" - created1, _ := time.Parse(time.RFC3339Nano, timeStr1) - timeStr2 := "2025-03-03T03:03:03.000000000Z" - created2, _ := time.Parse(time.RFC3339Nano, timeStr2) - image1 := imageInspect("image1", "foo:1", 12345, timeStr1) - image2 := imageInspect("image2", "bar:2", 67890, timeStr2) - api.EXPECT().ImageInspect(anyCancellableContext(), "foo:1").Return(image1, nil).MaxTimes(2) - api.EXPECT().ImageInspect(anyCancellableContext(), "bar:2").Return(image2, nil) - c1 := containerDetail("service1", "123", "running", "foo:1") - c2 := containerDetail("service1", "456", "running", "bar:2") - c2.Ports = []container.Port{{PublicPort: 80, PrivatePort: 90, IP: "localhost"}} - c3 := containerDetail("service2", "789", "exited", "foo:1") - api.EXPECT().ContainerList(t.Context(), listOpts).Return([]container.Summary{c1, c2, c3}, nil) - - images, err := tested.Images(t.Context(), strings.ToLower(testProject), compose.ImagesOptions{}) - - expected := map[string]compose.ImageSummary{ - "123": { - ID: "image1", - Repository: "foo", - Tag: "1", - Size: 12345, - Created: &created1, - }, - "456": { - ID: "image2", - Repository: "bar", - Tag: "2", - Size: 67890, - Created: &created2, - }, - "789": { - ID: "image1", - Repository: "foo", - Tag: "1", - Size: 12345, - Created: &created1, - }, - } - assert.NilError(t, err) - assert.DeepEqual(t, images, expected) -} - -func imageInspect(id string, imageReference string, size int64, created string) image.InspectResponse { - return image.InspectResponse{ - ID: id, - RepoTags: []string{ - "someRepo:someTag", - imageReference, - }, - Size: size, - Created: created, - } -} - -func containerDetail(service string, id string, status string, imageName string) container.Summary { - return container.Summary{ - ID: id, - Names: []string{"/" + id}, - Image: imageName, - Labels: containerLabels(service, false), - State: status, - } -} diff --git a/pkg/compose/kill.go b/pkg/compose/kill.go deleted file mode 100644 index 63d369902af..00000000000 --- a/pkg/compose/kill.go +++ /dev/null @@ -1,74 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strings" - - "github.com/docker/docker/api/types/container" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Kill(ctx context.Context, projectName string, options api.KillOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.kill(ctx, strings.ToLower(projectName), options) - }, "kill", s.events) -} - -func (s *composeService) kill(ctx context.Context, projectName string, options api.KillOptions) error { - services := options.Services - - var containers Containers - containers, err := s.getContainers(ctx, projectName, oneOffInclude, options.All, services...) - if err != nil { - return err - } - - project := options.Project - if project == nil { - project, err = s.getProjectWithResources(ctx, containers, projectName) - if err != nil { - return err - } - } - - if !options.RemoveOrphans { - containers = containers.filter(isService(project.ServiceNames()...)) - } - if len(containers) == 0 { - return api.ErrNoResources - } - - eg, ctx := errgroup.WithContext(ctx) - containers.forEach(func(ctr container.Summary) { - eg.Go(func() error { - eventName := getContainerProgressName(ctr) - s.events.On(killingEvent(eventName)) - err := s.apiClient().ContainerKill(ctx, ctr.ID, options.Signal) - if err != nil { - s.events.On(errorEvent(eventName, "Error while Killing")) - return err - } - s.events.On(killedEvent(eventName)) - return nil - }) - }) - return eg.Wait() -} diff --git a/pkg/compose/kill_test.go b/pkg/compose/kill_test.go deleted file mode 100644 index f27606b7f2e..00000000000 --- a/pkg/compose/kill_test.go +++ /dev/null @@ -1,148 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "path/filepath" - "strings" - "testing" - - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/api/types/volume" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - compose "github.com/docker/compose/v5/pkg/api" -) - -const testProject = "testProject" - -func TestKillAll(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - name := strings.ToLower(testProject) - - api.EXPECT().ContainerList(t.Context(), container.ListOptions{ - Filters: filters.NewArgs(projectFilter(name), hasConfigHashLabel()), - }).Return( - []container.Summary{testContainer("service1", "123", false), testContainer("service1", "456", false), testContainer("service2", "789", false)}, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{ - {ID: "abc123", Name: "testProject_default"}, - }, nil) - api.EXPECT().ContainerKill(anyCancellableContext(), "123", "").Return(nil) - api.EXPECT().ContainerKill(anyCancellableContext(), "456", "").Return(nil) - api.EXPECT().ContainerKill(anyCancellableContext(), "789", "").Return(nil) - - err = tested.Kill(t.Context(), name, compose.KillOptions{}) - assert.NilError(t, err) -} - -func TestKillSignal(t *testing.T) { - const serviceName = "service1" - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - name := strings.ToLower(testProject) - listOptions := container.ListOptions{ - Filters: filters.NewArgs(projectFilter(name), serviceFilter(serviceName), hasConfigHashLabel()), - } - - api.EXPECT().ContainerList(t.Context(), listOptions).Return([]container.Summary{testContainer(serviceName, "123", false)}, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{ - {ID: "abc123", Name: "testProject_default"}, - }, nil) - api.EXPECT().ContainerKill(anyCancellableContext(), "123", "SIGTERM").Return(nil) - - err = tested.Kill(t.Context(), name, compose.KillOptions{Services: []string{serviceName}, Signal: "SIGTERM"}) - assert.NilError(t, err) -} - -func testContainer(service string, id string, oneOff bool) container.Summary { - // canonical docker names in the API start with a leading slash, some - // parts of Compose code will attempt to strip this off, so make sure - // it's consistently present - name := "/" + strings.TrimPrefix(id, "/") - return container.Summary{ - ID: id, - Names: []string{name}, - Labels: containerLabels(service, oneOff), - State: container.StateExited, - } -} - -func containerLabels(service string, oneOff bool) map[string]string { - workingdir := "/src/pkg/compose/testdata" - composefile := filepath.Join(workingdir, "compose.yaml") - labels := map[string]string{ - compose.ServiceLabel: service, - compose.ConfigFilesLabel: composefile, - compose.WorkingDirLabel: workingdir, - compose.ProjectLabel: strings.ToLower(testProject), - } - if oneOff { - labels[compose.OneoffLabel] = "True" - } - return labels -} - -func anyCancellableContext() gomock.Matcher { - //nolint:forbidigo // This creates a context type for gomock matching, not for actual test usage - ctxWithCancel, cancel := context.WithCancel(context.Background()) - cancel() - return gomock.AssignableToTypeOf(ctxWithCancel) -} - -func projectFilterListOpt(withOneOff bool) container.ListOptions { - filter := filters.NewArgs( - projectFilter(strings.ToLower(testProject)), - hasConfigHashLabel(), - ) - if !withOneOff { - filter.Add("label", fmt.Sprintf("%s=False", compose.OneoffLabel)) - } - return container.ListOptions{ - Filters: filter, - All: true, - } -} diff --git a/pkg/compose/loader.go b/pkg/compose/loader.go deleted file mode 100644 index 9a0699da7c6..00000000000 --- a/pkg/compose/loader.go +++ /dev/null @@ -1,157 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "os" - "strings" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/loader" - "github.com/compose-spec/compose-go/v2/types" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/remote" - "github.com/docker/compose/v5/pkg/utils" -) - -// LoadProject implements api.Compose.LoadProject -// It loads and validates a Compose project from configuration files. -func (s *composeService) LoadProject(ctx context.Context, options api.ProjectLoadOptions) (*types.Project, error) { - // Setup remote loaders (Git, OCI) - remoteLoaders := s.createRemoteLoaders(options) - - projectOptions, err := s.buildProjectOptions(options, remoteLoaders) - if err != nil { - return nil, err - } - - // Register all user-provided listeners (e.g., for metrics collection) - for _, listener := range options.LoadListeners { - if listener != nil { - projectOptions.WithListeners(listener) - } - } - - if options.Compatibility || utils.StringToBool(projectOptions.Environment[api.ComposeCompatibility]) { - api.Separator = "_" - } - - project, err := projectOptions.LoadProject(ctx) - if err != nil { - return nil, err - } - - // Post-processing: service selection, environment resolution, etc. - project, err = s.postProcessProject(project, options) - if err != nil { - return nil, err - } - - return project, nil -} - -// createRemoteLoaders creates Git and OCI remote loaders if not in offline mode -func (s *composeService) createRemoteLoaders(options api.ProjectLoadOptions) []loader.ResourceLoader { - if options.Offline { - return nil - } - git := remote.NewGitRemoteLoader(s.dockerCli, options.Offline) - oci := remote.NewOCIRemoteLoader(s.dockerCli, options.Offline, options.OCI) - return []loader.ResourceLoader{git, oci} -} - -// buildProjectOptions constructs compose-go ProjectOptions from API options -func (s *composeService) buildProjectOptions(options api.ProjectLoadOptions, remoteLoaders []loader.ResourceLoader) (*cli.ProjectOptions, error) { - opts := []cli.ProjectOptionsFn{ - cli.WithWorkingDirectory(options.WorkingDir), - cli.WithOsEnv, - } - - // Add PWD if not present - if _, present := os.LookupEnv("PWD"); !present { - if pwd, err := os.Getwd(); err == nil { - opts = append(opts, cli.WithEnv([]string{"PWD=" + pwd})) - } - } - - // Add remote loaders - for _, r := range remoteLoaders { - opts = append(opts, cli.WithResourceLoader(r)) - } - - opts = append(opts, - // Load PWD/.env if present and no explicit --env-file has been set - cli.WithEnvFiles(options.EnvFiles...), - // read dot env file to populate project environment - cli.WithDotEnv, - // get compose file path set by COMPOSE_FILE - cli.WithConfigFileEnv, - // if none was selected, get default compose.yaml file from current dir or parent folder - cli.WithDefaultConfigPath, - // .. and then, a project directory != PWD maybe has been set so let's load .env file - cli.WithEnvFiles(options.EnvFiles...), //nolint:gocritic // intentionally applying cli.WithEnvFiles twice. - cli.WithDotEnv, //nolint:gocritic // intentionally applying cli.WithDotEnv twice. - // eventually COMPOSE_PROFILES should have been set - cli.WithDefaultProfiles(options.Profiles...), - cli.WithName(options.ProjectName), - ) - - return cli.NewProjectOptions(options.ConfigPaths, append(options.ProjectOptionsFns, opts...)...) -} - -// postProcessProject applies post-loading transformations to the project -func (s *composeService) postProcessProject(project *types.Project, options api.ProjectLoadOptions) (*types.Project, error) { - if project.Name == "" { - return nil, errors.New("project name can't be empty. Use ProjectName option to set a valid name") - } - - project, err := project.WithServicesEnabled(options.Services...) - if err != nil { - return nil, err - } - - // Add custom labels - for name, s := range project.Services { - s.CustomLabels = map[string]string{ - api.ProjectLabel: project.Name, - api.ServiceLabel: name, - api.VersionLabel: api.ComposeVersion, - api.WorkingDirLabel: project.WorkingDir, - api.ConfigFilesLabel: strings.Join(project.ComposeFiles, ","), - api.OneoffLabel: "False", - } - if len(options.EnvFiles) != 0 { - s.CustomLabels[api.EnvironmentFileLabel] = strings.Join(options.EnvFiles, ",") - } - project.Services[name] = s - } - - project, err = project.WithSelectedServices(options.Services) - if err != nil { - return nil, err - } - - // Remove unnecessary resources if not All - if !options.All { - project = project.WithoutUnnecessaryResources() - } - - return project, nil -} diff --git a/pkg/compose/loader_test.go b/pkg/compose/loader_test.go deleted file mode 100644 index 8006d4169d0..00000000000 --- a/pkg/compose/loader_test.go +++ /dev/null @@ -1,322 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "os" - "path/filepath" - "testing" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestLoadProject_Basic(t *testing.T) { - // Create a temporary compose file - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -name: test-project -services: - web: - image: nginx:latest - ports: - - "8080:80" - db: - image: postgres:latest - environment: - POSTGRES_PASSWORD: secret -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Load the project - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - }) - - // Assertions - require.NoError(t, err) - assert.NotNil(t, project) - assert.Equal(t, "test-project", project.Name) - assert.Len(t, project.Services, 2) - assert.Contains(t, project.Services, "web") - assert.Contains(t, project.Services, "db") - - // Check labels were applied - webService := project.Services["web"] - assert.Equal(t, "test-project", webService.CustomLabels[api.ProjectLabel]) - assert.Equal(t, "web", webService.CustomLabels[api.ServiceLabel]) -} - -func TestLoadProject_WithEnvironmentResolution(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -services: - app: - image: myapp:latest - environment: - - TEST_VAR=${TEST_VAR} - - LITERAL_VAR=literal_value -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - // Set environment variable - t.Setenv("TEST_VAR", "resolved_value") - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Test with environment resolution (default) - t.Run("WithResolution", func(t *testing.T) { - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - }) - require.NoError(t, err) - - appService := project.Services["app"] - // Environment should be resolved - assert.NotNil(t, appService.Environment["TEST_VAR"]) - assert.Equal(t, "resolved_value", *appService.Environment["TEST_VAR"]) - assert.NotNil(t, appService.Environment["LITERAL_VAR"]) - assert.Equal(t, "literal_value", *appService.Environment["LITERAL_VAR"]) - }) - - // Test without environment resolution - t.Run("WithoutResolution", func(t *testing.T) { - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - ProjectOptionsFns: []cli.ProjectOptionsFn{cli.WithoutEnvironmentResolution}, - }) - require.NoError(t, err) - - appService := project.Services["app"] - // Environment should NOT be resolved, keeping raw values - // Note: This depends on compose-go behavior, which may still have some resolution - assert.NotNil(t, appService.Environment) - }) -} - -func TestLoadProject_ServiceSelection(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -services: - web: - image: nginx:latest - db: - image: postgres:latest - cache: - image: redis:latest -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Load only specific services - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - Services: []string{"web", "db"}, - }) - - require.NoError(t, err) - assert.Len(t, project.Services, 2) - assert.Contains(t, project.Services, "web") - assert.Contains(t, project.Services, "db") - assert.NotContains(t, project.Services, "cache") -} - -func TestLoadProject_WithProfiles(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -services: - web: - image: nginx:latest - debug: - image: busybox:latest - profiles: ["debug"] -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Without debug profile - t.Run("WithoutProfile", func(t *testing.T) { - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - }) - require.NoError(t, err) - assert.Len(t, project.Services, 1) - assert.Contains(t, project.Services, "web") - }) - - // With debug profile - t.Run("WithProfile", func(t *testing.T) { - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - Profiles: []string{"debug"}, - }) - require.NoError(t, err) - assert.Len(t, project.Services, 2) - assert.Contains(t, project.Services, "web") - assert.Contains(t, project.Services, "debug") - }) -} - -func TestLoadProject_WithLoadListeners(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -services: - web: - image: nginx:latest -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Track events received - var events []string - listener := func(event string, metadata map[string]any) { - events = append(events, event) - } - - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - LoadListeners: []api.LoadListener{listener}, - }) - - require.NoError(t, err) - assert.NotNil(t, project) - - // Listeners should have been called (exact events depend on compose-go implementation) - // The slice itself is always initialized (non-nil), even if empty - _ = events // events may or may not have entries depending on compose-go behavior -} - -func TestLoadProject_ProjectNameInference(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -services: - web: - image: nginx:latest -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Without explicit project name - t.Run("InferredName", func(t *testing.T) { - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - }) - require.NoError(t, err) - // Project name should be inferred from directory - assert.NotEmpty(t, project.Name) - }) - - // With explicit project name - t.Run("ExplicitName", func(t *testing.T) { - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - ProjectName: "my-custom-project", - }) - require.NoError(t, err) - assert.Equal(t, "my-custom-project", project.Name) - }) -} - -func TestLoadProject_Compatibility(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -services: - web: - image: nginx:latest -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // With compatibility mode - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - Compatibility: true, - }) - - require.NoError(t, err) - assert.NotNil(t, project) - // In compatibility mode, separator should be "_" - assert.Equal(t, "_", api.Separator) - - // Reset separator - api.Separator = "-" -} - -func TestLoadProject_InvalidComposeFile(t *testing.T) { - tmpDir := t.TempDir() - composeFile := filepath.Join(tmpDir, "compose.yaml") - composeContent := ` -this is not valid yaml: [[[ -` - err := os.WriteFile(composeFile, []byte(composeContent), 0o644) - require.NoError(t, err) - - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Should return an error for invalid YAML - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{composeFile}, - }) - - require.Error(t, err) - assert.Nil(t, project) -} - -func TestLoadProject_MissingComposeFile(t *testing.T) { - service, err := NewComposeService(nil) - require.NoError(t, err) - - // Should return an error for missing file - project, err := service.LoadProject(t.Context(), api.ProjectLoadOptions{ - ConfigPaths: []string{"/nonexistent/compose.yaml"}, - }) - - require.Error(t, err) - assert.Nil(t, project) -} diff --git a/pkg/compose/logs.go b/pkg/compose/logs.go deleted file mode 100644 index c633f5c311f..00000000000 --- a/pkg/compose/logs.go +++ /dev/null @@ -1,148 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "io" - - "github.com/containerd/errdefs" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/pkg/stdcopy" - "github.com/sirupsen/logrus" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -func (s *composeService) Logs( - ctx context.Context, - projectName string, - consumer api.LogConsumer, - options api.LogOptions, -) error { - var containers Containers - var err error - - if options.Index > 0 { - ctr, err := s.getSpecifiedContainer(ctx, projectName, oneOffExclude, true, options.Services[0], options.Index) - if err != nil { - return err - } - containers = append(containers, ctr) - } else { - containers, err = s.getContainers(ctx, projectName, oneOffExclude, true, options.Services...) - if err != nil { - return err - } - } - - if options.Project != nil && len(options.Services) == 0 { - // we run with an explicit compose.yaml, so only consider services defined in this file - options.Services = options.Project.ServiceNames() - containers = containers.filter(isService(options.Services...)) - } - - eg, ctx := errgroup.WithContext(ctx) - for _, ctr := range containers { - eg.Go(func() error { - err := s.logContainer(ctx, consumer, ctr, options) - if errdefs.IsNotImplemented(err) { - logrus.Warnf("Can't retrieve logs for %q: %s", getCanonicalContainerName(ctr), err.Error()) - return nil - } - return err - }) - } - - if options.Follow { - printer := newLogPrinter(consumer) - - monitor := newMonitor(s.apiClient(), projectName) - if len(options.Services) > 0 { - monitor.withServices(options.Services) - } else if options.Project != nil { - monitor.withServices(options.Project.ServiceNames()) - } - monitor.withListener(printer.HandleEvent) - monitor.withListener(func(event api.ContainerEvent) { - if event.Type == api.ContainerEventStarted { - eg.Go(func() error { - ctr, err := s.apiClient().ContainerInspect(ctx, event.ID) - if err != nil { - return err - } - - err = s.doLogContainer(ctx, consumer, event.Source, ctr, api.LogOptions{ - Follow: options.Follow, - Since: ctr.State.StartedAt, - Until: options.Until, - Tail: options.Tail, - Timestamps: options.Timestamps, - }) - if errdefs.IsNotImplemented(err) { - // ignore - return nil - } - return err - }) - } - }) - eg.Go(func() error { - // pass ctx so monitor will immediately stop on SIGINT - return monitor.Start(ctx) - }) - } - - return eg.Wait() -} - -func (s *composeService) logContainer(ctx context.Context, consumer api.LogConsumer, c container.Summary, options api.LogOptions) error { - ctr, err := s.apiClient().ContainerInspect(ctx, c.ID) - if err != nil { - return err - } - name := getContainerNameWithoutProject(c) - return s.doLogContainer(ctx, consumer, name, ctr, options) -} - -func (s *composeService) doLogContainer(ctx context.Context, consumer api.LogConsumer, name string, ctr container.InspectResponse, options api.LogOptions) error { - r, err := s.apiClient().ContainerLogs(ctx, ctr.ID, container.LogsOptions{ - ShowStdout: true, - ShowStderr: true, - Follow: options.Follow, - Since: options.Since, - Until: options.Until, - Tail: options.Tail, - Timestamps: options.Timestamps, - }) - if err != nil { - return err - } - defer r.Close() //nolint:errcheck - - w := utils.GetWriter(func(line string) { - consumer.Log(name, line) - }) - if ctr.Config.Tty { - _, err = io.Copy(w, r) - } else { - _, err = stdcopy.StdCopy(w, w, r) - } - return err -} diff --git a/pkg/compose/logs_test.go b/pkg/compose/logs_test.go deleted file mode 100644 index b5823c908da..00000000000 --- a/pkg/compose/logs_test.go +++ /dev/null @@ -1,191 +0,0 @@ -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "io" - "strings" - "sync" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - containerType "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/pkg/stdcopy" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - "go.uber.org/mock/gomock" - - compose "github.com/docker/compose/v5/pkg/api" -) - -func TestComposeService_Logs_Demux(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - require.NoError(t, err) - - name := strings.ToLower(testProject) - - api.EXPECT().ContainerList(t.Context(), containerType.ListOptions{ - All: true, - Filters: filters.NewArgs(oneOffFilter(false), projectFilter(name), hasConfigHashLabel()), - }).Return( - []containerType.Summary{ - testContainer("service", "c", false), - }, - nil, - ) - - api.EXPECT(). - ContainerInspect(anyCancellableContext(), "c"). - Return(containerType.InspectResponse{ - ContainerJSONBase: &containerType.ContainerJSONBase{ID: "c"}, - Config: &containerType.Config{Tty: false}, - }, nil) - c1Reader, c1Writer := io.Pipe() - t.Cleanup(func() { - _ = c1Reader.Close() - _ = c1Writer.Close() - }) - c1Stdout := stdcopy.NewStdWriter(c1Writer, stdcopy.Stdout) - c1Stderr := stdcopy.NewStdWriter(c1Writer, stdcopy.Stderr) - go func() { - _, err := c1Stdout.Write([]byte("hello stdout\n")) - assert.NoError(t, err, "Writing to fake stdout") - _, err = c1Stderr.Write([]byte("hello stderr\n")) - assert.NoError(t, err, "Writing to fake stderr") - _ = c1Writer.Close() - }() - api.EXPECT().ContainerLogs(anyCancellableContext(), "c", gomock.Any()). - Return(c1Reader, nil) - - opts := compose.LogOptions{ - Project: &types.Project{ - Services: types.Services{ - "service": {Name: "service"}, - }, - }, - } - - consumer := &testLogConsumer{} - err = tested.Logs(t.Context(), name, consumer, opts) - require.NoError(t, err) - - require.Equal( - t, - []string{"hello stdout", "hello stderr"}, - consumer.LogsForContainer("c"), - ) -} - -// TestComposeService_Logs_ServiceFiltering ensures that we do not include -// logs from out-of-scope services based on the Compose file vs actual state. -// -// NOTE(milas): This test exists because each method is currently duplicating -// a lot of the project/service filtering logic. We should consider moving it -// to an earlier point in the loading process, at which point this test could -// safely be removed. -func TestComposeService_Logs_ServiceFiltering(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - require.NoError(t, err) - - name := strings.ToLower(testProject) - - api.EXPECT().ContainerList(t.Context(), containerType.ListOptions{ - All: true, - Filters: filters.NewArgs(oneOffFilter(false), projectFilter(name), hasConfigHashLabel()), - }).Return( - []containerType.Summary{ - testContainer("serviceA", "c1", false), - testContainer("serviceA", "c2", false), - // serviceB will be filtered out by the project definition to - // ensure we ignore "orphan" containers - testContainer("serviceB", "c3", false), - testContainer("serviceC", "c4", false), - }, - nil, - ) - - for _, id := range []string{"c1", "c2", "c4"} { - api.EXPECT(). - ContainerInspect(anyCancellableContext(), id). - Return( - containerType.InspectResponse{ - ContainerJSONBase: &containerType.ContainerJSONBase{ID: id}, - Config: &containerType.Config{Tty: true}, - }, - nil, - ) - api.EXPECT().ContainerLogs(anyCancellableContext(), id, gomock.Any()). - Return(io.NopCloser(strings.NewReader("hello "+id+"\n")), nil). - Times(1) - } - - // this simulates passing `--filename` with a Compose file that does NOT - // reference `serviceB` even though it has running services for this proj - proj := &types.Project{ - Services: types.Services{ - "serviceA": {Name: "serviceA"}, - "serviceC": {Name: "serviceC"}, - }, - } - consumer := &testLogConsumer{} - opts := compose.LogOptions{ - Project: proj, - } - err = tested.Logs(t.Context(), name, consumer, opts) - require.NoError(t, err) - - require.Equal(t, []string{"hello c1"}, consumer.LogsForContainer("c1")) - require.Equal(t, []string{"hello c2"}, consumer.LogsForContainer("c2")) - require.Empty(t, consumer.LogsForContainer("c3")) - require.Equal(t, []string{"hello c4"}, consumer.LogsForContainer("c4")) -} - -type testLogConsumer struct { - mu sync.Mutex - // logs is keyed by container ID; values are log lines - logs map[string][]string -} - -func (l *testLogConsumer) Log(containerName, message string) { - l.mu.Lock() - defer l.mu.Unlock() - if l.logs == nil { - l.logs = make(map[string][]string) - } - l.logs[containerName] = append(l.logs[containerName], message) -} - -func (l *testLogConsumer) Err(containerName, message string) { - l.Log(containerName, message) -} - -func (l *testLogConsumer) Status(containerName, msg string) {} - -func (l *testLogConsumer) LogsForContainer(containerName string) []string { - l.mu.Lock() - defer l.mu.Unlock() - return l.logs[containerName] -} diff --git a/pkg/compose/ls.go b/pkg/compose/ls.go deleted file mode 100644 index b999e828bdc..00000000000 --- a/pkg/compose/ls.go +++ /dev/null @@ -1,136 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "slices" - "sort" - "strings" - - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) List(ctx context.Context, opts api.ListOptions) ([]api.Stack, error) { - list, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - Filters: filters.NewArgs(hasProjectLabelFilter(), hasConfigHashLabel()), - All: opts.All, - }) - if err != nil { - return nil, err - } - - return containersToStacks(list) -} - -func containersToStacks(containers []container.Summary) ([]api.Stack, error) { - containersByLabel, keys, err := groupContainerByLabel(containers, api.ProjectLabel) - if err != nil { - return nil, err - } - var projects []api.Stack - for _, project := range keys { - configFiles, err := combinedConfigFiles(containersByLabel[project]) - if err != nil { - logrus.Warn(err.Error()) - configFiles = "N/A" - } - - projects = append(projects, api.Stack{ - ID: project, - Name: project, - Status: combinedStatus(containerToState(containersByLabel[project])), - ConfigFiles: configFiles, - }) - } - return projects, nil -} - -func combinedConfigFiles(containers []container.Summary) (string, error) { - configFiles := []string{} - - for _, c := range containers { - files, ok := c.Labels[api.ConfigFilesLabel] - if !ok { - return "", fmt.Errorf("no label %q set on container %q of compose project", api.ConfigFilesLabel, c.ID) - } - - for f := range strings.SplitSeq(files, ",") { - if !slices.Contains(configFiles, f) { - configFiles = append(configFiles, f) - } - } - } - - return strings.Join(configFiles, ","), nil -} - -func containerToState(containers []container.Summary) []string { - statuses := []string{} - for _, c := range containers { - statuses = append(statuses, c.State) - } - return statuses -} - -func combinedStatus(statuses []string) string { - nbByStatus := map[string]int{} - keys := []string{} - for _, status := range statuses { - nb, ok := nbByStatus[status] - if !ok { - nb = 0 - keys = append(keys, status) - } - nbByStatus[status] = nb + 1 - } - sort.Strings(keys) - result := "" - for _, status := range keys { - nb := nbByStatus[status] - if result != "" { - result += ", " - } - result += fmt.Sprintf("%s(%d)", status, nb) - } - return result -} - -func groupContainerByLabel(containers []container.Summary, labelName string) (map[string][]container.Summary, []string, error) { - containersByLabel := map[string][]container.Summary{} - keys := []string{} - for _, c := range containers { - label, ok := c.Labels[labelName] - if !ok { - return nil, nil, fmt.Errorf("no label %q set on container %q of compose project", labelName, c.ID) - } - labelContainers, ok := containersByLabel[label] - if !ok { - labelContainers = []container.Summary{} - keys = append(keys, label) - } - labelContainers = append(labelContainers, c) - containersByLabel[label] = labelContainers - } - sort.Strings(keys) - return containersByLabel, keys, nil -} diff --git a/pkg/compose/ls_test.go b/pkg/compose/ls_test.go deleted file mode 100644 index b444abd3476..00000000000 --- a/pkg/compose/ls_test.go +++ /dev/null @@ -1,122 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "fmt" - "testing" - - "github.com/docker/docker/api/types/container" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestContainersToStacks(t *testing.T) { - containers := []container.Summary{ - { - ID: "service1", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project1", api.ConfigFilesLabel: "/home/docker-compose.yaml"}, - }, - { - ID: "service2", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project1", api.ConfigFilesLabel: "/home/docker-compose.yaml"}, - }, - { - ID: "service3", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project2", api.ConfigFilesLabel: "/home/project2-docker-compose.yaml"}, - }, - } - stacks, err := containersToStacks(containers) - assert.NilError(t, err) - assert.DeepEqual(t, stacks, []api.Stack{ - { - ID: "project1", - Name: "project1", - Status: "running(2)", - ConfigFiles: "/home/docker-compose.yaml", - }, - { - ID: "project2", - Name: "project2", - Status: "running(1)", - ConfigFiles: "/home/project2-docker-compose.yaml", - }, - }) -} - -func TestStacksMixedStatus(t *testing.T) { - assert.Equal(t, combinedStatus([]string{"running"}), "running(1)") - assert.Equal(t, combinedStatus([]string{"running", "running", "running"}), "running(3)") - assert.Equal(t, combinedStatus([]string{"running", "exited", "running"}), "exited(1), running(2)") -} - -func TestCombinedConfigFiles(t *testing.T) { - containersByLabel := map[string][]container.Summary{ - "project1": { - { - ID: "service1", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project1", api.ConfigFilesLabel: "/home/docker-compose.yaml"}, - }, - { - ID: "service2", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project1", api.ConfigFilesLabel: "/home/docker-compose.yaml"}, - }, - }, - "project2": { - { - ID: "service3", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project2", api.ConfigFilesLabel: "/home/project2-docker-compose.yaml"}, - }, - }, - "project3": { - { - ID: "service4", - State: "running", - Labels: map[string]string{api.ProjectLabel: "project3"}, - }, - }, - } - - testData := map[string]struct { - ConfigFiles string - Error error - }{ - "project1": {ConfigFiles: "/home/docker-compose.yaml", Error: nil}, - "project2": {ConfigFiles: "/home/project2-docker-compose.yaml", Error: nil}, - "project3": {ConfigFiles: "", Error: fmt.Errorf("no label %q set on container %q of compose project", api.ConfigFilesLabel, "service4")}, - } - - for project, containers := range containersByLabel { - configFiles, err := combinedConfigFiles(containers) - - expected := testData[project] - - if expected.Error != nil { - assert.Error(t, err, expected.Error.Error()) - } else { - assert.Equal(t, err, expected.Error) - } - assert.Equal(t, configFiles, expected.ConfigFiles) - } -} diff --git a/pkg/compose/model.go b/pkg/compose/model.go deleted file mode 100644 index e4614d199c3..00000000000 --- a/pkg/compose/model.go +++ /dev/null @@ -1,294 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bufio" - "context" - "encoding/json" - "fmt" - "os/exec" - "slices" - "strconv" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/docker/cli/cli-plugins/manager" - "github.com/docker/docker/api/types/versions" - "github.com/spf13/cobra" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) ensureModels(ctx context.Context, project *types.Project, quietPull bool) error { - if len(project.Models) == 0 { - return nil - } - - mdlAPI, err := s.newModelAPI(project) - if err != nil { - return err - } - defer mdlAPI.Close() - availableModels, err := mdlAPI.ListModels(ctx) - - eg, ctx := errgroup.WithContext(ctx) - eg.Go(func() error { - return mdlAPI.SetModelVariables(ctx, project) - }) - - for name, config := range project.Models { - if config.Name == "" { - config.Name = name - } - eg.Go(func() error { - if !slices.Contains(availableModels, config.Model) { - err = mdlAPI.PullModel(ctx, config, quietPull, s.events) - if err != nil { - return err - } - } - return mdlAPI.ConfigureModel(ctx, config, s.events) - }) - } - return eg.Wait() -} - -type modelAPI struct { - path string - env []string - prepare func(ctx context.Context, cmd *exec.Cmd) error - cleanup func() - version string -} - -func (s *composeService) newModelAPI(project *types.Project) (*modelAPI, error) { - dockerModel, err := manager.GetPlugin("model", s.dockerCli, &cobra.Command{}) - if err != nil { - if errdefs.IsNotFound(err) { - return nil, fmt.Errorf("'models' support requires Docker Model plugin") - } - return nil, err - } - if dockerModel.Err != nil { - return nil, fmt.Errorf("failed to load Docker Model plugin: %w", dockerModel.Err) - } - endpoint, cleanup, err := s.propagateDockerEndpoint() - if err != nil { - return nil, err - } - return &modelAPI{ - path: dockerModel.Path, - version: dockerModel.Version, - prepare: func(ctx context.Context, cmd *exec.Cmd) error { - return s.prepareShellOut(ctx, project.Environment, cmd) - }, - cleanup: cleanup, - env: append(project.Environment.Values(), endpoint...), - }, nil -} - -func (m *modelAPI) Close() { - m.cleanup() -} - -func (m *modelAPI) PullModel(ctx context.Context, model types.ModelConfig, quietPull bool, events api.EventProcessor) error { - events.On(api.Resource{ - ID: model.Name, - Status: api.Working, - Text: api.StatusPulling, - }) - - cmd := exec.CommandContext(ctx, m.path, "pull", model.Model) - err := m.prepare(ctx, cmd) - if err != nil { - return err - } - stream, err := cmd.StdoutPipe() - if err != nil { - return err - } - - err = cmd.Start() - if err != nil { - return err - } - - scanner := bufio.NewScanner(stream) - for scanner.Scan() { - msg := scanner.Text() - if msg == "" { - continue - } - - if !quietPull { - events.On(api.Resource{ - ID: model.Name, - Status: api.Working, - Text: api.StatusPulling, - }) - } - } - - err = cmd.Wait() - if err != nil { - events.On(errorEvent(model.Name, err.Error())) - } - events.On(api.Resource{ - ID: model.Name, - Status: api.Working, - Text: api.StatusPulled, - }) - return err -} - -func (m *modelAPI) ConfigureModel(ctx context.Context, config types.ModelConfig, events api.EventProcessor) error { - events.On(api.Resource{ - ID: config.Name, - Status: api.Working, - Text: api.StatusConfiguring, - }) - // configure [--context-size=<n>] MODEL [-- <runtime-flags...>] - args := []string{"configure"} - if config.ContextSize > 0 { - args = append(args, "--context-size", strconv.Itoa(config.ContextSize)) - } - args = append(args, config.Model) - // Only append RuntimeFlags if docker model CLI version is >= v1.0.6 - if len(config.RuntimeFlags) != 0 && m.supportsRuntimeFlags() { - args = append(args, "--") - args = append(args, config.RuntimeFlags...) - } - cmd := exec.CommandContext(ctx, m.path, args...) - err := m.prepare(ctx, cmd) - if err != nil { - return err - } - err = cmd.Run() - if err != nil { - events.On(errorEvent(config.Name, err.Error())) - return err - } - events.On(api.Resource{ - ID: config.Name, - Status: api.Done, - Text: api.StatusConfigured, - }) - return nil -} - -func (m *modelAPI) SetModelVariables(ctx context.Context, project *types.Project) error { - cmd := exec.CommandContext(ctx, m.path, "status", "--json") - err := m.prepare(ctx, cmd) - if err != nil { - return err - } - - statusOut, err := cmd.CombinedOutput() - if err != nil { - return fmt.Errorf("error checking docker-model status: %w", err) - } - type Status struct { - Endpoint string `json:"endpoint"` - } - - var status Status - err = json.Unmarshal(statusOut, &status) - if err != nil { - return err - } - - for _, service := range project.Services { - for ref, modelConfig := range service.Models { - model := project.Models[ref] - varPrefix := strings.ReplaceAll(strings.ToUpper(ref), "-", "_") - var variable string - if modelConfig != nil && modelConfig.ModelVariable != "" { - variable = modelConfig.ModelVariable - } else { - variable = varPrefix + "_MODEL" - } - service.Environment[variable] = &model.Model - - if modelConfig != nil && modelConfig.EndpointVariable != "" { - variable = modelConfig.EndpointVariable - } else { - variable = varPrefix + "_URL" - } - service.Environment[variable] = &status.Endpoint - } - } - return nil -} - -type Model struct { - Id string `json:"id"` - Tags []string `json:"tags"` - Created int `json:"created"` - Config struct { - Format string `json:"format"` - Quantization string `json:"quantization"` - Parameters string `json:"parameters"` - Architecture string `json:"architecture"` - Size string `json:"size"` - } `json:"config"` -} - -func (m *modelAPI) ListModels(ctx context.Context) ([]string, error) { - cmd := exec.CommandContext(ctx, m.path, "ls", "--json") - err := m.prepare(ctx, cmd) - if err != nil { - return nil, err - } - - output, err := cmd.CombinedOutput() - if err != nil { - return nil, fmt.Errorf("error checking available models: %w", err) - } - - type AvailableModel struct { - Id string `json:"id"` - Tags []string `json:"tags"` - Created int `json:"created"` - } - - models := []AvailableModel{} - err = json.Unmarshal(output, &models) - if err != nil { - return nil, fmt.Errorf("error unmarshalling available models: %w", err) - } - var availableModels []string - for _, model := range models { - availableModels = append(availableModels, model.Tags...) - } - return availableModels, nil -} - -// supportsRuntimeFlags checks if the docker model version supports runtime flags -// Runtime flags are supported in version >= v1.0.6 -func (m *modelAPI) supportsRuntimeFlags() bool { - // If version is not cached, don't append runtime flags to be safe - if m.version == "" { - return false - } - - // Strip 'v' prefix if present (e.g., "v1.0.6" -> "1.0.6") - versionStr := strings.TrimPrefix(m.version, "v") - return !versions.LessThan(versionStr, "1.0.6") -} diff --git a/pkg/compose/monitor.go b/pkg/compose/monitor.go deleted file mode 100644 index 70dd30ada00..00000000000 --- a/pkg/compose/monitor.go +++ /dev/null @@ -1,215 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strconv" - - "github.com/containerd/errdefs" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/events" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/client" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -type monitor struct { - apiClient client.APIClient - project string - // services tells us which service to consider and those we can ignore, maybe ran by a concurrent compose command - services map[string]bool - listeners []api.ContainerEventListener -} - -func newMonitor(apiClient client.APIClient, project string) *monitor { - return &monitor{ - apiClient: apiClient, - project: project, - services: map[string]bool{}, - } -} - -func (c *monitor) withServices(services []string) { - for _, name := range services { - c.services[name] = true - } -} - -// Start runs monitor to detect application events and return after termination -// -//nolint:gocyclo -func (c *monitor) Start(ctx context.Context) error { - // collect initial application container - initialState, err := c.apiClient.ContainerList(ctx, container.ListOptions{ - All: true, - Filters: filters.NewArgs( - projectFilter(c.project), - oneOffFilter(false), - hasConfigHashLabel(), - ), - }) - if err != nil { - return err - } - - // containers is the set if container IDs the application is based on - containers := utils.Set[string]{} - for _, ctr := range initialState { - if len(c.services) == 0 || c.services[ctr.Labels[api.ServiceLabel]] { - containers.Add(ctr.ID) - } - } - restarting := utils.Set[string]{} - - evtCh, errCh := c.apiClient.Events(ctx, events.ListOptions{ - Filters: filters.NewArgs( - filters.Arg("type", "container"), - projectFilter(c.project)), - }) - for { - if len(containers) == 0 { - return nil - } - select { - case <-ctx.Done(): - return nil - case err := <-errCh: - return err - case event := <-evtCh: - if len(c.services) > 0 && !c.services[event.Actor.Attributes[api.ServiceLabel]] { - continue - } - ctr, err := c.getContainerSummary(event) - if err != nil { - return err - } - - switch event.Action { - case events.ActionCreate: - if len(c.services) == 0 || c.services[ctr.Labels[api.ServiceLabel]] { - containers.Add(ctr.ID) - } - evtType := api.ContainerEventCreated - if _, ok := ctr.Labels[api.ContainerReplaceLabel]; ok { - evtType = api.ContainerEventRecreated - } - for _, listener := range c.listeners { - listener(newContainerEvent(event.TimeNano, ctr, evtType)) - } - logrus.Debugf("container %s created", ctr.Name) - case events.ActionStart: - restarted := restarting.Has(ctr.ID) - if restarted { - logrus.Debugf("container %s restarted", ctr.Name) - for _, listener := range c.listeners { - listener(newContainerEvent(event.TimeNano, ctr, api.ContainerEventStarted, func(e *api.ContainerEvent) { - e.Restarting = restarted - })) - } - } else { - logrus.Debugf("container %s started", ctr.Name) - for _, listener := range c.listeners { - listener(newContainerEvent(event.TimeNano, ctr, api.ContainerEventStarted)) - } - } - if len(c.services) == 0 || c.services[ctr.Labels[api.ServiceLabel]] { - containers.Add(ctr.ID) - } - case events.ActionRestart: - for _, listener := range c.listeners { - listener(newContainerEvent(event.TimeNano, ctr, api.ContainerEventRestarted)) - } - logrus.Debugf("container %s restarted", ctr.Name) - case events.ActionDie: - logrus.Debugf("container %s exited with code %d", ctr.Name, ctr.ExitCode) - inspect, err := c.apiClient.ContainerInspect(ctx, event.Actor.ID) - if errdefs.IsNotFound(err) { - // Source is already removed - } else if err != nil { - return err - } - - if inspect.State != nil && (inspect.State.Restarting || inspect.State.Running) { - // State.Restarting is set by engine when container is configured to restart on exit - // on ContainerRestart it doesn't (see https://github.com/moby/moby/issues/45538) - // container state still is reported as "running" - logrus.Debugf("container %s is restarting", ctr.Name) - restarting.Add(ctr.ID) - for _, listener := range c.listeners { - listener(newContainerEvent(event.TimeNano, ctr, api.ContainerEventExited, func(e *api.ContainerEvent) { - e.Restarting = true - })) - } - } else { - for _, listener := range c.listeners { - listener(newContainerEvent(event.TimeNano, ctr, api.ContainerEventExited)) - } - containers.Remove(ctr.ID) - } - } - } - } -} - -func newContainerEvent(timeNano int64, ctr *api.ContainerSummary, eventType int, opts ...func(e *api.ContainerEvent)) api.ContainerEvent { - name := ctr.Name - defaultName := getDefaultContainerName(ctr.Project, ctr.Labels[api.ServiceLabel], ctr.Labels[api.ContainerNumberLabel]) - if name == defaultName { - // remove project- prefix - name = name[len(ctr.Project)+1:] - } - - event := api.ContainerEvent{ - Type: eventType, - Container: ctr, - Time: timeNano, - Source: name, - ID: ctr.ID, - Service: ctr.Service, - ExitCode: ctr.ExitCode, - } - for _, opt := range opts { - opt(&event) - } - return event -} - -func (c *monitor) getContainerSummary(event events.Message) (*api.ContainerSummary, error) { - ctr := &api.ContainerSummary{ - ID: event.Actor.ID, - Name: event.Actor.Attributes["name"], - Project: c.project, - Service: event.Actor.Attributes[api.ServiceLabel], - Labels: event.Actor.Attributes, // More than just labels, but that'c the closest the API gives us - } - if ec, ok := event.Actor.Attributes["exitCode"]; ok { - exitCode, err := strconv.Atoi(ec) - if err != nil { - return nil, err - } - ctr.ExitCode = exitCode - } - return ctr, nil -} - -func (c *monitor) withListener(listener api.ContainerEventListener) { - c.listeners = append(c.listeners, listener) -} diff --git a/pkg/compose/pause.go b/pkg/compose/pause.go deleted file mode 100644 index 35d476eeac2..00000000000 --- a/pkg/compose/pause.go +++ /dev/null @@ -1,87 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strings" - - "github.com/docker/docker/api/types/container" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Pause(ctx context.Context, projectName string, options api.PauseOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.pause(ctx, strings.ToLower(projectName), options) - }, "pause", s.events) -} - -func (s *composeService) pause(ctx context.Context, projectName string, options api.PauseOptions) error { - containers, err := s.getContainers(ctx, projectName, oneOffExclude, false, options.Services...) - if err != nil { - return err - } - - if options.Project != nil { - containers = containers.filter(isService(options.Project.ServiceNames()...)) - } - - eg, ctx := errgroup.WithContext(ctx) - containers.forEach(func(container container.Summary) { - eg.Go(func() error { - err := s.apiClient().ContainerPause(ctx, container.ID) - if err == nil { - eventName := getContainerProgressName(container) - s.events.On(newEvent(eventName, api.Done, "Paused")) - } - return err - }) - }) - return eg.Wait() -} - -func (s *composeService) UnPause(ctx context.Context, projectName string, options api.PauseOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.unPause(ctx, strings.ToLower(projectName), options) - }, "unpause", s.events) -} - -func (s *composeService) unPause(ctx context.Context, projectName string, options api.PauseOptions) error { - containers, err := s.getContainers(ctx, projectName, oneOffExclude, false, options.Services...) - if err != nil { - return err - } - - if options.Project != nil { - containers = containers.filter(isService(options.Project.ServiceNames()...)) - } - - eg, ctx := errgroup.WithContext(ctx) - containers.forEach(func(ctr container.Summary) { - eg.Go(func() error { - err = s.apiClient().ContainerUnpause(ctx, ctr.ID) - if err == nil { - eventName := getContainerProgressName(ctr) - s.events.On(newEvent(eventName, api.Done, "Unpaused")) - } - return err - }) - }) - return eg.Wait() -} diff --git a/pkg/compose/plugins.go b/pkg/compose/plugins.go deleted file mode 100644 index 159394f6bf1..00000000000 --- a/pkg/compose/plugins.go +++ /dev/null @@ -1,283 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "context" - "encoding/json" - "errors" - "fmt" - "io" - "os" - "os/exec" - "path/filepath" - "strings" - "sync" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/docker/cli/cli-plugins/manager" - "github.com/docker/cli/cli/config" - "github.com/sirupsen/logrus" - "github.com/spf13/cobra" - - "github.com/docker/compose/v5/pkg/api" -) - -type JsonMessage struct { - Type string `json:"type"` - Message string `json:"message"` -} - -const ( - ErrorType = "error" - InfoType = "info" - SetEnvType = "setenv" - DebugType = "debug" - providerMetadataDirectory = "compose/providers" -) - -var mux sync.Mutex - -func (s *composeService) runPlugin(ctx context.Context, project *types.Project, service types.ServiceConfig, command string) error { - provider := *service.Provider - - plugin, err := s.getPluginBinaryPath(provider.Type) - if err != nil { - return err - } - - cmd, err := s.setupPluginCommand(ctx, project, service, plugin, command) - if err != nil { - return err - } - - variables, err := s.executePlugin(cmd, command, service) - if err != nil { - return err - } - - mux.Lock() - defer mux.Unlock() - for name, s := range project.Services { - if _, ok := s.DependsOn[service.Name]; ok { - prefix := strings.ToUpper(service.Name) + "_" - for key, val := range variables { - s.Environment[prefix+key] = &val - } - project.Services[name] = s - } - } - return nil -} - -func (s *composeService) executePlugin(cmd *exec.Cmd, command string, service types.ServiceConfig) (types.Mapping, error) { - var action string - switch command { - case "up": - s.events.On(creatingEvent(service.Name)) - action = "create" - case "down": - s.events.On(removingEvent(service.Name)) - action = "remove" - default: - return nil, fmt.Errorf("unsupported plugin command: %s", command) - } - - stdout, err := cmd.StdoutPipe() - if err != nil { - return nil, err - } - - err = cmd.Start() - if err != nil { - return nil, err - } - - decoder := json.NewDecoder(stdout) - defer func() { _ = stdout.Close() }() - - variables := types.Mapping{} - - for { - var msg JsonMessage - err = decoder.Decode(&msg) - if errors.Is(err, io.EOF) { - break - } - if err != nil { - return nil, err - } - switch msg.Type { - case ErrorType: - s.events.On(newEvent(service.Name, api.Error, msg.Message)) - return nil, errors.New(msg.Message) - case InfoType: - s.events.On(newEvent(service.Name, api.Working, msg.Message)) - case SetEnvType: - key, val, found := strings.Cut(msg.Message, "=") - if !found { - return nil, fmt.Errorf("invalid response from plugin: %s", msg.Message) - } - variables[key] = val - case DebugType: - logrus.Debugf("%s: %s", service.Name, msg.Message) - default: - return nil, fmt.Errorf("invalid response from plugin: %s", msg.Type) - } - } - - err = cmd.Wait() - if err != nil { - s.events.On(errorEvent(service.Name, err.Error())) - return nil, fmt.Errorf("failed to %s service provider: %s", action, err.Error()) - } - switch command { - case "up": - s.events.On(createdEvent(service.Name)) - case "down": - s.events.On(removedEvent(service.Name)) - } - return variables, nil -} - -func (s *composeService) getPluginBinaryPath(provider string) (path string, err error) { - if provider == "compose" { - return "", errors.New("'compose' is not a valid provider type") - } - plugin, err := manager.GetPlugin(provider, s.dockerCli, &cobra.Command{}) - if err == nil { - path = plugin.Path - } - if errdefs.IsNotFound(err) { - path, err = exec.LookPath(executable(provider)) - } - return path, err -} - -func (s *composeService) setupPluginCommand(ctx context.Context, project *types.Project, service types.ServiceConfig, path, command string) (*exec.Cmd, error) { - cmdOptionsMetadata := s.getPluginMetadata(path, service.Provider.Type, project) - var currentCommandMetadata CommandMetadata - switch command { - case "up": - currentCommandMetadata = cmdOptionsMetadata.Up - case "down": - currentCommandMetadata = cmdOptionsMetadata.Down - } - - provider := *service.Provider - commandMetadataIsEmpty := cmdOptionsMetadata.IsEmpty() - if err := currentCommandMetadata.CheckRequiredParameters(provider); !commandMetadataIsEmpty && err != nil { - return nil, err - } - - args := []string{"compose", fmt.Sprintf("--project-name=%s", project.Name), command} - for k, v := range provider.Options { - for _, value := range v { - if _, ok := currentCommandMetadata.GetParameter(k); commandMetadataIsEmpty || ok { - args = append(args, fmt.Sprintf("--%s=%s", k, value)) - } - } - } - args = append(args, service.Name) - - cmd := exec.CommandContext(ctx, path, args...) - - err := s.prepareShellOut(ctx, project.Environment, cmd) - if err != nil { - return nil, err - } - return cmd, nil -} - -func (s *composeService) getPluginMetadata(path, command string, project *types.Project) ProviderMetadata { - cmd := exec.Command(path, "compose", "metadata") - err := s.prepareShellOut(context.Background(), project.Environment, cmd) - if err != nil { - logrus.Debugf("failed to prepare plugin metadata command: %v", err) - return ProviderMetadata{} - } - stdout := &bytes.Buffer{} - cmd.Stdout = stdout - - if err := cmd.Run(); err != nil { - logrus.Debugf("failed to start plugin metadata command: %v", err) - return ProviderMetadata{} - } - - var metadata ProviderMetadata - if err := json.Unmarshal(stdout.Bytes(), &metadata); err != nil { - output, _ := io.ReadAll(stdout) - logrus.Debugf("failed to decode plugin metadata: %v - %s", err, output) - return ProviderMetadata{} - } - // Save metadata into docker home directory to be used by Docker LSP tool - // Just log the error as it's not a critical error for the main flow - metadataDir := filepath.Join(config.Dir(), providerMetadataDirectory) - if err := os.MkdirAll(metadataDir, 0o700); err == nil { - metadataFilePath := filepath.Join(metadataDir, command+".json") - if err := os.WriteFile(metadataFilePath, stdout.Bytes(), 0o600); err != nil { - logrus.Debugf("failed to save plugin metadata: %v", err) - } - } else { - logrus.Debugf("failed to create plugin metadata directory: %v", err) - } - return metadata -} - -type ProviderMetadata struct { - Description string `json:"description"` - Up CommandMetadata `json:"up"` - Down CommandMetadata `json:"down"` -} - -func (p ProviderMetadata) IsEmpty() bool { - return p.Description == "" && p.Up.Parameters == nil && p.Down.Parameters == nil -} - -type CommandMetadata struct { - Parameters []ParameterMetadata `json:"parameters"` -} - -type ParameterMetadata struct { - Name string `json:"name"` - Description string `json:"description"` - Required bool `json:"required"` - Type string `json:"type"` - Default string `json:"default,omitempty"` -} - -func (c CommandMetadata) GetParameter(paramName string) (ParameterMetadata, bool) { - for _, p := range c.Parameters { - if p.Name == paramName { - return p, true - } - } - return ParameterMetadata{}, false -} - -func (c CommandMetadata) CheckRequiredParameters(provider types.ServiceProviderConfig) error { - for _, p := range c.Parameters { - if p.Required { - if _, ok := provider.Options[p.Name]; !ok { - return fmt.Errorf("required parameter %q is missing from provider %q definition", p.Name, provider.Type) - } - } - } - return nil -} diff --git a/pkg/compose/plugins_windows.go b/pkg/compose/plugins_windows.go deleted file mode 100644 index 327b3e4c102..00000000000 --- a/pkg/compose/plugins_windows.go +++ /dev/null @@ -1,23 +0,0 @@ -//go:build windows - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -func executable(s string) string { - return s + ".exe" -} diff --git a/pkg/compose/port.go b/pkg/compose/port.go deleted file mode 100644 index ac64b4dfef1..00000000000 --- a/pkg/compose/port.go +++ /dev/null @@ -1,55 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "strings" - - "github.com/docker/docker/api/types/container" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Port(ctx context.Context, projectName string, service string, port uint16, options api.PortOptions) (string, int, error) { - projectName = strings.ToLower(projectName) - ctr, err := s.getSpecifiedContainer(ctx, projectName, oneOffInclude, false, service, options.Index) - if err != nil { - return "", 0, err - } - for _, p := range ctr.Ports { - if p.PrivatePort == port && p.Type == options.Protocol { - return p.IP, int(p.PublicPort), nil - } - } - return "", 0, portNotFoundError(options.Protocol, port, ctr) -} - -func portNotFoundError(protocol string, port uint16, ctr container.Summary) error { - formatPort := func(protocol string, port uint16) string { - return fmt.Sprintf("%d/%s", port, protocol) - } - - var containerPorts []string - for _, p := range ctr.Ports { - containerPorts = append(containerPorts, formatPort(p.Type, p.PublicPort)) - } - - name := strings.TrimPrefix(ctr.Names[0], "/") - return fmt.Errorf("no port %s for container %s: %s", formatPort(protocol, port), name, strings.Join(containerPorts, ", ")) -} diff --git a/pkg/compose/printer.go b/pkg/compose/printer.go deleted file mode 100644 index 6fe6b9fda6e..00000000000 --- a/pkg/compose/printer.go +++ /dev/null @@ -1,57 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "fmt" - - "github.com/docker/compose/v5/pkg/api" -) - -// logPrinter watch application containers and collect their logs -type logPrinter interface { - HandleEvent(event api.ContainerEvent) -} - -type printer struct { - consumer api.LogConsumer -} - -// newLogPrinter builds a LogPrinter passing containers logs to LogConsumer -func newLogPrinter(consumer api.LogConsumer) logPrinter { - printer := printer{ - consumer: consumer, - } - return &printer -} - -func (p *printer) HandleEvent(event api.ContainerEvent) { - switch event.Type { - case api.ContainerEventExited: - if event.Restarting { - p.consumer.Status(event.Source, fmt.Sprintf("exited with code %d (restarting)", event.ExitCode)) - } else { - p.consumer.Status(event.Source, fmt.Sprintf("exited with code %d", event.ExitCode)) - } - case api.ContainerEventRecreated: - p.consumer.Status(event.Container.Labels[api.ContainerReplaceLabel], "has been recreated") - case api.ContainerEventLog, api.HookEventLog: - p.consumer.Log(event.Source, event.Line) - case api.ContainerEventErr: - p.consumer.Err(event.Source, event.Line) - } -} diff --git a/pkg/compose/progress.go b/pkg/compose/progress.go deleted file mode 100644 index 26f9b5d8590..00000000000 --- a/pkg/compose/progress.go +++ /dev/null @@ -1,176 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - - "github.com/docker/compose/v5/pkg/api" -) - -type progressFunc func(context.Context) error - -func Run(ctx context.Context, pf progressFunc, operation string, bus api.EventProcessor) error { - bus.Start(ctx, operation) - err := pf(ctx) - bus.Done(operation, err != nil) - return err -} - -// errorEvent creates a new Error Resource with message -func errorEvent(id string, msg string) api.Resource { - return api.Resource{ - ID: id, - Status: api.Error, - Text: api.StatusError, - Details: msg, - } -} - -// errorEventf creates a new Error Resource with format message -func errorEventf(id string, msg string, args ...any) api.Resource { - return errorEvent(id, fmt.Sprintf(msg, args...)) -} - -// creatingEvent creates a new Create in progress Resource -func creatingEvent(id string) api.Resource { - return newEvent(id, api.Working, api.StatusCreating) -} - -// startingEvent creates a new Starting in progress Resource -func startingEvent(id string) api.Resource { - return newEvent(id, api.Working, api.StatusStarting) -} - -// startedEvent creates a new Started in progress Resource -func startedEvent(id string) api.Resource { - return newEvent(id, api.Done, api.StatusStarted) -} - -// waiting creates a new waiting event -func waiting(id string) api.Resource { - return newEvent(id, api.Working, api.StatusWaiting) -} - -// healthy creates a new healthy event -func healthy(id string) api.Resource { - return newEvent(id, api.Done, api.StatusHealthy) -} - -// exited creates a new exited event -func exited(id string) api.Resource { - return newEvent(id, api.Done, api.StatusExited) -} - -// restartingEvent creates a new Restarting in progress Resource -func restartingEvent(id string) api.Resource { - return newEvent(id, api.Working, api.StatusRestarting) -} - -// runningEvent creates a new Running in progress Resource -func runningEvent(id string) api.Resource { - return newEvent(id, api.Done, api.StatusRunning) -} - -// createdEvent creates a new Created (done) Resource -func createdEvent(id string) api.Resource { - return newEvent(id, api.Done, api.StatusCreated) -} - -// stoppingEvent creates a new Stopping in progress Resource -func stoppingEvent(id string) api.Resource { - return newEvent(id, api.Working, api.StatusStopping) -} - -// stoppedEvent creates a new Stopping in progress Resource -func stoppedEvent(id string) api.Resource { - return newEvent(id, api.Done, api.StatusStopped) -} - -// killingEvent creates a new Killing in progress Resource -func killingEvent(id string) api.Resource { - return newEvent(id, api.Working, api.StatusKilling) -} - -// killedEvent creates a new Killed in progress Resource -func killedEvent(id string) api.Resource { - return newEvent(id, api.Done, api.StatusKilled) -} - -// removingEvent creates a new Removing in progress Resource -func removingEvent(id string) api.Resource { - return newEvent(id, api.Working, api.StatusRemoving) -} - -// removedEvent creates a new removed (done) Resource -func removedEvent(id string) api.Resource { - return newEvent(id, api.Done, api.StatusRemoved) -} - -// buildingEvent creates a new Building in progress Resource -func buildingEvent(id string) api.Resource { - return newEvent("Image "+id, api.Working, api.StatusBuilding) -} - -// builtEvent creates a new built (done) Resource -func builtEvent(id string) api.Resource { - return newEvent("Image "+id, api.Done, api.StatusBuilt) -} - -// pullingEvent creates a new pulling (in progress) Resource -func pullingEvent(id string) api.Resource { - return newEvent("Image "+id, api.Working, api.StatusPulling) -} - -// pulledEvent creates a new pulled (done) Resource -func pulledEvent(id string) api.Resource { - return newEvent("Image "+id, api.Done, api.StatusPulled) -} - -// skippedEvent creates a new Skipped Resource -func skippedEvent(id string, reason string) api.Resource { - return api.Resource{ - ID: id, - Status: api.Warning, - Text: "Skipped: " + reason, - } -} - -// newEvent new event -func newEvent(id string, status api.EventStatus, text string, reason ...string) api.Resource { - r := api.Resource{ - ID: id, - Status: status, - Text: text, - } - if len(reason) > 0 { - r.Details = reason[0] - } - return r -} - -type ignore struct{} - -func (q *ignore) Start(_ context.Context, _ string) { -} - -func (q *ignore) Done(_ string, _ bool) { -} - -func (q *ignore) On(_ ...api.Resource) { -} diff --git a/pkg/compose/ps.go b/pkg/compose/ps.go deleted file mode 100644 index fb32358aebe..00000000000 --- a/pkg/compose/ps.go +++ /dev/null @@ -1,128 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "sort" - "strings" - - "github.com/docker/docker/api/types/container" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Ps(ctx context.Context, projectName string, options api.PsOptions) ([]api.ContainerSummary, error) { - projectName = strings.ToLower(projectName) - oneOff := oneOffExclude - if options.All { - oneOff = oneOffInclude - } - containers, err := s.getContainers(ctx, projectName, oneOff, options.All, options.Services...) - if err != nil { - return nil, err - } - - if len(options.Services) != 0 { - containers = containers.filter(isService(options.Services...)) - } - summary := make([]api.ContainerSummary, len(containers)) - eg, ctx := errgroup.WithContext(ctx) - for i, ctr := range containers { - eg.Go(func() error { - publishers := make([]api.PortPublisher, len(ctr.Ports)) - sort.Slice(ctr.Ports, func(i, j int) bool { - return ctr.Ports[i].PrivatePort < ctr.Ports[j].PrivatePort - }) - for i, p := range ctr.Ports { - publishers[i] = api.PortPublisher{ - URL: p.IP, - TargetPort: int(p.PrivatePort), - PublishedPort: int(p.PublicPort), - Protocol: p.Type, - } - } - - inspect, err := s.apiClient().ContainerInspect(ctx, ctr.ID) - if err != nil { - return err - } - - var ( - health container.HealthStatus - exitCode int - ) - if inspect.State != nil { - switch inspect.State.Status { - case container.StateRunning: - if inspect.State.Health != nil { - health = inspect.State.Health.Status - } - case container.StateExited, container.StateDead: - exitCode = inspect.State.ExitCode - } - } - - var ( - local int - mounts []string - ) - for _, m := range ctr.Mounts { - name := m.Name - if name == "" { - name = m.Source - } - if m.Driver == "local" { - local++ - } - mounts = append(mounts, name) - } - - var networks []string - if ctr.NetworkSettings != nil { - for k := range ctr.NetworkSettings.Networks { - networks = append(networks, k) - } - } - - summary[i] = api.ContainerSummary{ - ID: ctr.ID, - Name: getCanonicalContainerName(ctr), - Names: ctr.Names, - Image: ctr.Image, - Project: ctr.Labels[api.ProjectLabel], - Service: ctr.Labels[api.ServiceLabel], - Command: ctr.Command, - State: ctr.State, - Status: ctr.Status, - Created: ctr.Created, - Labels: ctr.Labels, - SizeRw: ctr.SizeRw, - SizeRootFs: ctr.SizeRootFs, - Mounts: mounts, - LocalVolumes: local, - Networks: networks, - Health: health, - ExitCode: exitCode, - Publishers: publishers, - } - return nil - }) - } - return summary, eg.Wait() -} diff --git a/pkg/compose/ps_test.go b/pkg/compose/ps_test.go deleted file mode 100644 index b006d2f57fa..00000000000 --- a/pkg/compose/ps_test.go +++ /dev/null @@ -1,114 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "strings" - "testing" - - containerType "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - compose "github.com/docker/compose/v5/pkg/api" -) - -func TestPs(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - args := filters.NewArgs(projectFilter(strings.ToLower(testProject)), hasConfigHashLabel()) - args.Add("label", "com.docker.compose.oneoff=False") - listOpts := containerType.ListOptions{Filters: args, All: false} - c1, inspect1 := containerDetails("service1", "123", containerType.StateRunning, containerType.Healthy, 0) - c2, inspect2 := containerDetails("service1", "456", containerType.StateRunning, "", 0) - c2.Ports = []containerType.Port{{PublicPort: 80, PrivatePort: 90, IP: "localhost"}} - c3, inspect3 := containerDetails("service2", "789", containerType.StateExited, "", 130) - api.EXPECT().ContainerList(t.Context(), listOpts).Return([]containerType.Summary{c1, c2, c3}, nil) - api.EXPECT().ContainerInspect(anyCancellableContext(), "123").Return(inspect1, nil) - api.EXPECT().ContainerInspect(anyCancellableContext(), "456").Return(inspect2, nil) - api.EXPECT().ContainerInspect(anyCancellableContext(), "789").Return(inspect3, nil) - - containers, err := tested.Ps(t.Context(), strings.ToLower(testProject), compose.PsOptions{}) - - expected := []compose.ContainerSummary{ - { - ID: "123", Name: "123", Names: []string{"/123"}, Image: "foo", Project: strings.ToLower(testProject), Service: "service1", - State: containerType.StateRunning, - Health: containerType.Healthy, - Publishers: []compose.PortPublisher{}, - Labels: map[string]string{ - compose.ProjectLabel: strings.ToLower(testProject), - compose.ConfigFilesLabel: "/src/pkg/compose/testdata/compose.yaml", - compose.WorkingDirLabel: "/src/pkg/compose/testdata", - compose.ServiceLabel: "service1", - }, - }, - { - ID: "456", Name: "456", Names: []string{"/456"}, Image: "foo", Project: strings.ToLower(testProject), Service: "service1", - State: containerType.StateRunning, - Health: "", - Publishers: []compose.PortPublisher{{URL: "localhost", TargetPort: 90, PublishedPort: 80}}, - Labels: map[string]string{ - compose.ProjectLabel: strings.ToLower(testProject), - compose.ConfigFilesLabel: "/src/pkg/compose/testdata/compose.yaml", - compose.WorkingDirLabel: "/src/pkg/compose/testdata", - compose.ServiceLabel: "service1", - }, - }, - { - ID: "789", Name: "789", Names: []string{"/789"}, Image: "foo", Project: strings.ToLower(testProject), Service: "service2", - State: containerType.StateExited, - Health: "", - ExitCode: 130, - Publishers: []compose.PortPublisher{}, - Labels: map[string]string{ - compose.ProjectLabel: strings.ToLower(testProject), - compose.ConfigFilesLabel: "/src/pkg/compose/testdata/compose.yaml", - compose.WorkingDirLabel: "/src/pkg/compose/testdata", - compose.ServiceLabel: "service2", - }, - }, - } - assert.NilError(t, err) - assert.DeepEqual(t, containers, expected) -} - -func containerDetails(service string, id string, status containerType.ContainerState, health containerType.HealthStatus, exitCode int) (containerType.Summary, containerType.InspectResponse) { - ctr := containerType.Summary{ - ID: id, - Names: []string{"/" + id}, - Image: "foo", - Labels: containerLabels(service, false), - State: status, - } - inspect := containerType.InspectResponse{ - ContainerJSONBase: &containerType.ContainerJSONBase{ - State: &containerType.State{ - Status: status, - Health: &containerType.Health{Status: health}, - ExitCode: exitCode, - }, - }, - } - return ctr, inspect -} diff --git a/pkg/compose/publish.go b/pkg/compose/publish.go deleted file mode 100644 index fb466607559..00000000000 --- a/pkg/compose/publish.go +++ /dev/null @@ -1,505 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "bytes" - "context" - "crypto/sha256" - "encoding/json" - "errors" - "fmt" - "io" - "os" - "strings" - - "github.com/DefangLabs/secret-detector/pkg/scanner" - "github.com/DefangLabs/secret-detector/pkg/secrets" - "github.com/compose-spec/compose-go/v2/loader" - "github.com/compose-spec/compose-go/v2/types" - "github.com/distribution/reference" - "github.com/opencontainers/go-digest" - "github.com/opencontainers/image-spec/specs-go" - v1 "github.com/opencontainers/image-spec/specs-go/v1" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/internal/oci" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/compose/transform" -) - -func (s *composeService) Publish(ctx context.Context, project *types.Project, repository string, options api.PublishOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.publish(ctx, project, repository, options) - }, "publish", s.events) -} - -//nolint:gocyclo -func (s *composeService) publish(ctx context.Context, project *types.Project, repository string, options api.PublishOptions) error { - project, err := project.WithProfiles([]string{"*"}) - if err != nil { - return err - } - accept, err := s.preChecks(project, options) - if err != nil { - return err - } - if !accept { - return nil - } - err = s.Push(ctx, project, api.PushOptions{IgnoreFailures: true, ImageMandatory: true}) - if err != nil { - return err - } - - layers, err := s.createLayers(ctx, project, options) - if err != nil { - return err - } - - s.events.On(api.Resource{ - ID: repository, - Text: "publishing", - Status: api.Working, - }) - if logrus.IsLevelEnabled(logrus.DebugLevel) { - logrus.Debug("publishing layers") - for _, layer := range layers { - indent, _ := json.MarshalIndent(layer, "", " ") - fmt.Println(string(indent)) - } - } - if !s.dryRun { - named, err := reference.ParseDockerRef(repository) - if err != nil { - return err - } - - var insecureRegistries []string - if options.InsecureRegistry { - insecureRegistries = append(insecureRegistries, reference.Domain(named)) - } - - resolver := oci.NewResolver(s.configFile(), insecureRegistries...) - - descriptor, err := oci.PushManifest(ctx, resolver, named, layers, options.OCIVersion) - if err != nil { - s.events.On(api.Resource{ - ID: repository, - Text: "publishing", - Status: api.Error, - }) - return err - } - - if options.Application { - manifests := []v1.Descriptor{} - for _, service := range project.Services { - ref, err := reference.ParseDockerRef(service.Image) - if err != nil { - return err - } - - manifest, err := oci.Copy(ctx, resolver, ref, named) - if err != nil { - return err - } - manifests = append(manifests, manifest) - } - - descriptor.Data = nil - index, err := json.Marshal(v1.Index{ - Versioned: specs.Versioned{SchemaVersion: 2}, - MediaType: v1.MediaTypeImageIndex, - Manifests: manifests, - Subject: &descriptor, - Annotations: map[string]string{ - "com.docker.compose.version": api.ComposeVersion, - }, - }) - if err != nil { - return err - } - imagesDescriptor := v1.Descriptor{ - MediaType: v1.MediaTypeImageIndex, - ArtifactType: oci.ComposeProjectArtifactType, - Digest: digest.FromString(string(index)), - Size: int64(len(index)), - Annotations: map[string]string{ - "com.docker.compose.version": api.ComposeVersion, - }, - Data: index, - } - err = oci.Push(ctx, resolver, reference.TrimNamed(named), imagesDescriptor) - if err != nil { - return err - } - } - } - s.events.On(api.Resource{ - ID: repository, - Text: "published", - Status: api.Done, - }) - return nil -} - -func (s *composeService) createLayers(ctx context.Context, project *types.Project, options api.PublishOptions) ([]v1.Descriptor, error) { - var layers []v1.Descriptor - extFiles := map[string]string{} - envFiles := map[string]string{} - for _, file := range project.ComposeFiles { - data, err := processFile(ctx, file, project, extFiles, envFiles) - if err != nil { - return nil, err - } - - layerDescriptor := oci.DescriptorForComposeFile(file, data) - layers = append(layers, layerDescriptor) - } - - extLayers, err := processExtends(ctx, project, extFiles) - if err != nil { - return nil, err - } - layers = append(layers, extLayers...) - - if options.WithEnvironment { - layers = append(layers, envFileLayers(envFiles)...) - } - - if options.ResolveImageDigests { - yaml, err := s.generateImageDigestsOverride(ctx, project) - if err != nil { - return nil, err - } - - layerDescriptor := oci.DescriptorForComposeFile("image-digests.yaml", yaml) - layers = append(layers, layerDescriptor) - } - return layers, nil -} - -func processExtends(ctx context.Context, project *types.Project, extFiles map[string]string) ([]v1.Descriptor, error) { - var layers []v1.Descriptor - moreExtFiles := map[string]string{} - for xf, hash := range extFiles { - data, err := processFile(ctx, xf, project, moreExtFiles, nil) - if err != nil { - return nil, err - } - - layerDescriptor := oci.DescriptorForComposeFile(hash, data) - layerDescriptor.Annotations["com.docker.compose.extends"] = "true" - layers = append(layers, layerDescriptor) - } - for f, hash := range moreExtFiles { - if _, ok := extFiles[f]; ok { - delete(moreExtFiles, f) - } - extFiles[f] = hash - } - if len(moreExtFiles) > 0 { - extLayers, err := processExtends(ctx, project, moreExtFiles) - if err != nil { - return nil, err - } - layers = append(layers, extLayers...) - } - return layers, nil -} - -func processFile(ctx context.Context, file string, project *types.Project, extFiles map[string]string, envFiles map[string]string) ([]byte, error) { - f, err := os.ReadFile(file) - if err != nil { - return nil, err - } - - base, err := loader.LoadWithContext(ctx, types.ConfigDetails{ - WorkingDir: project.WorkingDir, - Environment: project.Environment, - ConfigFiles: []types.ConfigFile{ - { - Filename: file, - Content: f, - }, - }, - }, func(options *loader.Options) { - options.SkipValidation = true - options.SkipExtends = true - options.SkipConsistencyCheck = true - options.ResolvePaths = true - options.SkipInclude = true - options.Profiles = project.Profiles - }) - if err != nil { - return nil, err - } - for name, service := range base.Services { - for i, envFile := range service.EnvFiles { - hash := fmt.Sprintf("%x.env", sha256.Sum256([]byte(envFile.Path))) - envFiles[envFile.Path] = hash - f, err = transform.ReplaceEnvFile(f, name, i, hash) - if err != nil { - return nil, err - } - } - - if service.Extends == nil { - continue - } - xf := service.Extends.File - if xf == "" { - continue - } - if _, err = os.Stat(service.Extends.File); os.IsNotExist(err) { - // No local file, while we loaded the project successfully: This is actually a remote resource - continue - } - - hash := fmt.Sprintf("%x.yaml", sha256.Sum256([]byte(xf))) - extFiles[xf] = hash - - f, err = transform.ReplaceExtendsFile(f, name, hash) - if err != nil { - return nil, err - } - } - return f, nil -} - -func (s *composeService) generateImageDigestsOverride(ctx context.Context, project *types.Project) ([]byte, error) { - project, err := project.WithImagesResolved(ImageDigestResolver(ctx, s.configFile(), s.apiClient())) - if err != nil { - return nil, err - } - override := types.Project{ - Services: types.Services{}, - } - for name, service := range project.Services { - override.Services[name] = types.ServiceConfig{ - Image: service.Image, - } - } - return override.MarshalYAML() -} - -func (s *composeService) preChecks(project *types.Project, options api.PublishOptions) (bool, error) { - if ok, err := s.checkOnlyBuildSection(project); !ok || err != nil { - return false, err - } - bindMounts := s.checkForBindMount(project) - if len(bindMounts) > 0 { - b := strings.Builder{} - b.WriteString("you are about to publish bind mounts declaration within your OCI artifact.\n" + - "only the bind mount declarations will be added to the OCI artifact (not content)\n" + - "please double check that you are not mounting potential user's sensitive directories or data\n") - for key, val := range bindMounts { - b.WriteString(key) - for _, v := range val { - b.WriteString(v.String()) - b.WriteRune('\n') - } - } - b.WriteString("Are you ok to publish these bind mount declarations?") - confirm, err := s.prompt(b.String(), false) - if err != nil || !confirm { - return false, err - } - } - detectedSecrets, err := s.checkForSensitiveData(project) - if err != nil { - return false, err - } - if len(detectedSecrets) > 0 { - b := strings.Builder{} - b.WriteString("you are about to publish sensitive data within your OCI artifact.\n" + - "please double check that you are not leaking sensitive data\n") - for _, val := range detectedSecrets { - b.WriteString(val.Type) - b.WriteRune('\n') - b.WriteString(fmt.Sprintf("%q: %s\n", val.Key, val.Value)) - } - b.WriteString("Are you ok to publish these sensitive data?") - confirm, err := s.prompt(b.String(), false) - if err != nil || !confirm { - return false, err - } - } - err = s.checkEnvironmentVariables(project, options) - if err != nil { - return false, err - } - return true, nil -} - -func (s *composeService) checkEnvironmentVariables(project *types.Project, options api.PublishOptions) error { - errorList := map[string][]string{} - - for _, service := range project.Services { - if len(service.EnvFiles) > 0 { - errorList[service.Name] = append(errorList[service.Name], fmt.Sprintf("service %q has env_file declared.", service.Name)) - } - } - - if !options.WithEnvironment && len(errorList) > 0 { - errorMsgSuffix := "To avoid leaking sensitive data, you must either explicitly allow the sending of environment variables by using the --with-env flag,\n" + - "or remove sensitive data from your Compose configuration" - var errorMsg strings.Builder - for _, errors := range errorList { - for _, err := range errors { - errorMsg.WriteString(fmt.Sprintf("%s\n", err)) - } - } - return fmt.Errorf("%s%s", errorMsg.String(), errorMsgSuffix) - - } - return nil -} - -func envFileLayers(files map[string]string) []v1.Descriptor { - var layers []v1.Descriptor - for file, hash := range files { - f, err := os.ReadFile(file) - if err != nil { - // if we can't read the file, skip to the next one - continue - } - layerDescriptor := oci.DescriptorForEnvFile(hash, f) - layers = append(layers, layerDescriptor) - } - return layers -} - -func (s *composeService) checkOnlyBuildSection(project *types.Project) (bool, error) { - errorList := []string{} - for _, service := range project.Services { - if service.Image == "" && service.Build != nil { - errorList = append(errorList, service.Name) - } - } - if len(errorList) > 0 { - var errMsg strings.Builder - errMsg.WriteString("your Compose stack cannot be published as it only contains a build section for service(s):\n") - for _, serviceInError := range errorList { - errMsg.WriteString(fmt.Sprintf("- %q\n", serviceInError)) - } - return false, errors.New(errMsg.String()) - } - return true, nil -} - -func (s *composeService) checkForBindMount(project *types.Project) map[string][]types.ServiceVolumeConfig { - allFindings := map[string][]types.ServiceVolumeConfig{} - for serviceName, config := range project.Services { - bindMounts := []types.ServiceVolumeConfig{} - for _, volume := range config.Volumes { - if volume.Type == types.VolumeTypeBind { - bindMounts = append(bindMounts, volume) - } - } - if len(bindMounts) > 0 { - allFindings[serviceName] = bindMounts - } - } - return allFindings -} - -func (s *composeService) checkForSensitiveData(project *types.Project) ([]secrets.DetectedSecret, error) { - var allFindings []secrets.DetectedSecret - scan := scanner.NewDefaultScanner() - // Check all compose files - for _, file := range project.ComposeFiles { - in, err := composeFileAsByteReader(file, project) - if err != nil { - return nil, err - } - - findings, err := scan.ScanReader(in) - if err != nil { - return nil, fmt.Errorf("failed to scan compose file %s: %w", file, err) - } - allFindings = append(allFindings, findings...) - } - for _, service := range project.Services { - // Check env files - for _, envFile := range service.EnvFiles { - findings, err := scan.ScanFile(envFile.Path) - if err != nil { - return nil, fmt.Errorf("failed to scan env file %s: %w", envFile.Path, err) - } - allFindings = append(allFindings, findings...) - } - } - - // Check configs defined by files - for _, config := range project.Configs { - if config.File != "" { - findings, err := scan.ScanFile(config.File) - if err != nil { - return nil, fmt.Errorf("failed to scan config file %s: %w", config.File, err) - } - allFindings = append(allFindings, findings...) - } - } - - // Check secrets defined by files - for _, secret := range project.Secrets { - if secret.File != "" { - findings, err := scan.ScanFile(secret.File) - if err != nil { - return nil, fmt.Errorf("failed to scan secret file %s: %w", secret.File, err) - } - allFindings = append(allFindings, findings...) - } - } - - return allFindings, nil -} - -func composeFileAsByteReader(filePath string, project *types.Project) (io.Reader, error) { - composeFile, err := os.ReadFile(filePath) - if err != nil { - return nil, fmt.Errorf("failed to open compose file %s: %w", filePath, err) - } - base, err := loader.LoadWithContext(context.TODO(), types.ConfigDetails{ - WorkingDir: project.WorkingDir, - Environment: project.Environment, - ConfigFiles: []types.ConfigFile{ - { - Filename: filePath, - Content: composeFile, - }, - }, - }, func(options *loader.Options) { - options.SkipValidation = true - options.SkipExtends = true - options.SkipConsistencyCheck = true - options.ResolvePaths = true - options.SkipInterpolation = true - options.SkipResolveEnvironment = true - }) - if err != nil { - return nil, err - } - - in, err := base.MarshalYAML() - if err != nil { - return nil, err - } - return bytes.NewBuffer(in), nil -} diff --git a/pkg/compose/publish_test.go b/pkg/compose/publish_test.go deleted file mode 100644 index 8f91f663e69..00000000000 --- a/pkg/compose/publish_test.go +++ /dev/null @@ -1,102 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "slices" - "testing" - - "github.com/compose-spec/compose-go/v2/loader" - "github.com/compose-spec/compose-go/v2/types" - "github.com/google/go-cmp/cmp" - v1 "github.com/opencontainers/image-spec/specs-go/v1" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/internal" - "github.com/docker/compose/v5/pkg/api" -) - -func Test_createLayers(t *testing.T) { - project, err := loader.LoadWithContext(t.Context(), types.ConfigDetails{ - WorkingDir: "testdata/publish/", - Environment: types.Mapping{}, - ConfigFiles: []types.ConfigFile{ - { - Filename: "testdata/publish/compose.yaml", - }, - }, - }) - assert.NilError(t, err) - project.ComposeFiles = []string{"testdata/publish/compose.yaml"} - - service := &composeService{} - layers, err := service.createLayers(t.Context(), project, api.PublishOptions{ - WithEnvironment: true, - }) - assert.NilError(t, err) - - published := string(layers[0].Data) - assert.Equal(t, published, `name: test -services: - test: - extends: - file: f8f9ede3d201ec37d5a5e3a77bbadab79af26035e53135e19571f50d541d390c.yaml - service: foo - - string: - image: test - env_file: 5efca9cdbac9f5394c6c2e2094b1b42661f988f57fcab165a0bf72b205451af3.env - - list: - image: test - env_file: - - 5efca9cdbac9f5394c6c2e2094b1b42661f988f57fcab165a0bf72b205451af3.env - - mapping: - image: test - env_file: - - path: 5efca9cdbac9f5394c6c2e2094b1b42661f988f57fcab165a0bf72b205451af3.env -`) - - expectedLayers := []v1.Descriptor{ - { - MediaType: "application/vnd.docker.compose.file+yaml", - Annotations: map[string]string{ - "com.docker.compose.file": "compose.yaml", - "com.docker.compose.version": internal.Version, - }, - }, - { - MediaType: "application/vnd.docker.compose.file+yaml", - Annotations: map[string]string{ - "com.docker.compose.extends": "true", - "com.docker.compose.file": "f8f9ede3d201ec37d5a5e3a77bbadab79af26035e53135e19571f50d541d390c", - "com.docker.compose.version": internal.Version, - }, - }, - { - MediaType: "application/vnd.docker.compose.envfile", - Annotations: map[string]string{ - "com.docker.compose.envfile": "5efca9cdbac9f5394c6c2e2094b1b42661f988f57fcab165a0bf72b205451af3", - "com.docker.compose.version": internal.Version, - }, - }, - } - assert.DeepEqual(t, expectedLayers, layers, cmp.FilterPath(func(path cmp.Path) bool { - return !slices.Contains([]string{".Data", ".Digest", ".Size"}, path.String()) - }, cmp.Ignore())) -} diff --git a/pkg/compose/pull.go b/pkg/compose/pull.go deleted file mode 100644 index bcc466646a6..00000000000 --- a/pkg/compose/pull.go +++ /dev/null @@ -1,454 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "encoding/base64" - "encoding/json" - "errors" - "fmt" - "io" - "strings" - "sync" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/distribution/reference" - "github.com/docker/cli/cli/config/configfile" - clitypes "github.com/docker/cli/cli/config/types" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/client" - "github.com/docker/docker/pkg/jsonmessage" - "github.com/opencontainers/go-digest" - "github.com/sirupsen/logrus" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/internal/registry" - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Pull(ctx context.Context, project *types.Project, options api.PullOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.pull(ctx, project, options) - }, "pull", s.events) -} - -func (s *composeService) pull(ctx context.Context, project *types.Project, opts api.PullOptions) error { //nolint:gocyclo - images, err := s.getLocalImagesDigests(ctx, project) - if err != nil { - return err - } - - eg, ctx := errgroup.WithContext(ctx) - eg.SetLimit(s.maxConcurrency) - - var ( - mustBuild []string - pullErrors = make([]error, len(project.Services)) - imagesBeingPulled = map[string]string{} - ) - - i := 0 - for name, service := range project.Services { - if service.Image == "" { - s.events.On(api.Resource{ - ID: name, - Status: api.Done, - Text: "Skipped", - Details: "No image to be pulled", - }) - continue - } - - switch service.PullPolicy { - case types.PullPolicyNever, types.PullPolicyBuild: - s.events.On(api.Resource{ - ID: "Image " + service.Image, - Status: api.Done, - Text: "Skipped", - }) - continue - case types.PullPolicyMissing, types.PullPolicyIfNotPresent: - if imageAlreadyPresent(service.Image, images) { - s.events.On(api.Resource{ - ID: "Image " + service.Image, - Status: api.Done, - Text: "Skipped", - Details: "Image is already present locally", - }) - continue - } - } - - if service.Build != nil && opts.IgnoreBuildable { - s.events.On(api.Resource{ - ID: "Image " + service.Image, - Status: api.Done, - Text: "Skipped", - Details: "Image can be built", - }) - continue - } - - if _, ok := imagesBeingPulled[service.Image]; ok { - continue - } - - imagesBeingPulled[service.Image] = service.Name - - idx := i - eg.Go(func() error { - _, err := s.pullServiceImage(ctx, service, opts.Quiet, project.Environment["DOCKER_DEFAULT_PLATFORM"]) - if err != nil { - pullErrors[idx] = err - if service.Build != nil { - mustBuild = append(mustBuild, service.Name) - } - if !opts.IgnoreFailures && service.Build == nil { - if s.dryRun { - s.events.On(errorEventf("Image "+service.Image, - "error pulling image: %s", service.Image)) - } - // fail fast if image can't be pulled nor built - return err - } - } - return nil - }) - i++ - } - - err = eg.Wait() - - if len(mustBuild) > 0 { - logrus.Warnf("WARNING: Some service image(s) must be built from source by running:\n docker compose build %s", strings.Join(mustBuild, " ")) - } - - if err != nil { - return err - } - if opts.IgnoreFailures { - return nil - } - return errors.Join(pullErrors...) -} - -func imageAlreadyPresent(serviceImage string, localImages map[string]api.ImageSummary) bool { - normalizedImage, err := reference.ParseDockerRef(serviceImage) - if err != nil { - return false - } - switch refType := normalizedImage.(type) { - case reference.NamedTagged: - _, ok := localImages[serviceImage] - return ok && refType.Tag() != "latest" - default: - _, ok := localImages[serviceImage] - return ok - } -} - -func getUnwrappedErrorMessage(err error) string { - derr := errors.Unwrap(err) - if derr != nil { - return getUnwrappedErrorMessage(derr) - } - return err.Error() -} - -func (s *composeService) pullServiceImage(ctx context.Context, service types.ServiceConfig, quietPull bool, defaultPlatform string) (string, error) { - resource := "Image " + service.Image - s.events.On(pullingEvent(service.Image)) - ref, err := reference.ParseNormalizedNamed(service.Image) - if err != nil { - return "", err - } - - encodedAuth, err := encodedAuth(ref, s.configFile()) - if err != nil { - return "", err - } - - platform := service.Platform - if platform == "" { - platform = defaultPlatform - } - - stream, err := s.apiClient().ImagePull(ctx, service.Image, image.PullOptions{ - RegistryAuth: encodedAuth, - Platform: platform, - }) - - if ctx.Err() != nil { - s.events.On(api.Resource{ - ID: resource, - Status: api.Warning, - Text: "Interrupted", - }) - return "", nil - } - - // check if has error and the service has a build section - // then the status should be warning instead of error - if err != nil && service.Build != nil { - s.events.On(api.Resource{ - ID: resource, - Status: api.Warning, - Text: getUnwrappedErrorMessage(err), - }) - return "", err - } - - if err != nil { - s.events.On(errorEvent(resource, getUnwrappedErrorMessage(err))) - return "", err - } - - dec := json.NewDecoder(stream) - for { - var jm jsonmessage.JSONMessage - if err := dec.Decode(&jm); err != nil { - if errors.Is(err, io.EOF) { - break - } - return "", err - } - if jm.Error != nil { - return "", errors.New(jm.Error.Message) - } - if !quietPull { - toPullProgressEvent(resource, jm, s.events) - } - } - s.events.On(pulledEvent(service.Image)) - - inspected, err := s.apiClient().ImageInspect(ctx, service.Image) - if err != nil { - return "", err - } - return inspected.ID, nil -} - -// ImageDigestResolver creates a func able to resolve image digest from a docker ref, -func ImageDigestResolver(ctx context.Context, file *configfile.ConfigFile, apiClient client.APIClient) func(named reference.Named) (digest.Digest, error) { - return func(named reference.Named) (digest.Digest, error) { - auth, err := encodedAuth(named, file) - if err != nil { - return "", err - } - inspect, err := apiClient.DistributionInspect(ctx, named.String(), auth) - if err != nil { - return "", - fmt.Errorf("failed to resolve digest for %s: %w", named.String(), err) - } - return inspect.Descriptor.Digest, nil - } -} - -type authProvider interface { - GetAuthConfig(registryHostname string) (clitypes.AuthConfig, error) -} - -func encodedAuth(ref reference.Named, configFile authProvider) (string, error) { - authConfig, err := configFile.GetAuthConfig(registry.GetAuthConfigKey(reference.Domain(ref))) - if err != nil { - return "", err - } - - buf, err := json.Marshal(authConfig) - if err != nil { - return "", err - } - return base64.URLEncoding.EncodeToString(buf), nil -} - -func (s *composeService) pullRequiredImages(ctx context.Context, project *types.Project, images map[string]api.ImageSummary, quietPull bool) error { - needPull := map[string]types.ServiceConfig{} - for name, service := range project.Services { - pull, err := mustPull(service, images) - if err != nil { - return err - } - if pull { - needPull[name] = service - } - for i, vol := range service.Volumes { - if vol.Type == types.VolumeTypeImage { - if _, ok := images[vol.Source]; !ok { - // Hack: create a fake ServiceConfig so we pull missing volume image - n := fmt.Sprintf("%s:volume %d", name, i) - needPull[n] = types.ServiceConfig{ - Name: n, - Image: vol.Source, - } - } - } - } - - } - if len(needPull) == 0 { - return nil - } - - eg, ctx := errgroup.WithContext(ctx) - eg.SetLimit(s.maxConcurrency) - pulledImages := map[string]api.ImageSummary{} - var mutex sync.Mutex - for name, service := range needPull { - eg.Go(func() error { - id, err := s.pullServiceImage(ctx, service, quietPull, project.Environment["DOCKER_DEFAULT_PLATFORM"]) - mutex.Lock() - defer mutex.Unlock() - pulledImages[name] = api.ImageSummary{ - ID: id, - Repository: service.Image, - LastTagTime: time.Now(), - } - if err != nil && isServiceImageToBuild(service, project.Services) { - // image can be built, so we can ignore pull failure - return nil - } - return err - }) - } - err := eg.Wait() - for i, service := range needPull { - if pulledImages[i].ID != "" { - images[service.Image] = pulledImages[i] - } - } - return err -} - -func mustPull(service types.ServiceConfig, images map[string]api.ImageSummary) (bool, error) { - if service.Provider != nil { - return false, nil - } - if service.Image == "" { - return false, nil - } - policy, duration, err := service.GetPullPolicy() - if err != nil { - return false, err - } - switch policy { - case types.PullPolicyAlways: - // force pull - return true, nil - case types.PullPolicyNever, types.PullPolicyBuild: - return false, nil - case types.PullPolicyRefresh: - img, ok := images[service.Image] - if !ok { - return true, nil - } - return time.Now().After(img.LastTagTime.Add(duration)), nil - default: // Pull if missing - _, ok := images[service.Image] - return !ok, nil - } -} - -func isServiceImageToBuild(service types.ServiceConfig, services types.Services) bool { - if service.Build != nil { - return true - } - - if service.Image == "" { - // N.B. this should be impossible as service must have either `build` or `image` (or both) - return false - } - - // look through the other services to see if another has a build definition for the same - // image name - for _, svc := range services { - if svc.Image == service.Image && svc.Build != nil { - return true - } - } - return false -} - -const ( - PreparingPhase = "Preparing" - WaitingPhase = "waiting" - PullingFsPhase = "Pulling fs layer" - DownloadingPhase = "Downloading" - DownloadCompletePhase = "Download complete" - ExtractingPhase = "Extracting" - VerifyingChecksumPhase = "Verifying Checksum" - AlreadyExistsPhase = "Already exists" - PullCompletePhase = "Pull complete" -) - -func toPullProgressEvent(parent string, jm jsonmessage.JSONMessage, events api.EventProcessor) { - if jm.ID == "" || jm.Progress == nil { - return - } - - var ( - progress string - total int64 - percent int - current int64 - status = api.Working - ) - - progress = jm.Progress.String() - - switch jm.Status { - case PreparingPhase, WaitingPhase, PullingFsPhase: - percent = 0 - case DownloadingPhase, ExtractingPhase, VerifyingChecksumPhase: - if jm.Progress != nil { - current = jm.Progress.Current - total = jm.Progress.Total - if jm.Progress.Total > 0 { - percent = int(jm.Progress.Current * 100 / jm.Progress.Total) - if percent > 100 { - percent = 100 - } - } - } - case DownloadCompletePhase, AlreadyExistsPhase, PullCompletePhase: - status = api.Done - percent = 100 - } - - if strings.Contains(jm.Status, "Image is up to date") || - strings.Contains(jm.Status, "Downloaded newer image") { - status = api.Done - percent = 100 - } - - if jm.Error != nil { - status = api.Error - progress = jm.Error.Message - } - - events.On(api.Resource{ - ID: jm.ID, - ParentID: parent, - Current: current, - Total: total, - Percent: percent, - Status: status, - Text: jm.Status, - Details: progress, - }) -} diff --git a/pkg/compose/push.go b/pkg/compose/push.go deleted file mode 100644 index a1e9b2686e4..00000000000 --- a/pkg/compose/push.go +++ /dev/null @@ -1,184 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "encoding/base64" - "encoding/json" - "errors" - "fmt" - "io" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/distribution/reference" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/pkg/jsonmessage" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/internal/registry" - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Push(ctx context.Context, project *types.Project, options api.PushOptions) error { - if options.Quiet { - return s.push(ctx, project, options) - } - return Run(ctx, func(ctx context.Context) error { - return s.push(ctx, project, options) - }, "push", s.events) -} - -func (s *composeService) push(ctx context.Context, project *types.Project, options api.PushOptions) error { - eg, ctx := errgroup.WithContext(ctx) - eg.SetLimit(s.maxConcurrency) - - for _, service := range project.Services { - if service.Build == nil || service.Image == "" { - if options.ImageMandatory && service.Image == "" && service.Provider == nil { - return fmt.Errorf("%q attribute is mandatory to push an image for service %q", "service.image", service.Name) - } - s.events.On(api.Resource{ - ID: service.Name, - Status: api.Done, - Text: "Skipped", - }) - continue - } - tags := []string{service.Image} - if service.Build != nil { - tags = append(tags, service.Build.Tags...) - } - - for _, tag := range tags { - eg.Go(func() error { - s.events.On(newEvent(tag, api.Working, "Pushing")) - err := s.pushServiceImage(ctx, tag, options.Quiet) - if err != nil { - if !options.IgnoreFailures { - s.events.On(newEvent(tag, api.Error, err.Error())) - return err - } - s.events.On(newEvent(tag, api.Warning, err.Error())) - } else { - s.events.On(newEvent(tag, api.Done, "Pushed")) - } - return nil - }) - } - } - return eg.Wait() -} - -func (s *composeService) pushServiceImage(ctx context.Context, tag string, quietPush bool) error { - ref, err := reference.ParseNormalizedNamed(tag) - if err != nil { - return err - } - - authConfig, err := s.configFile().GetAuthConfig(registry.GetAuthConfigKey(reference.Domain(ref))) - if err != nil { - return err - } - - buf, err := json.Marshal(authConfig) - if err != nil { - return err - } - - stream, err := s.apiClient().ImagePush(ctx, tag, image.PushOptions{ - RegistryAuth: base64.URLEncoding.EncodeToString(buf), - }) - if err != nil { - return err - } - dec := json.NewDecoder(stream) - for { - var jm jsonmessage.JSONMessage - if err := dec.Decode(&jm); err != nil { - if errors.Is(err, io.EOF) { - break - } - return err - } - if jm.Error != nil { - return errors.New(jm.Error.Message) - } - - if !quietPush { - toPushProgressEvent(tag, jm, s.events) - } - } - - return nil -} - -func toPushProgressEvent(prefix string, jm jsonmessage.JSONMessage, events api.EventProcessor) { - if jm.ID == "" { - // skipped - return - } - var ( - text string - status = api.Working - total int64 - current int64 - percent int - ) - if isDone(jm) { - status = api.Done - percent = 100 - } - if jm.Error != nil { - status = api.Error - text = jm.Error.Message - } - if jm.Progress != nil { - text = jm.Progress.String() - if jm.Progress.Total != 0 { - current = jm.Progress.Current - total = jm.Progress.Total - if jm.Progress.Total > 0 { - percent = int(jm.Progress.Current * 100 / jm.Progress.Total) - if percent > 100 { - percent = 100 - } - } - } - } - - events.On(api.Resource{ - ParentID: prefix, - ID: jm.ID, - Text: text, - Status: status, - Current: current, - Total: total, - Percent: percent, - }) -} - -func isDone(msg jsonmessage.JSONMessage) bool { - // TODO there should be a better way to detect push is done than such a status message check - switch strings.ToLower(msg.Status) { - case "pushed", "layer already exists": - return true - default: - return false - } -} diff --git a/pkg/compose/remove.go b/pkg/compose/remove.go deleted file mode 100644 index 3f63c9a6cce..00000000000 --- a/pkg/compose/remove.go +++ /dev/null @@ -1,115 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "strings" - - "github.com/docker/docker/api/types/container" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Remove(ctx context.Context, projectName string, options api.RemoveOptions) error { - projectName = strings.ToLower(projectName) - - if options.Stop { - err := s.Stop(ctx, projectName, api.StopOptions{ - Services: options.Services, - Project: options.Project, - }) - if err != nil { - return err - } - } - - containers, err := s.getContainers(ctx, projectName, oneOffExclude, true, options.Services...) - if err != nil { - if api.IsNotFoundError(err) { - _, _ = fmt.Fprintln(s.stderr(), "No stopped containers") - return nil - } - return err - } - - if options.Project != nil { - containers = containers.filter(isService(options.Project.ServiceNames()...)) - } - - var stoppedContainers Containers - for _, ctr := range containers { - // We have to inspect containers, as State reported by getContainers suffers a race condition - inspected, err := s.apiClient().ContainerInspect(ctx, ctr.ID) - if api.IsNotFoundError(err) { - // Already removed. Maybe configured with auto-remove - continue - } - if err != nil { - return err - } - if !inspected.State.Running || (options.Stop && s.dryRun) { - stoppedContainers = append(stoppedContainers, ctr) - } - } - - var names []string - stoppedContainers.forEach(func(c container.Summary) { - names = append(names, getCanonicalContainerName(c)) - }) - - if len(names) == 0 { - return api.ErrNoResources - } - - msg := fmt.Sprintf("Going to remove %s", strings.Join(names, ", ")) - if options.Force { - _, _ = fmt.Fprintln(s.stdout(), msg) - } else { - confirm, err := s.prompt(msg, false) - if err != nil { - return err - } - if !confirm { - return nil - } - } - return Run(ctx, func(ctx context.Context) error { - return s.remove(ctx, stoppedContainers, options) - }, "remove", s.events) -} - -func (s *composeService) remove(ctx context.Context, containers Containers, options api.RemoveOptions) error { - eg, ctx := errgroup.WithContext(ctx) - for _, ctr := range containers { - eg.Go(func() error { - eventName := getContainerProgressName(ctr) - s.events.On(removingEvent(eventName)) - err := s.apiClient().ContainerRemove(ctx, ctr.ID, container.RemoveOptions{ - RemoveVolumes: options.Volumes, - Force: options.Force, - }) - if err == nil { - s.events.On(removedEvent(eventName)) - } - return err - }) - } - return eg.Wait() -} diff --git a/pkg/compose/restart.go b/pkg/compose/restart.go deleted file mode 100644 index ca73b962896..00000000000 --- a/pkg/compose/restart.go +++ /dev/null @@ -1,114 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -func (s *composeService) Restart(ctx context.Context, projectName string, options api.RestartOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.restart(ctx, strings.ToLower(projectName), options) - }, "restart", s.events) -} - -func (s *composeService) restart(ctx context.Context, projectName string, options api.RestartOptions) error { //nolint:gocyclo - containers, err := s.getContainers(ctx, projectName, oneOffExclude, true) - if err != nil { - return err - } - - project := options.Project - if project == nil { - project, err = s.getProjectWithResources(ctx, containers, projectName) - if err != nil { - return err - } - } - - if options.NoDeps { - project, err = project.WithSelectedServices(options.Services, types.IgnoreDependencies) - if err != nil { - return err - } - } - - // ignore depends_on relations which are not impacted by restarting service or not required - project, err = project.WithServicesTransform(func(_ string, s types.ServiceConfig) (types.ServiceConfig, error) { - for name, r := range s.DependsOn { - if !r.Restart { - delete(s.DependsOn, name) - } - } - return s, nil - }) - if err != nil { - return err - } - - if len(options.Services) != 0 { - project, err = project.WithSelectedServices(options.Services, types.IncludeDependents) - if err != nil { - return err - } - } - - return InDependencyOrder(ctx, project, func(c context.Context, service string) error { - config := project.Services[service] - err = s.waitDependencies(ctx, project, service, config.DependsOn, containers, 0) - if err != nil { - return err - } - - eg, ctx := errgroup.WithContext(ctx) - for _, ctr := range containers.filter(isService(service)) { - eg.Go(func() error { - def := project.Services[service] - for _, hook := range def.PreStop { - err = s.runHook(ctx, ctr, def, hook, nil) - if err != nil { - return err - } - } - eventName := getContainerProgressName(ctr) - s.events.On(restartingEvent(eventName)) - timeout := utils.DurationSecondToInt(options.Timeout) - err = s.apiClient().ContainerRestart(ctx, ctr.ID, container.StopOptions{Timeout: timeout}) - if err != nil { - return err - } - s.events.On(startedEvent(eventName)) - for _, hook := range def.PostStart { - err = s.runHook(ctx, ctr, def, hook, nil) - if err != nil { - return err - } - } - return nil - }) - } - return eg.Wait() - }) -} diff --git a/pkg/compose/run.go b/pkg/compose/run.go deleted file mode 100644 index e5a68fd0447..00000000000 --- a/pkg/compose/run.go +++ /dev/null @@ -1,227 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "os" - "os/signal" - "slices" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli" - cmd "github.com/docker/cli/cli/command/container" - "github.com/docker/docker/pkg/stringid" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) RunOneOffContainer(ctx context.Context, project *types.Project, opts api.RunOptions) (int, error) { - containerID, err := s.prepareRun(ctx, project, opts) - if err != nil { - return 0, err - } - - // remove cancellable context signal handler so we can forward signals to container without compose from exiting - signal.Reset() - - sigc := make(chan os.Signal, 128) - signal.Notify(sigc) - go cmd.ForwardAllSignals(ctx, s.apiClient(), containerID, sigc) - defer signal.Stop(sigc) - - err = cmd.RunStart(ctx, s.dockerCli, &cmd.StartOptions{ - OpenStdin: !opts.Detach && opts.Interactive, - Attach: !opts.Detach, - Containers: []string{containerID}, - DetachKeys: s.configFile().DetachKeys, - }) - var stErr cli.StatusError - if errors.As(err, &stErr) { - return stErr.StatusCode, nil - } - return 0, err -} - -func (s *composeService) prepareRun(ctx context.Context, project *types.Project, opts api.RunOptions) (string, error) { - // Temporary implementation of use_api_socket until we get actual support inside docker engine - project, err := s.useAPISocket(project) - if err != nil { - return "", err - } - - err = Run(ctx, func(ctx context.Context) error { - return s.startDependencies(ctx, project, opts) - }, "run", s.events) - if err != nil { - return "", err - } - - service, err := project.GetService(opts.Service) - if err != nil { - return "", err - } - - applyRunOptions(project, &service, opts) - - if err := s.stdin().CheckTty(opts.Interactive, service.Tty); err != nil { - return "", err - } - - slug := stringid.GenerateRandomID() - if service.ContainerName == "" { - service.ContainerName = fmt.Sprintf("%[1]s%[4]s%[2]s%[4]srun%[4]s%[3]s", project.Name, service.Name, stringid.TruncateID(slug), api.Separator) - } - one := 1 - service.Scale = &one - service.Restart = "" - if service.Deploy != nil { - service.Deploy.RestartPolicy = nil - } - service.CustomLabels = service.CustomLabels. - Add(api.SlugLabel, slug). - Add(api.OneoffLabel, "True") - - // Only ensure image exists for the target service, dependencies were already handled by startDependencies - buildOpts := prepareBuildOptions(opts) - if err := s.ensureImagesExists(ctx, project, buildOpts, opts.QuietPull); err != nil { // all dependencies already checked, but might miss service img - return "", err - } - - observedState, err := s.getContainers(ctx, project.Name, oneOffInclude, true) - if err != nil { - return "", err - } - - if !opts.NoDeps { - if err := s.waitDependencies(ctx, project, service.Name, service.DependsOn, observedState, 0); err != nil { - return "", err - } - } - createOpts := createOptions{ - AutoRemove: opts.AutoRemove, - AttachStdin: opts.Interactive, - UseNetworkAliases: opts.UseNetworkAliases, - Labels: mergeLabels(service.Labels, service.CustomLabels), - } - - err = newConvergence(project.ServiceNames(), observedState, nil, nil, s).resolveServiceReferences(&service) - if err != nil { - return "", err - } - - err = s.ensureModels(ctx, project, opts.QuietPull) - if err != nil { - return "", err - } - - created, err := s.createContainer(ctx, project, service, service.ContainerName, -1, createOpts) - if err != nil { - return "", err - } - - ctr, err := s.apiClient().ContainerInspect(ctx, created.ID) - if err != nil { - return "", err - } - - err = s.injectSecrets(ctx, project, service, ctr.ID) - if err != nil { - return created.ID, err - } - - err = s.injectConfigs(ctx, project, service, ctr.ID) - return created.ID, err -} - -func prepareBuildOptions(opts api.RunOptions) *api.BuildOptions { - if opts.Build == nil { - return nil - } - // Create a copy of build options and restrict to only the target service - buildOptsCopy := *opts.Build - buildOptsCopy.Services = []string{opts.Service} - return &buildOptsCopy -} - -func applyRunOptions(project *types.Project, service *types.ServiceConfig, opts api.RunOptions) { - service.Tty = opts.Tty - service.StdinOpen = opts.Interactive - service.ContainerName = opts.Name - - if len(opts.Command) > 0 { - service.Command = opts.Command - } - if opts.User != "" { - service.User = opts.User - } - - if len(opts.CapAdd) > 0 { - service.CapAdd = append(service.CapAdd, opts.CapAdd...) - service.CapDrop = slices.DeleteFunc(service.CapDrop, func(e string) bool { return slices.Contains(opts.CapAdd, e) }) - } - if len(opts.CapDrop) > 0 { - service.CapDrop = append(service.CapDrop, opts.CapDrop...) - service.CapAdd = slices.DeleteFunc(service.CapAdd, func(e string) bool { return slices.Contains(opts.CapDrop, e) }) - } - if opts.WorkingDir != "" { - service.WorkingDir = opts.WorkingDir - } - if opts.Entrypoint != nil { - service.Entrypoint = opts.Entrypoint - if len(opts.Command) == 0 { - service.Command = []string{} - } - } - if len(opts.Environment) > 0 { - cmdEnv := types.NewMappingWithEquals(opts.Environment) - serviceOverrideEnv := cmdEnv.Resolve(func(s string) (string, bool) { - v, ok := envResolver(project.Environment)(s) - return v, ok - }).RemoveEmpty() - if service.Environment == nil { - service.Environment = types.MappingWithEquals{} - } - service.Environment.OverrideBy(serviceOverrideEnv) - } - for k, v := range opts.Labels { - service.Labels = service.Labels.Add(k, v) - } -} - -func (s *composeService) startDependencies(ctx context.Context, project *types.Project, options api.RunOptions) error { - project = project.WithServicesDisabled(options.Service) - - err := s.Create(ctx, project, api.CreateOptions{ - Build: options.Build, - IgnoreOrphans: options.IgnoreOrphans, - RemoveOrphans: options.RemoveOrphans, - QuietPull: options.QuietPull, - }) - if err != nil { - return err - } - - if len(project.Services) > 0 { - return s.Start(ctx, project.Name, api.StartOptions{ - Project: project, - }) - } - return nil -} diff --git a/pkg/compose/scale.go b/pkg/compose/scale.go deleted file mode 100644 index 4ef8134a264..00000000000 --- a/pkg/compose/scale.go +++ /dev/null @@ -1,35 +0,0 @@ -/* -Copyright 2020 Docker Compose CLI authors - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package compose - -import ( - "context" - - "github.com/compose-spec/compose-go/v2/types" - - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Scale(ctx context.Context, project *types.Project, options api.ScaleOptions) error { - return Run(ctx, tracing.SpanWrapFunc("project/scale", tracing.ProjectOptions(ctx, project), func(ctx context.Context) error { - err := s.create(ctx, project, api.CreateOptions{Services: options.Services}) - if err != nil { - return err - } - return s.start(ctx, project.Name, api.StartOptions{Project: project, Services: options.Services}, nil) - }), "scale", s.events) -} diff --git a/pkg/compose/secrets.go b/pkg/compose/secrets.go deleted file mode 100644 index 72bc4b5c8e8..00000000000 --- a/pkg/compose/secrets.go +++ /dev/null @@ -1,179 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "archive/tar" - "bytes" - "context" - "fmt" - "strconv" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/docker/api/types/container" -) - -type mountType string - -const ( - secretMount mountType = "secret" - configMount mountType = "config" -) - -func (s *composeService) injectSecrets(ctx context.Context, project *types.Project, service types.ServiceConfig, id string) error { - return s.injectFileReferences(ctx, project, service, id, secretMount) -} - -func (s *composeService) injectConfigs(ctx context.Context, project *types.Project, service types.ServiceConfig, id string) error { - return s.injectFileReferences(ctx, project, service, id, configMount) -} - -func (s *composeService) injectFileReferences(ctx context.Context, project *types.Project, service types.ServiceConfig, id string, mountType mountType) error { - mounts, sources := s.getFilesAndMap(project, service, mountType) - - for _, mount := range mounts { - content, err := s.resolveFileContent(project, sources[mount.Source], mountType) - if err != nil { - return err - } - if content == "" { - continue - } - - if service.ReadOnly { - return fmt.Errorf("cannot create %s %q in read-only service %s: `file` is the sole supported option", mountType, sources[mount.Source].Name, service.Name) - } - - s.setDefaultTarget(&mount, mountType) - - if err := s.copyFileToContainer(ctx, id, content, mount); err != nil { - return err - } - } - return nil -} - -func (s *composeService) getFilesAndMap(project *types.Project, service types.ServiceConfig, mountType mountType) ([]types.FileReferenceConfig, map[string]types.FileObjectConfig) { - var files []types.FileReferenceConfig - var fileMap map[string]types.FileObjectConfig - - switch mountType { - case secretMount: - files = make([]types.FileReferenceConfig, len(service.Secrets)) - for i, config := range service.Secrets { - files[i] = types.FileReferenceConfig(config) - } - fileMap = make(map[string]types.FileObjectConfig) - for k, v := range project.Secrets { - fileMap[k] = types.FileObjectConfig(v) - } - case configMount: - files = make([]types.FileReferenceConfig, len(service.Configs)) - for i, config := range service.Configs { - files[i] = types.FileReferenceConfig(config) - } - fileMap = make(map[string]types.FileObjectConfig) - for k, v := range project.Configs { - fileMap[k] = types.FileObjectConfig(v) - } - } - return files, fileMap -} - -func (s *composeService) resolveFileContent(project *types.Project, source types.FileObjectConfig, mountType mountType) (string, error) { - if source.Content != "" { - // inlined, or already resolved by include - return source.Content, nil - } - if source.Environment != "" { - env, ok := project.Environment[source.Environment] - if !ok { - return "", fmt.Errorf("environment variable %q required by %s %q is not set", source.Environment, mountType, source.Name) - } - return env, nil - } - return "", nil -} - -func (s *composeService) setDefaultTarget(file *types.FileReferenceConfig, mountType mountType) { - if file.Target == "" { - if mountType == secretMount { - file.Target = "/run/secrets/" + file.Source - } else { - file.Target = "/" + file.Source - } - } else if mountType == secretMount && !isAbsTarget(file.Target) { - file.Target = "/run/secrets/" + file.Target - } -} - -func (s *composeService) copyFileToContainer(ctx context.Context, id, content string, file types.FileReferenceConfig) error { - b, err := createTar(content, file) - if err != nil { - return err - } - - return s.apiClient().CopyToContainer(ctx, id, "/", &b, container.CopyToContainerOptions{ - CopyUIDGID: file.UID != "" || file.GID != "", - }) -} - -func createTar(env string, config types.FileReferenceConfig) (bytes.Buffer, error) { - value := []byte(env) - b := bytes.Buffer{} - tarWriter := tar.NewWriter(&b) - mode := types.FileMode(0o444) - if config.Mode != nil { - mode = *config.Mode - } - - var uid, gid int - if config.UID != "" { - v, err := strconv.Atoi(config.UID) - if err != nil { - return b, err - } - uid = v - } - if config.GID != "" { - v, err := strconv.Atoi(config.GID) - if err != nil { - return b, err - } - gid = v - } - - header := &tar.Header{ - Name: config.Target, - Size: int64(len(value)), - Mode: int64(mode), - ModTime: time.Now(), - Uid: uid, - Gid: gid, - } - err := tarWriter.WriteHeader(header) - if err != nil { - return bytes.Buffer{}, err - } - _, err = tarWriter.Write(value) - if err != nil { - return bytes.Buffer{}, err - } - err = tarWriter.Close() - return b, err -} diff --git a/pkg/compose/shellout.go b/pkg/compose/shellout.go deleted file mode 100644 index ceea0788214..00000000000 --- a/pkg/compose/shellout.go +++ /dev/null @@ -1,90 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "os" - "os/exec" - "path/filepath" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli-plugins/metadata" - "github.com/docker/cli/cli/command" - "github.com/docker/cli/cli/flags" - "github.com/docker/docker/client" - "go.opentelemetry.io/otel" - "go.opentelemetry.io/otel/propagation" - - "github.com/docker/compose/v5/internal" -) - -// prepareShellOut prepare a shell-out command to be ran by Compose -func (s *composeService) prepareShellOut(gctx context.Context, env types.Mapping, cmd *exec.Cmd) error { - env = env.Clone() - // remove DOCKER_CLI_PLUGIN... variable so a docker-cli plugin will detect it run standalone - delete(env, metadata.ReexecEnvvar) - - // propagate opentelemetry context to child process, see https://github.com/open-telemetry/oteps/blob/main/text/0258-env-context-baggage-carriers.md - carrier := propagation.MapCarrier{} - otel.GetTextMapPropagator().Inject(gctx, &carrier) - env.Merge(types.Mapping(carrier)) - - cmd.Env = env.Values() - return nil -} - -// propagateDockerEndpoint produces DOCKER_* env vars for a child CLI plugin to target the same docker endpoint -// `cleanup` func MUST be called after child process completion to enforce removal of cert files -func (s *composeService) propagateDockerEndpoint() ([]string, func(), error) { - cleanup := func() {} - env := types.Mapping{} - - env[command.EnvOverrideContext] = s.dockerCli.CurrentContext() - env["USER_AGENT"] = "compose/" + internal.Version - - endpoint := s.dockerCli.DockerEndpoint() - env[client.EnvOverrideHost] = endpoint.Host - if endpoint.TLSData != nil { - certs, err := os.MkdirTemp("", "compose") - if err != nil { - return nil, cleanup, err - } - cleanup = func() { - _ = os.RemoveAll(certs) - } - env[client.EnvOverrideCertPath] = certs - env["DOCKER_TLS"] = "1" - if !endpoint.SkipTLSVerify { - env[client.EnvTLSVerify] = "1" - } - - err = os.WriteFile(filepath.Join(certs, flags.DefaultKeyFile), endpoint.TLSData.Key, 0o600) - if err != nil { - return nil, cleanup, err - } - err = os.WriteFile(filepath.Join(certs, flags.DefaultCertFile), endpoint.TLSData.Cert, 0o600) - if err != nil { - return nil, cleanup, err - } - err = os.WriteFile(filepath.Join(certs, flags.DefaultCaFile), endpoint.TLSData.CA, 0o600) - if err != nil { - return nil, cleanup, err - } - } - return env.Values(), cleanup, nil -} diff --git a/pkg/compose/start.go b/pkg/compose/start.go deleted file mode 100644 index eeb232fa66d..00000000000 --- a/pkg/compose/start.go +++ /dev/null @@ -1,115 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - containerType "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Start(ctx context.Context, projectName string, options api.StartOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.start(ctx, strings.ToLower(projectName), options, nil) - }, "start", s.events) -} - -func (s *composeService) start(ctx context.Context, projectName string, options api.StartOptions, listener api.ContainerEventListener) error { - project := options.Project - if project == nil { - var containers Containers - containers, err := s.getContainers(ctx, projectName, oneOffExclude, true) - if err != nil { - return err - } - - project, err = s.projectFromName(containers, projectName, options.AttachTo...) - if err != nil { - return err - } - } - - var containers Containers - containers, err := s.apiClient().ContainerList(ctx, containerType.ListOptions{ - Filters: filters.NewArgs( - projectFilter(project.Name), - oneOffFilter(false), - ), - All: true, - }) - if err != nil { - return err - } - - err = InDependencyOrder(ctx, project, func(c context.Context, name string) error { - service, err := project.GetService(name) - if err != nil { - return err - } - - return s.startService(ctx, project, service, containers, listener, options.WaitTimeout) - }) - if err != nil { - return err - } - - if options.Wait { - depends := types.DependsOnConfig{} - for _, s := range project.Services { - depends[s.Name] = types.ServiceDependency{ - Condition: getDependencyCondition(s, project), - Required: true, - } - } - if options.WaitTimeout > 0 { - withTimeout, cancel := context.WithTimeout(ctx, options.WaitTimeout) - ctx = withTimeout - defer cancel() - } - - err = s.waitDependencies(ctx, project, project.Name, depends, containers, 0) - if err != nil { - if errors.Is(ctx.Err(), context.DeadlineExceeded) { - return fmt.Errorf("application not healthy after %s", options.WaitTimeout) - } - return err - } - } - - return nil -} - -// getDependencyCondition checks if service is depended on by other services -// with service_completed_successfully condition, and applies that condition -// instead, or --wait will never finish waiting for one-shot containers -func getDependencyCondition(service types.ServiceConfig, project *types.Project) string { - for _, services := range project.Services { - for dependencyService, dependencyConfig := range services.DependsOn { - if dependencyService == service.Name && dependencyConfig.Condition == types.ServiceConditionCompletedSuccessfully { - return types.ServiceConditionCompletedSuccessfully - } - } - } - return ServiceConditionRunningOrHealthy -} diff --git a/pkg/compose/stop.go b/pkg/compose/stop.go deleted file mode 100644 index 79272513c0b..00000000000 --- a/pkg/compose/stop.go +++ /dev/null @@ -1,58 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "slices" - "strings" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Stop(ctx context.Context, projectName string, options api.StopOptions) error { - return Run(ctx, func(ctx context.Context) error { - return s.stop(ctx, strings.ToLower(projectName), options, nil) - }, "stop", s.events) -} - -func (s *composeService) stop(ctx context.Context, projectName string, options api.StopOptions, event api.ContainerEventListener) error { - containers, err := s.getContainers(ctx, projectName, oneOffExclude, true) - if err != nil { - return err - } - - project := options.Project - if project == nil { - project, err = s.getProjectWithResources(ctx, containers, projectName) - if err != nil { - return err - } - } - - if len(options.Services) == 0 { - options.Services = project.ServiceNames() - } - - return InReverseDependencyOrder(ctx, project, func(c context.Context, service string) error { - if !slices.Contains(options.Services, service) { - return nil - } - serv := project.Services[service] - return s.stopContainers(ctx, &serv, containers.filter(isService(service)).filter(isNotOneOff), options.Timeout, event) - }) -} diff --git a/pkg/compose/stop_test.go b/pkg/compose/stop_test.go deleted file mode 100644 index adecd1d615b..00000000000 --- a/pkg/compose/stop_test.go +++ /dev/null @@ -1,68 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "strings" - "testing" - "time" - - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/api/types/volume" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - compose "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/utils" -) - -func TestStopTimeout(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - api, cli := prepareMocks(mockCtrl) - tested, err := NewComposeService(cli) - assert.NilError(t, err) - - api.EXPECT().ContainerList(gomock.Any(), projectFilterListOpt(false)).Return( - []container.Summary{ - testContainer("service1", "123", false), - testContainer("service1", "456", false), - testContainer("service2", "789", false), - }, nil) - api.EXPECT().VolumeList( - gomock.Any(), - volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject))), - }). - Return(volume.ListResponse{}, nil) - api.EXPECT().NetworkList(gomock.Any(), network.ListOptions{Filters: filters.NewArgs(projectFilter(strings.ToLower(testProject)))}). - Return([]network.Summary{}, nil) - - timeout := 2 * time.Second - stopConfig := container.StopOptions{Timeout: utils.DurationSecondToInt(&timeout)} - api.EXPECT().ContainerStop(gomock.Any(), "123", stopConfig).Return(nil) - api.EXPECT().ContainerStop(gomock.Any(), "456", stopConfig).Return(nil) - api.EXPECT().ContainerStop(gomock.Any(), "789", stopConfig).Return(nil) - - err = tested.Stop(t.Context(), strings.ToLower(testProject), compose.StopOptions{ - Timeout: &timeout, - }) - assert.NilError(t, err) -} diff --git a/pkg/compose/suffix_unix.go b/pkg/compose/suffix_unix.go deleted file mode 100644 index 59595384ca4..00000000000 --- a/pkg/compose/suffix_unix.go +++ /dev/null @@ -1,23 +0,0 @@ -//go:build !windows - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -func executable(s string) string { - return s -} diff --git a/pkg/compose/testdata/compose.yaml b/pkg/compose/testdata/compose.yaml deleted file mode 100644 index 4e3e6cb98c1..00000000000 --- a/pkg/compose/testdata/compose.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - service1: - image: nginx - service2: - image: mysql diff --git a/pkg/compose/testdata/publish/common.yaml b/pkg/compose/testdata/publish/common.yaml deleted file mode 100644 index ce048e36623..00000000000 --- a/pkg/compose/testdata/publish/common.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - foo: - image: bar diff --git a/pkg/compose/testdata/publish/compose.yaml b/pkg/compose/testdata/publish/compose.yaml deleted file mode 100644 index 9c9f3659b5e..00000000000 --- a/pkg/compose/testdata/publish/compose.yaml +++ /dev/null @@ -1,20 +0,0 @@ -name: test -services: - test: - extends: - file: common.yaml - service: foo - - string: - image: test - env_file: test.env - - list: - image: test - env_file: - - test.env - - mapping: - image: test - env_file: - - path: test.env diff --git a/pkg/compose/testdata/publish/test.env b/pkg/compose/testdata/publish/test.env deleted file mode 100644 index 6e1f61b59ea..00000000000 --- a/pkg/compose/testdata/publish/test.env +++ /dev/null @@ -1 +0,0 @@ -HELLO=WORLD \ No newline at end of file diff --git a/pkg/compose/top.go b/pkg/compose/top.go deleted file mode 100644 index d2efdc1bc8c..00000000000 --- a/pkg/compose/top.go +++ /dev/null @@ -1,65 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strings" - - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Top(ctx context.Context, projectName string, services []string) ([]api.ContainerProcSummary, error) { - projectName = strings.ToLower(projectName) - var containers Containers - containers, err := s.getContainers(ctx, projectName, oneOffInclude, false) - if err != nil { - return nil, err - } - if len(services) > 0 { - containers = containers.filter(isService(services...)) - } - summary := make([]api.ContainerProcSummary, len(containers)) - eg, ctx := errgroup.WithContext(ctx) - for i, ctr := range containers { - eg.Go(func() error { - topContent, err := s.apiClient().ContainerTop(ctx, ctr.ID, []string{}) - if err != nil { - return err - } - name := getCanonicalContainerName(ctr) - s := api.ContainerProcSummary{ - ID: ctr.ID, - Name: name, - Processes: topContent.Processes, - Titles: topContent.Titles, - Service: name, - } - if service, exists := ctr.Labels[api.ServiceLabel]; exists { - s.Service = service - } - if replica, exists := ctr.Labels[api.ContainerNumberLabel]; exists { - s.Replica = replica - } - summary[i] = s - return nil - }) - } - return summary, eg.Wait() -} diff --git a/pkg/compose/transform/replace.go b/pkg/compose/transform/replace.go deleted file mode 100644 index 36266a066f4..00000000000 --- a/pkg/compose/transform/replace.go +++ /dev/null @@ -1,149 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package transform - -import ( - "fmt" - - "go.yaml.in/yaml/v4" -) - -// ReplaceExtendsFile changes value for service.extends.file in input yaml stream, preserving formatting -func ReplaceExtendsFile(in []byte, service string, value string) ([]byte, error) { - var doc yaml.Node - err := yaml.Unmarshal(in, &doc) - if err != nil { - return nil, err - } - if doc.Kind != yaml.DocumentNode { - return nil, fmt.Errorf("expected document kind %v, got %v", yaml.DocumentNode, doc.Kind) - } - root := doc.Content[0] - if root.Kind != yaml.MappingNode { - return nil, fmt.Errorf("expected document root to be a mapping, got %v", root.Kind) - } - - services, err := getMapping(root, "services") - if err != nil { - return nil, err - } - - target, err := getMapping(services, service) - if err != nil { - return nil, err - } - - extends, err := getMapping(target, "extends") - if err != nil { - return nil, err - } - - file, err := getMapping(extends, "file") - if err != nil { - return nil, err - } - - // we've found target `file` yaml node. Let's replace value in stream at node position - return replace(in, file.Line, file.Column, value), nil -} - -// ReplaceEnvFile changes value for service.extends.env_file in input yaml stream, preserving formatting -func ReplaceEnvFile(in []byte, service string, i int, value string) ([]byte, error) { - var doc yaml.Node - err := yaml.Unmarshal(in, &doc) - if err != nil { - return nil, err - } - if doc.Kind != yaml.DocumentNode { - return nil, fmt.Errorf("expected document kind %v, got %v", yaml.DocumentNode, doc.Kind) - } - root := doc.Content[0] - if root.Kind != yaml.MappingNode { - return nil, fmt.Errorf("expected document root to be a mapping, got %v", root.Kind) - } - - services, err := getMapping(root, "services") - if err != nil { - return nil, err - } - - target, err := getMapping(services, service) - if err != nil { - return nil, err - } - - envFile, err := getMapping(target, "env_file") - if err != nil { - return nil, err - } - - // env_file can be either a string, sequence of strings, or sequence of mappings with path attribute - if envFile.Kind == yaml.SequenceNode { - envFile = envFile.Content[i] - if envFile.Kind == yaml.MappingNode { - envFile, err = getMapping(envFile, "path") - if err != nil { - return nil, err - } - } - return replace(in, envFile.Line, envFile.Column, value), nil - } else { - return replace(in, envFile.Line, envFile.Column, value), nil - } -} - -func getMapping(root *yaml.Node, key string) (*yaml.Node, error) { - var node *yaml.Node - l := len(root.Content) - for i := 0; i < l; i += 2 { - k := root.Content[i] - if k.Kind != yaml.ScalarNode || k.Tag != "!!str" { - return nil, fmt.Errorf("expected mapping key to be a string, got %v %v", root.Kind, k.Tag) - } - if k.Value == key { - node = root.Content[i+1] - return node, nil - } - } - return nil, fmt.Errorf("key %v not found", key) -} - -// replace changes yaml node value in stream at position, preserving content -func replace(in []byte, line int, column int, value string) []byte { - var out []byte - l := 1 - pos := 0 - for _, b := range in { - if b == '\n' { - l++ - if l == line { - break - } - } - pos++ - } - pos += column - out = append(out, in[0:pos]...) - out = append(out, []byte(value)...) - for ; pos < len(in); pos++ { - if in[pos] == '\n' { - break - } - } - out = append(out, in[pos:]...) - return out -} diff --git a/pkg/compose/transform/replace_test.go b/pkg/compose/transform/replace_test.go deleted file mode 100644 index 6f7477e0350..00000000000 --- a/pkg/compose/transform/replace_test.go +++ /dev/null @@ -1,85 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package transform - -import ( - "reflect" - "testing" - - "gotest.tools/v3/assert" -) - -func TestReplace(t *testing.T) { - tests := []struct { - name string - in string - want string - }{ - { - name: "simple", - in: `services: - test: - extends: - file: foo.yaml - service: foo -`, - want: `services: - test: - extends: - file: REPLACED - service: foo -`, - }, - { - name: "last line", - in: `services: - test: - extends: - service: foo - file: foo.yaml -`, - want: `services: - test: - extends: - service: foo - file: REPLACED -`, - }, - { - name: "last line no CR", - in: `services: - test: - extends: - service: foo - file: foo.yaml`, - want: `services: - test: - extends: - service: foo - file: REPLACED`, - }, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - got, err := ReplaceExtendsFile([]byte(tt.in), "test", "REPLACED") - assert.NilError(t, err) - if !reflect.DeepEqual(got, []byte(tt.want)) { - t.Errorf("ReplaceExtendsFile() got = %v, want %v", got, tt.want) - } - }) - } -} diff --git a/pkg/compose/up.go b/pkg/compose/up.go deleted file mode 100644 index d5eb4d87691..00000000000 --- a/pkg/compose/up.go +++ /dev/null @@ -1,303 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "os" - "os/signal" - "slices" - "sync" - "sync/atomic" - "syscall" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/containerd/errdefs" - "github.com/docker/cli/cli" - "github.com/eiannone/keyboard" - "github.com/sirupsen/logrus" - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/cmd/formatter" - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Up(ctx context.Context, project *types.Project, options api.UpOptions) error { //nolint:gocyclo - err := Run(ctx, tracing.SpanWrapFunc("project/up", tracing.ProjectOptions(ctx, project), func(ctx context.Context) error { - err := s.create(ctx, project, options.Create) - if err != nil { - return err - } - if options.Start.Attach == nil { - return s.start(ctx, project.Name, options.Start, nil) - } - return nil - }), "up", s.events) - if err != nil { - return err - } - - if options.Start.Attach == nil { - return err - } - if s.dryRun { - _, _ = fmt.Fprintln(s.stdout(), "end of 'compose up' output, interactive run is not supported in dry-run mode") - return err - } - - // if we get a second signal during shutdown, we kill the services - // immediately, so the channel needs to have sufficient capacity or - // we might miss a signal while setting up the second channel read - // (this is also why signal.Notify is used vs signal.NotifyContext) - signalChan := make(chan os.Signal, 2) - signal.Notify(signalChan, syscall.SIGINT, syscall.SIGTERM) - defer signal.Stop(signalChan) - var isTerminated atomic.Bool - - var ( - logConsumer = options.Start.Attach - navigationMenu *formatter.LogKeyboard - kEvents <-chan keyboard.KeyEvent - ) - if options.Start.NavigationMenu { - kEvents, err = keyboard.GetKeys(100) - if err != nil { - logrus.Warnf("could not start menu, an error occurred while starting: %v", err) - options.Start.NavigationMenu = false - } else { - defer keyboard.Close() //nolint:errcheck - isDockerDesktopActive, err := s.isDesktopIntegrationActive(ctx) - if err != nil { - return err - } - tracing.KeyboardMetrics(ctx, options.Start.NavigationMenu, isDockerDesktopActive) - navigationMenu = formatter.NewKeyboardManager(isDockerDesktopActive, signalChan) - logConsumer = navigationMenu.Decorate(logConsumer) - } - } - - watcher, err := NewWatcher(project, options, s.watch, logConsumer) - if err != nil && options.Start.Watch { - return err - } - - if navigationMenu != nil && watcher != nil { - navigationMenu.EnableWatch(options.Start.Watch, watcher) - } - - printer := newLogPrinter(logConsumer) - - // global context to handle canceling goroutines - globalCtx, cancel := context.WithCancel(ctx) - defer cancel() - - if navigationMenu != nil { - navigationMenu.EnableDetach(cancel) - } - - var ( - eg errgroup.Group - mu sync.Mutex - errs []error - ) - - appendErr := func(err error) { - if err != nil { - mu.Lock() - errs = append(errs, err) - mu.Unlock() - } - } - - eg.Go(func() error { - first := true - gracefulTeardown := func() { - first = false - s.events.On(newEvent(api.ResourceCompose, api.Working, api.StatusStopping, "Gracefully Stopping... press Ctrl+C again to force")) - eg.Go(func() error { - err = s.stop(context.WithoutCancel(globalCtx), project.Name, api.StopOptions{ - Services: options.Create.Services, - Project: project, - }, printer.HandleEvent) - appendErr(err) - return nil - }) - isTerminated.Store(true) - } - - for { - select { - case <-globalCtx.Done(): - if watcher != nil { - return watcher.Stop() - } - return nil - case <-ctx.Done(): - if first { - gracefulTeardown() - } - case <-signalChan: - if first { - _ = keyboard.Close() - gracefulTeardown() - break - } - eg.Go(func() error { - err := s.kill(context.WithoutCancel(globalCtx), project.Name, api.KillOptions{ - Services: options.Create.Services, - Project: project, - All: true, - }) - // Ignore errors indicating that some of the containers were already stopped or removed. - if errdefs.IsNotFound(err) || errdefs.IsConflict(err) || errors.Is(err, api.ErrNoResources) { - return nil - } - - appendErr(err) - return nil - }) - return nil - case event := <-kEvents: - navigationMenu.HandleKeyEvents(globalCtx, event, project, options) - } - } - }) - - if options.Start.Watch && watcher != nil { - if err := watcher.Start(globalCtx); err != nil { - // cancel the global context to terminate background goroutines - cancel() - _ = eg.Wait() - return err - } - } - - monitor := newMonitor(s.apiClient(), project.Name) - if len(options.Start.Services) > 0 { - monitor.withServices(options.Start.Services) - } else { - // Start.AttachTo have been already curated with only the services to monitor - monitor.withServices(options.Start.AttachTo) - } - monitor.withListener(printer.HandleEvent) - - var exitCode int - if options.Start.OnExit != api.CascadeIgnore { - once := true - // detect first container to exit to trigger application shutdown - monitor.withListener(func(event api.ContainerEvent) { - if once && event.Type == api.ContainerEventExited { - if options.Start.OnExit == api.CascadeFail && event.ExitCode == 0 { - return - } - once = false - exitCode = event.ExitCode - s.events.On(newEvent(api.ResourceCompose, api.Working, api.StatusStopping, "Aborting on container exit...")) - eg.Go(func() error { - err = s.stop(context.WithoutCancel(globalCtx), project.Name, api.StopOptions{ - Services: options.Create.Services, - Project: project, - }, printer.HandleEvent) - appendErr(err) - return nil - }) - } - }) - } - - if options.Start.ExitCodeFrom != "" { - once := true - // capture exit code from first container to exit with selected service - monitor.withListener(func(event api.ContainerEvent) { - if once && event.Type == api.ContainerEventExited && event.Service == options.Start.ExitCodeFrom { - exitCode = event.ExitCode - once = false - } - }) - } - - containers, err := s.attach(globalCtx, project, printer.HandleEvent, options.Start.AttachTo) - if err != nil { - cancel() - _ = eg.Wait() - return err - } - attached := make([]string, len(containers)) - for i, ctr := range containers { - attached[i] = ctr.ID - } - - monitor.withListener(func(event api.ContainerEvent) { - if event.Type != api.ContainerEventStarted { - return - } - if slices.Contains(attached, event.ID) && !event.Restarting { - return - } - eg.Go(func() error { - ctr, err := s.apiClient().ContainerInspect(globalCtx, event.ID) - if err != nil { - appendErr(err) - return nil - } - - err = s.doLogContainer(globalCtx, options.Start.Attach, event.Source, ctr, api.LogOptions{ - Follow: true, - Since: ctr.State.StartedAt, - }) - if errdefs.IsNotImplemented(err) { - // container may be configured with logging_driver: none - // as container already started, we might miss the very first logs. But still better than none - err := s.doAttachContainer(globalCtx, event.Service, event.ID, event.Source, printer.HandleEvent) - appendErr(err) - return nil - } - appendErr(err) - return nil - }) - }) - - eg.Go(func() error { - err := monitor.Start(globalCtx) - // cancel the global context to terminate signal-handler goroutines - cancel() - appendErr(err) - return nil - }) - - // We use the parent context without cancellation as we manage sigterm to stop the stack - err = s.start(context.WithoutCancel(ctx), project.Name, options.Start, printer.HandleEvent) - if err != nil && !isTerminated.Load() { // Ignore error if the process is terminated - cancel() - _ = eg.Wait() - return err - } - - _ = eg.Wait() - err = errors.Join(errs...) - if exitCode != 0 { - errMsg := "" - if err != nil { - errMsg = err.Error() - } - return cli.StatusError{StatusCode: exitCode, Status: errMsg} - } - return err -} diff --git a/pkg/compose/viz.go b/pkg/compose/viz.go deleted file mode 100644 index cf8c4401254..00000000000 --- a/pkg/compose/viz.go +++ /dev/null @@ -1,141 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "strconv" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - - "github.com/docker/compose/v5/pkg/api" -) - -// maps a service with the services it depends on -type vizGraph map[*types.ServiceConfig][]*types.ServiceConfig - -func (s *composeService) Viz(_ context.Context, project *types.Project, opts api.VizOptions) (string, error) { - graph := make(vizGraph) - for _, service := range project.Services { - graph[&service] = make([]*types.ServiceConfig, 0, len(service.DependsOn)) - for dependencyName := range service.DependsOn { - // no error should be returned since dependencyName should exist - dependency, _ := project.GetService(dependencyName) - graph[&service] = append(graph[&service], &dependency) - } - } - - // build graphviz graph - var graphBuilder strings.Builder - - // graph name - graphBuilder.WriteString("digraph ") - writeQuoted(&graphBuilder, project.Name) - graphBuilder.WriteString(" {\n") - - // graph layout - // dot is the perfect layout for this use case since graph is directed and hierarchical - graphBuilder.WriteString(opts.Indentation + "layout=dot;\n") - - addNodes(&graphBuilder, graph, project.Name, &opts) - graphBuilder.WriteByte('\n') - - addEdges(&graphBuilder, graph, &opts) - graphBuilder.WriteString("}\n") - - return graphBuilder.String(), nil -} - -// addNodes adds the corresponding graphviz representation of all the nodes in the given graph to the graphBuilder -// returns the same graphBuilder -func addNodes(graphBuilder *strings.Builder, graph vizGraph, projectName string, opts *api.VizOptions) *strings.Builder { - for serviceNode := range graph { - // write: - // "service name" [style="filled" label<service name - graphBuilder.WriteString(opts.Indentation) - writeQuoted(graphBuilder, serviceNode.Name) - graphBuilder.WriteString(" [style=\"filled\" label=<") - graphBuilder.WriteString(serviceNode.Name) - graphBuilder.WriteString("") - - if opts.IncludeNetworks && len(serviceNode.Networks) > 0 { - graphBuilder.WriteString("") - graphBuilder.WriteString(" Networks:") - for _, networkName := range serviceNode.NetworksByPriority() { - graphBuilder.WriteString(" ") - graphBuilder.WriteString(networkName) - } - graphBuilder.WriteString("") - } - - if opts.IncludePorts && len(serviceNode.Ports) > 0 { - graphBuilder.WriteString("") - graphBuilder.WriteString(" Ports:") - for _, portConfig := range serviceNode.Ports { - graphBuilder.WriteString(" ") - if portConfig.HostIP != "" { - graphBuilder.WriteString(portConfig.HostIP) - graphBuilder.WriteByte(':') - } - graphBuilder.WriteString(portConfig.Published) - graphBuilder.WriteByte(':') - graphBuilder.WriteString(strconv.Itoa(int(portConfig.Target))) - graphBuilder.WriteString(" (") - graphBuilder.WriteString(portConfig.Protocol) - graphBuilder.WriteString(", ") - graphBuilder.WriteString(portConfig.Mode) - graphBuilder.WriteString(")") - } - graphBuilder.WriteString("") - } - - if opts.IncludeImageName { - graphBuilder.WriteString("") - graphBuilder.WriteString(" Image: ") - graphBuilder.WriteString(api.GetImageNameOrDefault(*serviceNode, projectName)) - graphBuilder.WriteString("") - } - - graphBuilder.WriteString(">];\n") - } - - return graphBuilder -} - -// addEdges adds the corresponding graphviz representation of all edges in the given graph to the graphBuilder -// returns the same graphBuilder -func addEdges(graphBuilder *strings.Builder, graph vizGraph, opts *api.VizOptions) *strings.Builder { - for parent, children := range graph { - for _, child := range children { - graphBuilder.WriteString(opts.Indentation) - writeQuoted(graphBuilder, parent.Name) - graphBuilder.WriteString(" -> ") - writeQuoted(graphBuilder, child.Name) - graphBuilder.WriteString(";\n") - } - } - - return graphBuilder -} - -// writeQuoted writes "str" to builder -func writeQuoted(builder *strings.Builder, str string) { - builder.WriteByte('"') - builder.WriteString(str) - builder.WriteByte('"') -} diff --git a/pkg/compose/viz_test.go b/pkg/compose/viz_test.go deleted file mode 100644 index dad8bac36a3..00000000000 --- a/pkg/compose/viz_test.go +++ /dev/null @@ -1,216 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "strconv" - "testing" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - "go.uber.org/mock/gomock" - - compose "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/mocks" -) - -func TestViz(t *testing.T) { - project := types.Project{ - Name: "viz-test", - WorkingDir: "/home", - Services: types.Services{ - "service1": { - Name: "service1", - Image: "image-for-service1", - Ports: []types.ServicePortConfig{ - { - Published: "80", - Target: 80, - Protocol: "tcp", - }, - { - Published: "53", - Target: 533, - Protocol: "udp", - }, - }, - Networks: map[string]*types.ServiceNetworkConfig{ - "internal": nil, - }, - }, - "service2": { - Name: "service2", - Image: "image-for-service2", - Ports: []types.ServicePortConfig{}, - }, - "service3": { - Name: "service3", - Image: "some-image", - DependsOn: map[string]types.ServiceDependency{ - "service2": {}, - "service1": {}, - }, - }, - "service4": { - Name: "service4", - Image: "another-image", - DependsOn: map[string]types.ServiceDependency{ - "service3": {}, - }, - Ports: []types.ServicePortConfig{ - { - Published: "8080", - Target: 80, - }, - }, - Networks: map[string]*types.ServiceNetworkConfig{ - "external": nil, - }, - }, - "With host IP": { - Name: "With host IP", - Image: "user/image-name", - DependsOn: map[string]types.ServiceDependency{ - "service1": {}, - }, - Ports: []types.ServicePortConfig{ - { - Published: "8888", - Target: 8080, - HostIP: "127.0.0.1", - }, - }, - }, - }, - Networks: types.Networks{ - "internal": types.NetworkConfig{}, - "external": types.NetworkConfig{}, - "not-used": types.NetworkConfig{}, - }, - Volumes: nil, - Secrets: nil, - Configs: nil, - Extensions: nil, - ComposeFiles: nil, - Environment: nil, - DisabledServices: nil, - Profiles: nil, - } - - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - cli := mocks.NewMockCli(mockCtrl) - tested, err := NewComposeService(cli) - require.NoError(t, err) - - t.Run("viz (no ports, networks or image)", func(t *testing.T) { - graphStr, err := tested.Viz(t.Context(), &project, compose.VizOptions{ - Indentation: " ", - IncludePorts: false, - IncludeImageName: false, - IncludeNetworks: false, - }) - require.NoError(t, err, "viz command failed") - - // check indentation - assert.Contains(t, graphStr, "\n ", graphStr) - assert.NotContains(t, graphStr, "\n ", graphStr) - - // check digraph name - assert.Contains(t, graphStr, "digraph \""+project.Name+"\"", graphStr) - - // check nodes - for _, service := range project.Services { - assert.Contains(t, graphStr, "\""+service.Name+"\" [style=\"filled\"", graphStr) - } - - // check node attributes - assert.NotContains(t, graphStr, "Networks", graphStr) - assert.NotContains(t, graphStr, "Image", graphStr) - assert.NotContains(t, graphStr, "Ports", graphStr) - - // check edges that SHOULD exist in the generated graph - allowedEdges := make(map[string][]string) - for name, service := range project.Services { - allowed := make([]string, 0, len(service.DependsOn)) - for depName := range service.DependsOn { - allowed = append(allowed, depName) - } - allowedEdges[name] = allowed - } - for serviceName, dependencies := range allowedEdges { - for _, dependencyName := range dependencies { - assert.Contains(t, graphStr, "\""+serviceName+"\" -> \""+dependencyName+"\"", graphStr) - } - } - - // check edges that SHOULD NOT exist in the generated graph - forbiddenEdges := make(map[string][]string) - for name, service := range project.Services { - forbiddenEdges[name] = make([]string, 0, len(project.ServiceNames())-len(service.DependsOn)) - for _, serviceName := range project.ServiceNames() { - _, edgeExists := service.DependsOn[serviceName] - if !edgeExists { - forbiddenEdges[name] = append(forbiddenEdges[name], serviceName) - } - } - } - for serviceName, forbiddenDeps := range forbiddenEdges { - for _, forbiddenDep := range forbiddenDeps { - assert.NotContains(t, graphStr, "\""+serviceName+"\" -> \""+forbiddenDep+"\"") - } - } - }) - - t.Run("viz (with ports, networks and image)", func(t *testing.T) { - graphStr, err := tested.Viz(t.Context(), &project, compose.VizOptions{ - Indentation: "\t", - IncludePorts: true, - IncludeImageName: true, - IncludeNetworks: true, - }) - require.NoError(t, err, "viz command failed") - - // check indentation - assert.Contains(t, graphStr, "\n\t", graphStr) - assert.NotContains(t, graphStr, "\n\t\t", graphStr) - - // check digraph name - assert.Contains(t, graphStr, "digraph \""+project.Name+"\"", graphStr) - - // check nodes - for _, service := range project.Services { - assert.Contains(t, graphStr, "\""+service.Name+"\" [style=\"filled\"", graphStr) - } - - // check node attributes - assert.Contains(t, graphStr, "Networks", graphStr) - assert.Contains(t, graphStr, ">internal<", graphStr) - assert.Contains(t, graphStr, ">external<", graphStr) - assert.Contains(t, graphStr, "Image", graphStr) - for _, service := range project.Services { - assert.Contains(t, graphStr, ">"+service.Image+"<", graphStr) - } - assert.Contains(t, graphStr, "Ports", graphStr) - for _, service := range project.Services { - for _, portConfig := range service.Ports { - assert.NotContains(t, graphStr, ">"+portConfig.Published+":"+strconv.Itoa(int(portConfig.Target))+"<", graphStr) - } - } - }) -} diff --git a/pkg/compose/volumes.go b/pkg/compose/volumes.go deleted file mode 100644 index 84e42b37c30..00000000000 --- a/pkg/compose/volumes.go +++ /dev/null @@ -1,83 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "slices" - - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/volume" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Volumes(ctx context.Context, project string, options api.VolumesOptions) ([]api.VolumesSummary, error) { - allContainers, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - Filters: filters.NewArgs(projectFilter(project)), - }) - if err != nil { - return nil, err - } - - var containers []container.Summary - - if len(options.Services) > 0 { - // filter service containers - for _, c := range allContainers { - if slices.Contains(options.Services, c.Labels[api.ServiceLabel]) { - containers = append(containers, c) - } - } - } else { - containers = allContainers - } - - volumesResponse, err := s.apiClient().VolumeList(ctx, volume.ListOptions{ - Filters: filters.NewArgs(projectFilter(project)), - }) - if err != nil { - return nil, err - } - - projectVolumes := volumesResponse.Volumes - - if len(options.Services) == 0 { - return projectVolumes, nil - } - - var volumes []api.VolumesSummary - - // create a name lookup of volumes used by containers - serviceVolumes := make(map[string]bool) - - for _, container := range containers { - for _, mount := range container.Mounts { - serviceVolumes[mount.Name] = true - } - } - - // append if volumes in this project are in serviceVolumes - for _, v := range projectVolumes { - if serviceVolumes[v.Name] { - volumes = append(volumes, v) - } - } - - return volumes, nil -} diff --git a/pkg/compose/volumes_test.go b/pkg/compose/volumes_test.go deleted file mode 100644 index 85a838eaeed..00000000000 --- a/pkg/compose/volumes_test.go +++ /dev/null @@ -1,85 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "testing" - - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/volume" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestVolumes(t *testing.T) { - mockCtrl := gomock.NewController(t) - defer mockCtrl.Finish() - - mockApi, mockCli := prepareMocks(mockCtrl) - tested := composeService{ - dockerCli: mockCli, - } - - // Create test volumes - vol1 := &volume.Volume{Name: testProject + "_vol1"} - vol2 := &volume.Volume{Name: testProject + "_vol2"} - vol3 := &volume.Volume{Name: testProject + "_vol3"} - - // Create test containers with volume mounts - c1 := container.Summary{ - Labels: map[string]string{api.ServiceLabel: "service1"}, - Mounts: []container.MountPoint{ - {Name: testProject + "_vol1"}, - {Name: testProject + "_vol2"}, - }, - } - c2 := container.Summary{ - Labels: map[string]string{api.ServiceLabel: "service2"}, - Mounts: []container.MountPoint{ - {Name: testProject + "_vol3"}, - }, - } - - args := filters.NewArgs(projectFilter(testProject)) - listOpts := container.ListOptions{Filters: args} - volumeListArgs := filters.NewArgs(projectFilter(testProject)) - volumeListOpts := volume.ListOptions{Filters: volumeListArgs} - volumeReturn := volume.ListResponse{ - Volumes: []*volume.Volume{vol1, vol2, vol3}, - } - containerReturn := []container.Summary{c1, c2} - - mockApi.EXPECT().ContainerList(t.Context(), listOpts).Times(2).Return(containerReturn, nil) - mockApi.EXPECT().VolumeList(t.Context(), volumeListOpts).Times(2).Return(volumeReturn, nil) - - // Test without service filter - should return all project volumes - volumeOptions := api.VolumesOptions{} - volumes, err := tested.Volumes(t.Context(), testProject, volumeOptions) - expected := []api.VolumesSummary{vol1, vol2, vol3} - assert.NilError(t, err) - assert.DeepEqual(t, volumes, expected) - - // Test with service filter - should only return volumes used by service1 - volumeOptions = api.VolumesOptions{Services: []string{"service1"}} - volumes, err = tested.Volumes(t.Context(), testProject, volumeOptions) - expected = []api.VolumesSummary{vol1, vol2} - assert.NilError(t, err) - assert.DeepEqual(t, volumes, expected) -} diff --git a/pkg/compose/wait.go b/pkg/compose/wait.go deleted file mode 100644 index 30413cb533e..00000000000 --- a/pkg/compose/wait.go +++ /dev/null @@ -1,67 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - - "golang.org/x/sync/errgroup" - - "github.com/docker/compose/v5/pkg/api" -) - -func (s *composeService) Wait(ctx context.Context, projectName string, options api.WaitOptions) (int64, error) { - containers, err := s.getContainers(ctx, projectName, oneOffInclude, false, options.Services...) - if err != nil { - return 0, err - } - if len(containers) == 0 { - return 0, fmt.Errorf("no containers for project %q", projectName) - } - - eg, waitCtx := errgroup.WithContext(ctx) - var statusCode int64 - for _, ctr := range containers { - eg.Go(func() error { - var err error - resultC, errC := s.apiClient().ContainerWait(waitCtx, ctr.ID, "") - - select { - case result := <-resultC: - _, _ = fmt.Fprintf(s.stdout(), "container %q exited with status code %d\n", ctr.ID, result.StatusCode) - statusCode = result.StatusCode - case err = <-errC: - } - - return err - }) - } - - err = eg.Wait() - if err != nil { - return 42, err // Ignore abort flag in case of error in wait - } - - if options.DownProjectOnContainerExit { - return statusCode, s.Down(ctx, projectName, api.DownOptions{ - RemoveOrphans: true, - }) - } - - return statusCode, err -} diff --git a/pkg/compose/watch.go b/pkg/compose/watch.go deleted file mode 100644 index 77575bcdca2..00000000000 --- a/pkg/compose/watch.go +++ /dev/null @@ -1,858 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "errors" - "fmt" - "io" - "io/fs" - "os" - "path" - "path/filepath" - "slices" - "strconv" - "strings" - gsync "sync" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/compose-spec/compose-go/v2/utils" - ccli "github.com/docker/cli/cli/command/container" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/go-viper/mapstructure/v2" - "github.com/moby/buildkit/util/progress/progressui" - "github.com/sirupsen/logrus" - "golang.org/x/sync/errgroup" - - pathutil "github.com/docker/compose/v5/internal/paths" - "github.com/docker/compose/v5/internal/sync" - "github.com/docker/compose/v5/internal/tracing" - "github.com/docker/compose/v5/pkg/api" - cutils "github.com/docker/compose/v5/pkg/utils" - "github.com/docker/compose/v5/pkg/watch" -) - -type WatchFunc func(ctx context.Context, project *types.Project, options api.WatchOptions) (func() error, error) - -type Watcher struct { - project *types.Project - options api.WatchOptions - watchFn WatchFunc - stopFn func() - errCh chan error -} - -func NewWatcher(project *types.Project, options api.UpOptions, w WatchFunc, consumer api.LogConsumer) (*Watcher, error) { - for i := range project.Services { - service := project.Services[i] - - if service.Develop != nil && service.Develop.Watch != nil { - build := options.Create.Build - return &Watcher{ - project: project, - options: api.WatchOptions{ - LogTo: consumer, - Build: build, - }, - watchFn: w, - errCh: make(chan error), - }, nil - } - } - // none of the services is eligible to watch - return nil, fmt.Errorf("none of the selected services is configured for watch, see https://docs.docker.com/compose/how-tos/file-watch/") -} - -// ensure state changes are atomic -var mx gsync.Mutex - -func (w *Watcher) Start(ctx context.Context) error { - mx.Lock() - defer mx.Unlock() - ctx, cancelFunc := context.WithCancel(ctx) - w.stopFn = cancelFunc - wait, err := w.watchFn(ctx, w.project, w.options) - if err != nil { - go func() { - w.errCh <- err - }() - return err - } - go func() { - w.errCh <- wait() - }() - return nil -} - -func (w *Watcher) Stop() error { - mx.Lock() - defer mx.Unlock() - if w.stopFn == nil { - return nil - } - w.stopFn() - w.stopFn = nil - err := <-w.errCh - return err -} - -// getSyncImplementation returns an appropriate sync implementation for the -// project. -// -// Currently, an implementation that batches files and transfers them using -// the Moby `Untar` API. -func (s *composeService) getSyncImplementation(project *types.Project) (sync.Syncer, error) { - var useTar bool - if useTarEnv, ok := os.LookupEnv("COMPOSE_EXPERIMENTAL_WATCH_TAR"); ok { - useTar, _ = strconv.ParseBool(useTarEnv) - } else { - useTar = true - } - if !useTar { - return nil, errors.New("no available sync implementation") - } - - return sync.NewTar(project.Name, tarDockerClient{s: s}), nil -} - -func (s *composeService) Watch(ctx context.Context, project *types.Project, options api.WatchOptions) error { - wait, err := s.watch(ctx, project, options) - if err != nil { - return err - } - return wait() -} - -type watchRule struct { - types.Trigger - include watch.PathMatcher - ignore watch.PathMatcher - service string -} - -func (r watchRule) Matches(event watch.FileEvent) *sync.PathMapping { - hostPath := string(event) - if !pathutil.IsChild(r.Path, hostPath) { - return nil - } - included, err := r.include.Matches(hostPath) - if err != nil { - logrus.Warnf("error include matching %q: %v", hostPath, err) - return nil - } - if !included { - logrus.Debugf("%s is not matching include pattern", hostPath) - return nil - } - isIgnored, err := r.ignore.Matches(hostPath) - if err != nil { - logrus.Warnf("error ignore matching %q: %v", hostPath, err) - return nil - } - - if isIgnored { - logrus.Debugf("%s is matching ignore pattern", hostPath) - return nil - } - - var containerPath string - if r.Target != "" { - rel, err := filepath.Rel(r.Path, hostPath) - if err != nil { - logrus.Warnf("error making %s relative to %s: %v", hostPath, r.Path, err) - return nil - } - // always use Unix-style paths for inside the container - containerPath = path.Join(r.Target, filepath.ToSlash(rel)) - } - return &sync.PathMapping{ - HostPath: hostPath, - ContainerPath: containerPath, - } -} - -func (s *composeService) watch(ctx context.Context, project *types.Project, options api.WatchOptions) (func() error, error) { //nolint: gocyclo - var err error - if project, err = project.WithSelectedServices(options.Services); err != nil { - return nil, err - } - syncer, err := s.getSyncImplementation(project) - if err != nil { - return nil, err - } - eg, ctx := errgroup.WithContext(ctx) - - var ( - rules []watchRule - paths []string - ) - for serviceName, service := range project.Services { - config, err := loadDevelopmentConfig(service, project) - if err != nil { - return nil, err - } - - if service.Develop != nil { - config = service.Develop - } - - if config == nil { - continue - } - - for _, trigger := range config.Watch { - if trigger.Action == types.WatchActionRebuild { - if service.Build == nil { - return nil, fmt.Errorf("can't watch service %q with action %s without a build context", service.Name, types.WatchActionRebuild) - } - if options.Build == nil { - return nil, fmt.Errorf("--no-build is incompatible with watch action %s in service %s", types.WatchActionRebuild, service.Name) - } - // set the service to always be built - watch triggers `Up()` when it receives a rebuild event - service.PullPolicy = types.PullPolicyBuild - project.Services[serviceName] = service - } - } - - for _, trigger := range config.Watch { - if isSync(trigger) && checkIfPathAlreadyBindMounted(trigger.Path, service.Volumes) { - logrus.Warnf("path '%s' also declared by a bind mount volume, this path won't be monitored!\n", trigger.Path) - continue - } else { - shouldInitialSync := trigger.InitialSync - - // Check legacy extension attribute for backward compatibility - if !shouldInitialSync { - var legacyInitialSync bool - success, err := trigger.Extensions.Get("x-initialSync", &legacyInitialSync) - if err == nil && success && legacyInitialSync { - shouldInitialSync = true - logrus.Warnf("x-initialSync is DEPRECATED, please use the official `initial_sync` attribute\n") - } - } - - if shouldInitialSync && isSync(trigger) { - // Need to check initial files are in container that are meant to be synced from watch action - err := s.initialSync(ctx, project, service, trigger, syncer) - if err != nil { - return nil, err - } - } - } - paths = append(paths, trigger.Path) - } - - serviceWatchRules, err := getWatchRules(config, service) - if err != nil { - return nil, err - } - rules = append(rules, serviceWatchRules...) - } - - if len(paths) == 0 { - return nil, fmt.Errorf("none of the selected services is configured for watch, consider setting a 'develop' section") - } - - watcher, err := watch.NewWatcher(paths) - if err != nil { - return nil, err - } - - err = watcher.Start() - if err != nil { - return nil, err - } - - eg.Go(func() error { - return s.watchEvents(ctx, project, options, watcher, syncer, rules) - }) - options.LogTo.Log(api.WatchLogger, "Watch enabled") - - return func() error { - err := eg.Wait() - if werr := watcher.Close(); werr != nil { - logrus.Debugf("Error closing Watcher: %v", werr) - } - return err - }, nil -} - -func getWatchRules(config *types.DevelopConfig, service types.ServiceConfig) ([]watchRule, error) { - var rules []watchRule - - dockerIgnores, err := watch.LoadDockerIgnore(service.Build) - if err != nil { - return nil, err - } - - // add a hardcoded set of ignores on top of what came from .dockerignore - // some of this should likely be configurable (e.g. there could be cases - // where you want `.git` to be synced) but this is suitable for now - dotGitIgnore, err := watch.NewDockerPatternMatcher("/", []string{".git/"}) - if err != nil { - return nil, err - } - - for _, trigger := range config.Watch { - ignore, err := watch.NewDockerPatternMatcher(trigger.Path, trigger.Ignore) - if err != nil { - return nil, err - } - - var include watch.PathMatcher - if len(trigger.Include) == 0 { - include = watch.AnyMatcher{} - } else { - include, err = watch.NewDockerPatternMatcher(trigger.Path, trigger.Include) - if err != nil { - return nil, err - } - } - - rules = append(rules, watchRule{ - Trigger: trigger, - include: include, - ignore: watch.NewCompositeMatcher( - dockerIgnores, - watch.EphemeralPathMatcher(), - dotGitIgnore, - ignore, - ), - service: service.Name, - }) - } - return rules, nil -} - -func isSync(trigger types.Trigger) bool { - return trigger.Action == types.WatchActionSync || trigger.Action == types.WatchActionSyncRestart -} - -func (s *composeService) watchEvents(ctx context.Context, project *types.Project, options api.WatchOptions, watcher watch.Notify, syncer sync.Syncer, rules []watchRule) error { - ctx, cancel := context.WithCancel(ctx) - defer cancel() - - // debounce and group filesystem events so that we capture IDE saving many files as one "batch" event - batchEvents := watch.BatchDebounceEvents(ctx, s.clock, watcher.Events()) - - for { - select { - case <-ctx.Done(): - options.LogTo.Log(api.WatchLogger, "Watch disabled") - // Ensure watcher is closed to release resources - _ = watcher.Close() - return nil - case err, open := <-watcher.Errors(): - if err != nil { - options.LogTo.Err(api.WatchLogger, "Watch disabled with errors: "+err.Error()) - } - if open { - continue - } - _ = watcher.Close() - return err - case batch, ok := <-batchEvents: - if !ok { - options.LogTo.Log(api.WatchLogger, "Watch disabled") - _ = watcher.Close() - return nil - } - if len(batch) > 1000 { - logrus.Warnf("Very large batch of file changes detected: %d files. This may impact performance.", len(batch)) - options.LogTo.Log(api.WatchLogger, "Large batch of file changes detected. If you just switched branches, this is expected.") - } - start := time.Now() - logrus.Debugf("batch start: count[%d]", len(batch)) - err := s.handleWatchBatch(ctx, project, options, batch, rules, syncer) - if err != nil { - logrus.Warnf("Error handling changed files: %v", err) - // If context was canceled, exit immediately - if ctx.Err() != nil { - _ = watcher.Close() - return ctx.Err() - } - } - logrus.Debugf("batch complete: duration[%s] count[%d]", time.Since(start), len(batch)) - } - } -} - -func loadDevelopmentConfig(service types.ServiceConfig, project *types.Project) (*types.DevelopConfig, error) { - var config types.DevelopConfig - y, ok := service.Extensions["x-develop"] - if !ok { - return nil, nil - } - logrus.Warnf("x-develop is DEPRECATED, please use the official `develop` attribute") - err := mapstructure.Decode(y, &config) - if err != nil { - return nil, err - } - baseDir, err := filepath.EvalSymlinks(project.WorkingDir) - if err != nil { - return nil, fmt.Errorf("resolving symlink for %q: %w", project.WorkingDir, err) - } - - for i, trigger := range config.Watch { - if !filepath.IsAbs(trigger.Path) { - trigger.Path = filepath.Join(baseDir, trigger.Path) - } - if p, err := filepath.EvalSymlinks(trigger.Path); err == nil { - // this might fail because the path doesn't exist, etc. - trigger.Path = p - } - trigger.Path = filepath.Clean(trigger.Path) - if trigger.Path == "" { - return nil, errors.New("watch rules MUST define a path") - } - - if trigger.Action == types.WatchActionRebuild && service.Build == nil { - return nil, fmt.Errorf("service %s doesn't have a build section, can't apply %s on watch", types.WatchActionRebuild, service.Name) - } - if trigger.Action == types.WatchActionSyncExec && len(trigger.Exec.Command) == 0 { - return nil, fmt.Errorf("can't watch with action %q on service %s without a command", types.WatchActionSyncExec, service.Name) - } - - config.Watch[i] = trigger - } - return &config, nil -} - -func checkIfPathAlreadyBindMounted(watchPath string, volumes []types.ServiceVolumeConfig) bool { - for _, volume := range volumes { - if volume.Bind != nil { - relPath, err := filepath.Rel(volume.Source, watchPath) - if err == nil && !strings.HasPrefix(relPath, "..") { - return true - } - } - } - return false -} - -type tarDockerClient struct { - s *composeService -} - -func (t tarDockerClient) ContainersForService(ctx context.Context, projectName string, serviceName string) ([]container.Summary, error) { - containers, err := t.s.getContainers(ctx, projectName, oneOffExclude, true, serviceName) - if err != nil { - return nil, err - } - return containers, nil -} - -func (t tarDockerClient) Exec(ctx context.Context, containerID string, cmd []string, in io.Reader) error { - execCfg := container.ExecOptions{ - Cmd: cmd, - AttachStdout: false, - AttachStderr: true, - AttachStdin: in != nil, - Tty: false, - } - execCreateResp, err := t.s.apiClient().ContainerExecCreate(ctx, containerID, execCfg) - if err != nil { - return err - } - - startCheck := container.ExecStartOptions{Tty: false, Detach: false} - conn, err := t.s.apiClient().ContainerExecAttach(ctx, execCreateResp.ID, startCheck) - if err != nil { - return err - } - defer conn.Close() - - var eg errgroup.Group - if in != nil { - eg.Go(func() error { - defer func() { - _ = conn.CloseWrite() - }() - _, err := io.Copy(conn.Conn, in) - return err - }) - } - eg.Go(func() error { - _, err := io.Copy(t.s.stdout(), conn.Reader) - return err - }) - - err = t.s.apiClient().ContainerExecStart(ctx, execCreateResp.ID, startCheck) - if err != nil { - return err - } - - // although the errgroup is not tied directly to the context, the operations - // in it are reading/writing to the connection, which is tied to the context, - // so they won't block indefinitely - if err := eg.Wait(); err != nil { - return err - } - - execResult, err := t.s.apiClient().ContainerExecInspect(ctx, execCreateResp.ID) - if err != nil { - return err - } - if execResult.Running { - return errors.New("process still running") - } - if execResult.ExitCode != 0 { - return fmt.Errorf("exit code %d", execResult.ExitCode) - } - return nil -} - -func (t tarDockerClient) Untar(ctx context.Context, id string, archive io.ReadCloser) error { - return t.s.apiClient().CopyToContainer(ctx, id, "/", archive, container.CopyToContainerOptions{ - CopyUIDGID: true, - }) -} - -//nolint:gocyclo -func (s *composeService) handleWatchBatch(ctx context.Context, project *types.Project, options api.WatchOptions, batch []watch.FileEvent, rules []watchRule, syncer sync.Syncer) error { - var ( - restart = map[string]bool{} - syncfiles = map[string][]*sync.PathMapping{} - exec = map[string][]int{} - rebuild = map[string]bool{} - ) - for _, event := range batch { - for i, rule := range rules { - mapping := rule.Matches(event) - if mapping == nil { - continue - } - - switch rule.Action { - case types.WatchActionRebuild: - rebuild[rule.service] = true - case types.WatchActionSync: - syncfiles[rule.service] = append(syncfiles[rule.service], mapping) - case types.WatchActionRestart: - restart[rule.service] = true - case types.WatchActionSyncRestart: - syncfiles[rule.service] = append(syncfiles[rule.service], mapping) - restart[rule.service] = true - case types.WatchActionSyncExec: - syncfiles[rule.service] = append(syncfiles[rule.service], mapping) - // We want to run exec hooks only once after syncfiles if multiple file events match - // as we can't compare ServiceHook to sort and compact a slice, collect rule indexes - exec[rule.service] = append(exec[rule.service], i) - } - } - } - - logrus.Debugf("watch actions: rebuild %d sync %d restart %d", len(rebuild), len(syncfiles), len(restart)) - - if len(rebuild) > 0 { - err := s.rebuild(ctx, project, utils.MapKeys(rebuild), options) - if err != nil { - return err - } - } - - for serviceName, pathMappings := range syncfiles { - writeWatchSyncMessage(options.LogTo, serviceName, pathMappings) - err := syncer.Sync(ctx, serviceName, pathMappings) - if err != nil { - return err - } - } - if len(restart) > 0 { - services := utils.MapKeys(restart) - err := s.restart(ctx, project.Name, api.RestartOptions{ - Services: services, - Project: project, - NoDeps: false, - }) - if err != nil { - return err - } - options.LogTo.Log( - api.WatchLogger, - fmt.Sprintf("service(s) %q restarted", services)) - } - - eg, ctx := errgroup.WithContext(ctx) - for service, rulesToExec := range exec { - slices.Sort(rulesToExec) - for _, i := range slices.Compact(rulesToExec) { - err := s.exec(ctx, project, service, rules[i].Exec, eg) - if err != nil { - return err - } - } - } - return eg.Wait() -} - -func (s *composeService) exec(ctx context.Context, project *types.Project, serviceName string, x types.ServiceHook, eg *errgroup.Group) error { - containers, err := s.getContainers(ctx, project.Name, oneOffExclude, false, serviceName) - if err != nil { - return err - } - for _, c := range containers { - eg.Go(func() error { - exec := ccli.NewExecOptions() - exec.User = x.User - exec.Privileged = x.Privileged - exec.Command = x.Command - exec.Workdir = x.WorkingDir - exec.DetachKeys = s.configFile().DetachKeys - for _, v := range x.Environment.ToMapping().Values() { - err := exec.Env.Set(v) - if err != nil { - return err - } - } - return ccli.RunExec(ctx, s.dockerCli, c.ID, exec) - }) - } - return nil -} - -func (s *composeService) rebuild(ctx context.Context, project *types.Project, services []string, options api.WatchOptions) error { - options.LogTo.Log(api.WatchLogger, fmt.Sprintf("Rebuilding service(s) %q after changes were detected...", services)) - // restrict the build to ONLY this service, not any of its dependencies - options.Build.Services = services - options.Build.Progress = string(progressui.PlainMode) - options.Build.Out = cutils.GetWriter(func(line string) { - options.LogTo.Log(api.WatchLogger, line) - }) - - var ( - imageNameToIdMap map[string]string - err error - ) - err = tracing.SpanWrapFunc("project/build", tracing.ProjectOptions(ctx, project), - func(ctx context.Context) error { - imageNameToIdMap, err = s.build(ctx, project, *options.Build, nil) - return err - })(ctx) - if err != nil { - options.LogTo.Log(api.WatchLogger, fmt.Sprintf("Build failed. Error: %v", err)) - return err - } - - if options.Prune { - s.pruneDanglingImagesOnRebuild(ctx, project.Name, imageNameToIdMap) - } - - options.LogTo.Log(api.WatchLogger, fmt.Sprintf("service(s) %q successfully built", services)) - - err = s.create(ctx, project, api.CreateOptions{ - Services: services, - Inherit: true, - Recreate: api.RecreateForce, - }) - if err != nil { - options.LogTo.Log(api.WatchLogger, fmt.Sprintf("Failed to recreate services after update. Error: %v", err)) - return err - } - - p, err := project.WithSelectedServices(services, types.IncludeDependents) - if err != nil { - return err - } - err = s.start(ctx, project.Name, api.StartOptions{ - Project: p, - Services: services, - AttachTo: services, - }, nil) - if err != nil { - options.LogTo.Log(api.WatchLogger, fmt.Sprintf("Application failed to start after update. Error: %v", err)) - } - return nil -} - -// writeWatchSyncMessage prints out a message about the sync for the changed paths. -func writeWatchSyncMessage(log api.LogConsumer, serviceName string, pathMappings []*sync.PathMapping) { - if logrus.IsLevelEnabled(logrus.DebugLevel) { - hostPathsToSync := make([]string, len(pathMappings)) - for i := range pathMappings { - hostPathsToSync[i] = pathMappings[i].HostPath - } - log.Log( - api.WatchLogger, - fmt.Sprintf( - "Syncing service %q after changes were detected: %s", - serviceName, - strings.Join(hostPathsToSync, ", "), - ), - ) - } else { - log.Log( - api.WatchLogger, - fmt.Sprintf("Syncing service %q after %d changes were detected", serviceName, len(pathMappings)), - ) - } -} - -func (s *composeService) pruneDanglingImagesOnRebuild(ctx context.Context, projectName string, imageNameToIdMap map[string]string) { - images, err := s.apiClient().ImageList(ctx, image.ListOptions{ - Filters: filters.NewArgs( - filters.Arg("dangling", "true"), - filters.Arg("label", api.ProjectLabel+"="+projectName), - ), - }) - if err != nil { - logrus.Debugf("Failed to list images: %v", err) - return - } - - for _, img := range images { - if _, ok := imageNameToIdMap[img.ID]; !ok { - _, err := s.apiClient().ImageRemove(ctx, img.ID, image.RemoveOptions{}) - if err != nil { - logrus.Debugf("Failed to remove image %s: %v", img.ID, err) - } - } - } -} - -// Walks develop.watch.path and checks which files should be copied inside the container -// ignores develop.watch.ignore, Dockerfile, compose files, bind mounted paths and .git -func (s *composeService) initialSync(ctx context.Context, project *types.Project, service types.ServiceConfig, trigger types.Trigger, syncer sync.Syncer) error { - dockerIgnores, err := watch.LoadDockerIgnore(service.Build) - if err != nil { - return err - } - - dotGitIgnore, err := watch.NewDockerPatternMatcher("/", []string{".git/"}) - if err != nil { - return err - } - - triggerIgnore, err := watch.NewDockerPatternMatcher(trigger.Path, trigger.Ignore) - if err != nil { - return err - } - // FIXME .dockerignore - ignoreInitialSync := watch.NewCompositeMatcher( - dockerIgnores, - watch.EphemeralPathMatcher(), - dotGitIgnore, - triggerIgnore) - - pathsToCopy, err := s.initialSyncFiles(ctx, project, service, trigger, ignoreInitialSync) - if err != nil { - return err - } - - return syncer.Sync(ctx, service.Name, pathsToCopy) -} - -// Syncs files from develop.watch.path if thy have been modified after the image has been created -// -//nolint:gocyclo -func (s *composeService) initialSyncFiles(ctx context.Context, project *types.Project, service types.ServiceConfig, trigger types.Trigger, ignore watch.PathMatcher) ([]*sync.PathMapping, error) { - fi, err := os.Stat(trigger.Path) - if err != nil { - return nil, err - } - timeImageCreated, err := s.imageCreatedTime(ctx, project, service.Name) - if err != nil { - return nil, err - } - var pathsToCopy []*sync.PathMapping - switch mode := fi.Mode(); { - case mode.IsDir(): - // process directory - err = filepath.WalkDir(trigger.Path, func(path string, d fs.DirEntry, err error) error { - if err != nil { - // handle possible path err, just in case... - return err - } - if trigger.Path == path { - // walk starts at the root directory - return nil - } - if shouldIgnore(filepath.Base(path), ignore) || checkIfPathAlreadyBindMounted(path, service.Volumes) { - // By definition sync ignores bind mounted paths - if d.IsDir() { - // skip folder - return fs.SkipDir - } - return nil // skip file - } - info, err := d.Info() - if err != nil { - return err - } - if !d.IsDir() { - if info.ModTime().Before(timeImageCreated) { - // skip file if it was modified before image creation - return nil - } - rel, err := filepath.Rel(trigger.Path, path) - if err != nil { - return err - } - // only copy files (and not full directories) - pathsToCopy = append(pathsToCopy, &sync.PathMapping{ - HostPath: path, - ContainerPath: filepath.Join(trigger.Target, rel), - }) - } - return nil - }) - case mode.IsRegular(): - // process file - if fi.ModTime().After(timeImageCreated) && !shouldIgnore(filepath.Base(trigger.Path), ignore) && !checkIfPathAlreadyBindMounted(trigger.Path, service.Volumes) { - pathsToCopy = append(pathsToCopy, &sync.PathMapping{ - HostPath: trigger.Path, - ContainerPath: trigger.Target, - }) - } - } - return pathsToCopy, err -} - -func shouldIgnore(name string, ignore watch.PathMatcher) bool { - shouldIgnore, _ := ignore.Matches(name) - // ignore files that match any ignore pattern - return shouldIgnore -} - -// gets the image creation time for a service -func (s *composeService) imageCreatedTime(ctx context.Context, project *types.Project, serviceName string) (time.Time, error) { - containers, err := s.apiClient().ContainerList(ctx, container.ListOptions{ - All: true, - Filters: filters.NewArgs( - filters.Arg("label", fmt.Sprintf("%s=%s", api.ProjectLabel, project.Name)), - filters.Arg("label", fmt.Sprintf("%s=%s", api.ServiceLabel, serviceName))), - }) - if err != nil { - return time.Now(), err - } - if len(containers) == 0 { - return time.Now(), fmt.Errorf("could not get created time for service's image") - } - - img, err := s.apiClient().ImageInspect(ctx, containers[0].ImageID) - if err != nil { - return time.Now(), err - } - // Need to get the oldest one? - timeCreated, err := time.Parse(time.RFC3339Nano, img.Created) - if err != nil { - return time.Now(), err - } - return timeCreated, nil -} diff --git a/pkg/compose/watch_test.go b/pkg/compose/watch_test.go deleted file mode 100644 index 956ca4e27e7..00000000000 --- a/pkg/compose/watch_test.go +++ /dev/null @@ -1,188 +0,0 @@ -/* - - Copyright 2020 Docker Compose CLI authors - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package compose - -import ( - "context" - "fmt" - "os" - "testing" - "time" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/streams" - "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/jonboulle/clockwork" - "github.com/stretchr/testify/require" - "go.uber.org/mock/gomock" - "gotest.tools/v3/assert" - - "github.com/docker/compose/v5/internal/sync" - "github.com/docker/compose/v5/pkg/api" - "github.com/docker/compose/v5/pkg/mocks" - "github.com/docker/compose/v5/pkg/watch" -) - -type testWatcher struct { - events chan watch.FileEvent - errors chan error -} - -func (t testWatcher) Start() error { - return nil -} - -func (t testWatcher) Close() error { - return nil -} - -func (t testWatcher) Events() chan watch.FileEvent { - return t.events -} - -func (t testWatcher) Errors() chan error { - return t.errors -} - -type stdLogger struct{} - -func (s stdLogger) Log(containerName, message string) { - fmt.Printf("%s: %s\n", containerName, message) -} - -func (s stdLogger) Err(containerName, message string) { - fmt.Fprintf(os.Stderr, "%s: %s\n", containerName, message) -} - -func (s stdLogger) Status(containerName, msg string) { - fmt.Printf("%s: %s\n", containerName, msg) -} - -func TestWatch_Sync(t *testing.T) { - mockCtrl := gomock.NewController(t) - cli := mocks.NewMockCli(mockCtrl) - cli.EXPECT().Err().Return(streams.NewOut(os.Stderr)).AnyTimes() - apiClient := mocks.NewMockAPIClient(mockCtrl) - apiClient.EXPECT().ContainerList(gomock.Any(), gomock.Any()).Return([]container.Summary{ - testContainer("test", "123", false), - }, nil).AnyTimes() - // we expect the image to be pruned - apiClient.EXPECT().ImageList(gomock.Any(), image.ListOptions{ - Filters: filters.NewArgs( - filters.Arg("dangling", "true"), - filters.Arg("label", api.ProjectLabel+"=myProjectName"), - ), - }).Return([]image.Summary{ - {ID: "123"}, - {ID: "456"}, - }, nil).Times(1) - apiClient.EXPECT().ImageRemove(gomock.Any(), "123", image.RemoveOptions{}).Times(1) - apiClient.EXPECT().ImageRemove(gomock.Any(), "456", image.RemoveOptions{}).Times(1) - // - cli.EXPECT().Client().Return(apiClient).AnyTimes() - - ctx, cancelFunc := context.WithCancel(t.Context()) - t.Cleanup(cancelFunc) - - proj := types.Project{ - Name: "myProjectName", - Services: types.Services{ - "test": { - Name: "test", - }, - }, - } - - watcher := testWatcher{ - events: make(chan watch.FileEvent), - errors: make(chan error), - } - - syncer := newFakeSyncer() - clock := clockwork.NewFakeClock() - go func() { - service := composeService{ - dockerCli: cli, - clock: clock, - } - rules, err := getWatchRules(&types.DevelopConfig{ - Watch: []types.Trigger{ - { - Path: "/sync", - Action: "sync", - Target: "/work", - Ignore: []string{"ignore"}, - }, - { - Path: "/rebuild", - Action: "rebuild", - }, - }, - }, types.ServiceConfig{Name: "test"}) - assert.NilError(t, err) - - err = service.watchEvents(ctx, &proj, api.WatchOptions{ - Build: &api.BuildOptions{}, - LogTo: stdLogger{}, - Prune: true, - }, watcher, syncer, rules) - assert.NilError(t, err) - }() - - watcher.Events() <- watch.NewFileEvent("/sync/changed") - watcher.Events() <- watch.NewFileEvent("/sync/changed/sub") - err := clock.BlockUntilContext(ctx, 3) - assert.NilError(t, err) - clock.Advance(watch.QuietPeriod) - select { - case actual := <-syncer.synced: - require.ElementsMatch(t, []*sync.PathMapping{ - {HostPath: "/sync/changed", ContainerPath: "/work/changed"}, - {HostPath: "/sync/changed/sub", ContainerPath: "/work/changed/sub"}, - }, actual) - case <-time.After(100 * time.Millisecond): - t.Error("timeout") - } - - watcher.Events() <- watch.NewFileEvent("/rebuild") - watcher.Events() <- watch.NewFileEvent("/sync/changed") - err = clock.BlockUntilContext(ctx, 4) - assert.NilError(t, err) - clock.Advance(watch.QuietPeriod) - select { - case batch := <-syncer.synced: - t.Fatalf("received unexpected events: %v", batch) - case <-time.After(100 * time.Millisecond): - // expected - } - // TODO: there's not a great way to assert that the rebuild attempt happened -} - -type fakeSyncer struct { - synced chan []*sync.PathMapping -} - -func newFakeSyncer() *fakeSyncer { - return &fakeSyncer{ - synced: make(chan []*sync.PathMapping), - } -} - -func (f *fakeSyncer) Sync(ctx context.Context, service string, paths []*sync.PathMapping) error { - f.synced <- paths - return nil -} diff --git a/pkg/dryrun/dryrunclient.go b/pkg/dryrun/dryrunclient.go deleted file mode 100644 index 6755ae96b6a..00000000000 --- a/pkg/dryrun/dryrunclient.go +++ /dev/null @@ -1,692 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package dryrun - -import ( - "bytes" - "context" - "crypto/rand" - "encoding/json" - "errors" - "fmt" - "io" - "net" - "net/http" - "runtime" - "strings" - "sync" - - "github.com/docker/buildx/builder" - "github.com/docker/buildx/util/imagetools" - "github.com/docker/cli/cli/command" - moby "github.com/docker/docker/api/types" - "github.com/docker/docker/api/types/build" - "github.com/docker/docker/api/types/checkpoint" - containerType "github.com/docker/docker/api/types/container" - "github.com/docker/docker/api/types/events" - "github.com/docker/docker/api/types/filters" - "github.com/docker/docker/api/types/image" - "github.com/docker/docker/api/types/network" - "github.com/docker/docker/api/types/registry" - "github.com/docker/docker/api/types/swarm" - "github.com/docker/docker/api/types/system" - "github.com/docker/docker/api/types/volume" - "github.com/docker/docker/client" - "github.com/docker/docker/pkg/jsonmessage" - specs "github.com/opencontainers/image-spec/specs-go/v1" -) - -var _ client.APIClient = &DryRunClient{} - -// DryRunClient implements APIClient by delegating to implementation functions. This allows lazy init and per-method overrides -type DryRunClient struct { - apiClient client.APIClient - containers []containerType.Summary - execs sync.Map - resolver *imagetools.Resolver -} - -type execDetails struct { - container string - command []string -} - -// NewDryRunClient produces a DryRunClient -func NewDryRunClient(apiClient client.APIClient, cli command.Cli) (*DryRunClient, error) { - b, err := builder.New(cli, builder.WithSkippedValidation()) - if err != nil { - return nil, err - } - configFile, err := b.ImageOpt() - if err != nil { - return nil, err - } - return &DryRunClient{ - apiClient: apiClient, - containers: []containerType.Summary{}, - execs: sync.Map{}, - resolver: imagetools.New(configFile), - }, nil -} - -func getCallingFunction() string { - pc, _, _, _ := runtime.Caller(2) - fullName := runtime.FuncForPC(pc).Name() - return fullName[strings.LastIndex(fullName, ".")+1:] -} - -// All methods and functions which need to be overridden for dry run. - -func (d *DryRunClient) ContainerAttach(ctx context.Context, container string, options containerType.AttachOptions) (moby.HijackedResponse, error) { - return moby.HijackedResponse{}, errors.New("interactive run is not supported in dry-run mode") -} - -func (d *DryRunClient) ContainerCreate(ctx context.Context, config *containerType.Config, hostConfig *containerType.HostConfig, - networkingConfig *network.NetworkingConfig, platform *specs.Platform, containerName string, -) (containerType.CreateResponse, error) { - d.containers = append(d.containers, containerType.Summary{ - ID: containerName, - Names: []string{containerName}, - Labels: config.Labels, - HostConfig: struct { - NetworkMode string `json:",omitempty"` - Annotations map[string]string `json:",omitempty"` - }{}, - }) - return containerType.CreateResponse{ID: containerName}, nil -} - -func (d *DryRunClient) ContainerInspect(ctx context.Context, container string) (containerType.InspectResponse, error) { - containerJSON, err := d.apiClient.ContainerInspect(ctx, container) - if err != nil { - id := "dryRunId" - for _, c := range d.containers { - if c.ID == container { - id = container - } - } - return containerType.InspectResponse{ - ContainerJSONBase: &containerType.ContainerJSONBase{ - ID: id, - Name: container, - State: &containerType.State{ - Status: containerType.StateRunning, // needed for --wait option - Health: &containerType.Health{ - Status: containerType.Healthy, // needed for healthcheck control - }, - }, - }, - Mounts: nil, - Config: &containerType.Config{}, - NetworkSettings: &containerType.NetworkSettings{}, - }, nil - } - return containerJSON, err -} - -func (d *DryRunClient) ContainerKill(ctx context.Context, container, signal string) error { - return nil -} - -func (d *DryRunClient) ContainerList(ctx context.Context, options containerType.ListOptions) ([]containerType.Summary, error) { - caller := getCallingFunction() - switch caller { - case "start": - return d.containers, nil - case "getContainers": - if len(d.containers) == 0 { - var err error - d.containers, err = d.apiClient.ContainerList(ctx, options) - return d.containers, err - } - } - return d.apiClient.ContainerList(ctx, options) -} - -func (d *DryRunClient) ContainerPause(ctx context.Context, container string) error { - return nil -} - -func (d *DryRunClient) ContainerRemove(ctx context.Context, container string, options containerType.RemoveOptions) error { - return nil -} - -func (d *DryRunClient) ContainerRename(ctx context.Context, container, newContainerName string) error { - return nil -} - -func (d *DryRunClient) ContainerRestart(ctx context.Context, container string, options containerType.StopOptions) error { - return nil -} - -func (d *DryRunClient) ContainerStart(ctx context.Context, container string, options containerType.StartOptions) error { - return nil -} - -func (d *DryRunClient) ContainerStop(ctx context.Context, container string, options containerType.StopOptions) error { - return nil -} - -func (d *DryRunClient) ContainerUnpause(ctx context.Context, container string) error { - return nil -} - -func (d *DryRunClient) CopyFromContainer(ctx context.Context, container, srcPath string) (io.ReadCloser, containerType.PathStat, error) { - rc := io.NopCloser(strings.NewReader("")) - if _, err := d.ContainerStatPath(ctx, container, srcPath); err != nil { - return rc, containerType.PathStat{}, fmt.Errorf("could not find the file %s in container %s", srcPath, container) - } - return rc, containerType.PathStat{}, nil -} - -func (d *DryRunClient) CopyToContainer(ctx context.Context, container, path string, content io.Reader, options containerType.CopyToContainerOptions) error { - return nil -} - -func (d *DryRunClient) ImageBuild(ctx context.Context, reader io.Reader, options build.ImageBuildOptions) (build.ImageBuildResponse, error) { - rc := io.NopCloser(bytes.NewReader(nil)) - - return build.ImageBuildResponse{ - Body: rc, - }, nil -} - -func (d *DryRunClient) ImageInspect(ctx context.Context, imageName string, options ...client.ImageInspectOption) (image.InspectResponse, error) { - caller := getCallingFunction() - switch caller { - case "pullServiceImage", "buildContainerVolumes": - return image.InspectResponse{ID: "dryRunId"}, nil - default: - return d.apiClient.ImageInspect(ctx, imageName, options...) - } -} - -// Deprecated: Use [DryRunClient.ImageInspect] instead; raw response can be obtained by [client.ImageInspectWithRawResponse] option. -func (d *DryRunClient) ImageInspectWithRaw(ctx context.Context, imageName string) (image.InspectResponse, []byte, error) { - var buf bytes.Buffer - resp, err := d.ImageInspect(ctx, imageName, client.ImageInspectWithRawResponse(&buf)) - if err != nil { - return image.InspectResponse{}, nil, err - } - return resp, buf.Bytes(), err -} - -func (d *DryRunClient) ImagePull(ctx context.Context, ref string, options image.PullOptions) (io.ReadCloser, error) { - if _, _, err := d.resolver.Resolve(ctx, ref); err != nil { - return nil, err - } - rc := io.NopCloser(strings.NewReader("")) - return rc, nil -} - -func (d *DryRunClient) ImagePush(ctx context.Context, ref string, options image.PushOptions) (io.ReadCloser, error) { - if _, _, err := d.resolver.Resolve(ctx, ref); err != nil { - return nil, err - } - jsonMessage, err := json.Marshal(&jsonmessage.JSONMessage{ - Status: "Pushed", - Progress: &jsonmessage.JSONProgress{ - Current: 100, - Total: 100, - Start: 0, - HideCounts: false, - Units: "Mb", - }, - ID: ref, - }) - if err != nil { - return nil, err - } - rc := io.NopCloser(bytes.NewReader(jsonMessage)) - return rc, nil -} - -func (d *DryRunClient) ImageRemove(ctx context.Context, imageName string, options image.RemoveOptions) ([]image.DeleteResponse, error) { - return nil, nil -} - -func (d *DryRunClient) NetworkConnect(ctx context.Context, networkName, container string, config *network.EndpointSettings) error { - return nil -} - -func (d *DryRunClient) NetworkCreate(ctx context.Context, name string, options network.CreateOptions) (network.CreateResponse, error) { - return network.CreateResponse{ - ID: name, - Warning: "", - }, nil -} - -func (d *DryRunClient) NetworkDisconnect(ctx context.Context, networkName, container string, force bool) error { - return nil -} - -func (d *DryRunClient) NetworkRemove(ctx context.Context, networkName string) error { - return nil -} - -func (d *DryRunClient) VolumeCreate(ctx context.Context, options volume.CreateOptions) (volume.Volume, error) { - return volume.Volume{ - ClusterVolume: nil, - Driver: options.Driver, - Labels: options.Labels, - Name: options.Name, - Options: options.DriverOpts, - }, nil -} - -func (d *DryRunClient) VolumeRemove(ctx context.Context, volumeID string, force bool) error { - return nil -} - -func (d *DryRunClient) ContainerExecCreate(ctx context.Context, container string, config containerType.ExecOptions) (containerType.ExecCreateResponse, error) { - b := make([]byte, 32) - _, _ = rand.Read(b) - id := fmt.Sprintf("%x", b) - d.execs.Store(id, execDetails{ - container: container, - command: config.Cmd, - }) - return containerType.ExecCreateResponse{ - ID: id, - }, nil -} - -func (d *DryRunClient) ContainerExecStart(ctx context.Context, execID string, config containerType.ExecStartOptions) error { - _, ok := d.execs.LoadAndDelete(execID) - if !ok { - return fmt.Errorf("invalid exec ID %q", execID) - } - return nil -} - -// Functions delegated to original APIClient (not used by Compose or not modifying the Compose stack) - -func (d *DryRunClient) ConfigList(ctx context.Context, options swarm.ConfigListOptions) ([]swarm.Config, error) { - return d.apiClient.ConfigList(ctx, options) -} - -func (d *DryRunClient) ConfigCreate(ctx context.Context, config swarm.ConfigSpec) (swarm.ConfigCreateResponse, error) { - return d.apiClient.ConfigCreate(ctx, config) -} - -func (d *DryRunClient) ConfigRemove(ctx context.Context, id string) error { - return d.apiClient.ConfigRemove(ctx, id) -} - -func (d *DryRunClient) ConfigInspectWithRaw(ctx context.Context, name string) (swarm.Config, []byte, error) { - return d.apiClient.ConfigInspectWithRaw(ctx, name) -} - -func (d *DryRunClient) ConfigUpdate(ctx context.Context, id string, version swarm.Version, config swarm.ConfigSpec) error { - return d.apiClient.ConfigUpdate(ctx, id, version, config) -} - -func (d *DryRunClient) ContainerCommit(ctx context.Context, container string, options containerType.CommitOptions) (containerType.CommitResponse, error) { - return d.apiClient.ContainerCommit(ctx, container, options) -} - -func (d *DryRunClient) ContainerDiff(ctx context.Context, container string) ([]containerType.FilesystemChange, error) { - return d.apiClient.ContainerDiff(ctx, container) -} - -func (d *DryRunClient) ContainerExecAttach(ctx context.Context, execID string, config containerType.ExecStartOptions) (moby.HijackedResponse, error) { - return moby.HijackedResponse{}, errors.New("interactive exec is not supported in dry-run mode") -} - -func (d *DryRunClient) ContainerExecInspect(ctx context.Context, execID string) (containerType.ExecInspect, error) { - return d.apiClient.ContainerExecInspect(ctx, execID) -} - -func (d *DryRunClient) ContainerExecResize(ctx context.Context, execID string, options containerType.ResizeOptions) error { - return d.apiClient.ContainerExecResize(ctx, execID, options) -} - -func (d *DryRunClient) ContainerExport(ctx context.Context, container string) (io.ReadCloser, error) { - return d.apiClient.ContainerExport(ctx, container) -} - -func (d *DryRunClient) ContainerInspectWithRaw(ctx context.Context, container string, getSize bool) (containerType.InspectResponse, []byte, error) { - return d.apiClient.ContainerInspectWithRaw(ctx, container, getSize) -} - -func (d *DryRunClient) ContainerLogs(ctx context.Context, container string, options containerType.LogsOptions) (io.ReadCloser, error) { - return d.apiClient.ContainerLogs(ctx, container, options) -} - -func (d *DryRunClient) ContainerResize(ctx context.Context, container string, options containerType.ResizeOptions) error { - return d.apiClient.ContainerResize(ctx, container, options) -} - -func (d *DryRunClient) ContainerStatPath(ctx context.Context, container, path string) (containerType.PathStat, error) { - return d.apiClient.ContainerStatPath(ctx, container, path) -} - -func (d *DryRunClient) ContainerStats(ctx context.Context, container string, stream bool) (containerType.StatsResponseReader, error) { - return d.apiClient.ContainerStats(ctx, container, stream) -} - -func (d *DryRunClient) ContainerStatsOneShot(ctx context.Context, container string) (containerType.StatsResponseReader, error) { - return d.apiClient.ContainerStatsOneShot(ctx, container) -} - -func (d *DryRunClient) ContainerTop(ctx context.Context, container string, arguments []string) (containerType.TopResponse, error) { - return d.apiClient.ContainerTop(ctx, container, arguments) -} - -func (d *DryRunClient) ContainerUpdate(ctx context.Context, container string, updateConfig containerType.UpdateConfig) (containerType.UpdateResponse, error) { - return d.apiClient.ContainerUpdate(ctx, container, updateConfig) -} - -func (d *DryRunClient) ContainerWait(ctx context.Context, container string, condition containerType.WaitCondition) (<-chan containerType.WaitResponse, <-chan error) { - return d.apiClient.ContainerWait(ctx, container, condition) -} - -func (d *DryRunClient) ContainersPrune(ctx context.Context, pruneFilters filters.Args) (containerType.PruneReport, error) { - return d.apiClient.ContainersPrune(ctx, pruneFilters) -} - -func (d *DryRunClient) DistributionInspect(ctx context.Context, imageName, encodedRegistryAuth string) (registry.DistributionInspect, error) { - return d.apiClient.DistributionInspect(ctx, imageName, encodedRegistryAuth) -} - -func (d *DryRunClient) BuildCachePrune(ctx context.Context, opts build.CachePruneOptions) (*build.CachePruneReport, error) { - return d.apiClient.BuildCachePrune(ctx, opts) -} - -func (d *DryRunClient) BuildCancel(ctx context.Context, id string) error { - return d.apiClient.BuildCancel(ctx, id) -} - -func (d *DryRunClient) ImageCreate(ctx context.Context, parentReference string, options image.CreateOptions) (io.ReadCloser, error) { - return d.apiClient.ImageCreate(ctx, parentReference, options) -} - -func (d *DryRunClient) ImageHistory(ctx context.Context, imageName string, options ...client.ImageHistoryOption) ([]image.HistoryResponseItem, error) { - return d.apiClient.ImageHistory(ctx, imageName, options...) -} - -func (d *DryRunClient) ImageImport(ctx context.Context, source image.ImportSource, ref string, options image.ImportOptions) (io.ReadCloser, error) { - return d.apiClient.ImageImport(ctx, source, ref, options) -} - -func (d *DryRunClient) ImageList(ctx context.Context, options image.ListOptions) ([]image.Summary, error) { - return d.apiClient.ImageList(ctx, options) -} - -func (d *DryRunClient) ImageLoad(ctx context.Context, input io.Reader, options ...client.ImageLoadOption) (image.LoadResponse, error) { - return d.apiClient.ImageLoad(ctx, input, options...) -} - -func (d *DryRunClient) ImageSearch(ctx context.Context, term string, options registry.SearchOptions) ([]registry.SearchResult, error) { - return d.apiClient.ImageSearch(ctx, term, options) -} - -func (d *DryRunClient) ImageSave(ctx context.Context, images []string, options ...client.ImageSaveOption) (io.ReadCloser, error) { - return d.apiClient.ImageSave(ctx, images, options...) -} - -func (d *DryRunClient) ImageTag(ctx context.Context, imageName, ref string) error { - return d.apiClient.ImageTag(ctx, imageName, ref) -} - -func (d *DryRunClient) ImagesPrune(ctx context.Context, pruneFilter filters.Args) (image.PruneReport, error) { - return d.apiClient.ImagesPrune(ctx, pruneFilter) -} - -func (d *DryRunClient) NodeInspectWithRaw(ctx context.Context, nodeID string) (swarm.Node, []byte, error) { - return d.apiClient.NodeInspectWithRaw(ctx, nodeID) -} - -func (d *DryRunClient) NodeList(ctx context.Context, options swarm.NodeListOptions) ([]swarm.Node, error) { - return d.apiClient.NodeList(ctx, options) -} - -func (d *DryRunClient) NodeRemove(ctx context.Context, nodeID string, options swarm.NodeRemoveOptions) error { - return d.apiClient.NodeRemove(ctx, nodeID, options) -} - -func (d *DryRunClient) NodeUpdate(ctx context.Context, nodeID string, version swarm.Version, node swarm.NodeSpec) error { - return d.apiClient.NodeUpdate(ctx, nodeID, version, node) -} - -func (d *DryRunClient) NetworkInspect(ctx context.Context, networkName string, options network.InspectOptions) (network.Inspect, error) { - return d.apiClient.NetworkInspect(ctx, networkName, options) -} - -func (d *DryRunClient) NetworkInspectWithRaw(ctx context.Context, networkName string, options network.InspectOptions) (network.Inspect, []byte, error) { - return d.apiClient.NetworkInspectWithRaw(ctx, networkName, options) -} - -func (d *DryRunClient) NetworkList(ctx context.Context, options network.ListOptions) ([]network.Inspect, error) { - return d.apiClient.NetworkList(ctx, options) -} - -func (d *DryRunClient) NetworksPrune(ctx context.Context, pruneFilter filters.Args) (network.PruneReport, error) { - return d.apiClient.NetworksPrune(ctx, pruneFilter) -} - -func (d *DryRunClient) PluginList(ctx context.Context, filter filters.Args) (moby.PluginsListResponse, error) { - return d.apiClient.PluginList(ctx, filter) -} - -func (d *DryRunClient) PluginRemove(ctx context.Context, name string, options moby.PluginRemoveOptions) error { - return d.apiClient.PluginRemove(ctx, name, options) -} - -func (d *DryRunClient) PluginEnable(ctx context.Context, name string, options moby.PluginEnableOptions) error { - return d.apiClient.PluginEnable(ctx, name, options) -} - -func (d *DryRunClient) PluginDisable(ctx context.Context, name string, options moby.PluginDisableOptions) error { - return d.apiClient.PluginDisable(ctx, name, options) -} - -func (d *DryRunClient) PluginInstall(ctx context.Context, name string, options moby.PluginInstallOptions) (io.ReadCloser, error) { - return d.apiClient.PluginInstall(ctx, name, options) -} - -func (d *DryRunClient) PluginUpgrade(ctx context.Context, name string, options moby.PluginInstallOptions) (io.ReadCloser, error) { - return d.apiClient.PluginUpgrade(ctx, name, options) -} - -func (d *DryRunClient) PluginPush(ctx context.Context, name string, registryAuth string) (io.ReadCloser, error) { - return d.apiClient.PluginPush(ctx, name, registryAuth) -} - -func (d *DryRunClient) PluginSet(ctx context.Context, name string, args []string) error { - return d.apiClient.PluginSet(ctx, name, args) -} - -func (d *DryRunClient) PluginInspectWithRaw(ctx context.Context, name string) (*moby.Plugin, []byte, error) { - return d.apiClient.PluginInspectWithRaw(ctx, name) -} - -func (d *DryRunClient) PluginCreate(ctx context.Context, createContext io.Reader, options moby.PluginCreateOptions) error { - return d.apiClient.PluginCreate(ctx, createContext, options) -} - -func (d *DryRunClient) ServiceCreate(ctx context.Context, service swarm.ServiceSpec, options swarm.ServiceCreateOptions) (swarm.ServiceCreateResponse, error) { - return d.apiClient.ServiceCreate(ctx, service, options) -} - -func (d *DryRunClient) ServiceInspectWithRaw(ctx context.Context, serviceID string, options swarm.ServiceInspectOptions) (swarm.Service, []byte, error) { - return d.apiClient.ServiceInspectWithRaw(ctx, serviceID, options) -} - -func (d *DryRunClient) ServiceList(ctx context.Context, options swarm.ServiceListOptions) ([]swarm.Service, error) { - return d.apiClient.ServiceList(ctx, options) -} - -func (d *DryRunClient) ServiceRemove(ctx context.Context, serviceID string) error { - return d.apiClient.ServiceRemove(ctx, serviceID) -} - -func (d *DryRunClient) ServiceUpdate(ctx context.Context, serviceID string, version swarm.Version, service swarm.ServiceSpec, options swarm.ServiceUpdateOptions) (swarm.ServiceUpdateResponse, error) { - return d.apiClient.ServiceUpdate(ctx, serviceID, version, service, options) -} - -func (d *DryRunClient) ServiceLogs(ctx context.Context, serviceID string, options containerType.LogsOptions) (io.ReadCloser, error) { - return d.apiClient.ServiceLogs(ctx, serviceID, options) -} - -func (d *DryRunClient) TaskLogs(ctx context.Context, taskID string, options containerType.LogsOptions) (io.ReadCloser, error) { - return d.apiClient.TaskLogs(ctx, taskID, options) -} - -func (d *DryRunClient) TaskInspectWithRaw(ctx context.Context, taskID string) (swarm.Task, []byte, error) { - return d.apiClient.TaskInspectWithRaw(ctx, taskID) -} - -func (d *DryRunClient) TaskList(ctx context.Context, options swarm.TaskListOptions) ([]swarm.Task, error) { - return d.apiClient.TaskList(ctx, options) -} - -func (d *DryRunClient) SwarmInit(ctx context.Context, req swarm.InitRequest) (string, error) { - return d.apiClient.SwarmInit(ctx, req) -} - -func (d *DryRunClient) SwarmJoin(ctx context.Context, req swarm.JoinRequest) error { - return d.apiClient.SwarmJoin(ctx, req) -} - -func (d *DryRunClient) SwarmGetUnlockKey(ctx context.Context) (swarm.UnlockKeyResponse, error) { - return d.apiClient.SwarmGetUnlockKey(ctx) -} - -func (d *DryRunClient) SwarmUnlock(ctx context.Context, req swarm.UnlockRequest) error { - return d.apiClient.SwarmUnlock(ctx, req) -} - -func (d *DryRunClient) SwarmLeave(ctx context.Context, force bool) error { - return d.apiClient.SwarmLeave(ctx, force) -} - -func (d *DryRunClient) SwarmInspect(ctx context.Context) (swarm.Swarm, error) { - return d.apiClient.SwarmInspect(ctx) -} - -func (d *DryRunClient) SwarmUpdate(ctx context.Context, version swarm.Version, swarmSpec swarm.Spec, flags swarm.UpdateFlags) error { - return d.apiClient.SwarmUpdate(ctx, version, swarmSpec, flags) -} - -func (d *DryRunClient) SecretList(ctx context.Context, options swarm.SecretListOptions) ([]swarm.Secret, error) { - return d.apiClient.SecretList(ctx, options) -} - -func (d *DryRunClient) SecretCreate(ctx context.Context, secret swarm.SecretSpec) (swarm.SecretCreateResponse, error) { - return d.apiClient.SecretCreate(ctx, secret) -} - -func (d *DryRunClient) SecretRemove(ctx context.Context, id string) error { - return d.apiClient.SecretRemove(ctx, id) -} - -func (d *DryRunClient) SecretInspectWithRaw(ctx context.Context, name string) (swarm.Secret, []byte, error) { - return d.apiClient.SecretInspectWithRaw(ctx, name) -} - -func (d *DryRunClient) SecretUpdate(ctx context.Context, id string, version swarm.Version, secret swarm.SecretSpec) error { - return d.apiClient.SecretUpdate(ctx, id, version, secret) -} - -func (d *DryRunClient) Events(ctx context.Context, options events.ListOptions) (<-chan events.Message, <-chan error) { - return d.apiClient.Events(ctx, options) -} - -func (d *DryRunClient) Info(ctx context.Context) (system.Info, error) { - return d.apiClient.Info(ctx) -} - -func (d *DryRunClient) RegistryLogin(ctx context.Context, auth registry.AuthConfig) (registry.AuthenticateOKBody, error) { - return d.apiClient.RegistryLogin(ctx, auth) -} - -func (d *DryRunClient) DiskUsage(ctx context.Context, options moby.DiskUsageOptions) (moby.DiskUsage, error) { - return d.apiClient.DiskUsage(ctx, options) -} - -func (d *DryRunClient) Ping(ctx context.Context) (moby.Ping, error) { - return d.apiClient.Ping(ctx) -} - -func (d *DryRunClient) VolumeInspect(ctx context.Context, volumeID string) (volume.Volume, error) { - return d.apiClient.VolumeInspect(ctx, volumeID) -} - -func (d *DryRunClient) VolumeInspectWithRaw(ctx context.Context, volumeID string) (volume.Volume, []byte, error) { - return d.apiClient.VolumeInspectWithRaw(ctx, volumeID) -} - -func (d *DryRunClient) VolumeList(ctx context.Context, opts volume.ListOptions) (volume.ListResponse, error) { - return d.apiClient.VolumeList(ctx, opts) -} - -func (d *DryRunClient) VolumesPrune(ctx context.Context, pruneFilter filters.Args) (volume.PruneReport, error) { - return d.apiClient.VolumesPrune(ctx, pruneFilter) -} - -func (d *DryRunClient) VolumeUpdate(ctx context.Context, volumeID string, version swarm.Version, options volume.UpdateOptions) error { - return d.apiClient.VolumeUpdate(ctx, volumeID, version, options) -} - -func (d *DryRunClient) ClientVersion() string { - return d.apiClient.ClientVersion() -} - -func (d *DryRunClient) DaemonHost() string { - return d.apiClient.DaemonHost() -} - -func (d *DryRunClient) HTTPClient() *http.Client { - return d.apiClient.HTTPClient() -} - -func (d *DryRunClient) ServerVersion(ctx context.Context) (moby.Version, error) { - return d.apiClient.ServerVersion(ctx) -} - -func (d *DryRunClient) NegotiateAPIVersion(ctx context.Context) { - d.apiClient.NegotiateAPIVersion(ctx) -} - -func (d *DryRunClient) NegotiateAPIVersionPing(ping moby.Ping) { - d.apiClient.NegotiateAPIVersionPing(ping) -} - -func (d *DryRunClient) DialHijack(ctx context.Context, url, proto string, meta map[string][]string) (net.Conn, error) { - return d.apiClient.DialHijack(ctx, url, proto, meta) -} - -func (d *DryRunClient) Dialer() func(context.Context) (net.Conn, error) { - return d.apiClient.Dialer() -} - -func (d *DryRunClient) Close() error { - return d.apiClient.Close() -} - -func (d *DryRunClient) CheckpointCreate(ctx context.Context, container string, options checkpoint.CreateOptions) error { - return d.apiClient.CheckpointCreate(ctx, container, options) -} - -func (d *DryRunClient) CheckpointDelete(ctx context.Context, container string, options checkpoint.DeleteOptions) error { - return d.apiClient.CheckpointDelete(ctx, container, options) -} - -func (d *DryRunClient) CheckpointList(ctx context.Context, container string, options checkpoint.ListOptions) ([]checkpoint.Summary, error) { - return d.apiClient.CheckpointList(ctx, container, options) -} diff --git a/pkg/e2e/assert.go b/pkg/e2e/assert.go deleted file mode 100644 index 25547203dbe..00000000000 --- a/pkg/e2e/assert.go +++ /dev/null @@ -1,45 +0,0 @@ -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "encoding/json" - "strings" - "testing" - - "github.com/stretchr/testify/require" -) - -// RequireServiceState ensures that the container is in the expected state -// (running or exited). -func RequireServiceState(t testing.TB, cli *CLI, service string, state string) { - t.Helper() - psRes := cli.RunDockerComposeCmd(t, "ps", "--all", "--format=json", service) - var svc map[string]any - require.NoError(t, json.Unmarshal([]byte(psRes.Stdout()), &svc), - "Invalid `compose ps` JSON: command output: %s", - psRes.Combined()) - - require.Equal(t, service, svc["Service"], - "Found ps output for unexpected service") - require.Equalf(t, - strings.ToLower(state), - strings.ToLower(svc["State"].(string)), - "Service %q (%s) not in expected state", - service, svc["Name"], - ) -} diff --git a/pkg/e2e/bridge_test.go b/pkg/e2e/bridge_test.go deleted file mode 100644 index c4c99b8d292..00000000000 --- a/pkg/e2e/bridge_test.go +++ /dev/null @@ -1,61 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "path/filepath" - "strings" - "testing" - - "gotest.tools/v3/assert" -) - -func TestConvertAndTransformList(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "bridge" - const bridgeImageVersion = "v0.0.3" - tmpDir := t.TempDir() - - t.Run("kubernetes manifests", func(t *testing.T) { - kubedir := filepath.Join(tmpDir, "kubernetes") - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/bridge/compose.yaml", "--project-name", projectName, "bridge", "convert", - "--output", kubedir, "--transformation", fmt.Sprintf("docker/compose-bridge-kubernetes:%s", bridgeImageVersion)) - assert.NilError(t, res.Error) - assert.Equal(t, res.ExitCode, 0) - res = c.RunCmd(t, "diff", "-r", kubedir, "./fixtures/bridge/expected-kubernetes") - assert.NilError(t, res.Error, res.Combined()) - }) - - t.Run("helm charts", func(t *testing.T) { - helmDir := filepath.Join(tmpDir, "helm") - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/bridge/compose.yaml", "--project-name", projectName, "bridge", "convert", - "--output", helmDir, "--transformation", fmt.Sprintf("docker/compose-bridge-helm:%s", bridgeImageVersion)) - assert.NilError(t, res.Error) - assert.Equal(t, res.ExitCode, 0) - res = c.RunCmd(t, "diff", "-r", helmDir, "./fixtures/bridge/expected-helm") - assert.NilError(t, res.Error, res.Combined()) - }) - - t.Run("list transformers images", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "bridge", "transformations", - "ls") - assert.Assert(t, strings.Contains(res.Stdout(), "docker/compose-bridge-helm"), res.Combined()) - assert.Assert(t, strings.Contains(res.Stdout(), "docker/compose-bridge-kubernetes"), res.Combined()) - }) -} diff --git a/pkg/e2e/build_test.go b/pkg/e2e/build_test.go deleted file mode 100644 index 1fc3ac87667..00000000000 --- a/pkg/e2e/build_test.go +++ /dev/null @@ -1,641 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "net/http" - "os" - "regexp" - "runtime" - "strconv" - "strings" - "testing" - "time" - - "github.com/stretchr/testify/require" - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" - "gotest.tools/v3/poll" -) - -func TestLocalComposeBuild(t *testing.T) { - for _, env := range []string{"DOCKER_BUILDKIT=0", "DOCKER_BUILDKIT=1"} { - c := NewCLI(t, WithEnv(strings.Split(env, ",")...)) - - t.Run(env+" build named and unnamed images", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "-f", "build-test-nginx") - c.RunDockerOrExitError(t, "rmi", "-f", "custom-nginx") - - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "build") - - res.Assert(t, icmd.Expected{Out: "COPY static /usr/share/nginx/html"}) - c.RunDockerCmd(t, "image", "inspect", "build-test-nginx") - c.RunDockerCmd(t, "image", "inspect", "custom-nginx") - }) - - t.Run(env+" build with build-arg", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "-f", "build-test-nginx") - c.RunDockerOrExitError(t, "rmi", "-f", "custom-nginx") - - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "build", "--build-arg", "FOO=BAR") - - res := c.RunDockerCmd(t, "image", "inspect", "build-test-nginx") - res.Assert(t, icmd.Expected{Out: `"FOO": "BAR"`}) - }) - - t.Run(env+" build with build-arg set by env", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "-f", "build-test-nginx") - c.RunDockerOrExitError(t, "rmi", "-f", "custom-nginx") - - icmd.RunCmd(c.NewDockerComposeCmd(t, - "--project-directory", - "fixtures/build-test", - "build", - "--build-arg", - "FOO"), - func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "FOO=BAR") - }).Assert(t, icmd.Success) - - res := c.RunDockerCmd(t, "image", "inspect", "build-test-nginx") - res.Assert(t, icmd.Expected{Out: `"FOO": "BAR"`}) - }) - - t.Run(env+" build with multiple build-args ", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "-f", "multi-args-multiargs") - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/build-test/multi-args", "build") - - icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "DOCKER_BUILDKIT=0") - }) - - res := c.RunDockerCmd(t, "image", "inspect", "multi-args-multiargs") - res.Assert(t, icmd.Expected{Out: `"RESULT": "SUCCESS"`}) - }) - - t.Run(env+" build as part of up", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "-f", "build-test-nginx") - c.RunDockerOrExitError(t, "rmi", "-f", "custom-nginx") - - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "up", "-d") - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "down") - }) - - res.Assert(t, icmd.Expected{Out: "COPY static /usr/share/nginx/html"}) - res.Assert(t, icmd.Expected{Out: "COPY static2 /usr/share/nginx/html"}) - - output := HTTPGetWithRetry(t, "http://localhost:8070", http.StatusOK, 2*time.Second, 20*time.Second) - assert.Assert(t, strings.Contains(output, "Hello from Nginx container")) - - c.RunDockerCmd(t, "image", "inspect", "build-test-nginx") - c.RunDockerCmd(t, "image", "inspect", "custom-nginx") - }) - - t.Run(env+" no rebuild when up again", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "up", "-d") - - assert.Assert(t, !strings.Contains(res.Stdout(), "COPY static")) - }) - - t.Run(env+" rebuild when up --build", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "up", "-d", "--build") - - res.Assert(t, icmd.Expected{Out: "COPY static /usr/share/nginx/html"}) - res.Assert(t, icmd.Expected{Out: "COPY static2 /usr/share/nginx/html"}) - }) - - t.Run(env+" build --push ignored for unnamed images", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "build", "--push", "nginx") - assert.Assert(t, !strings.Contains(res.Stdout(), "failed to push"), res.Stdout()) - }) - - t.Run(env+" build --quiet", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "build", "--quiet") - res.Assert(t, icmd.Expected{Out: ""}) - }) - - t.Run(env+" cleanup build project", func(t *testing.T) { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test", "down") - c.RunDockerOrExitError(t, "rmi", "-f", "build-test-nginx") - c.RunDockerOrExitError(t, "rmi", "-f", "custom-nginx") - }) - } -} - -func TestBuildSSH(t *testing.T) { - if runtime.GOOS == "windows" { - t.Skip("Running on Windows. Skipping...") - } - c := NewParallelCLI(t) - - t.Run("build failed with ssh default value", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test", "build", "--ssh", "") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "invalid empty ssh agent socket: make sure SSH_AUTH_SOCK is set", - }) - }) - - t.Run("build succeed with ssh from Compose file", func(t *testing.T) { - c.RunDockerOrExitError(t, "rmi", "build-test-ssh") - - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/ssh", "build") - c.RunDockerCmd(t, "image", "inspect", "build-test-ssh") - }) - - t.Run("build succeed with ssh from CLI", func(t *testing.T) { - c.RunDockerOrExitError(t, "rmi", "build-test-ssh") - - c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/ssh/compose-without-ssh.yaml", "--project-directory", - "fixtures/build-test/ssh", "build", "--no-cache", "--ssh", "fake-ssh=./fixtures/build-test/ssh/fake_rsa") - c.RunDockerCmd(t, "image", "inspect", "build-test-ssh") - }) - - /* - FIXME disabled waiting for https://github.com/moby/buildkit/issues/5558 - t.Run("build failed with wrong ssh key id from CLI", func(t *testing.T) { - c.RunDockerOrExitError(t, "rmi", "build-test-ssh") - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/build-test/ssh/compose-without-ssh.yaml", - "--project-directory", "fixtures/build-test/ssh", "build", "--no-cache", "--ssh", - "wrong-ssh=./fixtures/build-test/ssh/fake_rsa") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "unset ssh forward key fake-ssh", - }) - }) - */ - - t.Run("build succeed as part of up with ssh from Compose file", func(t *testing.T) { - c.RunDockerOrExitError(t, "rmi", "build-test-ssh") - - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/ssh", "up", "-d", "--build") - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/ssh", "down") - }) - c.RunDockerCmd(t, "image", "inspect", "build-test-ssh") - }) -} - -func TestBuildSecrets(t *testing.T) { - if runtime.GOOS == "windows" { - t.Skip("skipping test on windows") - } - c := NewParallelCLI(t) - - t.Run("build with secrets", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "build-test-secret") - - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/build-test/secrets", "build") - - res := icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "SOME_SECRET=bar") - }) - - res.Assert(t, icmd.Success) - }) -} - -func TestBuildTags(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("build with tags", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "build-test-tags") - - c.RunDockerComposeCmd(t, "--project-directory", "./fixtures/build-test/tags", "build", "--no-cache") - - res := c.RunDockerCmd(t, "image", "inspect", "build-test-tags") - expectedOutput := `"RepoTags": [ - "docker/build-test-tags:1.0.0", - "build-test-tags:latest", - "other-image-name:v1.0.0" - ], -` - res.Assert(t, icmd.Expected{Out: expectedOutput}) - }) -} - -func TestBuildImageDependencies(t *testing.T) { - doTest := func(t *testing.T, cli *CLI, args ...string) { - resetState := func() { - cli.RunDockerComposeCmd(t, "down", "--rmi=all", "-t=0") - res := cli.RunDockerOrExitError(t, "image", "rm", "build-dependencies-service") - if res.Error != nil { - require.Contains(t, res.Stderr(), `No such image: build-dependencies-service`) - } - } - resetState() - t.Cleanup(resetState) - - // the image should NOT exist now - res := cli.RunDockerOrExitError(t, "image", "inspect", "build-dependencies-service") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "No such image: build-dependencies-service", - }) - - res = cli.RunDockerComposeCmd(t, args...) - t.Log(res.Combined()) - - res = cli.RunDockerCmd(t, - "image", "inspect", "--format={{ index .RepoTags 0 }}", - "build-dependencies-service") - res.Assert(t, icmd.Expected{Out: "build-dependencies-service:latest"}) - - res = cli.RunDockerComposeCmd(t, "down", "-t0", "--rmi=all", "--remove-orphans") - t.Log(res.Combined()) - - res = cli.RunDockerOrExitError(t, "image", "inspect", "build-dependencies-service") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "No such image: build-dependencies-service", - }) - } - - t.Run("ClassicBuilder", func(t *testing.T) { - cli := NewCLI(t, WithEnv( - "DOCKER_BUILDKIT=0", - "COMPOSE_FILE=./fixtures/build-dependencies/classic.yaml", - )) - doTest(t, cli, "build") - doTest(t, cli, "build", "--with-dependencies", "service") - }) - - t.Run("Bake by additional contexts", func(t *testing.T) { - cli := NewCLI(t, WithEnv( - "DOCKER_BUILDKIT=1", "COMPOSE_BAKE=1", - "COMPOSE_FILE=./fixtures/build-dependencies/compose.yaml", - )) - doTest(t, cli, "--verbose", "build") - doTest(t, cli, "--verbose", "build", "service") - doTest(t, cli, "--verbose", "up", "--build", "service") - }) -} - -func TestBuildPlatformsWithCorrectBuildxConfig(t *testing.T) { - if runtime.GOOS == "windows" { - t.Skip("Running on Windows. Skipping...") - } - c := NewParallelCLI(t) - - // declare builder - result := c.RunDockerCmd(t, "buildx", "create", "--name", "build-platform", "--use", "--bootstrap") - assert.NilError(t, result.Error) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/platforms", "down") - _ = c.RunDockerCmd(t, "buildx", "rm", "-f", "build-platform") - }) - - t.Run("platform not supported by builder", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", - "-f", "fixtures/build-test/platforms/compose-unsupported-platform.yml", "build") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "no match for platform", - }) - }) - - t.Run("multi-arch build ok", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", "build") - assert.NilError(t, res.Error, res.Stderr()) - res.Assert(t, icmd.Expected{Out: "I am building for linux/arm64"}) - res.Assert(t, icmd.Expected{Out: "I am building for linux/amd64"}) - }) - - t.Run("multi-arch multi service builds ok", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", - "-f", "fixtures/build-test/platforms/compose-multiple-platform-builds.yaml", "build") - assert.NilError(t, res.Error, res.Stderr()) - res.Assert(t, icmd.Expected{Out: "I'm Service A and I am building for linux/arm64"}) - res.Assert(t, icmd.Expected{Out: "I'm Service A and I am building for linux/amd64"}) - res.Assert(t, icmd.Expected{Out: "I'm Service B and I am building for linux/arm64"}) - res.Assert(t, icmd.Expected{Out: "I'm Service B and I am building for linux/amd64"}) - res.Assert(t, icmd.Expected{Out: "I'm Service C and I am building for linux/arm64"}) - res.Assert(t, icmd.Expected{Out: "I'm Service C and I am building for linux/amd64"}) - }) - - t.Run("multi-arch up --build", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", "up", "--build", "--menu=false") - assert.NilError(t, res.Error, res.Stderr()) - res.Assert(t, icmd.Expected{Out: "platforms-1 exited with code 0"}) - }) - - t.Run("use DOCKER_DEFAULT_PLATFORM value when up --build", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/build-test/platforms", "up", "--build", "--menu=false") - res := icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "DOCKER_DEFAULT_PLATFORM=linux/amd64") - }) - assert.NilError(t, res.Error, res.Stderr()) - res.Assert(t, icmd.Expected{Out: "I am building for linux/amd64"}) - assert.Assert(t, !strings.Contains(res.Stdout(), "I am building for linux/arm64")) - }) - - t.Run("use service platform value when no build platforms defined ", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", - "-f", "fixtures/build-test/platforms/compose-service-platform-and-no-build-platforms.yaml", "build") - assert.NilError(t, res.Error, res.Stderr()) - res.Assert(t, icmd.Expected{Out: "I am building for linux/386"}) - }) -} - -func TestBuildPrivileged(t *testing.T) { - c := NewParallelCLI(t) - - // declare builder - result := c.RunDockerCmd(t, "buildx", "create", "--name", "build-privileged", "--use", "--bootstrap", "--buildkitd-flags", - `'--allow-insecure-entitlement=security.insecure'`) - assert.NilError(t, result.Error) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/privileged", "down") - _ = c.RunDockerCmd(t, "buildx", "rm", "-f", "build-privileged") - }) - - t.Run("use build privileged mode to run insecure build command", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/privileged", "build") - capEffRe := regexp.MustCompile("CapEff:\t([0-9a-f]+)") - matches := capEffRe.FindStringSubmatch(res.Stdout()) - assert.Equal(t, 2, len(matches), "Did not match CapEff in output, matches: %v", matches) - - capEff, err := strconv.ParseUint(matches[1], 16, 64) - assert.NilError(t, err, "Parsing CapEff: %s", matches[1]) - - // NOTE: can't use constant from x/sys/unix or tests won't compile on macOS/Windows - // #define CAP_SYS_ADMIN 21 - // https://github.com/torvalds/linux/blob/v6.1/include/uapi/linux/capability.h#L278 - const capSysAdmin = 0x15 - if capEff&capSysAdmin != capSysAdmin { - t.Fatalf("CapEff %s is missing CAP_SYS_ADMIN", matches[1]) - } - }) -} - -func TestBuildPlatformsStandardErrors(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("no platform support with Classic Builder", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/build-test/platforms", "build") - - res := icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "DOCKER_BUILDKIT=0") - }) - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "the classic builder doesn't support multi-arch build, set DOCKER_BUILDKIT=1 to use BuildKit", - }) - }) - - t.Run("builder does not support multi-arch", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", "build") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "Multi-platform build is not supported for the docker driver.", - }) - }) - - t.Run("service platform not defined in platforms build section", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test/platforms", - "-f", "fixtures/build-test/platforms/compose-service-platform-not-in-build-platforms.yaml", "build") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: `service.build.platforms MUST include service.platform "linux/riscv64"`, - }) - }) - - t.Run("DOCKER_DEFAULT_PLATFORM value not defined in platforms build section", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/build-test/platforms", "build") - res := icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "DOCKER_DEFAULT_PLATFORM=windows/amd64") - }) - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: `service "platforms" build.platforms does not support value set by DOCKER_DEFAULT_PLATFORM: windows/amd64`, - }) - }) - - t.Run("no privileged support with Classic Builder", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/build-test/privileged", "build") - - res := icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "DOCKER_BUILDKIT=0") - }) - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "the classic builder doesn't support privileged mode, set DOCKER_BUILDKIT=1 to use BuildKit", - }) - }) -} - -func TestBuildBuilder(t *testing.T) { - c := NewParallelCLI(t) - builderName := "build-with-builder" - // declare builder - result := c.RunDockerCmd(t, "buildx", "create", "--name", builderName, "--use", "--bootstrap") - assert.NilError(t, result.Error) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/", "down") - _ = c.RunDockerCmd(t, "buildx", "rm", "-f", builderName) - }) - - t.Run("use specific builder to run build command", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test", "build", "--builder", builderName) - assert.NilError(t, res.Error, res.Stderr()) - }) - - t.Run("error when using specific builder to run build command", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/build-test", "build", "--builder", "unknown-builder") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: fmt.Sprintf(`no builder %q found`, "unknown-builder"), - }) - }) -} - -func TestBuildEntitlements(t *testing.T) { - c := NewParallelCLI(t) - - // declare builder - result := c.RunDockerCmd(t, "buildx", "create", "--name", "build-insecure", "--use", "--bootstrap", "--buildkitd-flags", - `'--allow-insecure-entitlement=security.insecure'`) - assert.NilError(t, result.Error) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/entitlements", "down") - _ = c.RunDockerCmd(t, "buildx", "rm", "-f", "build-insecure") - }) - - t.Run("use build privileged mode to run insecure build command", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/build-test/entitlements", "build") - capEffRe := regexp.MustCompile("CapEff:\t([0-9a-f]+)") - matches := capEffRe.FindStringSubmatch(res.Stdout()) - assert.Equal(t, 2, len(matches), "Did not match CapEff in output, matches: %v", matches) - - capEff, err := strconv.ParseUint(matches[1], 16, 64) - assert.NilError(t, err, "Parsing CapEff: %s", matches[1]) - - // NOTE: can't use constant from x/sys/unix or tests won't compile on macOS/Windows - // #define CAP_SYS_ADMIN 21 - // https://github.com/torvalds/linux/blob/v6.1/include/uapi/linux/capability.h#L278 - const capSysAdmin = 0x15 - if capEff&capSysAdmin != capSysAdmin { - t.Fatalf("CapEff %s is missing CAP_SYS_ADMIN", matches[1]) - } - }) -} - -func TestBuildDependsOn(t *testing.T) { - c := NewParallelCLI(t) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/build-dependencies/compose-depends_on.yaml", "down", "--rmi=local") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/build-dependencies/compose-depends_on.yaml", "--progress=plain", "up", "test2") - out := res.Combined() - assert.Check(t, strings.Contains(out, "test1 Built")) -} - -func TestBuildSubset(t *testing.T) { - c := NewParallelCLI(t) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/subset/compose.yaml", "down", "--rmi=local") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/subset/compose.yaml", "build", "main") - out := res.Combined() - assert.Check(t, strings.Contains(out, "main Built")) -} - -func TestBuildDependentImage(t *testing.T) { - c := NewParallelCLI(t) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/dependencies/compose.yaml", "down", "--rmi=local") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/dependencies/compose.yaml", "build", "firstbuild") - out := res.Combined() - assert.Check(t, strings.Contains(out, "firstbuild Built")) - - res = c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/dependencies/compose.yaml", "build", "secondbuild") - out = res.Combined() - assert.Check(t, strings.Contains(out, "secondbuild Built")) -} - -func TestBuildSubDependencies(t *testing.T) { - c := NewParallelCLI(t) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/sub-dependencies/compose.yaml", "down", "--rmi=local") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/sub-dependencies/compose.yaml", "build", "main") - out := res.Combined() - assert.Check(t, strings.Contains(out, "main Built")) - - res = c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/sub-dependencies/compose.yaml", "up", "--build", "main") - out = res.Combined() - assert.Check(t, strings.Contains(out, "main Built")) -} - -func TestBuildLongOutputLine(t *testing.T) { - c := NewParallelCLI(t) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/long-output-line/compose.yaml", "down", "--rmi=local") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/long-output-line/compose.yaml", "build", "long-line") - out := res.Combined() - assert.Check(t, strings.Contains(out, "long-line Built")) - - res = c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/long-output-line/compose.yaml", "up", "--build", "long-line") - out = res.Combined() - assert.Check(t, strings.Contains(out, "long-line Built")) -} - -func TestBuildDependentImageWithProfile(t *testing.T) { - c := NewParallelCLI(t) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/profiles/compose.yaml", "down", "--rmi=local") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/build-test/profiles/compose.yaml", "build", "secret-build-test") - out := res.Combined() - assert.Check(t, strings.Contains(out, "secret-build-test Built")) -} - -func TestBuildTLS(t *testing.T) { - t.Helper() - - c := NewParallelCLI(t) - const dindBuilder = "e2e-dind-builder" - tmp := t.TempDir() - - t.Cleanup(func() { - c.RunDockerCmd(t, "rm", "-f", dindBuilder) - c.RunDockerCmd(t, "context", "rm", dindBuilder) - }) - - c.RunDockerCmd(t, "run", "--name", dindBuilder, "--privileged", "-p", "2376:2376", "-d", "docker:dind") - - poll.WaitOn(t, func(_ poll.LogT) poll.Result { - res := c.RunDockerCmd(t, "logs", dindBuilder) - if strings.Contains(res.Combined(), "API listen on [::]:2376") { - return poll.Success() - } - return poll.Continue("waiting for Docker daemon to be running") - }, poll.WithTimeout(10*time.Second)) - - time.Sleep(1 * time.Second) // wait for dind setup - c.RunDockerCmd(t, "cp", dindBuilder+":/certs/client", tmp) - - c.RunDockerCmd(t, "context", "create", dindBuilder, "--docker", - fmt.Sprintf("host=tcp://localhost:2376,ca=%s/client/ca.pem,cert=%s/client/cert.pem,key=%s/client/key.pem,skip-tls-verify=1", tmp, tmp, tmp)) - - cmd := c.NewDockerComposeCmd(t, "-f", "fixtures/build-test/minimal/compose.yaml", "build") - cmd.Env = append(cmd.Env, "DOCKER_CONTEXT="+dindBuilder) - cmd.Stdout = os.Stdout - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{Err: "Built"}) -} - -func TestBuildEscaped(t *testing.T) { - c := NewParallelCLI(t) - - res := c.RunDockerComposeCmd(t, "--project-directory", "./fixtures/build-test/escaped", "build", "--no-cache", "foo") - res.Assert(t, icmd.Expected{Out: "foo is ${bar}"}) - - res = c.RunDockerComposeCmd(t, "--project-directory", "./fixtures/build-test/escaped", "build", "--no-cache", "echo") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "--project-directory", "./fixtures/build-test/escaped", "build", "--no-cache", "arg") - res.Assert(t, icmd.Success) -} diff --git a/pkg/e2e/cancel_test.go b/pkg/e2e/cancel_test.go deleted file mode 100644 index 64f3ff609a9..00000000000 --- a/pkg/e2e/cancel_test.go +++ /dev/null @@ -1,99 +0,0 @@ -//go:build !windows - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "context" - "fmt" - "os/exec" - "strings" - "syscall" - "testing" - "time" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" - - "github.com/docker/compose/v5/pkg/utils" -) - -func TestComposeCancel(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("metrics on cancel Compose build", func(t *testing.T) { - const buildProjectPath = "fixtures/build-infinite/compose.yaml" - - ctx, cancel := context.WithCancel(t.Context()) - defer cancel() - - // require a separate groupID from the process running tests, in order to simulate ctrl+C from a terminal. - // sending kill signal - var stdout, stderr utils.SafeBuffer - cmd, err := StartWithNewGroupID( - ctx, - c.NewDockerComposeCmd(t, "-f", buildProjectPath, "build", "--progress", "plain"), - &stdout, - &stderr, - ) - assert.NilError(t, err) - processDone := make(chan error, 1) - go func() { - defer close(processDone) - processDone <- cmd.Wait() - }() - - c.WaitForCondition(t, func() (bool, string) { - out := stdout.String() - errors := stderr.String() - return strings.Contains(out, - "RUN sleep infinity"), fmt.Sprintf("'RUN sleep infinity' not found in : \n%s\nStderr: \n%s\n", out, - errors) - }, 30*time.Second, 1*time.Second) - - // simulate Ctrl-C : send signal to processGroup, children will have same groupId by default - err = syscall.Kill(-cmd.Process.Pid, syscall.SIGINT) - assert.NilError(t, err) - - select { - case <-ctx.Done(): - t.Fatal("test context canceled") - case err := <-processDone: - // TODO(milas): Compose should really not return exit code 130 here, - // this is an old hack for the compose-cli wrapper - assert.Error(t, err, "exit status 130", - "STDOUT:\n%s\nSTDERR:\n%s\n", stdout.String(), stderr.String()) - case <-time.After(10 * time.Second): - t.Fatal("timeout waiting for Compose exit") - } - }) -} - -func StartWithNewGroupID(ctx context.Context, command icmd.Cmd, stdout *utils.SafeBuffer, stderr *utils.SafeBuffer) (*exec.Cmd, error) { - cmd := exec.CommandContext(ctx, command.Command[0], command.Command[1:]...) - cmd.Env = command.Env - cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} - if stdout != nil { - cmd.Stdout = stdout - } - if stderr != nil { - cmd.Stderr = stderr - } - err := cmd.Start() - return cmd, err -} diff --git a/pkg/e2e/cascade_test.go b/pkg/e2e/cascade_test.go deleted file mode 100644 index 5bb23e1db54..00000000000 --- a/pkg/e2e/cascade_test.go +++ /dev/null @@ -1,55 +0,0 @@ -//go:build !windows - -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - - "gotest.tools/v3/assert" -) - -func TestCascadeStop(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-cascade-stop" - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/cascade/compose.yaml", "--project-name", projectName, - "up", "--abort-on-container-exit") - assert.Assert(t, strings.Contains(res.Combined(), "exit-1 exited with code 0"), res.Combined()) - // no --exit-code-from, so this is not an error - assert.Equal(t, res.ExitCode, 0) -} - -func TestCascadeFail(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-cascade-fail" - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "./fixtures/cascade/compose.yaml", "--project-name", projectName, - "up", "--abort-on-container-failure") - assert.Assert(t, strings.Contains(res.Combined(), "exit-1 exited with code 0"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "fail-1 exited with code 111"), res.Combined()) - // failing exit code should be propagated - assert.Equal(t, res.ExitCode, 111) -} diff --git a/pkg/e2e/commit_test.go b/pkg/e2e/commit_test.go deleted file mode 100644 index 0daf130545b..00000000000 --- a/pkg/e2e/commit_test.go +++ /dev/null @@ -1,93 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" -) - -func TestCommit(t *testing.T) { - const projectName = "e2e-commit-service" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/commit/compose.yaml", "--project-name", projectName, "up", "-d", "service") - - c.RunDockerComposeCmd( - t, - "--project-name", - projectName, - "commit", - "-a", - "John Hannibal Smith <hannibal@a-team.com>", - "-c", - "ENV DEBUG=true", - "-m", - "sample commit", - "service", - "service:latest", - ) -} - -func TestCommitWithReplicas(t *testing.T) { - const projectName = "e2e-commit-service-with-replicas" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/commit/compose.yaml", "--project-name", projectName, "up", "-d", "service-with-replicas") - - c.RunDockerComposeCmd( - t, - "--project-name", - projectName, - "commit", - "-a", - "John Hannibal Smith <hannibal@a-team.com>", - "-c", - "ENV DEBUG=true", - "-m", - "sample commit", - "--index=1", - "service-with-replicas", - "service-with-replicas:1", - ) - c.RunDockerComposeCmd( - t, - "--project-name", - projectName, - "commit", - "-a", - "John Hannibal Smith <hannibal@a-team.com>", - "-c", - "ENV DEBUG=true", - "-m", - "sample commit", - "--index=2", - "service-with-replicas", - "service-with-replicas:2", - ) -} diff --git a/pkg/e2e/compose_environment_test.go b/pkg/e2e/compose_environment_test.go deleted file mode 100644 index 62d6c9ebc5e..00000000000 --- a/pkg/e2e/compose_environment_test.go +++ /dev/null @@ -1,245 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestEnvPriority(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("up", func(t *testing.T) { - c.RunDockerOrExitError(t, "rmi", "env-compose-priority") - c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose-with-env.yaml", - "up", "-d", "--build") - }) - - // Full options activated - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From OS Environment) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("compose file priority", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose-with-env.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - cmd.Env = append(cmd.Env, "WHEREAMI=shell") - res := icmd.RunCmd(cmd) - assert.Equal(t, strings.TrimSpace(res.Stdout()), "shell") - }) - - // Full options activated - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("compose file priority", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose-with-env.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override", - "run", "--rm", "-e", "WHEREAMI=shell", "env-compose-priority") - res := icmd.RunCmd(cmd) - assert.Equal(t, strings.TrimSpace(res.Stdout()), "shell") - }) - - // No Compose file, all other options - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From OS Environment) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("shell priority", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - cmd.Env = append(cmd.Env, "WHEREAMI=shell") - res := icmd.RunCmd(cmd) - assert.Equal(t, strings.TrimSpace(res.Stdout()), "shell") - }) - - // No Compose file, all other options with env variable from OS environment - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From environment) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("shell priority file with default value", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override.with.default", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - cmd.Env = append(cmd.Env, "WHEREAMI=shell") - res := icmd.RunCmd(cmd) - assert.Equal(t, strings.TrimSpace(res.Stdout()), "shell") - }) - - // No Compose file, all other options with env variable from OS environment - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From environment default value from file in --env-file) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("shell priority implicitly set", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override.with.default", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - res := icmd.RunCmd(cmd) - assert.Equal(t, strings.TrimSpace(res.Stdout()), "EnvFileDefaultValue") - }) - - // No Compose file, all other options with env variable from OS environment - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From environment default value from file in COMPOSE_ENV_FILES) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("shell priority from COMPOSE_ENV_FILES variable", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - cmd.Env = append(cmd.Env, "COMPOSE_ENV_FILES=./fixtures/environment/env-priority/.env.override.with.default") - res := icmd.RunCmd(cmd) - stdout := res.Stdout() - assert.Equal(t, strings.TrimSpace(stdout), "EnvFileDefaultValue") - }) - - // No Compose file and env variable pass to the run command - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("shell priority from run command", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override", - "run", "--rm", "-e", "WHEREAMI=shell-run", "env-compose-priority") - assert.Equal(t, strings.TrimSpace(res.Stdout()), "shell-run") - }) - - // No Compose file & no env variable but override env file - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From environment patched by .env as a default --env-file value) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("override env file from compose", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose-with-env-file.yaml", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - assert.Equal(t, strings.TrimSpace(res.Stdout()), "Env File") - }) - - // No Compose file & no env variable but override by default env file - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From environment patched by --env-file value) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("override env file", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.override", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - assert.Equal(t, strings.TrimSpace(res.Stdout()), "override") - }) - - // No Compose file & no env variable but override env file - // 1. Command Line (docker compose run --env <KEY[=VAL]>) <-- Result expected (From environment patched by --env-file value) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive - // 5. Variable is not defined - t.Run("env file", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - assert.Equal(t, strings.TrimSpace(res.Stdout()), "Env File") - }) - - // No Compose file & no env variable, using an empty override env file - // 1. Command Line (docker compose run --env <KEY[=VAL]>) - // 2. Compose File (service::environment section) - // 3. Compose File (service::env_file section file) - // 4. Container Image ENV directive <-- Result expected - // 5. Variable is not defined - t.Run("use Dockerfile", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-priority/compose.yaml", - "--env-file", "./fixtures/environment/env-priority/.env.empty", - "run", "--rm", "-e", "WHEREAMI", "env-compose-priority") - assert.Equal(t, strings.TrimSpace(res.Stdout()), "Dockerfile") - }) - - t.Run("down", func(t *testing.T) { - c.RunDockerComposeCmd(t, "--project-name", "env-priority", "down") - }) -} - -func TestEnvInterpolation(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("shell priority from run command", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/environment/env-interpolation/compose.yaml", "config") - cmd.Env = append(cmd.Env, "WHEREAMI=shell") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{Out: `IMAGE: default_env:shell`}) - }) - - t.Run("shell priority from run command using default value fallback", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-interpolation-default-value/compose.yaml", "config"). - Assert(t, icmd.Expected{Out: `IMAGE: default_env:EnvFileDefaultValue`}) - }) -} - -func TestCommentsInEnvFile(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("comments in env files", func(t *testing.T) { - c.RunDockerOrExitError(t, "rmi", "env-file-comments") - - c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-file-comments/compose.yaml", "up", "-d", "--build") - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/env-file-comments/compose.yaml", - "run", "--rm", "-e", "COMMENT", "-e", "NO_COMMENT", "env-file-comments") - - res.Assert(t, icmd.Expected{Out: `COMMENT=1234`}) - res.Assert(t, icmd.Expected{Out: `NO_COMMENT=1234#5`}) - - c.RunDockerComposeCmd(t, "--project-name", "env-file-comments", "down", "--rmi", "all") - }) -} - -func TestUnsetEnv(t *testing.T) { - c := NewParallelCLI(t) - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", "empty-variable", "down", "--rmi", "all") - }) - - t.Run("override env variable", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/empty-variable/compose.yaml", "build") - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/empty-variable/compose.yaml", - "run", "-e", "EMPTY=hello", "--rm", "empty-variable") - res.Assert(t, icmd.Expected{Out: `=hello=`}) - }) - - t.Run("unset env variable", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/environment/empty-variable/compose.yaml", - "run", "--rm", "empty-variable") - res.Assert(t, icmd.Expected{Out: `==`}) - }) -} diff --git a/pkg/e2e/compose_exec_test.go b/pkg/e2e/compose_exec_test.go deleted file mode 100644 index 92b7e7ac74b..00000000000 --- a/pkg/e2e/compose_exec_test.go +++ /dev/null @@ -1,94 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestLocalComposeExec(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "compose-e2e-exec" - - cmdArgs := func(cmd string, args ...string) []string { - ret := []string{"--project-directory", "fixtures/simple-composefile", "--project-name", projectName, cmd} - ret = append(ret, args...) - return ret - } - - cleanup := func() { - c.RunDockerComposeCmd(t, cmdArgs("down", "--timeout=0")...) - } - cleanup() - t.Cleanup(cleanup) - - c.RunDockerComposeCmd(t, cmdArgs("up", "-d")...) - - t.Run("exec true", func(t *testing.T) { - c.RunDockerComposeCmd(t, cmdArgs("exec", "simple", "/bin/true")...) - }) - - t.Run("exec false", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, cmdArgs("exec", "simple", "/bin/false")...) - res.Assert(t, icmd.Expected{ExitCode: 1}) - }) - - t.Run("exec with env set", func(t *testing.T) { - res := icmd.RunCmd(c.NewDockerComposeCmd(t, cmdArgs("exec", "-e", "FOO", "simple", "/usr/bin/env")...), - func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "FOO=BAR") - }) - res.Assert(t, icmd.Expected{Out: "FOO=BAR"}) - }) - - t.Run("exec without env set", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, cmdArgs("exec", "-e", "FOO", "simple", "/usr/bin/env")...) - assert.Check(t, !strings.Contains(res.Stdout(), "FOO="), res.Combined()) - }) -} - -func TestLocalComposeExecOneOff(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "compose-e2e-exec-one-off" - defer c.cleanupWithDown(t, projectName) - cmdArgs := func(cmd string, args ...string) []string { - ret := []string{"--project-directory", "fixtures/simple-composefile", "--project-name", projectName, cmd} - ret = append(ret, args...) - return ret - } - - c.RunDockerComposeCmd(t, cmdArgs("run", "-d", "simple")...) - - t.Run("exec in one-off container", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, cmdArgs("exec", "-e", "FOO", "simple", "/usr/bin/env")...) - assert.Check(t, !strings.Contains(res.Stdout(), "FOO="), res.Combined()) - }) - - t.Run("exec with index", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, cmdArgs("exec", "--index", "1", "-e", "FOO", "simple", "/usr/bin/env")...) - res.Assert(t, icmd.Expected{ExitCode: 1, Err: "service \"simple\" is not running container #1"}) - }) - cmdResult := c.RunDockerCmd(t, "ps", "-q", "--filter", "label=com.docker.compose.project=compose-e2e-exec-one-off").Stdout() - containerIDs := strings.Split(cmdResult, "\n") - _ = c.RunDockerOrExitError(t, append([]string{"stop"}, containerIDs...)...) -} diff --git a/pkg/e2e/compose_run_build_once_test.go b/pkg/e2e/compose_run_build_once_test.go deleted file mode 100644 index f9726bb3b31..00000000000 --- a/pkg/e2e/compose_run_build_once_test.go +++ /dev/null @@ -1,100 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "crypto/rand" - "encoding/hex" - "fmt" - "regexp" - "strings" - "testing" - - "gotest.tools/v3/assert" -) - -// TestRunBuildOnce tests that services with pull_policy: build are only built once -// when using 'docker compose run', even when they are dependencies. -// This addresses a bug where dependencies were built twice: once in startDependencies -// and once in ensureImagesExists. -func TestRunBuildOnce(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("dependency with pull_policy build is built only once", func(t *testing.T) { - projectName := randomProjectName("build-once") - _ = c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once.yaml", "down", "--rmi", "local", "--remove-orphans", "-v") - res := c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once.yaml", "--verbose", "run", "--build", "--rm", "curl") - - output := res.Stdout() - - nginxBuilds := countServiceBuilds(output, projectName, "nginx") - - assert.Equal(t, nginxBuilds, 1, "nginx should build once, built %d times\nOutput:\n%s", nginxBuilds, output) - assert.Assert(t, strings.Contains(res.Stdout(), "curl service")) - - c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once.yaml", "down", "--remove-orphans") - }) - - t.Run("nested dependencies build only once each", func(t *testing.T) { - projectName := randomProjectName("build-nested") - _ = c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once-nested.yaml", "down", "--rmi", "local", "--remove-orphans", "-v") - res := c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once-nested.yaml", "--verbose", "run", "--build", "--rm", "app") - - output := res.Stdout() - - dbBuilds := countServiceBuilds(output, projectName, "db") - apiBuilds := countServiceBuilds(output, projectName, "api") - appBuilds := countServiceBuilds(output, projectName, "app") - - assert.Equal(t, dbBuilds, 1, "db should build once, built %d times\nOutput:\n%s", dbBuilds, output) - assert.Equal(t, apiBuilds, 1, "api should build once, built %d times\nOutput:\n%s", apiBuilds, output) - assert.Equal(t, appBuilds, 1, "app should build once, built %d times\nOutput:\n%s", appBuilds, output) - assert.Assert(t, strings.Contains(output, "App running")) - - c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once-nested.yaml", "down", "--rmi", "local", "--remove-orphans", "-v") - }) - - t.Run("service with no dependencies builds once", func(t *testing.T) { - projectName := randomProjectName("build-simple") - _ = c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once-no-deps.yaml", "down", "--rmi", "local", "--remove-orphans") - res := c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once-no-deps.yaml", "run", "--build", "--rm", "simple") - - output := res.Stdout() - - simpleBuilds := countServiceBuilds(output, projectName, "simple") - - assert.Equal(t, simpleBuilds, 1, "simple should build once, built %d times\nOutput:\n%s", simpleBuilds, output) - assert.Assert(t, strings.Contains(res.Stdout(), "Simple service")) - - c.RunDockerComposeCmd(t, "-p", projectName, "-f", "./fixtures/run-test/build-once-no-deps.yaml", "down", "--remove-orphans") - }) -} - -// countServiceBuilds counts how many times a service was built by matching -// the "naming to *{projectName}-{serviceName}* done" pattern in the output -func countServiceBuilds(output, projectName, serviceName string) int { - pattern := regexp.MustCompile(`naming to .*` + regexp.QuoteMeta(projectName) + `-` + regexp.QuoteMeta(serviceName) + `.* done`) - return len(pattern.FindAllString(output, -1)) -} - -// randomProjectName generates a unique project name for parallel test execution -// Format: prefix-<8 random hex chars> (e.g., "build-once-3f4a9b2c") -func randomProjectName(prefix string) string { - b := make([]byte, 4) // 4 bytes = 8 hex chars - rand.Read(b) //nolint:errcheck - return fmt.Sprintf("%s-%s", prefix, hex.EncodeToString(b)) -} diff --git a/pkg/e2e/compose_run_test.go b/pkg/e2e/compose_run_test.go deleted file mode 100644 index 7ee2313aa2b..00000000000 --- a/pkg/e2e/compose_run_test.go +++ /dev/null @@ -1,281 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "os" - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestLocalComposeRun(t *testing.T) { - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "run-test") - - t.Run("compose run", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "back") - lines := Lines(res.Stdout()) - assert.Equal(t, lines[len(lines)-1], "Hello there!!", res.Stdout()) - assert.Assert(t, !strings.Contains(res.Combined(), "orphan")) - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "back", "echo", - "Hello one more time") - lines = Lines(res.Stdout()) - assert.Equal(t, lines[len(lines)-1], "Hello one more time", res.Stdout()) - assert.Assert(t, strings.Contains(res.Combined(), "orphan")) - }) - - t.Run("check run container exited", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps", "--all") - lines := Lines(res.Stdout()) - var runContainerID string - var truncatedSlug string - for _, line := range lines { - fields := strings.Fields(line) - containerID := fields[len(fields)-1] - assert.Assert(t, !strings.HasPrefix(containerID, "run-test-front")) - if strings.HasPrefix(containerID, "run-test-back") { - // only the one-off container for back service - assert.Assert(t, strings.HasPrefix(containerID, "run-test-back-run-"), containerID) - truncatedSlug = strings.Replace(containerID, "run-test-back-run-", "", 1) - runContainerID = containerID - } - if strings.HasPrefix(containerID, "run-test-db-1") { - assert.Assert(t, strings.Contains(line, "Up"), line) - } - } - assert.Assert(t, runContainerID != "") - res = c.RunDockerCmd(t, "inspect", runContainerID) - res.Assert(t, icmd.Expected{Out: ` "Status": "exited"`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.project": "run-test"`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.oneoff": "True",`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.slug": "` + truncatedSlug}) - }) - - t.Run("compose run --rm", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "--rm", "back", "echo", - "Hello again") - lines := Lines(res.Stdout()) - assert.Equal(t, lines[len(lines)-1], "Hello again", res.Stdout()) - - res = c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, strings.Contains(res.Stdout(), "run-test-back"), res.Stdout()) - }) - - t.Run("down", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "down", "--remove-orphans") - res := c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, !strings.Contains(res.Stdout(), "run-test"), res.Stdout()) - }) - - t.Run("compose run --volumes", func(t *testing.T) { - wd, err := os.Getwd() - assert.NilError(t, err) - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "--volumes", wd+":/foo", - "back", "/bin/sh", "-c", "ls /foo") - res.Assert(t, icmd.Expected{Out: "compose_run_test.go"}) - - res = c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, strings.Contains(res.Stdout(), "run-test-back"), res.Stdout()) - }) - - t.Run("compose run --publish", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/ports.yaml", "run", "--publish", "8081:80", "-d", "back", - "/bin/sh", "-c", "sleep 1") - res := c.RunDockerCmd(t, "ps") - assert.Assert(t, strings.Contains(res.Stdout(), "8081->80/tcp"), res.Stdout()) - assert.Assert(t, !strings.Contains(res.Stdout(), "8082->80/tcp"), res.Stdout()) - }) - - t.Run("compose run --service-ports", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/ports.yaml", "run", "--service-ports", "-d", "back", - "/bin/sh", "-c", "sleep 1") - res := c.RunDockerCmd(t, "ps") - assert.Assert(t, strings.Contains(res.Stdout(), "8082->80/tcp"), res.Stdout()) - }) - - t.Run("compose run orphan", func(t *testing.T) { - // Use different compose files to get an orphan container - c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/orphan.yaml", "run", "simple") - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "back", "echo", "Hello") - assert.Assert(t, strings.Contains(res.Combined(), "orphan")) - - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "back", "echo", "Hello") - res = icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "COMPOSE_IGNORE_ORPHANS=True") - }) - assert.Assert(t, !strings.Contains(res.Combined(), "orphan")) - }) - - t.Run("down", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "down") - icmd.RunCmd(cmd, func(c *icmd.Cmd) { - c.Env = append(c.Env, "COMPOSE_REMOVE_ORPHANS=True") - }) - res := c.RunDockerCmd(t, "ps", "--all") - - assert.Assert(t, !strings.Contains(res.Stdout(), "run-test"), res.Stdout()) - }) - - t.Run("run starts only container and dependencies", func(t *testing.T) { - // ensure that even if another service is up run does not start it: https://github.com/docker/compose/issues/9459 - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/deps.yaml", "up", "service_b", "--menu=false") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/deps.yaml", "run", "service_a") - assert.Assert(t, strings.Contains(res.Combined(), "shared_dep"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "service_b"), res.Combined()) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/deps.yaml", "down", "--remove-orphans") - }) - - t.Run("run without dependencies", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/deps.yaml", "run", "--no-deps", "service_a") - assert.Assert(t, !strings.Contains(res.Combined(), "shared_dep"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "service_b"), res.Combined()) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/deps.yaml", "down", "--remove-orphans") - }) - - t.Run("run with not required dependency", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/dependencies/deps-not-required.yaml", "run", "foo") - assert.Assert(t, strings.Contains(res.Combined(), "foo"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "bar"), res.Combined()) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/dependencies/deps-not-required.yaml", "down", "--remove-orphans") - }) - - t.Run("--quiet-pull", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/quiet-pull.yaml", "down", "--remove-orphans", "--rmi", "all") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/quiet-pull.yaml", "run", "--quiet-pull", "backend") - assert.Assert(t, !strings.Contains(res.Combined(), "Pull complete"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "Pulled"), res.Combined()) - }) - - t.Run("COMPOSE_PROGRESS quiet", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/quiet-pull.yaml", "down", "--remove-orphans", "--rmi", "all") - res.Assert(t, icmd.Success) - - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/run-test/quiet-pull.yaml", "run", "backend") - res = icmd.RunCmd(cmd, func(c *icmd.Cmd) { - c.Env = append(c.Env, "COMPOSE_PROGRESS=quiet") - }) - assert.Assert(t, !strings.Contains(res.Combined(), "Pull complete"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "Pulled"), res.Combined()) - }) - - t.Run("--pull", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/pull.yaml", "down", "--remove-orphans", "--rmi", "all") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/pull.yaml", "run", "--pull", "always", "backend") - assert.Assert(t, strings.Contains(res.Combined(), "Image nginx Pulling"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "Image nginx Pulled"), res.Combined()) - }) - - t.Run("compose run --env-from-file", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "--env-from-file", "./fixtures/run-test/run.env", - "front", "env") - res.Assert(t, icmd.Expected{Out: "FOO=BAR"}) - }) - - t.Run("compose run -rm with stop signal", func(t *testing.T) { - projectName := "run-test" - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "-f", "./fixtures/ps-test/compose.yaml", "run", "--rm", "-d", "nginx") - res.Assert(t, icmd.Success) - - res = c.RunDockerCmd(t, "ps", "--quiet", "--filter", "name=run-test-nginx") - containerID := strings.TrimSpace(res.Stdout()) - - res = c.RunDockerCmd(t, "stop", containerID) - res.Assert(t, icmd.Success) - res = c.RunDockerCmd(t, "ps", "--all", "--filter", "name=run-test-nginx", "--format", "'{{.Names}}'") - assert.Assert(t, !strings.Contains(res.Stdout(), "run-test-nginx"), res.Stdout()) - }) - - t.Run("compose run --env", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "--env", "FOO=BAR", - "front", "env") - res.Assert(t, icmd.Expected{Out: "FOO=BAR"}) - }) - - t.Run("compose run --build", func(t *testing.T) { - c.cleanupWithDown(t, "run-test", "--rmi=local") - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/run-test/compose.yaml", "run", "build", "echo", "hello world") - res.Assert(t, icmd.Expected{Out: "hello world"}) - }) - - t.Run("compose run with piped input detection", func(t *testing.T) { - if composeStandaloneMode { - t.Skip("Skipping test compose with piped input detection in standalone mode") - } - // Test that piped input is properly detected and TTY is automatically disabled - // This tests the logic added in run.go that checks dockerCli.In().IsTerminal() - cmd := c.NewCmd("sh", "-c", "echo 'piped-content' | docker compose -f ./fixtures/run-test/piped-test.yaml run --rm piped-test") - res := icmd.RunCmd(cmd) - - res.Assert(t, icmd.Expected{Out: "piped-content"}) - res.Assert(t, icmd.Success) - }) - - t.Run("compose run piped input should not allocate TTY", func(t *testing.T) { - if composeStandaloneMode { - t.Skip("Skipping test compose with piped input detection in standalone mode") - } - // Test that when stdin is piped, the container correctly detects no TTY - // This verifies that the automatic noTty=true setting works correctly - cmd := c.NewCmd("sh", "-c", "echo '' | docker compose -f ./fixtures/run-test/piped-test.yaml run --rm tty-test") - res := icmd.RunCmd(cmd) - - res.Assert(t, icmd.Expected{Out: "No TTY detected"}) - res.Assert(t, icmd.Success) - }) - - t.Run("compose run piped input with explicit --tty should fail", func(t *testing.T) { - if composeStandaloneMode { - t.Skip("Skipping test compose with piped input detection in standalone mode") - } - // Test that explicitly requesting TTY with piped input fails with proper error message - // This should trigger the "input device is not a TTY" error - cmd := c.NewCmd("sh", "-c", "echo 'test' | docker compose -f ./fixtures/run-test/piped-test.yaml run --rm --tty piped-test") - res := icmd.RunCmd(cmd) - - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "the input device is not a TTY", - }) - }) - - t.Run("compose run piped input with --no-TTY=false should fail", func(t *testing.T) { - if composeStandaloneMode { - t.Skip("Skipping test compose with piped input detection in standalone mode") - } - // Test that explicitly disabling --no-TTY (i.e., requesting TTY) with piped input fails - // This should also trigger the "input device is not a TTY" error - cmd := c.NewCmd("sh", "-c", "echo 'test' | docker compose -f ./fixtures/run-test/piped-test.yaml run --rm --no-TTY=false piped-test") - res := icmd.RunCmd(cmd) - - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "the input device is not a TTY", - }) - }) -} diff --git a/pkg/e2e/compose_test.go b/pkg/e2e/compose_test.go deleted file mode 100644 index 3b5a7341a70..00000000000 --- a/pkg/e2e/compose_test.go +++ /dev/null @@ -1,408 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "net/http" - "os" - "path/filepath" - "strings" - "testing" - "time" - - testify "github.com/stretchr/testify/assert" - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestLocalComposeUp(t *testing.T) { - // this test shares a fixture with TestCompatibility and can't run at the same time - c := NewCLI(t) - - const projectName = "compose-e2e-demo" - - t.Run("up", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/sentences/compose.yaml", "--project-name", projectName, "up", "-d") - }) - - t.Run("check accessing running app", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-p", projectName, "ps") - res.Assert(t, icmd.Expected{Out: `web`}) - - endpoint := "http://localhost:90" - output := HTTPGetWithRetry(t, endpoint+"/words/noun", http.StatusOK, 2*time.Second, 20*time.Second) - assert.Assert(t, strings.Contains(output, `"word":`)) - - res = c.RunDockerCmd(t, "network", "ls") - res.Assert(t, icmd.Expected{Out: projectName + "_default"}) - }) - - t.Run("top", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-p", projectName, "top") - output := res.Stdout() - head := []string{"UID", "PID", "PPID", "C", "STIME", "TTY", "TIME", "CMD"} - for _, h := range head { - assert.Assert(t, strings.Contains(output, h), output) - } - assert.Assert(t, strings.Contains(output, `java -Xmx8m -Xms8m -jar /app/words.jar`), output) - assert.Assert(t, strings.Contains(output, `/dispatcher`), output) - }) - - t.Run("check compose labels", func(t *testing.T) { - res := c.RunDockerCmd(t, "inspect", projectName+"-web-1") - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.container-number": "1"`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.project": "compose-e2e-demo"`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.oneoff": "False",`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.config-hash":`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.project.config_files":`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.project.working_dir":`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.service": "web"`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.version":`}) - - res = c.RunDockerCmd(t, "network", "inspect", projectName+"_default") - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.network": "default"`}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.project": `}) - res.Assert(t, icmd.Expected{Out: `"com.docker.compose.version": `}) - }) - - t.Run("check user labels", func(t *testing.T) { - res := c.RunDockerCmd(t, "inspect", projectName+"-web-1") - res.Assert(t, icmd.Expected{Out: `"my-label": "test"`}) - }) - - t.Run("check healthcheck output", func(t *testing.T) { - c.WaitForCmdResult(t, c.NewDockerComposeCmd(t, "-p", projectName, "ps", "--format", "json"), - IsHealthy(projectName+"-web-1"), - 5*time.Second, 1*time.Second) - - res := c.RunDockerComposeCmd(t, "-p", projectName, "ps") - assertServiceStatus(t, projectName, "web", "(healthy)", res.Stdout()) - }) - - t.Run("images", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-p", projectName, "images") - res.Assert(t, icmd.Expected{Out: `compose-e2e-demo-db-1 gtardif/sentences-db latest`}) - res.Assert(t, icmd.Expected{Out: `compose-e2e-demo-web-1 gtardif/sentences-web latest`}) - res.Assert(t, icmd.Expected{Out: `compose-e2e-demo-words-1 gtardif/sentences-api latest`}) - }) - - t.Run("down SERVICE", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "web") - - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "ps") - assert.Assert(t, !strings.Contains(res.Combined(), "compose-e2e-demo-web-1"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "compose-e2e-demo-db-1"), res.Combined()) - }) - - t.Run("down", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - t.Run("check containers after down", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, !strings.Contains(res.Combined(), projectName), res.Combined()) - }) - - t.Run("check networks after down", func(t *testing.T) { - res := c.RunDockerCmd(t, "network", "ls") - assert.Assert(t, !strings.Contains(res.Combined(), projectName), res.Combined()) - }) -} - -func TestDownComposefileInParentFolder(t *testing.T) { - c := NewParallelCLI(t) - - tmpFolder, err := os.MkdirTemp("fixtures/simple-composefile", "test-tmp") - assert.NilError(t, err) - defer os.Remove(tmpFolder) //nolint:errcheck - projectName := filepath.Base(tmpFolder) - - res := c.RunDockerComposeCmd(t, "--project-directory", tmpFolder, "up", "-d") - res.Assert(t, icmd.Expected{Err: "Started", ExitCode: 0}) - - res = c.RunDockerComposeCmd(t, "-p", projectName, "down") - res.Assert(t, icmd.Expected{Err: "Removed", ExitCode: 0}) -} - -func TestAttachRestart(t *testing.T) { - t.Skip("Skipping test until we can fix it") - - if _, ok := os.LookupEnv("CI"); ok { - t.Skip("Skipping test on CI... flaky") - } - c := NewParallelCLI(t) - - cmd := c.NewDockerComposeCmd(t, "--ansi=never", "--project-directory", "./fixtures/attach-restart", "up") - res := icmd.StartCmd(cmd) - defer c.RunDockerComposeCmd(t, "-p", "attach-restart", "down") - - c.WaitForCondition(t, func() (bool, string) { - debug := res.Combined() - return strings.Count(res.Stdout(), - "failing-1 exited with code 1") == 3, fmt.Sprintf("'failing-1 exited with code 1' not found 3 times in : \n%s\n", - debug) - }, 4*time.Minute, 2*time.Second) - - assert.Equal(t, strings.Count(res.Stdout(), "failing-1 | world"), 3, res.Combined()) -} - -func TestInitContainer(t *testing.T) { - c := NewParallelCLI(t) - - res := c.RunDockerComposeCmd(t, "--ansi=never", "--project-directory", "./fixtures/init-container", "up", "--menu=false") - defer c.RunDockerComposeCmd(t, "-p", "init-container", "down") - testify.Regexp(t, "foo-1 | hello(?m:.*)bar-1 | world", res.Stdout()) -} - -func TestRm(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "compose-e2e-rm" - - t.Run("up", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-composefile/compose.yaml", "-p", projectName, "up", "-d") - }) - - t.Run("rm --stop --force simple", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-composefile/compose.yaml", "-p", projectName, "rm", - "--stop", "--force", "simple") - res.Assert(t, icmd.Expected{Err: "Removed", ExitCode: 0}) - }) - - t.Run("check containers after rm", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, !strings.Contains(res.Combined(), projectName+"-simple"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), projectName+"-another"), res.Combined()) - }) - - t.Run("up (again)", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-composefile/compose.yaml", "-p", projectName, "up", "-d") - }) - - t.Run("rm ---stop --force <none>", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-composefile/compose.yaml", "-p", projectName, "rm", - "--stop", "--force") - res.Assert(t, icmd.Expected{ExitCode: 0}) - }) - - t.Run("check containers after rm", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, !strings.Contains(res.Combined(), projectName+"-simple"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), projectName+"-another"), res.Combined()) - }) - - t.Run("down", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-p", projectName, "down") - }) -} - -func TestCompatibility(t *testing.T) { - // this test shares a fixture with TestLocalComposeUp and can't run at the same time - c := NewCLI(t) - - const projectName = "compose-e2e-compatibility" - - t.Run("up", func(t *testing.T) { - c.RunDockerComposeCmd(t, "--compatibility", "-f", "./fixtures/sentences/compose.yaml", "--project-name", - projectName, "up", "-d") - }) - - t.Run("check container names", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps", "--format", "{{.Names}}") - res.Assert(t, icmd.Expected{Out: "compose-e2e-compatibility_web_1"}) - res.Assert(t, icmd.Expected{Out: "compose-e2e-compatibility_words_1"}) - res.Assert(t, icmd.Expected{Out: "compose-e2e-compatibility_db_1"}) - }) - - t.Run("down", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-p", projectName, "down") - }) -} - -func TestConfig(t *testing.T) { - const projectName = "compose-e2e-config" - c := NewParallelCLI(t) - - wd, err := os.Getwd() - assert.NilError(t, err) - - t.Run("up", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-build-test/compose.yaml", "-p", projectName, "config") - res.Assert(t, icmd.Expected{Out: fmt.Sprintf(`name: %s -services: - nginx: - build: - context: %s - dockerfile: Dockerfile - networks: - default: null -networks: - default: - name: compose-e2e-config_default -`, projectName, filepath.Join(wd, "fixtures", "simple-build-test", "nginx-build")), ExitCode: 0}) - }) -} - -func TestConfigInterpolate(t *testing.T) { - const projectName = "compose-e2e-config-interpolate" - c := NewParallelCLI(t) - - wd, err := os.Getwd() - assert.NilError(t, err) - - t.Run("config", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-build-test/compose-interpolate.yaml", "-p", projectName, "config", "--no-interpolate") - res.Assert(t, icmd.Expected{Out: fmt.Sprintf(`name: %s -networks: - default: - name: compose-e2e-config-interpolate_default -services: - nginx: - build: - context: %s - dockerfile: ${MYVAR} - networks: - default: null -`, projectName, filepath.Join(wd, "fixtures", "simple-build-test", "nginx-build")), ExitCode: 0}) - }) -} - -func TestStopWithDependenciesAttached(t *testing.T) { - const projectName = "compose-e2e-stop-with-deps" - c := NewParallelCLI(t, WithEnv("COMMAND=echo hello")) - - cleanup := func() { - c.RunDockerComposeCmd(t, "-p", projectName, "down", "--remove-orphans", "--timeout=0") - } - cleanup() - t.Cleanup(cleanup) - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/dependencies/compose.yaml", "-p", projectName, "up", "--attach-dependencies", "foo", "--menu=false") - res.Assert(t, icmd.Expected{Out: "exited with code 0"}) -} - -func TestRemoveOrphaned(t *testing.T) { - const projectName = "compose-e2e-remove-orphaned" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "-p", projectName, "down", "--remove-orphans", "--timeout=0") - } - cleanup() - t.Cleanup(cleanup) - - // run stack - c.RunDockerComposeCmd(t, "-f", "./fixtures/sentences/compose.yaml", "-p", projectName, "up", "-d") - - // down "web" service with orphaned removed - c.RunDockerComposeCmd(t, "-f", "./fixtures/sentences/compose.yaml", "-p", projectName, "down", "--remove-orphans", "web") - - // check "words" service has not been considered orphaned - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/sentences/compose.yaml", "-p", projectName, "ps", "--format", "{{.Name}}") - res.Assert(t, icmd.Expected{Out: fmt.Sprintf("%s-words-1", projectName)}) -} - -func TestComposeFileSetByDotEnv(t *testing.T) { - c := NewCLI(t) - defer c.cleanupWithDown(t, "dotenv") - - cmd := c.NewDockerComposeCmd(t, "config") - cmd.Dir = filepath.Join(".", "fixtures", "dotenv") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ - ExitCode: 0, - Out: "image: test:latest", - }) - res.Assert(t, icmd.Expected{ - Out: "image: enabled:profile", - }) -} - -func TestComposeFileSetByProjectDirectory(t *testing.T) { - c := NewCLI(t) - defer c.cleanupWithDown(t, "dotenv") - - dir := filepath.Join(".", "fixtures", "dotenv", "development") - cmd := c.NewDockerComposeCmd(t, "--project-directory", dir, "config") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ - ExitCode: 0, - Out: "image: backend:latest", - }) -} - -func TestComposeFileSetByEnvFile(t *testing.T) { - c := NewCLI(t) - defer c.cleanupWithDown(t, "dotenv") - - dotEnv, err := os.CreateTemp(t.TempDir(), ".env") - assert.NilError(t, err) - err = os.WriteFile(dotEnv.Name(), []byte(` -COMPOSE_FILE=fixtures/dotenv/development/compose.yaml -IMAGE_NAME=test -IMAGE_TAG=latest -COMPOSE_PROFILES=test -`), 0o700) - assert.NilError(t, err) - - cmd := c.NewDockerComposeCmd(t, "--env-file", dotEnv.Name(), "config") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ - Out: "image: test:latest", - }) - res.Assert(t, icmd.Expected{ - Out: "image: enabled:profile", - }) -} - -func TestNestedDotEnv(t *testing.T) { - c := NewCLI(t) - defer c.cleanupWithDown(t, "nested") - - cmd := c.NewDockerComposeCmd(t, "run", "echo") - cmd.Dir = filepath.Join(".", "fixtures", "nested") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ - ExitCode: 0, - Out: "root win=root", - }) - - cmd = c.NewDockerComposeCmd(t, "run", "echo") - cmd.Dir = filepath.Join(".", "fixtures", "nested", "sub") - defer c.cleanupWithDown(t, "nested") - res = icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ - ExitCode: 0, - Out: "root sub win=sub", - }) -} - -func TestUnnecessaryResources(t *testing.T) { - const projectName = "compose-e2e-unnecessary-resources" - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, projectName) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "./fixtures/external/compose.yaml", "-p", projectName, "up", "-d") - res.Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "network foo_bar declared as external, but could not be found", - }) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/external/compose.yaml", "-p", projectName, "up", "-d", "test") - // Should not fail as missing external network is not used -} diff --git a/pkg/e2e/compose_up_test.go b/pkg/e2e/compose_up_test.go deleted file mode 100644 index b00cb3c1861..00000000000 --- a/pkg/e2e/compose_up_test.go +++ /dev/null @@ -1,114 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - "time" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestUpWait(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-deps-wait" - - timeout := time.After(30 * time.Second) - done := make(chan bool) - go func() { - //nolint:nolintlint,testifylint // helper asserts inside goroutine; acceptable in this e2e test - res := c.RunDockerComposeCmd(t, "-f", "fixtures/dependencies/deps-completed-successfully.yaml", "--project-name", projectName, "up", "--wait", "-d") - assert.Assert(t, strings.Contains(res.Combined(), "e2e-deps-wait-oneshot-1"), res.Combined()) - done <- true - }() - - select { - case <-timeout: - t.Fatal("test did not finish in time") - case <-done: - break - } - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") -} - -func TestUpExitCodeFrom(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-exit-code-from" - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/start-fail/start-depends_on-long-lived.yaml", "--project-name", projectName, "up", "--menu=false", "--exit-code-from=failure", "failure") - res.Assert(t, icmd.Expected{ExitCode: 42}) - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--remove-orphans") -} - -func TestUpExitCodeFromContainerKilled(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-exit-code-from-kill" - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/start-fail/start-depends_on-long-lived.yaml", "--project-name", projectName, "up", "--menu=false", "--exit-code-from=test") - res.Assert(t, icmd.Expected{ExitCode: 143}) - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--remove-orphans") -} - -func TestPortRange(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-port-range" - - reset := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--remove-orphans", "--timeout=0") - } - reset() - t.Cleanup(reset) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/port-range/compose.yaml", "--project-name", projectName, "up", "-d") - res.Assert(t, icmd.Success) -} - -func TestStdoutStderr(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-stdout-stderr" - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/stdout-stderr/compose.yaml", "--project-name", projectName, "up", "--menu=false") - res.Assert(t, icmd.Expected{Out: "log to stdout", Err: "log to stderr"}) - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--remove-orphans") -} - -func TestLoggingDriver(t *testing.T) { - c := NewCLI(t) - const projectName = "e2e-logging-driver" - defer c.cleanupWithDown(t, projectName) - - host := "HOST=127.0.0.1" - res := c.RunDockerCmd(t, "info", "-f", "{{.OperatingSystem}}") - os := res.Stdout() - if strings.TrimSpace(os) == "Docker Desktop" { - host = "HOST=host.docker.internal" - } - - cmd := c.NewDockerComposeCmd(t, "-f", "fixtures/logging-driver/compose.yaml", "--project-name", projectName, "up", "-d") - cmd.Env = append(cmd.Env, host, "BAR=foo") - icmd.RunCmd(cmd).Assert(t, icmd.Success) - - cmd = c.NewDockerComposeCmd(t, "-f", "fixtures/logging-driver/compose.yaml", "--project-name", projectName, "up", "-d") - cmd.Env = append(cmd.Env, host, "BAR=zot") - icmd.RunCmd(cmd).Assert(t, icmd.Success) -} diff --git a/pkg/e2e/config_test.go b/pkg/e2e/config_test.go deleted file mode 100644 index 15d3e3d932a..00000000000 --- a/pkg/e2e/config_test.go +++ /dev/null @@ -1,70 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" - - "gotest.tools/v3/icmd" -) - -func TestLocalComposeConfig(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "compose-e2e-config" - - t.Run("yaml", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/config/compose.yaml", "--project-name", projectName, "config") - res.Assert(t, icmd.Expected{Out: ` - ports: - - mode: ingress - target: 80 - published: "8080" - protocol: tcp`}) - }) - - t.Run("json", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/config/compose.yaml", "--project-name", projectName, "config", "--format", "json") - res.Assert(t, icmd.Expected{Out: `"published": "8080"`}) - }) - - t.Run("--no-interpolate", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/config/compose.yaml", "--project-name", projectName, "config", "--no-interpolate") - res.Assert(t, icmd.Expected{Out: `- ${PORT:-8080}:80`}) - }) - - t.Run("--variables --format json", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/config/compose.yaml", "--project-name", projectName, "config", "--variables", "--format", "json") - res.Assert(t, icmd.Expected{Out: `{ - "PORT": { - "Name": "PORT", - "DefaultValue": "8080", - "PresenceValue": "", - "Required": false - } -}`}) - }) - - t.Run("--variables --format yaml", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/config/compose.yaml", "--project-name", projectName, "config", "--variables", "--format", "yaml") - res.Assert(t, icmd.Expected{Out: `PORT: - name: PORT - defaultvalue: "8080" - presencevalue: "" - required: false`}) - }) -} diff --git a/pkg/e2e/configs_test.go b/pkg/e2e/configs_test.go deleted file mode 100644 index c7d86f1c765..00000000000 --- a/pkg/e2e/configs_test.go +++ /dev/null @@ -1,57 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" - - "gotest.tools/v3/icmd" -) - -func TestConfigFromEnv(t *testing.T) { - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "configs") - - t.Run("config from file", func(t *testing.T) { - res := icmd.RunCmd(c.NewDockerComposeCmd(t, "-f", "./fixtures/configs/compose.yaml", "run", "from_file")) - res.Assert(t, icmd.Expected{Out: "This is my config file"}) - }) - - t.Run("config from env", func(t *testing.T) { - res := icmd.RunCmd(c.NewDockerComposeCmd(t, "-f", "./fixtures/configs/compose.yaml", "run", "from_env"), - func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "CONFIG=config") - }) - res.Assert(t, icmd.Expected{Out: "config"}) - }) - - t.Run("config inlined", func(t *testing.T) { - res := icmd.RunCmd(c.NewDockerComposeCmd(t, "-f", "./fixtures/configs/compose.yaml", "run", "inlined"), - func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "CONFIG=config") - }) - res.Assert(t, icmd.Expected{Out: "This is my config"}) - }) - - t.Run("custom target", func(t *testing.T) { - res := icmd.RunCmd(c.NewDockerComposeCmd(t, "-f", "./fixtures/configs/compose.yaml", "run", "target"), - func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "CONFIG=config") - }) - res.Assert(t, icmd.Expected{Out: "This is my config"}) - }) -} diff --git a/pkg/e2e/container_name_test.go b/pkg/e2e/container_name_test.go deleted file mode 100644 index 99de738c237..00000000000 --- a/pkg/e2e/container_name_test.go +++ /dev/null @@ -1,43 +0,0 @@ -//go:build !windows - -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" - - "gotest.tools/v3/icmd" -) - -func TestUpContainerNameConflict(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-container_name_conflict" - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/container_name/compose.yaml", "--project-name", projectName, "up") - res.Assert(t, icmd.Expected{ExitCode: 1, Err: `container name "test" is already in use`}) - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - c.RunDockerComposeCmd(t, "-f", "fixtures/container_name/compose.yaml", "--project-name", projectName, "up", "test") - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - c.RunDockerComposeCmd(t, "-f", "fixtures/container_name/compose.yaml", "--project-name", projectName, "up", "another_test") -} diff --git a/pkg/e2e/cp_test.go b/pkg/e2e/cp_test.go deleted file mode 100644 index 4b58c8b6d26..00000000000 --- a/pkg/e2e/cp_test.go +++ /dev/null @@ -1,114 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "os" - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestCopy(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "copy_e2e" - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "--project-name", projectName, "down") - - os.Remove("./fixtures/cp-test/from-default.txt") //nolint:errcheck - os.Remove("./fixtures/cp-test/from-indexed.txt") //nolint:errcheck - os.RemoveAll("./fixtures/cp-test/cp-folder2") //nolint:errcheck - }) - - t.Run("start service", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "--project-name", projectName, "up", - "--scale", "nginx=5", "-d") - }) - - t.Run("make sure service is running", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-p", projectName, "ps") - assertServiceStatus(t, projectName, "nginx", "Up", res.Stdout()) - }) - - t.Run("copy to container copies the file to the all containers by default", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "-p", projectName, "cp", - "./fixtures/cp-test/cp-me.txt", "nginx:/tmp/default.txt") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - output := c.RunDockerCmd(t, "exec", projectName+"-nginx-1", "cat", "/tmp/default.txt").Stdout() - assert.Assert(t, strings.Contains(output, `hello world`), output) - - output = c.RunDockerCmd(t, "exec", projectName+"-nginx-2", "cat", "/tmp/default.txt").Stdout() - assert.Assert(t, strings.Contains(output, `hello world`), output) - - output = c.RunDockerCmd(t, "exec", projectName+"-nginx-3", "cat", "/tmp/default.txt").Stdout() - assert.Assert(t, strings.Contains(output, `hello world`), output) - }) - - t.Run("copy to container with a given index copies the file to the given container", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "-p", projectName, "cp", "--index=3", - "./fixtures/cp-test/cp-me.txt", "nginx:/tmp/indexed.txt") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - output := c.RunDockerCmd(t, "exec", projectName+"-nginx-3", "cat", "/tmp/indexed.txt").Stdout() - assert.Assert(t, strings.Contains(output, `hello world`), output) - - res = c.RunDockerOrExitError(t, "exec", projectName+"-nginx-2", "cat", "/tmp/indexed.txt") - res.Assert(t, icmd.Expected{ExitCode: 1}) - }) - - t.Run("copy from a container copies the file to the host from the first container by default", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "-p", projectName, "cp", - "nginx:/tmp/default.txt", "./fixtures/cp-test/from-default.txt") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - data, err := os.ReadFile("./fixtures/cp-test/from-default.txt") - assert.NilError(t, err) - assert.Equal(t, `hello world`, string(data)) - }) - - t.Run("copy from a container with a given index copies the file to host", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "-p", projectName, "cp", "--index=3", - "nginx:/tmp/indexed.txt", "./fixtures/cp-test/from-indexed.txt") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - data, err := os.ReadFile("./fixtures/cp-test/from-indexed.txt") - assert.NilError(t, err) - assert.Equal(t, `hello world`, string(data)) - }) - - t.Run("copy to and from a container also work with folder", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "-p", projectName, "cp", - "./fixtures/cp-test/cp-folder", "nginx:/tmp") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - output := c.RunDockerCmd(t, "exec", projectName+"-nginx-1", "cat", "/tmp/cp-folder/cp-me.txt").Stdout() - assert.Assert(t, strings.Contains(output, `hello world from folder`), output) - - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/cp-test/compose.yaml", "-p", projectName, "cp", - "nginx:/tmp/cp-folder", "./fixtures/cp-test/cp-folder2") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - data, err := os.ReadFile("./fixtures/cp-test/cp-folder2/cp-me.txt") - assert.NilError(t, err) - assert.Equal(t, `hello world from folder`, string(data)) - }) -} diff --git a/pkg/e2e/e2e_config_plugin.go b/pkg/e2e/e2e_config_plugin.go deleted file mode 100644 index 84ca6eabcd9..00000000000 --- a/pkg/e2e/e2e_config_plugin.go +++ /dev/null @@ -1,21 +0,0 @@ -//go:build !standalone - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -const composeStandaloneMode = false diff --git a/pkg/e2e/e2e_config_standalone.go b/pkg/e2e/e2e_config_standalone.go deleted file mode 100644 index a4c4fe62943..00000000000 --- a/pkg/e2e/e2e_config_standalone.go +++ /dev/null @@ -1,21 +0,0 @@ -//go:build standalone - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -const composeStandaloneMode = true diff --git a/pkg/e2e/env_file_test.go b/pkg/e2e/env_file_test.go deleted file mode 100644 index d9ea1d4c2b6..00000000000 --- a/pkg/e2e/env_file_test.go +++ /dev/null @@ -1,53 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestRawEnvFile(t *testing.T) { - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "dotenv") - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/dotenv/raw.yaml", "run", "test") - assert.Equal(t, strings.TrimSpace(res.Stdout()), "'{\"key\": \"value\"}'") -} - -func TestUnusedMissingEnvFile(t *testing.T) { - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "unused_dotenv") - - c.RunDockerComposeCmd(t, "-f", "./fixtures/env_file/compose.yaml", "up", "-d", "serviceA") - - // Runtime operations should work even with missing env file - c.RunDockerComposeCmd(t, "-f", "./fixtures/env_file/compose.yaml", "ps") - c.RunDockerComposeCmd(t, "-f", "./fixtures/env_file/compose.yaml", "logs") - c.RunDockerComposeCmd(t, "-f", "./fixtures/env_file/compose.yaml", "down") -} - -func TestRunEnvFile(t *testing.T) { - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "run_dotenv") - - res := c.RunDockerComposeCmd(t, "--project-directory", "./fixtures/env_file", "run", "serviceC", "env") - res.Assert(t, icmd.Expected{Out: "FOO=BAR"}) -} diff --git a/pkg/e2e/exec_test.go b/pkg/e2e/exec_test.go deleted file mode 100644 index 16643a0f8a0..00000000000 --- a/pkg/e2e/exec_test.go +++ /dev/null @@ -1,47 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" - - "gotest.tools/v3/icmd" -) - -func TestExec(t *testing.T) { - const projectName = "e2e-exec" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/exec/compose.yaml", "--project-name", projectName, "run", "-d", "test", "cat") - - res := c.RunDockerComposeCmdNoCheck(t, "--project-name", projectName, "exec", "--index=1", "test", "ps") - res.Assert(t, icmd.Expected{Err: "service \"test\" is not running container #1", ExitCode: 1}) - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "exec", "test", "ps") - res.Assert(t, icmd.Expected{Out: "cat"}) // one-off container was selected - - c.RunDockerComposeCmd(t, "-f", "./fixtures/exec/compose.yaml", "--project-name", projectName, "up", "-d") - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "exec", "test", "ps") - res.Assert(t, icmd.Expected{Out: "tail"}) // service container was selected -} diff --git a/pkg/e2e/export_test.go b/pkg/e2e/export_test.go deleted file mode 100644 index baa0dc5b94c..00000000000 --- a/pkg/e2e/export_test.go +++ /dev/null @@ -1,50 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" -) - -func TestExport(t *testing.T) { - const projectName = "e2e-export-service" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/export/compose.yaml", "--project-name", projectName, "up", "-d", "service") - c.RunDockerComposeCmd(t, "--project-name", projectName, "export", "-o", "service.tar", "service") -} - -func TestExportWithReplicas(t *testing.T) { - const projectName = "e2e-export-service-with-replicas" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/export/compose.yaml", "--project-name", projectName, "up", "-d", "service-with-replicas") - c.RunDockerComposeCmd(t, "--project-name", projectName, "export", "-o", "r1.tar", "--index=1", "service-with-replicas") - c.RunDockerComposeCmd(t, "--project-name", projectName, "export", "-o", "r2.tar", "--index=2", "service-with-replicas") -} diff --git a/pkg/e2e/expose_test.go b/pkg/e2e/expose_test.go deleted file mode 100644 index 438f7b54900..00000000000 --- a/pkg/e2e/expose_test.go +++ /dev/null @@ -1,48 +0,0 @@ -//go:build !windows - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "os" - "path/filepath" - "testing" - - "gotest.tools/v3/assert" -) - -// see https://github.com/docker/compose/issues/13378 -func TestExposeRange(t *testing.T) { - c := NewParallelCLI(t) - - f := filepath.Join(t.TempDir(), "compose.yaml") - err := os.WriteFile(f, []byte(` -name: test-expose-range -services: - test: - image: alpine - expose: - - "9091-9092" -`), 0o644) - assert.NilError(t, err) - - t.Cleanup(func() { - c.cleanupWithDown(t, "test-expose-range") - }) - c.RunDockerComposeCmd(t, "-f", f, "up") -} diff --git a/pkg/e2e/fixtures/attach-restart/compose.yaml b/pkg/e2e/fixtures/attach-restart/compose.yaml deleted file mode 100644 index d92143677fd..00000000000 --- a/pkg/e2e/fixtures/attach-restart/compose.yaml +++ /dev/null @@ -1,8 +0,0 @@ -services: - failing: - image: alpine - command: sh -c "sleep 0.1 && echo world && /bin/false" - deploy: - restart_policy: - condition: "on-failure" - max_attempts: 2 diff --git a/pkg/e2e/fixtures/bridge/Dockerfile b/pkg/e2e/fixtures/bridge/Dockerfile deleted file mode 100644 index 4cdd9857779..00000000000 --- a/pkg/e2e/fixtures/bridge/Dockerfile +++ /dev/null @@ -1,18 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine -ENV ENV_FROM_DOCKERFILE=1 -EXPOSE 8081 -CMD ["echo", "Hello from Dockerfile"] diff --git a/pkg/e2e/fixtures/bridge/compose.yaml b/pkg/e2e/fixtures/bridge/compose.yaml deleted file mode 100644 index 4fbd9bd94cf..00000000000 --- a/pkg/e2e/fixtures/bridge/compose.yaml +++ /dev/null @@ -1,31 +0,0 @@ -services: - serviceA: - image: alpine - build: . - ports: - - 80:8080 - networks: - - private-network - configs: - - source: my-config - target: /etc/my-config1.txt - serviceB: - image: alpine - build: . - ports: - - 8081:8082 - secrets: - - my-secrets - networks: - - private-network - - public-network -configs: - my-config: - file: my-config.txt -secrets: - my-secrets: - file: not-so-secret.txt -networks: - private-network: - internal: true - public-network: {} diff --git a/pkg/e2e/fixtures/bridge/expected-helm/Chart.yaml b/pkg/e2e/fixtures/bridge/expected-helm/Chart.yaml deleted file mode 100755 index 44a00001138..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/Chart.yaml +++ /dev/null @@ -1,12 +0,0 @@ -#! Chart.yaml -apiVersion: v2 -name: bridge -version: 0.0.1 -# kubeVersion: >= 1.29.1 -description: A generated Helm Chart for bridge generated via compose-bridge. -type: application -keywords: - - bridge -appVersion: 'v0.0.1' -sources: -annotations: diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/0-bridge-namespace.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/0-bridge-namespace.yaml deleted file mode 100755 index 953ebe7bb12..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/0-bridge-namespace.yaml +++ /dev/null @@ -1,8 +0,0 @@ -#! 0-bridge-namespace.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Namespace -metadata: - name: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/bridge-configs.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/bridge-configs.yaml deleted file mode 100755 index 48e8b0cf6ac..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/bridge-configs.yaml +++ /dev/null @@ -1,12 +0,0 @@ -#! bridge-configs.yaml -# Generated code, do not edit -apiVersion: v1 -kind: ConfigMap -metadata: - name: {{ .Values.projectName }} - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge -data: - my-config: | - My config file diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/my-secrets-secret.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/my-secrets-secret.yaml deleted file mode 100755 index 63659713ba7..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/my-secrets-secret.yaml +++ /dev/null @@ -1,13 +0,0 @@ -#! my-secrets-secret.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Secret -metadata: - name: my-secrets - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.secret: my-secrets -data: - my-secrets: bm90LXNlY3JldA== -type: Opaque diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/private-network-network-policy.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/private-network-network-policy.yaml deleted file mode 100755 index 0300049be68..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/private-network-network-policy.yaml +++ /dev/null @@ -1,24 +0,0 @@ -#! private-network-network-policy.yaml -# Generated code, do not edit -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: private-network-network-policy - namespace: {{ .Values.namespace }} -spec: - podSelector: - matchLabels: - com.docker.compose.network.private-network: "true" - policyTypes: - - Ingress - - Egress - ingress: - - from: - - podSelector: - matchLabels: - com.docker.compose.network.private-network: "true" - egress: - - to: - - podSelector: - matchLabels: - com.docker.compose.network.private-network: "true" diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/public-network-network-policy.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/public-network-network-policy.yaml deleted file mode 100755 index da042b3e8c1..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/public-network-network-policy.yaml +++ /dev/null @@ -1,24 +0,0 @@ -#! public-network-network-policy.yaml -# Generated code, do not edit -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: public-network-network-policy - namespace: {{ .Values.namespace }} -spec: - podSelector: - matchLabels: - com.docker.compose.network.public-network: "true" - policyTypes: - - Ingress - - Egress - ingress: - - from: - - podSelector: - matchLabels: - com.docker.compose.network.public-network: "true" - egress: - - to: - - podSelector: - matchLabels: - com.docker.compose.network.public-network: "true" diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-deployment.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-deployment.yaml deleted file mode 100755 index afef74e8bad..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-deployment.yaml +++ /dev/null @@ -1,49 +0,0 @@ -#! serviceA-deployment.yaml -# Generated code, do not edit -apiVersion: apps/v1 -kind: Deployment -metadata: - name: servicea - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - app.kubernetes.io/managed-by: Helm -spec: - replicas: {{ .Values.deployment.defaultReplicas }} - selector: - matchLabels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - strategy: - type: {{ .Values.deployment.strategy }} - template: - metadata: - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - com.docker.compose.network.private-network: "true" - spec: - containers: - - name: servicea - image: {{ .Values.serviceA.image }} - imagePullPolicy: {{ .Values.serviceA.imagePullPolicy }} - resources: - limits: - cpu: {{ .Values.resources.defaultCpuLimit }} - memory: {{ .Values.resources.defaultMemoryLimit }} - ports: - - name: servicea-8080 - containerPort: 8080 - volumeMounts: - - name: etc-my-config1-txt - mountPath: /etc/my-config1.txt - subPath: my-config - readOnly: true - volumes: - - name: etc-my-config1-txt - configMap: - name: {{ .Values.projectName }} - items: - - key: my-config - path: my-config diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-expose.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-expose.yaml deleted file mode 100755 index 5d733bd2245..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-expose.yaml +++ /dev/null @@ -1,19 +0,0 @@ -#! serviceA-expose.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: servicea - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - app.kubernetes.io/managed-by: Helm -spec: - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - ports: - - name: servicea-8080 - port: 8080 - targetPort: servicea-8080 diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-service.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-service.yaml deleted file mode 100755 index 2138281ba93..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceA-service.yaml +++ /dev/null @@ -1,25 +0,0 @@ -# check if there is at least one published port - -#! serviceA-service.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: servicea-published - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - app.kubernetes.io/managed-by: Helm -spec: - type: {{ .Values.service.type }} - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - ports: - - name: servicea-80 - port: 80 - protocol: TCP - targetPort: servicea-8080 - -# check if there is at least one published port diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-deployment.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-deployment.yaml deleted file mode 100755 index 7ea9d998f7f..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-deployment.yaml +++ /dev/null @@ -1,50 +0,0 @@ -#! serviceB-deployment.yaml -# Generated code, do not edit -apiVersion: apps/v1 -kind: Deployment -metadata: - name: serviceb - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - app.kubernetes.io/managed-by: Helm -spec: - replicas: {{ .Values.deployment.defaultReplicas }} - selector: - matchLabels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - strategy: - type: {{ .Values.deployment.strategy }} - template: - metadata: - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - com.docker.compose.network.private-network: "true" - com.docker.compose.network.public-network: "true" - spec: - containers: - - name: serviceb - image: {{ .Values.serviceB.image }} - imagePullPolicy: {{ .Values.serviceB.imagePullPolicy }} - resources: - limits: - cpu: {{ .Values.resources.defaultCpuLimit }} - memory: {{ .Values.resources.defaultMemoryLimit }} - ports: - - name: serviceb-8082 - containerPort: 8082 - volumeMounts: - - name: run-secrets-my-secrets - mountPath: /run/secrets/my-secrets - subPath: my-secrets - readOnly: true - volumes: - - name: run-secrets-my-secrets - secret: - secretName: my-secrets - items: - - key: my-secrets - path: my-secrets diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-expose.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-expose.yaml deleted file mode 100755 index f413254dca0..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-expose.yaml +++ /dev/null @@ -1,19 +0,0 @@ -#! serviceB-expose.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: serviceb - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - app.kubernetes.io/managed-by: Helm -spec: - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - ports: - - name: serviceb-8082 - port: 8082 - targetPort: serviceb-8082 diff --git a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-service.yaml b/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-service.yaml deleted file mode 100755 index 6860f3d2804..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/templates/serviceB-service.yaml +++ /dev/null @@ -1,21 +0,0 @@ -#! serviceB-service.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: serviceb-published - namespace: {{ .Values.namespace }} - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - app.kubernetes.io/managed-by: Helm -spec: - type: {{ .Values.service.type }} - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - ports: - - name: serviceb-8081 - port: 8081 - protocol: TCP - targetPort: serviceb-8082 diff --git a/pkg/e2e/fixtures/bridge/expected-helm/values.yaml b/pkg/e2e/fixtures/bridge/expected-helm/values.yaml deleted file mode 100755 index 78315927666..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-helm/values.yaml +++ /dev/null @@ -1,30 +0,0 @@ -#! values.yaml -# Project Name -projectName: bridge -# Namespace -namespace: bridge -# Default deployment settings -deployment: - strategy: Recreate - defaultReplicas: 1 -# Default resource limits -resources: - defaultCpuLimit: "100m" - defaultMemoryLimit: "512Mi" -# Service settings -service: - type: LoadBalancer -# Storage settings -storage: - defaultStorageClass: "hostpath" - defaultSize: "100Mi" - defaultAccessMode: "ReadWriteOnce" -# Services variables -serviceA: - image: alpine - imagePullPolicy: IfNotPresent -serviceB: - image: alpine - imagePullPolicy: IfNotPresent - -# You can apply the same logic to loop on networks, volumes, secrets and configs... diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/0-bridge-namespace.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/0-bridge-namespace.yaml deleted file mode 100755 index 40e4b0e23f4..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/0-bridge-namespace.yaml +++ /dev/null @@ -1,8 +0,0 @@ -#! 0-bridge-namespace.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Namespace -metadata: - name: bridge - labels: - com.docker.compose.project: bridge diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/bridge-configs.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/bridge-configs.yaml deleted file mode 100755 index 822d2e076ef..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/bridge-configs.yaml +++ /dev/null @@ -1,12 +0,0 @@ -#! bridge-configs.yaml -# Generated code, do not edit -apiVersion: v1 -kind: ConfigMap -metadata: - name: bridge - namespace: bridge - labels: - com.docker.compose.project: bridge -data: - my-config: | - My config file diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/kustomization.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/kustomization.yaml deleted file mode 100755 index ff8428feae2..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/kustomization.yaml +++ /dev/null @@ -1,16 +0,0 @@ -#! kustomization.yaml -# Generated code, do not edit -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization -resources: - - 0-bridge-namespace.yaml - - bridge-configs.yaml - - my-secrets-secret.yaml - - private-network-network-policy.yaml - - public-network-network-policy.yaml - - serviceA-deployment.yaml - - serviceA-expose.yaml - - serviceA-service.yaml - - serviceB-deployment.yaml - - serviceB-expose.yaml - - serviceB-service.yaml diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/my-secrets-secret.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/my-secrets-secret.yaml deleted file mode 100755 index 559eba6a26e..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/my-secrets-secret.yaml +++ /dev/null @@ -1,13 +0,0 @@ -#! my-secrets-secret.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Secret -metadata: - name: my-secrets - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.secret: my-secrets -data: - my-secrets: bm90LXNlY3JldA== -type: Opaque diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/private-network-network-policy.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/private-network-network-policy.yaml deleted file mode 100755 index 3f59b22dd9d..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/private-network-network-policy.yaml +++ /dev/null @@ -1,24 +0,0 @@ -#! private-network-network-policy.yaml -# Generated code, do not edit -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: private-network-network-policy - namespace: bridge -spec: - podSelector: - matchLabels: - com.docker.compose.network.private-network: "true" - policyTypes: - - Ingress - - Egress - ingress: - - from: - - podSelector: - matchLabels: - com.docker.compose.network.private-network: "true" - egress: - - to: - - podSelector: - matchLabels: - com.docker.compose.network.private-network: "true" diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/public-network-network-policy.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/public-network-network-policy.yaml deleted file mode 100755 index 04913d4b9a7..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/public-network-network-policy.yaml +++ /dev/null @@ -1,24 +0,0 @@ -#! public-network-network-policy.yaml -# Generated code, do not edit -apiVersion: networking.k8s.io/v1 -kind: NetworkPolicy -metadata: - name: public-network-network-policy - namespace: bridge -spec: - podSelector: - matchLabels: - com.docker.compose.network.public-network: "true" - policyTypes: - - Ingress - - Egress - ingress: - - from: - - podSelector: - matchLabels: - com.docker.compose.network.public-network: "true" - egress: - - to: - - podSelector: - matchLabels: - com.docker.compose.network.public-network: "true" diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-deployment.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-deployment.yaml deleted file mode 100755 index 0779cf56268..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-deployment.yaml +++ /dev/null @@ -1,44 +0,0 @@ -#! serviceA-deployment.yaml -# Generated code, do not edit -apiVersion: apps/v1 -kind: Deployment -metadata: - name: servicea - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA -spec: - replicas: 1 - selector: - matchLabels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - strategy: - type: Recreate - template: - metadata: - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - com.docker.compose.network.private-network: "true" - spec: - containers: - - name: servicea - image: alpine - imagePullPolicy: IfNotPresent - ports: - - name: servicea-8080 - containerPort: 8080 - volumeMounts: - - name: etc-my-config1-txt - mountPath: /etc/my-config1.txt - subPath: my-config - readOnly: true - volumes: - - name: etc-my-config1-txt - configMap: - name: bridge - items: - - key: my-config - path: my-config diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-expose.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-expose.yaml deleted file mode 100755 index d0bd013ecff..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-expose.yaml +++ /dev/null @@ -1,18 +0,0 @@ -#! serviceA-expose.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: servicea - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA -spec: - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - ports: - - name: servicea-8080 - port: 8080 - targetPort: servicea-8080 diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-service.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-service.yaml deleted file mode 100755 index 628cf04189c..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceA-service.yaml +++ /dev/null @@ -1,23 +0,0 @@ -# check if there is at least one published port - -#! serviceA-service.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: servicea-published - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA -spec: - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceA - ports: - - name: servicea-80 - port: 80 - protocol: TCP - targetPort: servicea-8080 - -# check if there is at least one published port diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-deployment.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-deployment.yaml deleted file mode 100755 index 191720c2014..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-deployment.yaml +++ /dev/null @@ -1,45 +0,0 @@ -#! serviceB-deployment.yaml -# Generated code, do not edit -apiVersion: apps/v1 -kind: Deployment -metadata: - name: serviceb - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB -spec: - replicas: 1 - selector: - matchLabels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - strategy: - type: Recreate - template: - metadata: - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - com.docker.compose.network.private-network: "true" - com.docker.compose.network.public-network: "true" - spec: - containers: - - name: serviceb - image: alpine - imagePullPolicy: IfNotPresent - ports: - - name: serviceb-8082 - containerPort: 8082 - volumeMounts: - - name: run-secrets-my-secrets - mountPath: /run/secrets/my-secrets - subPath: my-secrets - readOnly: true - volumes: - - name: run-secrets-my-secrets - secret: - secretName: my-secrets - items: - - key: my-secrets - path: my-secrets diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-expose.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-expose.yaml deleted file mode 100755 index 2025868991d..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-expose.yaml +++ /dev/null @@ -1,18 +0,0 @@ -#! serviceB-expose.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: serviceb - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB -spec: - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - ports: - - name: serviceb-8082 - port: 8082 - targetPort: serviceb-8082 diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-service.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-service.yaml deleted file mode 100755 index 94104185871..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/base/serviceB-service.yaml +++ /dev/null @@ -1,19 +0,0 @@ -#! serviceB-service.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: serviceb-published - namespace: bridge - labels: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB -spec: - selector: - com.docker.compose.project: bridge - com.docker.compose.service: serviceB - ports: - - name: serviceb-8081 - port: 8081 - protocol: TCP - targetPort: serviceb-8082 diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/kustomization.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/kustomization.yaml deleted file mode 100755 index a192e45f0fe..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/kustomization.yaml +++ /dev/null @@ -1,9 +0,0 @@ -#! kustomization.yaml -# Generated code, do not edit -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization -resources: - - ../../base -patches: - - path: serviceA-service.yaml - - path: serviceB-service.yaml diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/serviceA-service.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/serviceA-service.yaml deleted file mode 100755 index 6453b5adba3..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/serviceA-service.yaml +++ /dev/null @@ -1,13 +0,0 @@ -# check if there is at least one published port - -#! serviceA-service.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: servicea-published - namespace: bridge -spec: - type: LoadBalancer - -# check if there is at least one published port diff --git a/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/serviceB-service.yaml b/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/serviceB-service.yaml deleted file mode 100755 index f21b674336b..00000000000 --- a/pkg/e2e/fixtures/bridge/expected-kubernetes/overlays/desktop/serviceB-service.yaml +++ /dev/null @@ -1,9 +0,0 @@ -#! serviceB-service.yaml -# Generated code, do not edit -apiVersion: v1 -kind: Service -metadata: - name: serviceb-published - namespace: bridge -spec: - type: LoadBalancer diff --git a/pkg/e2e/fixtures/bridge/my-config.txt b/pkg/e2e/fixtures/bridge/my-config.txt deleted file mode 100644 index 24d11e40bb8..00000000000 --- a/pkg/e2e/fixtures/bridge/my-config.txt +++ /dev/null @@ -1 +0,0 @@ -My config file \ No newline at end of file diff --git a/pkg/e2e/fixtures/bridge/not-so-secret.txt b/pkg/e2e/fixtures/bridge/not-so-secret.txt deleted file mode 100644 index 4e76a78aebb..00000000000 --- a/pkg/e2e/fixtures/bridge/not-so-secret.txt +++ /dev/null @@ -1 +0,0 @@ -not-secret \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-dependencies/base.dockerfile b/pkg/e2e/fixtures/build-dependencies/base.dockerfile deleted file mode 100644 index 9dce0b74f41..00000000000 --- a/pkg/e2e/fixtures/build-dependencies/base.dockerfile +++ /dev/null @@ -1,19 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine - -COPY hello.txt /hello.txt - -CMD [ "/bin/true" ] diff --git a/pkg/e2e/fixtures/build-dependencies/classic.yaml b/pkg/e2e/fixtures/build-dependencies/classic.yaml deleted file mode 100644 index b0dbbaad0a1..00000000000 --- a/pkg/e2e/fixtures/build-dependencies/classic.yaml +++ /dev/null @@ -1,14 +0,0 @@ -services: - base: - image: base - init: true - build: - context: . - dockerfile: base.dockerfile - service: - init: true - depends_on: - - base - build: - context: . - dockerfile: service.dockerfile diff --git a/pkg/e2e/fixtures/build-dependencies/compose-depends_on.yaml b/pkg/e2e/fixtures/build-dependencies/compose-depends_on.yaml deleted file mode 100644 index 90b2beaef18..00000000000 --- a/pkg/e2e/fixtures/build-dependencies/compose-depends_on.yaml +++ /dev/null @@ -1,15 +0,0 @@ -services: - test1: - pull_policy: build - build: - dockerfile_inline: FROM alpine - command: - - echo - - "test 1 success" - test2: - image: alpine - depends_on: - - test1 - command: - - echo - - "test 2 success" \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-dependencies/compose.yaml b/pkg/e2e/fixtures/build-dependencies/compose.yaml deleted file mode 100644 index 952a7199ef8..00000000000 --- a/pkg/e2e/fixtures/build-dependencies/compose.yaml +++ /dev/null @@ -1,13 +0,0 @@ -services: - base: - init: true - build: - context: . - dockerfile: base.dockerfile - service: - init: true - build: - context: . - additional_contexts: - base: "service:base" - dockerfile: service.dockerfile diff --git a/pkg/e2e/fixtures/build-dependencies/hello.txt b/pkg/e2e/fixtures/build-dependencies/hello.txt deleted file mode 100644 index 810e7ba64ac..00000000000 --- a/pkg/e2e/fixtures/build-dependencies/hello.txt +++ /dev/null @@ -1 +0,0 @@ -this file was copied from base -> service diff --git a/pkg/e2e/fixtures/build-dependencies/service.dockerfile b/pkg/e2e/fixtures/build-dependencies/service.dockerfile deleted file mode 100644 index 8c710b57321..00000000000 --- a/pkg/e2e/fixtures/build-dependencies/service.dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM base - -CMD [ "cat", "/hello.txt" ] diff --git a/pkg/e2e/fixtures/build-infinite/compose.yaml b/pkg/e2e/fixtures/build-infinite/compose.yaml deleted file mode 100644 index cdc1e169357..00000000000 --- a/pkg/e2e/fixtures/build-infinite/compose.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - service1: - build: service1 \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-infinite/service1/Dockerfile b/pkg/e2e/fixtures/build-infinite/service1/Dockerfile deleted file mode 100644 index 3fd64e7b655..00000000000 --- a/pkg/e2e/fixtures/build-infinite/service1/Dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM busybox - -RUN sleep infinity \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/compose.yaml b/pkg/e2e/fixtures/build-test/compose.yaml deleted file mode 100644 index 2db602a2d8f..00000000000 --- a/pkg/e2e/fixtures/build-test/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - nginx: - build: nginx-build - ports: - - 8070:80 - - nginx2: - build: nginx-build2 - image: custom-nginx diff --git a/pkg/e2e/fixtures/build-test/dependencies/compose.yaml b/pkg/e2e/fixtures/build-test/dependencies/compose.yaml deleted file mode 100644 index eb5de31f943..00000000000 --- a/pkg/e2e/fixtures/build-test/dependencies/compose.yaml +++ /dev/null @@ -1,26 +0,0 @@ -services: - firstbuild: - build: - dockerfile_inline: | - FROM alpine - additional_contexts: - dep1: service:dep1 - entrypoint: ["echo", "Hello from firstbuild"] - depends_on: - - dep1 - - secondbuild: - build: - dockerfile_inline: | - FROM alpine - additional_contexts: - dep1: service:dep1 - entrypoint: ["echo", "Hello from secondbuild"] - depends_on: - - dep1 - - dep1: - build: - dockerfile_inline: | - FROM alpine - entrypoint: ["echo", "Hello from dep1"] \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/entitlements/Dockerfile b/pkg/e2e/fixtures/build-test/entitlements/Dockerfile deleted file mode 100644 index a242eb52d4b..00000000000 --- a/pkg/e2e/fixtures/build-test/entitlements/Dockerfile +++ /dev/null @@ -1,19 +0,0 @@ -# syntax = docker/dockerfile:experimental - - -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine -RUN --security=insecure cat /proc/self/status | grep CapEff diff --git a/pkg/e2e/fixtures/build-test/entitlements/compose.yaml b/pkg/e2e/fixtures/build-test/entitlements/compose.yaml deleted file mode 100644 index 403529e6477..00000000000 --- a/pkg/e2e/fixtures/build-test/entitlements/compose.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - privileged-service: - build: - context: . - entitlements: - - security.insecure - diff --git a/pkg/e2e/fixtures/build-test/escaped/Dockerfile b/pkg/e2e/fixtures/build-test/escaped/Dockerfile deleted file mode 100644 index dd507f4fffd..00000000000 --- a/pkg/e2e/fixtures/build-test/escaped/Dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine -ARG foo -RUN echo foo is $foo diff --git a/pkg/e2e/fixtures/build-test/escaped/compose.yaml b/pkg/e2e/fixtures/build-test/escaped/compose.yaml deleted file mode 100644 index 2d0077b9e63..00000000000 --- a/pkg/e2e/fixtures/build-test/escaped/compose.yaml +++ /dev/null @@ -1,23 +0,0 @@ -services: - foo: - build: - context: . - args: - foo: $${bar} - - echo: - build: - dockerfile_inline: | - FROM bash - RUN <<'EOF' - echo $(seq 10) - EOF - - arg: - build: - args: - BOOL: "true" - dockerfile_inline: | - FROM alpine:latest - ARG BOOL - RUN /bin/$${BOOL} diff --git a/pkg/e2e/fixtures/build-test/long-output-line/Dockerfile b/pkg/e2e/fixtures/build-test/long-output-line/Dockerfile deleted file mode 100644 index 5227a491434..00000000000 --- a/pkg/e2e/fixtures/build-test/long-output-line/Dockerfile +++ /dev/null @@ -1,49 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -FROM alpine -# We generate warnings *on purpose* to bloat the JSON output of bake -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT -ARG AWS_SECRET_ACCESS_KEY=FAKE_TO_GENERATE_WARNING_OUTPUT \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/long-output-line/compose.yaml b/pkg/e2e/fixtures/build-test/long-output-line/compose.yaml deleted file mode 100644 index d0e2a4fffdc..00000000000 --- a/pkg/e2e/fixtures/build-test/long-output-line/compose.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - long-line: - build: - context: . - dockerfile: Dockerfile diff --git a/pkg/e2e/fixtures/build-test/minimal/Dockerfile b/pkg/e2e/fixtures/build-test/minimal/Dockerfile deleted file mode 100644 index 968515a22ed..00000000000 --- a/pkg/e2e/fixtures/build-test/minimal/Dockerfile +++ /dev/null @@ -1,16 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM scratch -COPY . . \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/minimal/compose.yaml b/pkg/e2e/fixtures/build-test/minimal/compose.yaml deleted file mode 100644 index 8362c60b900..00000000000 --- a/pkg/e2e/fixtures/build-test/minimal/compose.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - test: - build: . \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/multi-args/Dockerfile b/pkg/e2e/fixtures/build-test/multi-args/Dockerfile deleted file mode 100644 index b2c18cdd664..00000000000 --- a/pkg/e2e/fixtures/build-test/multi-args/Dockerfile +++ /dev/null @@ -1,19 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -ARG IMAGE=666 -ARG TAG=666 - -FROM ${IMAGE}:${TAG} -RUN echo "SUCCESS" diff --git a/pkg/e2e/fixtures/build-test/multi-args/compose.yaml b/pkg/e2e/fixtures/build-test/multi-args/compose.yaml deleted file mode 100644 index 8bedfd6384e..00000000000 --- a/pkg/e2e/fixtures/build-test/multi-args/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - multiargs: - build: - context: . - args: - IMAGE: alpine - TAG: latest - labels: - - RESULT=SUCCESS diff --git a/pkg/e2e/fixtures/build-test/nginx-build/Dockerfile b/pkg/e2e/fixtures/build-test/nginx-build/Dockerfile deleted file mode 100644 index dd79c0e4a31..00000000000 --- a/pkg/e2e/fixtures/build-test/nginx-build/Dockerfile +++ /dev/null @@ -1,19 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM nginx:alpine - -ARG FOO -LABEL FOO=$FOO -COPY static /usr/share/nginx/html diff --git a/pkg/e2e/fixtures/build-test/nginx-build/static/index.html b/pkg/e2e/fixtures/build-test/nginx-build/static/index.html deleted file mode 100644 index 914e3406cb9..00000000000 --- a/pkg/e2e/fixtures/build-test/nginx-build/static/index.html +++ /dev/null @@ -1,10 +0,0 @@ -<!doctype html> -<html lang="en"> -<head> - <meta charset="utf-8"> - <title>Static file 2 - - -

Hello from Nginx container

- - diff --git a/pkg/e2e/fixtures/build-test/nginx-build2/Dockerfile b/pkg/e2e/fixtures/build-test/nginx-build2/Dockerfile deleted file mode 100644 index f8a39facaf3..00000000000 --- a/pkg/e2e/fixtures/build-test/nginx-build2/Dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM nginx:alpine - -COPY static2 /usr/share/nginx/html diff --git a/pkg/e2e/fixtures/build-test/nginx-build2/static2/index.html b/pkg/e2e/fixtures/build-test/nginx-build2/static2/index.html deleted file mode 100644 index 914e3406cb9..00000000000 --- a/pkg/e2e/fixtures/build-test/nginx-build2/static2/index.html +++ /dev/null @@ -1,10 +0,0 @@ - - - - - Static file 2 - - -

Hello from Nginx container

- - diff --git a/pkg/e2e/fixtures/build-test/platforms/Dockerfile b/pkg/e2e/fixtures/build-test/platforms/Dockerfile deleted file mode 100644 index ef22c17f6a5..00000000000 --- a/pkg/e2e/fixtures/build-test/platforms/Dockerfile +++ /dev/null @@ -1,22 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM --platform=$BUILDPLATFORM golang:alpine AS build - -ARG TARGETPLATFORM -ARG BUILDPLATFORM -RUN echo "I am building for $TARGETPLATFORM, running on $BUILDPLATFORM" > /log - -FROM alpine -COPY --from=build /log /log diff --git a/pkg/e2e/fixtures/build-test/platforms/compose-multiple-platform-builds.yaml b/pkg/e2e/fixtures/build-test/platforms/compose-multiple-platform-builds.yaml deleted file mode 100644 index aac3a3db90d..00000000000 --- a/pkg/e2e/fixtures/build-test/platforms/compose-multiple-platform-builds.yaml +++ /dev/null @@ -1,23 +0,0 @@ -services: - serviceA: - image: build-test-platform-a:test - build: - context: ./contextServiceA - platforms: - - linux/amd64 - - linux/arm64 - serviceB: - image: build-test-platform-b:test - build: - context: ./contextServiceB - platforms: - - linux/amd64 - - linux/arm64 - serviceC: - image: build-test-platform-c:test - build: - context: ./contextServiceC - platforms: - - linux/amd64 - - linux/arm64 - diff --git a/pkg/e2e/fixtures/build-test/platforms/compose-service-platform-and-no-build-platforms.yaml b/pkg/e2e/fixtures/build-test/platforms/compose-service-platform-and-no-build-platforms.yaml deleted file mode 100644 index 3d0eafbfc24..00000000000 --- a/pkg/e2e/fixtures/build-test/platforms/compose-service-platform-and-no-build-platforms.yaml +++ /dev/null @@ -1,6 +0,0 @@ -services: - platforms: - image: build-test-platform:test - platform: linux/386 - build: - context: . diff --git a/pkg/e2e/fixtures/build-test/platforms/compose-service-platform-not-in-build-platforms.yaml b/pkg/e2e/fixtures/build-test/platforms/compose-service-platform-not-in-build-platforms.yaml deleted file mode 100644 index bed88fa51f3..00000000000 --- a/pkg/e2e/fixtures/build-test/platforms/compose-service-platform-not-in-build-platforms.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - platforms: - image: build-test-platform:test - platform: linux/riscv64 - build: - context: . - platforms: - - linux/amd64 - - linux/arm64 diff --git a/pkg/e2e/fixtures/build-test/platforms/compose-unsupported-platform.yml b/pkg/e2e/fixtures/build-test/platforms/compose-unsupported-platform.yml deleted file mode 100644 index e3342829168..00000000000 --- a/pkg/e2e/fixtures/build-test/platforms/compose-unsupported-platform.yml +++ /dev/null @@ -1,8 +0,0 @@ -services: - platforms: - image: build-test-platform:test - build: - context: . - platforms: - - unsupported/unsupported - - linux/amd64 diff --git a/pkg/e2e/fixtures/build-test/platforms/compose.yaml b/pkg/e2e/fixtures/build-test/platforms/compose.yaml deleted file mode 100644 index 73421f4793f..00000000000 --- a/pkg/e2e/fixtures/build-test/platforms/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - platforms: - image: build-test-platform:test - build: - context: . - platforms: - - linux/amd64 - - linux/arm64 - diff --git a/pkg/e2e/fixtures/build-test/platforms/contextServiceA/Dockerfile b/pkg/e2e/fixtures/build-test/platforms/contextServiceA/Dockerfile deleted file mode 100644 index 468b2b10dd6..00000000000 --- a/pkg/e2e/fixtures/build-test/platforms/contextServiceA/Dockerfile +++ /dev/null @@ -1,22 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM --platform=$BUILDPLATFORM golang:alpine AS build - -ARG TARGETPLATFORM -ARG BUILDPLATFORM -RUN echo "I'm Service A and I am building for $TARGETPLATFORM, running on $BUILDPLATFORM" > /log - -FROM alpine -COPY --from=build /log /log diff --git a/pkg/e2e/fixtures/build-test/platforms/contextServiceB/Dockerfile b/pkg/e2e/fixtures/build-test/platforms/contextServiceB/Dockerfile deleted file mode 100644 index cfa2ae34ad7..00000000000 --- a/pkg/e2e/fixtures/build-test/platforms/contextServiceB/Dockerfile +++ /dev/null @@ -1,22 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM --platform=$BUILDPLATFORM golang:alpine AS build - -ARG TARGETPLATFORM -ARG BUILDPLATFORM -RUN echo "I'm Service B and I am building for $TARGETPLATFORM, running on $BUILDPLATFORM" > /log - -FROM alpine -COPY --from=build /log /log diff --git a/pkg/e2e/fixtures/build-test/platforms/contextServiceC/Dockerfile b/pkg/e2e/fixtures/build-test/platforms/contextServiceC/Dockerfile deleted file mode 100644 index 3216f618295..00000000000 --- a/pkg/e2e/fixtures/build-test/platforms/contextServiceC/Dockerfile +++ /dev/null @@ -1,22 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM --platform=$BUILDPLATFORM golang:alpine AS build - -ARG TARGETPLATFORM -ARG BUILDPLATFORM -RUN echo "I'm Service C and I am building for $TARGETPLATFORM, running on $BUILDPLATFORM" > /log - -FROM alpine -COPY --from=build /log /log diff --git a/pkg/e2e/fixtures/build-test/privileged/Dockerfile b/pkg/e2e/fixtures/build-test/privileged/Dockerfile deleted file mode 100644 index a242eb52d4b..00000000000 --- a/pkg/e2e/fixtures/build-test/privileged/Dockerfile +++ /dev/null @@ -1,19 +0,0 @@ -# syntax = docker/dockerfile:experimental - - -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine -RUN --security=insecure cat /proc/self/status | grep CapEff diff --git a/pkg/e2e/fixtures/build-test/privileged/compose.yaml b/pkg/e2e/fixtures/build-test/privileged/compose.yaml deleted file mode 100644 index ead867cae83..00000000000 --- a/pkg/e2e/fixtures/build-test/privileged/compose.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - privileged-service: - build: - context: . - privileged: true diff --git a/pkg/e2e/fixtures/build-test/profiles/Dockerfile b/pkg/e2e/fixtures/build-test/profiles/Dockerfile deleted file mode 100644 index 94eb80e9c7d..00000000000 --- a/pkg/e2e/fixtures/build-test/profiles/Dockerfile +++ /dev/null @@ -1,19 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - - -FROM alpine -RUN --mount=type=secret,id=test-secret ls -la /run/secrets/; cp /run/secrets/test-secret /tmp - -CMD ["cat", "/tmp/test-secret"] \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/profiles/compose.yaml b/pkg/e2e/fixtures/build-test/profiles/compose.yaml deleted file mode 100644 index 877babc0e5f..00000000000 --- a/pkg/e2e/fixtures/build-test/profiles/compose.yaml +++ /dev/null @@ -1,12 +0,0 @@ -secrets: - test-secret: - file: test-secret.txt - -services: - secret-build-test: - profiles: ["test"] - build: - context: . - dockerfile: Dockerfile - secrets: - - test-secret \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/profiles/test-secret.txt b/pkg/e2e/fixtures/build-test/profiles/test-secret.txt deleted file mode 100644 index 78882121907..00000000000 --- a/pkg/e2e/fixtures/build-test/profiles/test-secret.txt +++ /dev/null @@ -1 +0,0 @@ -SECRET \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/secrets/.env b/pkg/e2e/fixtures/build-test/secrets/.env deleted file mode 100644 index 9f8bc4f5d23..00000000000 --- a/pkg/e2e/fixtures/build-test/secrets/.env +++ /dev/null @@ -1 +0,0 @@ -ANOTHER_SECRET=zot \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/secrets/Dockerfile b/pkg/e2e/fixtures/build-test/secrets/Dockerfile deleted file mode 100644 index 336673b050c..00000000000 --- a/pkg/e2e/fixtures/build-test/secrets/Dockerfile +++ /dev/null @@ -1,30 +0,0 @@ -# syntax=docker/dockerfile:1 - - -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine - -RUN echo "foo" > /tmp/expected -RUN --mount=type=secret,id=mysecret cat /run/secrets/mysecret > /tmp/actual -RUN diff /tmp/expected /tmp/actual - -RUN echo "bar" > /tmp/expected -RUN --mount=type=secret,id=build_secret cat /run/secrets/build_secret > tmp/actual -RUN diff --ignore-all-space /tmp/expected /tmp/actual - -RUN echo "zot" > /tmp/expected -RUN --mount=type=secret,id=dotenvsecret cat /run/secrets/dotenvsecret > tmp/actual -RUN diff --ignore-all-space /tmp/expected /tmp/actual diff --git a/pkg/e2e/fixtures/build-test/secrets/compose.yml b/pkg/e2e/fixtures/build-test/secrets/compose.yml deleted file mode 100644 index f041acf6101..00000000000 --- a/pkg/e2e/fixtures/build-test/secrets/compose.yml +++ /dev/null @@ -1,18 +0,0 @@ -services: - ssh: - image: build-test-secret - build: - context: . - secrets: - - mysecret - - dotenvsecret - - source: envsecret - target: build_secret - -secrets: - mysecret: - file: ./secret.txt - envsecret: - environment: SOME_SECRET - dotenvsecret: - environment: ANOTHER_SECRET diff --git a/pkg/e2e/fixtures/build-test/secrets/secret.txt b/pkg/e2e/fixtures/build-test/secrets/secret.txt deleted file mode 100644 index 257cc5642cb..00000000000 --- a/pkg/e2e/fixtures/build-test/secrets/secret.txt +++ /dev/null @@ -1 +0,0 @@ -foo diff --git a/pkg/e2e/fixtures/build-test/ssh/Dockerfile b/pkg/e2e/fixtures/build-test/ssh/Dockerfile deleted file mode 100644 index d2fe8e5b895..00000000000 --- a/pkg/e2e/fixtures/build-test/ssh/Dockerfile +++ /dev/null @@ -1,24 +0,0 @@ -# syntax=docker/dockerfile:1 - - -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine -RUN apk add --no-cache openssh-client - -WORKDIR /compose -COPY fake_rsa.pub /compose/ - -RUN --mount=type=ssh,id=fake-ssh,required=true diff <(ssh-add -L) <(cat /compose/fake_rsa.pub) \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/ssh/compose-without-ssh.yaml b/pkg/e2e/fixtures/build-test/ssh/compose-without-ssh.yaml deleted file mode 100644 index cce1bb88a30..00000000000 --- a/pkg/e2e/fixtures/build-test/ssh/compose-without-ssh.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - ssh: - image: build-test-ssh - build: - context: . diff --git a/pkg/e2e/fixtures/build-test/ssh/compose.yaml b/pkg/e2e/fixtures/build-test/ssh/compose.yaml deleted file mode 100644 index 2fd56ab1494..00000000000 --- a/pkg/e2e/fixtures/build-test/ssh/compose.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - ssh: - image: build-test-ssh - build: - context: . - ssh: - - fake-ssh=./fake_rsa diff --git a/pkg/e2e/fixtures/build-test/ssh/fake_rsa b/pkg/e2e/fixtures/build-test/ssh/fake_rsa deleted file mode 100644 index 0e797265bb0..00000000000 --- a/pkg/e2e/fixtures/build-test/ssh/fake_rsa +++ /dev/null @@ -1,49 +0,0 @@ ------BEGIN OPENSSH PRIVATE KEY----- -b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAACFwAAAAdzc2gtcn -NhAAAAAwEAAQAAAgEA7nJ4xAhJ7VwI63tuay3DCHaTXeEY92H6YNZ8ptAIBY0mUn6Gc9ms -94HvcAKemCJkO0fy6U2JOoST+q1YPAJf86NrIU41hZdzrw2QdqG/A3ja4VTAaOJbH9wafK -HpWLs6kyigGti3KSBabm4HARU8lgtRE6AuCC1+mw821FzTsMWMxRp/rKVxgsiMUsdd57WR -KOdn8TRm6NHcEsy7X7zAJ7+Ch/muGGCCk3Z9+YUzoVVtY/wGYmWXXj/NUzxnEq0XLyO8HC -+QU/9dWlh1OLmoMuxN1lYtHRFWWstCboKNsOcIiJsLKfQ1t4z4jXq5P7JTLE5Pngemrr4x -K21RFjVaGQpOjyQgZn1o0wAvy78KORwgN0Elwcb/XIKJepzzezCIyXlSafeXuHP+oMjM2s -2MXNHlMKv6Jwh4QYwUQ61+bAcPkcmIdltiAMNLcxYiqEud85EQQl9ciuhMKa0bZl1OEILw -VSIasEu9BEKVrz52ZZVLGMchqOV/4f1PqPEagnfnRYEttJ6AuaYUaJXvSQP6Zj4AFb6WrP -wEBIFOuAH9i4WtG52QAK6uc1wsPZlHm8J+VnTEBKFuGERu/uJBWPo43Lju8VrHuZU8QeON -ERKfJbc1EI9XpqWi+3VcWT0QJtxEGW2YmD505+cKNc31xwOtcqwogtwT0wnuj0BAf33HY3 -8AAAc465v1nOub9ZwAAAAHc3NoLXJzYQAAAgEA7nJ4xAhJ7VwI63tuay3DCHaTXeEY92H6 -YNZ8ptAIBY0mUn6Gc9ms94HvcAKemCJkO0fy6U2JOoST+q1YPAJf86NrIU41hZdzrw2Qdq -G/A3ja4VTAaOJbH9wafKHpWLs6kyigGti3KSBabm4HARU8lgtRE6AuCC1+mw821FzTsMWM -xRp/rKVxgsiMUsdd57WRKOdn8TRm6NHcEsy7X7zAJ7+Ch/muGGCCk3Z9+YUzoVVtY/wGYm -WXXj/NUzxnEq0XLyO8HC+QU/9dWlh1OLmoMuxN1lYtHRFWWstCboKNsOcIiJsLKfQ1t4z4 -jXq5P7JTLE5Pngemrr4xK21RFjVaGQpOjyQgZn1o0wAvy78KORwgN0Elwcb/XIKJepzzez -CIyXlSafeXuHP+oMjM2s2MXNHlMKv6Jwh4QYwUQ61+bAcPkcmIdltiAMNLcxYiqEud85EQ -Ql9ciuhMKa0bZl1OEILwVSIasEu9BEKVrz52ZZVLGMchqOV/4f1PqPEagnfnRYEttJ6Aua -YUaJXvSQP6Zj4AFb6WrPwEBIFOuAH9i4WtG52QAK6uc1wsPZlHm8J+VnTEBKFuGERu/uJB -WPo43Lju8VrHuZU8QeONERKfJbc1EI9XpqWi+3VcWT0QJtxEGW2YmD505+cKNc31xwOtcq -wogtwT0wnuj0BAf33HY38AAAADAQABAAACAGK7A0YoKHQfp5HZid7XE+ptLpewnKXR69os -9XAcszWZPETsHr/ZYcUaCApZC1Hy642gPPRdJnUUcDFblS1DzncTM0iXGZI3I69X7nkwf+ -bwI7EpZoIHN7P5bv4sDHKxE4/bQm/bS/u7abZP2JaaNHvsM6XsrSK1s7aAljNYPE71fVQf -pL3Xwyhj4bZk1n0asQA+0MsO541/V6BxJSR/AxFyOpoSyANP8sEcTw0CGl6zAJhlwj770b -E0uc+9MvCIuxDJuxnwl9Iv6nd+KQtT1FFBhvk4tXVTuG3fu6IGbKTTBLWLfRPiClv2AvSR -3CKDs+ykgFLu2BWCqtlQakLH1IW9DTkPExV4ZjkGCRWHEvmJxxOqL6B48tBjwa5gBuPJRA -aYRi15Z3sprsqCBfp+aHPkMXkkNGSe5ROj8lFFY/f50ZS/9DSlyuUURFLtIGe5XuPNJk7L -xJkYJAdNbgvk4IPgzsU2EuYvSja5mtuo3dVyEIRtsIAN4xl01edDAxHEow6ar4gZCtXnBb -WqeqchEi4zVTdkkuDP3SF362pktdY7Op0mS/yFd8LFrca3VCy2PqNhKvlxClRqM9Tlp9cY -qDuyS9AGT1QO4BMtvSJGFa3P+h76rQsNldC+nGa4wNWvpAUcT5NS8W9QnGp7ah/qOK07t7 -fwYYENeRaAK3OItBABAAABAFjyDlnERaZ+/23B+zN0kQhCvmiNS5HE2+ooR5ofX08F3Uar -VPevy9p6s2LA+AlXY1ZZ1k0p5MI+4TkAbcB/VXaxrRUw9633p9rAgyumFGhK3i0M4whOCO -MJxmlp5sz5Qea+YzIa9z0F4ZwwvdHt7cp5joYBZoQ+Kv9OUy4xCs1zZ4ZbEsakGBrtLiTo -H3odXSg0mXQf10Ae3WkvAJ8M1xL/z1ryFeCvyv1sGwEx+5gvmZ6nnuJEEuXUBlpOwhPlST -4X9VL7gmdH9OoHnhUn3q2JEBQdVTegGij9wvoYT1bdzwBN/Amisn29K9w1aNdrNbYUJ6PO -0kE2lotSJ11qD8MAAAEBAP6IRuU25yj7zv0mEsaRWoQ5v3fYKKn4C6Eg3DbzKXybZkLyX7 -6QlyO7uWf54SdXM7sQW8KoXaMu9qbo/o+4o3m7YfOY1MYeTz3yICYObVA7Fc9ZHwKzc1PB -dFNzy6/G+2niNQF3Q1Fjp31Ve9LwKJK8Kj/eUYZ3QiUIropkw4ppA8q3h+nkVGS23xSrTM -kGLugBjcnWUfuN0tKx/b5mqziRoyzr5u0qzFDtx97QAyETo/onFrd1bMGED2BHVyrCwtqI -p6SXo2uFzwm/nLtOMlmfpixNcK6dtql/brx3Lsu18a+0a42O5Q/TYRdRq8D60O16rUS/LN -sFOjIYSA3spnUAAAEBAO/Sc3NTarFylk+yhOTE8G9xDt5ndbY0gsfhM9D4byKlY4yYIvs+ -yQAq3UHgSoN2f087zNubXSNiLJ8TOIPpbk8MzdvjqcpmnBhHcd4V2FLe9+hC8zEBf8MPPf -Cy1kXdCZ0bDMLTdgONiDTIc/0YXhFLZherXNIF1o/7Pcnu6IPwMDl/gcG3H1ncDxaLqxAm -L29SDXLX2hH9k+YJr9kFaho7PZBAwNYnMooupROSbQ9/lmfCt09ep/83n5G0mo93uGkyV2 -1wcQw9X2ZT8eVHZ4ni3ACC6VYbUn2M3Z+e3tpGaYzKXd/yq0YyppoRvEaxM/ewXappUJul -Xsd/RqSc66MAAAAAAQID ------END OPENSSH PRIVATE KEY----- diff --git a/pkg/e2e/fixtures/build-test/ssh/fake_rsa.pub b/pkg/e2e/fixtures/build-test/ssh/fake_rsa.pub deleted file mode 100644 index e43add2b27e..00000000000 --- a/pkg/e2e/fixtures/build-test/ssh/fake_rsa.pub +++ /dev/null @@ -1 +0,0 @@ -ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDucnjECEntXAjre25rLcMIdpNd4Rj3Yfpg1nym0AgFjSZSfoZz2az3ge9wAp6YImQ7R/LpTYk6hJP6rVg8Al/zo2shTjWFl3OvDZB2ob8DeNrhVMBo4lsf3Bp8oelYuzqTKKAa2LcpIFpubgcBFTyWC1EToC4ILX6bDzbUXNOwxYzFGn+spXGCyIxSx13ntZEo52fxNGbo0dwSzLtfvMAnv4KH+a4YYIKTdn35hTOhVW1j/AZiZZdeP81TPGcSrRcvI7wcL5BT/11aWHU4uagy7E3WVi0dEVZay0Jugo2w5wiImwsp9DW3jPiNerk/slMsTk+eB6auvjErbVEWNVoZCk6PJCBmfWjTAC/Lvwo5HCA3QSXBxv9cgol6nPN7MIjJeVJp95e4c/6gyMzazYxc0eUwq/onCHhBjBRDrX5sBw+RyYh2W2IAw0tzFiKoS53zkRBCX1yK6EwprRtmXU4QgvBVIhqwS70EQpWvPnZllUsYxyGo5X/h/U+o8RqCd+dFgS20noC5phRole9JA/pmPgAVvpas/AQEgU64Af2Lha0bnZAArq5zXCw9mUebwn5WdMQEoW4YRG7+4kFY+jjcuO7xWse5lTxB440REp8ltzUQj1empaL7dVxZPRAm3EQZbZiYPnTn5wo1zfXHA61yrCiC3BPTCe6PQEB/fcdjfw== diff --git a/pkg/e2e/fixtures/build-test/sub-dependencies/compose.yaml b/pkg/e2e/fixtures/build-test/sub-dependencies/compose.yaml deleted file mode 100644 index 5662d689b3c..00000000000 --- a/pkg/e2e/fixtures/build-test/sub-dependencies/compose.yaml +++ /dev/null @@ -1,36 +0,0 @@ -services: - main: - build: - dockerfile_inline: | - FROM alpine - additional_contexts: - dep1: service:dep1 - dep2: service:dep2 - entrypoint: ["echo", "Hello from main"] - - dep1: - build: - dockerfile_inline: | - FROM alpine - additional_contexts: - subdep1: service:subdep1 - subdep2: service:subdep2 - entrypoint: ["echo", "Hello from dep1"] - - dep2: - build: - dockerfile_inline: | - FROM alpine - entrypoint: ["echo", "Hello from dep2"] - - subdep1: - build: - dockerfile_inline: | - FROM alpine - entrypoint: ["echo", "Hello from subdep1"] - - subdep2: - build: - dockerfile_inline: | - FROM alpine - entrypoint: ["echo", "Hello from subdep2"] \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/subset/compose.yaml b/pkg/e2e/fixtures/build-test/subset/compose.yaml deleted file mode 100644 index 6bae0f1266b..00000000000 --- a/pkg/e2e/fixtures/build-test/subset/compose.yaml +++ /dev/null @@ -1,14 +0,0 @@ -services: - main: - build: - dockerfile_inline: | - FROM alpine - entrypoint: ["echo", "Hello from main"] - depends_on: - - dep1 - - dep1: - build: - dockerfile_inline: | - FROM alpine - entrypoint: ["echo", "Hello from dep1"] \ No newline at end of file diff --git a/pkg/e2e/fixtures/build-test/tags/Dockerfile b/pkg/e2e/fixtures/build-test/tags/Dockerfile deleted file mode 100644 index 09b9df4ad83..00000000000 --- a/pkg/e2e/fixtures/build-test/tags/Dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM nginx:alpine - -RUN echo "SUCCESS" diff --git a/pkg/e2e/fixtures/build-test/tags/compose.yaml b/pkg/e2e/fixtures/build-test/tags/compose.yaml deleted file mode 100644 index de0178024f8..00000000000 --- a/pkg/e2e/fixtures/build-test/tags/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - nginx: - image: build-test-tags - build: - context: . - tags: - - docker.io/docker/build-test-tags:1.0.0 - - other-image-name:v1.0.0 - diff --git a/pkg/e2e/fixtures/cascade/compose.yaml b/pkg/e2e/fixtures/cascade/compose.yaml deleted file mode 100644 index fe79adb58a3..00000000000 --- a/pkg/e2e/fixtures/cascade/compose.yaml +++ /dev/null @@ -1,19 +0,0 @@ -services: - running: - image: alpine - command: sleep infinity - init: true - - exit: - image: alpine - command: /bin/true - depends_on: - running: - condition: service_started - - fail: - image: alpine - command: sh -c "return 111" - depends_on: - exit: - condition: service_completed_successfully diff --git a/pkg/e2e/fixtures/commit/compose.yaml b/pkg/e2e/fixtures/commit/compose.yaml deleted file mode 100644 index 28e4b15bd68..00000000000 --- a/pkg/e2e/fixtures/commit/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - service: - image: alpine - command: sleep infinity - service-with-replicas: - image: alpine - command: sleep infinity - deploy: - replicas: 3 \ No newline at end of file diff --git a/pkg/e2e/fixtures/compose-pull/duplicate-images/compose.yaml b/pkg/e2e/fixtures/compose-pull/duplicate-images/compose.yaml deleted file mode 100644 index 4a0d4c7b905..00000000000 --- a/pkg/e2e/fixtures/compose-pull/duplicate-images/compose.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - simple: - image: alpine:3.13 - command: top - another: - image: alpine:3.13 - command: top diff --git a/pkg/e2e/fixtures/compose-pull/image-present-locally/compose.yaml b/pkg/e2e/fixtures/compose-pull/image-present-locally/compose.yaml deleted file mode 100644 index a23cf58572b..00000000000 --- a/pkg/e2e/fixtures/compose-pull/image-present-locally/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - simple: - image: alpine:3.13.12 - pull_policy: missing - command: top - latest: - image: alpine:latest - pull_policy: missing - command: top diff --git a/pkg/e2e/fixtures/compose-pull/no-image-name-given/compose.yaml b/pkg/e2e/fixtures/compose-pull/no-image-name-given/compose.yaml deleted file mode 100644 index 69494e7a1c3..00000000000 --- a/pkg/e2e/fixtures/compose-pull/no-image-name-given/compose.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - no-image-service: - build: . diff --git a/pkg/e2e/fixtures/compose-pull/simple/compose.yaml b/pkg/e2e/fixtures/compose-pull/simple/compose.yaml deleted file mode 100644 index 2a5fd32a7e6..00000000000 --- a/pkg/e2e/fixtures/compose-pull/simple/compose.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - simple: - image: alpine:3.14 - command: top - another: - image: alpine:3.15 - command: top diff --git a/pkg/e2e/fixtures/compose-pull/unknown-image/Dockerfile b/pkg/e2e/fixtures/compose-pull/unknown-image/Dockerfile deleted file mode 100644 index fd3bd1dc704..00000000000 --- a/pkg/e2e/fixtures/compose-pull/unknown-image/Dockerfile +++ /dev/null @@ -1,16 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine:3.15 - diff --git a/pkg/e2e/fixtures/compose-pull/unknown-image/compose.yaml b/pkg/e2e/fixtures/compose-pull/unknown-image/compose.yaml deleted file mode 100644 index de40d09c49a..00000000000 --- a/pkg/e2e/fixtures/compose-pull/unknown-image/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - fail: - image: does_not_exists - can_build: - image: doesn_t_exists_either - build: . - valid: - image: alpine:3.15 - diff --git a/pkg/e2e/fixtures/config/compose.yaml b/pkg/e2e/fixtures/config/compose.yaml deleted file mode 100644 index 634f521b23b..00000000000 --- a/pkg/e2e/fixtures/config/compose.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - test: - image: test - ports: - - ${PORT:-8080}:80 \ No newline at end of file diff --git a/pkg/e2e/fixtures/configs/compose.yaml b/pkg/e2e/fixtures/configs/compose.yaml deleted file mode 100644 index 34d4827dda5..00000000000 --- a/pkg/e2e/fixtures/configs/compose.yaml +++ /dev/null @@ -1,33 +0,0 @@ -services: - from_env: - image: alpine - configs: - - source: from_env - command: cat /from_env - - from_file: - image: alpine - configs: - - source: from_file - command: cat /from_file - - inlined: - image: alpine - configs: - - source: inlined - command: cat /inlined - - target: - image: alpine - configs: - - source: inlined - target: /target - command: cat /target - -configs: - from_env: - environment: CONFIG - from_file: - file: config.txt - inlined: - content: This is my $CONFIG diff --git a/pkg/e2e/fixtures/configs/config.txt b/pkg/e2e/fixtures/configs/config.txt deleted file mode 100644 index 58b9a4061dc..00000000000 --- a/pkg/e2e/fixtures/configs/config.txt +++ /dev/null @@ -1 +0,0 @@ -This is my config file diff --git a/pkg/e2e/fixtures/container_name/compose.yaml b/pkg/e2e/fixtures/container_name/compose.yaml deleted file mode 100644 index c967b885d8e..00000000000 --- a/pkg/e2e/fixtures/container_name/compose.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - test: - image: alpine - container_name: test - command: /bin/true - - another_test: - image: alpine - container_name: test - command: /bin/true diff --git a/pkg/e2e/fixtures/cp-test/compose.yaml b/pkg/e2e/fixtures/cp-test/compose.yaml deleted file mode 100644 index f835ebeebfe..00000000000 --- a/pkg/e2e/fixtures/cp-test/compose.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - nginx: - image: nginx:alpine \ No newline at end of file diff --git a/pkg/e2e/fixtures/cp-test/cp-folder/cp-me.txt b/pkg/e2e/fixtures/cp-test/cp-folder/cp-me.txt deleted file mode 100644 index a97acd87634..00000000000 --- a/pkg/e2e/fixtures/cp-test/cp-folder/cp-me.txt +++ /dev/null @@ -1 +0,0 @@ -hello world from folder \ No newline at end of file diff --git a/pkg/e2e/fixtures/cp-test/cp-me.txt b/pkg/e2e/fixtures/cp-test/cp-me.txt deleted file mode 100644 index 95d09f2b101..00000000000 --- a/pkg/e2e/fixtures/cp-test/cp-me.txt +++ /dev/null @@ -1 +0,0 @@ -hello world \ No newline at end of file diff --git a/pkg/e2e/fixtures/dependencies/Dockerfile b/pkg/e2e/fixtures/dependencies/Dockerfile deleted file mode 100644 index fe5992a8df8..00000000000 --- a/pkg/e2e/fixtures/dependencies/Dockerfile +++ /dev/null @@ -1,16 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM busybox:1.35.0 -RUN echo "hello" \ No newline at end of file diff --git a/pkg/e2e/fixtures/dependencies/compose.yaml b/pkg/e2e/fixtures/dependencies/compose.yaml deleted file mode 100644 index 099c5f18a73..00000000000 --- a/pkg/e2e/fixtures/dependencies/compose.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - foo: - image: nginx:alpine - command: "${COMMAND}" - depends_on: - - bar - - bar: - image: nginx:alpine - scale: 2 diff --git a/pkg/e2e/fixtures/dependencies/dependency-exit.yaml b/pkg/e2e/fixtures/dependencies/dependency-exit.yaml deleted file mode 100644 index 7ba02ba7948..00000000000 --- a/pkg/e2e/fixtures/dependencies/dependency-exit.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - web: - image: nginx:alpine - depends_on: - db: - condition: service_healthy - db: - image: alpine - command: sh -c "exit 1" - diff --git a/pkg/e2e/fixtures/dependencies/deps-completed-successfully.yaml b/pkg/e2e/fixtures/dependencies/deps-completed-successfully.yaml deleted file mode 100644 index 5e65cb7a0a5..00000000000 --- a/pkg/e2e/fixtures/dependencies/deps-completed-successfully.yaml +++ /dev/null @@ -1,11 +0,0 @@ -services: - oneshot: - image: alpine - command: echo 'hello world' - longrunning: - image: alpine - init: true - depends_on: - oneshot: - condition: service_completed_successfully - command: sleep infinity diff --git a/pkg/e2e/fixtures/dependencies/deps-not-required.yaml b/pkg/e2e/fixtures/dependencies/deps-not-required.yaml deleted file mode 100644 index 44286846bd0..00000000000 --- a/pkg/e2e/fixtures/dependencies/deps-not-required.yaml +++ /dev/null @@ -1,11 +0,0 @@ -services: - foo: - image: bash - command: echo "foo" - depends_on: - bar: - required: false - condition: service_healthy - bar: - image: nginx:alpine - profiles: [not-required] \ No newline at end of file diff --git a/pkg/e2e/fixtures/dependencies/recreate-no-deps.yaml b/pkg/e2e/fixtures/dependencies/recreate-no-deps.yaml deleted file mode 100644 index 0b44c273dfb..00000000000 --- a/pkg/e2e/fixtures/dependencies/recreate-no-deps.yaml +++ /dev/null @@ -1,17 +0,0 @@ -version: '3.8' -services: - my-service: - image: alpine - command: tail -f /dev/null - init: true - depends_on: - nginx: {condition: service_healthy} - - nginx: - image: nginx:alpine - stop_signal: SIGTERM - healthcheck: - test: "echo | nc -w 5 localhost:80" - interval: 2s - timeout: 1s - retries: 10 diff --git a/pkg/e2e/fixtures/dependencies/service-image-depends-on.yaml b/pkg/e2e/fixtures/dependencies/service-image-depends-on.yaml deleted file mode 100644 index 3139978361e..00000000000 --- a/pkg/e2e/fixtures/dependencies/service-image-depends-on.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - foo: - image: built-image-dependency - build: - context: . - bar: - image: built-image-dependency - depends_on: - - foo diff --git a/pkg/e2e/fixtures/dotenv/.env b/pkg/e2e/fixtures/dotenv/.env deleted file mode 100644 index 1230f22dd41..00000000000 --- a/pkg/e2e/fixtures/dotenv/.env +++ /dev/null @@ -1,3 +0,0 @@ -COMPOSE_FILE="${COMPOSE_FILE:-development/compose.yaml}" -IMAGE_NAME=test -COMPOSE_PROFILES=test \ No newline at end of file diff --git a/pkg/e2e/fixtures/dotenv/.env.raw b/pkg/e2e/fixtures/dotenv/.env.raw deleted file mode 100644 index 306900800fc..00000000000 --- a/pkg/e2e/fixtures/dotenv/.env.raw +++ /dev/null @@ -1 +0,0 @@ -TEST_VAR='{"key": "value"}' \ No newline at end of file diff --git a/pkg/e2e/fixtures/dotenv/development/.env b/pkg/e2e/fixtures/dotenv/development/.env deleted file mode 100644 index 93690287799..00000000000 --- a/pkg/e2e/fixtures/dotenv/development/.env +++ /dev/null @@ -1,2 +0,0 @@ -IMAGE_NAME="${IMAGE_NAME:-backend}" -IMAGE_TAG="${IMAGE_TAG:-latest}" diff --git a/pkg/e2e/fixtures/dotenv/development/compose.yaml b/pkg/e2e/fixtures/dotenv/development/compose.yaml deleted file mode 100644 index 4731d635b60..00000000000 --- a/pkg/e2e/fixtures/dotenv/development/compose.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - backend: - image: $IMAGE_NAME:$IMAGE_TAG - test: - profiles: - - test - image: enabled:profile \ No newline at end of file diff --git a/pkg/e2e/fixtures/dotenv/raw.yaml b/pkg/e2e/fixtures/dotenv/raw.yaml deleted file mode 100644 index a65664273d3..00000000000 --- a/pkg/e2e/fixtures/dotenv/raw.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - test: - image: alpine - command: sh -c "echo $$TEST_VAR" - env_file: - - path: .env.raw - format: raw # parse without interpolation \ No newline at end of file diff --git a/pkg/e2e/fixtures/env-secret/child/compose.yaml b/pkg/e2e/fixtures/env-secret/child/compose.yaml deleted file mode 100644 index 6e4ab8213cf..00000000000 --- a/pkg/e2e/fixtures/env-secret/child/compose.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - included: - image: alpine - secrets: - - my-secret - command: cat /run/secrets/my-secret - -secrets: - my-secret: - environment: 'MY_SECRET' diff --git a/pkg/e2e/fixtures/env-secret/compose.yaml b/pkg/e2e/fixtures/env-secret/compose.yaml deleted file mode 100644 index 51052d36d21..00000000000 --- a/pkg/e2e/fixtures/env-secret/compose.yaml +++ /dev/null @@ -1,20 +0,0 @@ -include: - - path: child/compose.yaml - env_file: - - secret.env - -services: - foo: - image: alpine - secrets: - - source: secret - target: bar - uid: "1005" - gid: "1005" - mode: 0440 - command: cat /run/secrets/bar - -secrets: - secret: - environment: SECRET - diff --git a/pkg/e2e/fixtures/env-secret/secret.env b/pkg/e2e/fixtures/env-secret/secret.env deleted file mode 100644 index a195fd539b0..00000000000 --- a/pkg/e2e/fixtures/env-secret/secret.env +++ /dev/null @@ -1 +0,0 @@ -MY_SECRET='this-is-secret' diff --git a/pkg/e2e/fixtures/env_file/compose.yaml b/pkg/e2e/fixtures/env_file/compose.yaml deleted file mode 100644 index 9983f573841..00000000000 --- a/pkg/e2e/fixtures/env_file/compose.yaml +++ /dev/null @@ -1,14 +0,0 @@ -services: - serviceA: - image: nginx:latest - - serviceB: - image: nginx:latest - env_file: - - /doesnotexist/.env - - serviceC: - profiles: ["test"] - image: alpine - env_file: test.env - diff --git a/pkg/e2e/fixtures/env_file/test.env b/pkg/e2e/fixtures/env_file/test.env deleted file mode 100644 index 15c36f50ef7..00000000000 --- a/pkg/e2e/fixtures/env_file/test.env +++ /dev/null @@ -1 +0,0 @@ -FOO=BAR \ No newline at end of file diff --git a/pkg/e2e/fixtures/environment/empty-variable/Dockerfile b/pkg/e2e/fixtures/environment/empty-variable/Dockerfile deleted file mode 100644 index a7dac49e0fb..00000000000 --- a/pkg/e2e/fixtures/environment/empty-variable/Dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine -ENV EMPTY=not_empty -CMD ["sh", "-c", "echo \"=$EMPTY=\""] diff --git a/pkg/e2e/fixtures/environment/empty-variable/compose.yaml b/pkg/e2e/fixtures/environment/empty-variable/compose.yaml deleted file mode 100644 index 6ac057af32e..00000000000 --- a/pkg/e2e/fixtures/environment/empty-variable/compose.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - empty-variable: - build: - context: . - image: empty-variable - environment: - - EMPTY # expect to propagate value from user's env OR unset in container \ No newline at end of file diff --git a/pkg/e2e/fixtures/environment/env-file-comments/.env b/pkg/e2e/fixtures/environment/env-file-comments/.env deleted file mode 100644 index 068e52bee49..00000000000 --- a/pkg/e2e/fixtures/environment/env-file-comments/.env +++ /dev/null @@ -1,2 +0,0 @@ -COMMENT=1234#5 -NO_COMMENT="1234#5" diff --git a/pkg/e2e/fixtures/environment/env-file-comments/Dockerfile b/pkg/e2e/fixtures/environment/env-file-comments/Dockerfile deleted file mode 100644 index 6c6972d6a64..00000000000 --- a/pkg/e2e/fixtures/environment/env-file-comments/Dockerfile +++ /dev/null @@ -1,18 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine -ENV COMMENT=Dockerfile -ENV NO_COMMENT=Dockerfile -CMD ["sh", "-c", "printenv", "|", "grep", "COMMENT"] diff --git a/pkg/e2e/fixtures/environment/env-file-comments/compose.yaml b/pkg/e2e/fixtures/environment/env-file-comments/compose.yaml deleted file mode 100644 index 718968660a6..00000000000 --- a/pkg/e2e/fixtures/environment/env-file-comments/compose.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - env-file-comments: - build: - context: . - image: env-file-comments \ No newline at end of file diff --git a/pkg/e2e/fixtures/environment/env-interpolation-default-value/.env b/pkg/e2e/fixtures/environment/env-interpolation-default-value/.env deleted file mode 100644 index 79a91230afa..00000000000 --- a/pkg/e2e/fixtures/environment/env-interpolation-default-value/.env +++ /dev/null @@ -1 +0,0 @@ -IMAGE=default_env:${WHEREAMI:-EnvFileDefaultValue} diff --git a/pkg/e2e/fixtures/environment/env-interpolation-default-value/compose.yaml b/pkg/e2e/fixtures/environment/env-interpolation-default-value/compose.yaml deleted file mode 100644 index 4d02fcdaa96..00000000000 --- a/pkg/e2e/fixtures/environment/env-interpolation-default-value/compose.yaml +++ /dev/null @@ -1,6 +0,0 @@ -services: - env-interpolation: - image: bash - environment: - IMAGE: ${IMAGE} - command: echo "$IMAGE" diff --git a/pkg/e2e/fixtures/environment/env-interpolation/.env b/pkg/e2e/fixtures/environment/env-interpolation/.env deleted file mode 100644 index b3a1dfee365..00000000000 --- a/pkg/e2e/fixtures/environment/env-interpolation/.env +++ /dev/null @@ -1,2 +0,0 @@ -WHEREAMI=EnvFile -IMAGE=default_env:${WHEREAMI} diff --git a/pkg/e2e/fixtures/environment/env-interpolation/compose.yaml b/pkg/e2e/fixtures/environment/env-interpolation/compose.yaml deleted file mode 100644 index 7a4b3865f64..00000000000 --- a/pkg/e2e/fixtures/environment/env-interpolation/compose.yaml +++ /dev/null @@ -1,6 +0,0 @@ -services: - env-interpolation: - image: bash - environment: - IMAGE: ${IMAGE} - command: echo "$IMAGE" \ No newline at end of file diff --git a/pkg/e2e/fixtures/environment/env-priority/.env b/pkg/e2e/fixtures/environment/env-priority/.env deleted file mode 100644 index c93127ac675..00000000000 --- a/pkg/e2e/fixtures/environment/env-priority/.env +++ /dev/null @@ -1 +0,0 @@ -WHEREAMI=Env File diff --git a/pkg/e2e/fixtures/environment/env-priority/.env.override.with.default b/pkg/e2e/fixtures/environment/env-priority/.env.override.with.default deleted file mode 100644 index 35258b20ffb..00000000000 --- a/pkg/e2e/fixtures/environment/env-priority/.env.override.with.default +++ /dev/null @@ -1 +0,0 @@ -WHEREAMI=${WHEREAMI:-EnvFileDefaultValue} diff --git a/pkg/e2e/fixtures/environment/env-priority/Dockerfile b/pkg/e2e/fixtures/environment/env-priority/Dockerfile deleted file mode 100644 index 0901119f7df..00000000000 --- a/pkg/e2e/fixtures/environment/env-priority/Dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine -ENV WHEREAMI=Dockerfile -CMD ["printenv", "WHEREAMI"] diff --git a/pkg/e2e/fixtures/environment/env-priority/compose-with-env-file.yaml b/pkg/e2e/fixtures/environment/env-priority/compose-with-env-file.yaml deleted file mode 100644 index 4659830f2e6..00000000000 --- a/pkg/e2e/fixtures/environment/env-priority/compose-with-env-file.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - env-compose-priority: - image: env-compose-priority - build: - context: . - env_file: - - .env.override diff --git a/pkg/e2e/fixtures/environment/env-priority/compose-with-env.yaml b/pkg/e2e/fixtures/environment/env-priority/compose-with-env.yaml deleted file mode 100644 index d8cdc140c33..00000000000 --- a/pkg/e2e/fixtures/environment/env-priority/compose-with-env.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - env-compose-priority: - image: env-compose-priority - build: - context: . - environment: - WHEREAMI: "Compose File" diff --git a/pkg/e2e/fixtures/environment/env-priority/compose.yaml b/pkg/e2e/fixtures/environment/env-priority/compose.yaml deleted file mode 100644 index 9d107d857d5..00000000000 --- a/pkg/e2e/fixtures/environment/env-priority/compose.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - env-compose-priority: - image: env-compose-priority - build: - context: . diff --git a/pkg/e2e/fixtures/exec/compose.yaml b/pkg/e2e/fixtures/exec/compose.yaml deleted file mode 100644 index 8920173bf45..00000000000 --- a/pkg/e2e/fixtures/exec/compose.yaml +++ /dev/null @@ -1,4 +0,0 @@ -services: - test: - image: alpine - command: tail -f /dev/null \ No newline at end of file diff --git a/pkg/e2e/fixtures/export/compose.yaml b/pkg/e2e/fixtures/export/compose.yaml deleted file mode 100644 index 28e4b15bd68..00000000000 --- a/pkg/e2e/fixtures/export/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - service: - image: alpine - command: sleep infinity - service-with-replicas: - image: alpine - command: sleep infinity - deploy: - replicas: 3 \ No newline at end of file diff --git a/pkg/e2e/fixtures/external/compose.yaml b/pkg/e2e/fixtures/external/compose.yaml deleted file mode 100644 index 29b8b74a179..00000000000 --- a/pkg/e2e/fixtures/external/compose.yaml +++ /dev/null @@ -1,14 +0,0 @@ -services: - test: - image: nginx:alpine - - other: - image: nginx:alpine - networks: - test_network: - ipv4_address: 8.8.8.8 - -networks: - test_network: - external: true - name: foo_bar \ No newline at end of file diff --git a/pkg/e2e/fixtures/hooks/compose.yaml b/pkg/e2e/fixtures/hooks/compose.yaml deleted file mode 100644 index b45a65df063..00000000000 --- a/pkg/e2e/fixtures/hooks/compose.yaml +++ /dev/null @@ -1,14 +0,0 @@ -services: - sample: - image: nginx - volumes: - - data:/data - pre_stop: - - command: sh -c 'echo "In the pre-stop" >> /data/log.txt' - test: - image: nginx - post_start: - - command: sh -c 'echo env' -volumes: - data: - name: sample-data \ No newline at end of file diff --git a/pkg/e2e/fixtures/hooks/poststart/compose-error.yaml b/pkg/e2e/fixtures/hooks/poststart/compose-error.yaml deleted file mode 100644 index 2d5bf5c44da..00000000000 --- a/pkg/e2e/fixtures/hooks/poststart/compose-error.yaml +++ /dev/null @@ -1,6 +0,0 @@ -services: - - test: - image: nginx - post_start: - - command: sh -c 'command in error' diff --git a/pkg/e2e/fixtures/hooks/poststart/compose-success.yaml b/pkg/e2e/fixtures/hooks/poststart/compose-success.yaml deleted file mode 100644 index 7277945883c..00000000000 --- a/pkg/e2e/fixtures/hooks/poststart/compose-success.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - test: - image: nginx - post_start: - - command: sh -c 'echo env' diff --git a/pkg/e2e/fixtures/hooks/prestop/compose-error.yaml b/pkg/e2e/fixtures/hooks/prestop/compose-error.yaml deleted file mode 100644 index 1beb6c7e519..00000000000 --- a/pkg/e2e/fixtures/hooks/prestop/compose-error.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - sample: - image: nginx - volumes: - - data:/data - pre_stop: - - command: sh -c 'command in error' -volumes: - data: - name: sample-data \ No newline at end of file diff --git a/pkg/e2e/fixtures/hooks/prestop/compose-success.yaml b/pkg/e2e/fixtures/hooks/prestop/compose-success.yaml deleted file mode 100644 index 0d80582117c..00000000000 --- a/pkg/e2e/fixtures/hooks/prestop/compose-success.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - sample: - image: nginx - volumes: - - data:/data - pre_stop: - - command: sh -c 'echo "In the pre-stop" >> /data/log.txt' -volumes: - data: - name: sample-data \ No newline at end of file diff --git a/pkg/e2e/fixtures/image-volume-recreate/Dockerfile b/pkg/e2e/fixtures/image-volume-recreate/Dockerfile deleted file mode 100644 index ee0234de196..00000000000 --- a/pkg/e2e/fixtures/image-volume-recreate/Dockerfile +++ /dev/null @@ -1,21 +0,0 @@ -# syntax=docker/dockerfile:1 -# -# Copyright 2020 Docker Compose CLI authors -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -FROM alpine -WORKDIR /app -ARG CONTENT=initial -RUN echo "$CONTENT" > /app/content.txt diff --git a/pkg/e2e/fixtures/image-volume-recreate/compose.yaml b/pkg/e2e/fixtures/image-volume-recreate/compose.yaml deleted file mode 100644 index 7b816521979..00000000000 --- a/pkg/e2e/fixtures/image-volume-recreate/compose.yaml +++ /dev/null @@ -1,18 +0,0 @@ -services: - source: - build: - context: . - dockerfile: Dockerfile - image: image-volume-source - - consumer: - image: alpine - depends_on: - - source - command: ["cat", "/data/content.txt"] - volumes: - - type: image - source: image-volume-source - target: /data - image: - subpath: app diff --git a/pkg/e2e/fixtures/init-container/compose.yaml b/pkg/e2e/fixtures/init-container/compose.yaml deleted file mode 100644 index 275727edda9..00000000000 --- a/pkg/e2e/fixtures/init-container/compose.yaml +++ /dev/null @@ -1,11 +0,0 @@ -services: - foo: - image: alpine - command: "echo hello" - - bar: - image: alpine - command: "echo world" - depends_on: - foo: - condition: "service_completed_successfully" diff --git a/pkg/e2e/fixtures/ipam/compose.yaml b/pkg/e2e/fixtures/ipam/compose.yaml deleted file mode 100644 index 632690d2f17..00000000000 --- a/pkg/e2e/fixtures/ipam/compose.yaml +++ /dev/null @@ -1,13 +0,0 @@ -services: - foo: - image: alpine - init: true - entrypoint: ["sleep", "600"] - networks: - default: - ipv4_address: 10.1.0.100 # <-- Fixed IP address -networks: - default: - ipam: - config: - - subnet: 10.1.0.0/16 diff --git a/pkg/e2e/fixtures/ipc-test/compose.yaml b/pkg/e2e/fixtures/ipc-test/compose.yaml deleted file mode 100644 index 0659dd96a67..00000000000 --- a/pkg/e2e/fixtures/ipc-test/compose.yaml +++ /dev/null @@ -1,13 +0,0 @@ -services: - service: - image: alpine - command: top - ipc: "service:shareable" - container: - image: alpine - command: top - ipc: "container:ipc_mode_container" - shareable: - image: alpine - command: top - ipc: shareable diff --git a/pkg/e2e/fixtures/links/compose.yaml b/pkg/e2e/fixtures/links/compose.yaml deleted file mode 100644 index 8c182c4d24d..00000000000 --- a/pkg/e2e/fixtures/links/compose.yaml +++ /dev/null @@ -1,8 +0,0 @@ -services: - foo: - image: nginx:alpine - links: - - bar - - bar: - image: nginx:alpine diff --git a/pkg/e2e/fixtures/logging-driver/compose.yaml b/pkg/e2e/fixtures/logging-driver/compose.yaml deleted file mode 100644 index 37b3e8b3e4d..00000000000 --- a/pkg/e2e/fixtures/logging-driver/compose.yaml +++ /dev/null @@ -1,19 +0,0 @@ -services: - fluentbit: - image: fluent/fluent-bit:3.1.7-debug - ports: - - "24224:24224" - - "24224:24224/udp" - environment: - FOO: ${BAR} - - app: - image: nginx - depends_on: - fluentbit: - condition: service_started - restart: true - logging: - driver: fluentd - options: - fluentd-address: ${HOST:-127.0.0.1}:24224 diff --git a/pkg/e2e/fixtures/logs-test/cat.yaml b/pkg/e2e/fixtures/logs-test/cat.yaml deleted file mode 100644 index 76bd5a9ab64..00000000000 --- a/pkg/e2e/fixtures/logs-test/cat.yaml +++ /dev/null @@ -1,6 +0,0 @@ -services: - test: - image: alpine - command: cat /text_file.txt - volumes: - - ${FILE}:/text_file.txt diff --git a/pkg/e2e/fixtures/logs-test/compose.yaml b/pkg/e2e/fixtures/logs-test/compose.yaml deleted file mode 100644 index 5f9ba0e9dee..00000000000 --- a/pkg/e2e/fixtures/logs-test/compose.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - ping: - image: alpine - init: true - command: ping localhost -c ${REPEAT:-1} - hello: - image: alpine - command: echo hello - deploy: - replicas: 2 diff --git a/pkg/e2e/fixtures/logs-test/restart.yaml b/pkg/e2e/fixtures/logs-test/restart.yaml deleted file mode 100644 index 35a2abb3def..00000000000 --- a/pkg/e2e/fixtures/logs-test/restart.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - ping: - image: alpine - command: "sh -c 'ping -c 2 localhost && exit 1'" - restart: "on-failure:2" diff --git a/pkg/e2e/fixtures/model/compose.yaml b/pkg/e2e/fixtures/model/compose.yaml deleted file mode 100644 index f9eb8f6ee44..00000000000 --- a/pkg/e2e/fixtures/model/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - test: - image: alpine/curl - models: - - foo - -models: - foo: - model: ai/smollm2 diff --git a/pkg/e2e/fixtures/nested/.env b/pkg/e2e/fixtures/nested/.env deleted file mode 100644 index df2676a3740..00000000000 --- a/pkg/e2e/fixtures/nested/.env +++ /dev/null @@ -1,2 +0,0 @@ -ROOT=root -WIN=root \ No newline at end of file diff --git a/pkg/e2e/fixtures/nested/compose.yaml b/pkg/e2e/fixtures/nested/compose.yaml deleted file mode 100644 index d449f943f1c..00000000000 --- a/pkg/e2e/fixtures/nested/compose.yaml +++ /dev/null @@ -1,4 +0,0 @@ -services: - echo: - image: alpine - command: echo $ROOT $SUB win=$WIN diff --git a/pkg/e2e/fixtures/nested/sub/.env b/pkg/e2e/fixtures/nested/sub/.env deleted file mode 100644 index b930a819159..00000000000 --- a/pkg/e2e/fixtures/nested/sub/.env +++ /dev/null @@ -1,2 +0,0 @@ -SUB=sub -WIN=sub diff --git a/pkg/e2e/fixtures/network-alias/compose.yaml b/pkg/e2e/fixtures/network-alias/compose.yaml deleted file mode 100644 index f74726fe098..00000000000 --- a/pkg/e2e/fixtures/network-alias/compose.yaml +++ /dev/null @@ -1,13 +0,0 @@ -services: - - container1: - image: nginx - links: - - container2:container - - container2: - image: nginx - networks: - default: - aliases: - - alias-of-container2 diff --git a/pkg/e2e/fixtures/network-interface-name/compose.yaml b/pkg/e2e/fixtures/network-interface-name/compose.yaml deleted file mode 100644 index 701830a48a5..00000000000 --- a/pkg/e2e/fixtures/network-interface-name/compose.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - test: - image: alpine - command: ip link show - networks: - default: - interface_name: foobar diff --git a/pkg/e2e/fixtures/network-links/compose.yaml b/pkg/e2e/fixtures/network-links/compose.yaml deleted file mode 100644 index c09a33fcdaa..00000000000 --- a/pkg/e2e/fixtures/network-links/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - container1: - image: nginx - network_mode: bridge - container2: - image: nginx - network_mode: bridge - links: - - container1 diff --git a/pkg/e2e/fixtures/network-recreate/compose.yaml b/pkg/e2e/fixtures/network-recreate/compose.yaml deleted file mode 100644 index 06a0a3e634a..00000000000 --- a/pkg/e2e/fixtures/network-recreate/compose.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - web: - image: nginx - networks: - - test - -networks: - test: - labels: - - foo=${FOO:-foo} \ No newline at end of file diff --git a/pkg/e2e/fixtures/network-test/compose.subnet.yaml b/pkg/e2e/fixtures/network-test/compose.subnet.yaml deleted file mode 100644 index 46a358f6205..00000000000 --- a/pkg/e2e/fixtures/network-test/compose.subnet.yaml +++ /dev/null @@ -1,12 +0,0 @@ -services: - test: - image: nginx:alpine - networks: - - test - -networks: - test: - ipam: - config: - - subnet: ${SUBNET-172.99.0.0/16} - diff --git a/pkg/e2e/fixtures/network-test/compose.yaml b/pkg/e2e/fixtures/network-test/compose.yaml deleted file mode 100644 index 608007ec34e..00000000000 --- a/pkg/e2e/fixtures/network-test/compose.yaml +++ /dev/null @@ -1,39 +0,0 @@ -services: - mydb: - image: mariadb - network_mode: "service:db" - environment: - - MYSQL_ALLOW_EMPTY_PASSWORD=yes - db: - image: gtardif/sentences-db - init: true - networks: - - dbnet - - closesnetworkname1 - - closesnetworkname2 - words: - image: gtardif/sentences-api - init: true - ports: - - "8080:8080" - networks: - - dbnet - - servicenet - web: - image: gtardif/sentences-web - init: true - ports: - - "80:80" - labels: - - "my-label=test" - networks: - - servicenet - -networks: - dbnet: - servicenet: - name: microservices - closesnetworkname1: - name: closenamenet - closesnetworkname2: - name: closenamenet-2 diff --git a/pkg/e2e/fixtures/network-test/mac_address.yaml b/pkg/e2e/fixtures/network-test/mac_address.yaml deleted file mode 100644 index 60e3861a2d8..00000000000 --- a/pkg/e2e/fixtures/network-test/mac_address.yaml +++ /dev/null @@ -1,4 +0,0 @@ -services: - test: - image: nginx:alpine - mac_address: 00:e0:84:35:d0:e8 diff --git a/pkg/e2e/fixtures/no-deps/network-mode.yaml b/pkg/e2e/fixtures/no-deps/network-mode.yaml deleted file mode 100644 index aab03f5b12d..00000000000 --- a/pkg/e2e/fixtures/no-deps/network-mode.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - app: - image: nginx:alpine - network_mode: service:db - - db: - image: nginx:alpine diff --git a/pkg/e2e/fixtures/no-deps/volume-from.yaml b/pkg/e2e/fixtures/no-deps/volume-from.yaml deleted file mode 100644 index 96b35761e0f..00000000000 --- a/pkg/e2e/fixtures/no-deps/volume-from.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - app: - image: nginx:alpine - volumes_from: - - db - - db: - image: nginx:alpine - volumes: - - /var/data diff --git a/pkg/e2e/fixtures/orphans/.env b/pkg/e2e/fixtures/orphans/.env deleted file mode 100644 index 717e3306ba7..00000000000 --- a/pkg/e2e/fixtures/orphans/.env +++ /dev/null @@ -1 +0,0 @@ -COMPOSE_REMOVE_ORPHANS=true diff --git a/pkg/e2e/fixtures/orphans/compose.yaml b/pkg/e2e/fixtures/orphans/compose.yaml deleted file mode 100644 index 33dbac0d26a..00000000000 --- a/pkg/e2e/fixtures/orphans/compose.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - orphan: - profiles: [run] - image: alpine - command: echo hello - test: - image: nginx:alpine diff --git a/pkg/e2e/fixtures/pause/compose.yaml b/pkg/e2e/fixtures/pause/compose.yaml deleted file mode 100644 index 615fcad5883..00000000000 --- a/pkg/e2e/fixtures/pause/compose.yaml +++ /dev/null @@ -1,17 +0,0 @@ -services: - a: - image: nginx:alpine - ports: [80] - healthcheck: - test: wget --spider -S -T1 http://localhost:80 - interval: 1s - timeout: 1s - b: - image: nginx:alpine - ports: [80] - depends_on: - - a - healthcheck: - test: wget --spider -S -T1 http://localhost:80 - interval: 1s - timeout: 1s diff --git a/pkg/e2e/fixtures/port-range/compose.yaml b/pkg/e2e/fixtures/port-range/compose.yaml deleted file mode 100644 index 65f6fde6184..00000000000 --- a/pkg/e2e/fixtures/port-range/compose.yaml +++ /dev/null @@ -1,16 +0,0 @@ -services: - a: - image: nginx:alpine - scale: 5 - ports: - - "6005-6015:80" - - b: - image: nginx:alpine - ports: - - 80 - - c: - image: nginx:alpine - ports: - - 80 \ No newline at end of file diff --git a/pkg/e2e/fixtures/profiles/compose.yaml b/pkg/e2e/fixtures/profiles/compose.yaml deleted file mode 100644 index 144cd3cd79b..00000000000 --- a/pkg/e2e/fixtures/profiles/compose.yaml +++ /dev/null @@ -1,8 +0,0 @@ -services: - regular-service: - image: nginx:alpine - - profiled-service: - image: nginx:alpine - profiles: - - test-profile \ No newline at end of file diff --git a/pkg/e2e/fixtures/profiles/docker-compose.yaml b/pkg/e2e/fixtures/profiles/docker-compose.yaml deleted file mode 100644 index 134d7bbc9bf..00000000000 --- a/pkg/e2e/fixtures/profiles/docker-compose.yaml +++ /dev/null @@ -1,15 +0,0 @@ -services: - foo: - container_name: foo_c - profiles: [ test ] - image: alpine - depends_on: [ db ] - - bar: - container_name: bar_c - profiles: [ test ] - image: alpine - - db: - container_name: db_c - image: alpine \ No newline at end of file diff --git a/pkg/e2e/fixtures/profiles/test-profile.env b/pkg/e2e/fixtures/profiles/test-profile.env deleted file mode 100644 index efb732b1ed2..00000000000 --- a/pkg/e2e/fixtures/profiles/test-profile.env +++ /dev/null @@ -1 +0,0 @@ -COMPOSE_PROFILES=test-profile diff --git a/pkg/e2e/fixtures/project-volume-bind-test/docker-compose.yml b/pkg/e2e/fixtures/project-volume-bind-test/docker-compose.yml deleted file mode 100644 index 7e1795719df..00000000000 --- a/pkg/e2e/fixtures/project-volume-bind-test/docker-compose.yml +++ /dev/null @@ -1,14 +0,0 @@ -services: - frontend: - image: nginx - container_name: frontend - volumes: - - project-data:/data - -volumes: - project-data: - driver: local - driver_opts: - type: none - o: bind - device: "${TEST_DIR}" diff --git a/pkg/e2e/fixtures/providers/depends-on-multiple-providers.yaml b/pkg/e2e/fixtures/providers/depends-on-multiple-providers.yaml deleted file mode 100644 index 6faec7cdc5e..00000000000 --- a/pkg/e2e/fixtures/providers/depends-on-multiple-providers.yaml +++ /dev/null @@ -1,21 +0,0 @@ -services: - test: - image: alpine - command: env - depends_on: - - provider1 - - provider2 - provider1: - provider: - type: example-provider - options: - name: provider1 - type: test1 - size: 1 - provider2: - provider: - type: example-provider - options: - name: provider2 - type: test2 - size: 2 diff --git a/pkg/e2e/fixtures/ps-test/compose.yaml b/pkg/e2e/fixtures/ps-test/compose.yaml deleted file mode 100644 index 08781e6a2a6..00000000000 --- a/pkg/e2e/fixtures/ps-test/compose.yaml +++ /dev/null @@ -1,12 +0,0 @@ -services: - nginx: - image: nginx:latest - expose: - - '80' - - '443' - - '8080' - busybox: - image: busybox - command: busybox httpd -f -p 8000 - ports: - - '127.0.0.1:8001:8000' diff --git a/pkg/e2e/fixtures/publish/Dockerfile b/pkg/e2e/fixtures/publish/Dockerfile deleted file mode 100644 index b17b6b06175..00000000000 --- a/pkg/e2e/fixtures/publish/Dockerfile +++ /dev/null @@ -1,15 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM alpine:latest \ No newline at end of file diff --git a/pkg/e2e/fixtures/publish/common.yaml b/pkg/e2e/fixtures/publish/common.yaml deleted file mode 100644 index c8022b46873..00000000000 --- a/pkg/e2e/fixtures/publish/common.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - foo: - image: bar \ No newline at end of file diff --git a/pkg/e2e/fixtures/publish/compose-bind-mount.yml b/pkg/e2e/fixtures/publish/compose-bind-mount.yml deleted file mode 100644 index ecfc700d1c6..00000000000 --- a/pkg/e2e/fixtures/publish/compose-bind-mount.yml +++ /dev/null @@ -1,5 +0,0 @@ -services: - serviceA: - image: a - volumes: - - .:/user-data diff --git a/pkg/e2e/fixtures/publish/compose-build-only.yml b/pkg/e2e/fixtures/publish/compose-build-only.yml deleted file mode 100644 index e4736d983df..00000000000 --- a/pkg/e2e/fixtures/publish/compose-build-only.yml +++ /dev/null @@ -1,9 +0,0 @@ -services: - serviceA: - build: - context: . - dockerfile: Dockerfile - serviceB: - build: - context: . - dockerfile: Dockerfile diff --git a/pkg/e2e/fixtures/publish/compose-env-file.yml b/pkg/e2e/fixtures/publish/compose-env-file.yml deleted file mode 100644 index b438c71daba..00000000000 --- a/pkg/e2e/fixtures/publish/compose-env-file.yml +++ /dev/null @@ -1,7 +0,0 @@ -services: - serviceA: - image: "alpine:3.12" - env_file: - - publish.env - serviceB: - image: "alpine:3.12" diff --git a/pkg/e2e/fixtures/publish/compose-environment.yml b/pkg/e2e/fixtures/publish/compose-environment.yml deleted file mode 100644 index 27e3a4b31bf..00000000000 --- a/pkg/e2e/fixtures/publish/compose-environment.yml +++ /dev/null @@ -1,7 +0,0 @@ -services: - serviceA: - image: "alpine:3.12" - environment: - - "FOO=bar" - serviceB: - image: "alpine:3.12" diff --git a/pkg/e2e/fixtures/publish/compose-local-include.yml b/pkg/e2e/fixtures/publish/compose-local-include.yml deleted file mode 100644 index 1af74e926eb..00000000000 --- a/pkg/e2e/fixtures/publish/compose-local-include.yml +++ /dev/null @@ -1,6 +0,0 @@ -include: - - common.yaml - -services: - test: - image: test \ No newline at end of file diff --git a/pkg/e2e/fixtures/publish/compose-multi-env-config.yml b/pkg/e2e/fixtures/publish/compose-multi-env-config.yml deleted file mode 100644 index 35a75eab36a..00000000000 --- a/pkg/e2e/fixtures/publish/compose-multi-env-config.yml +++ /dev/null @@ -1,11 +0,0 @@ -services: - serviceA: - image: "alpine:3.12" - environment: - - "FOO=bar" - serviceB: - image: "alpine:3.12" - env_file: - - publish.env - environment: - - "BAR=baz" \ No newline at end of file diff --git a/pkg/e2e/fixtures/publish/compose-sensitive.yml b/pkg/e2e/fixtures/publish/compose-sensitive.yml deleted file mode 100644 index 68dd59b83e7..00000000000 --- a/pkg/e2e/fixtures/publish/compose-sensitive.yml +++ /dev/null @@ -1,20 +0,0 @@ -services: - serviceA: - image: "alpine:3.12" - environment: - - AWS_ACCESS_KEY_ID=A3TX1234567890ABCDEF - - AWS_SECRET_ACCESS_KEY=aws"12345+67890/abcdefghijklm+NOPQRSTUVWXYZ+" - configs: - - myconfig - serviceB: - image: "alpine:3.12" - env_file: - - publish-sensitive.env - secrets: - - mysecret -configs: - myconfig: - file: config.txt -secrets: - mysecret: - file: secret.txt \ No newline at end of file diff --git a/pkg/e2e/fixtures/publish/compose-with-extends.yml b/pkg/e2e/fixtures/publish/compose-with-extends.yml deleted file mode 100644 index f8c1349f5f5..00000000000 --- a/pkg/e2e/fixtures/publish/compose-with-extends.yml +++ /dev/null @@ -1,5 +0,0 @@ -services: - test: - extends: - file: common.yaml - service: foo diff --git a/pkg/e2e/fixtures/publish/config.txt b/pkg/e2e/fixtures/publish/config.txt deleted file mode 100644 index 32501ce3ed9..00000000000 --- a/pkg/e2e/fixtures/publish/config.txt +++ /dev/null @@ -1 +0,0 @@ -eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw \ No newline at end of file diff --git a/pkg/e2e/fixtures/publish/oci/compose-override.yaml b/pkg/e2e/fixtures/publish/oci/compose-override.yaml deleted file mode 100644 index c8947e610a8..00000000000 --- a/pkg/e2e/fixtures/publish/oci/compose-override.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - app: - env_file: test.env \ No newline at end of file diff --git a/pkg/e2e/fixtures/publish/oci/compose.yaml b/pkg/e2e/fixtures/publish/oci/compose.yaml deleted file mode 100644 index 369094bec14..00000000000 --- a/pkg/e2e/fixtures/publish/oci/compose.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - app: - extends: - file: extends.yaml - service: test \ No newline at end of file diff --git a/pkg/e2e/fixtures/publish/oci/extends.yaml b/pkg/e2e/fixtures/publish/oci/extends.yaml deleted file mode 100644 index 9184173d087..00000000000 --- a/pkg/e2e/fixtures/publish/oci/extends.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - test: - image: alpine \ No newline at end of file diff --git a/pkg/e2e/fixtures/publish/oci/test.env b/pkg/e2e/fixtures/publish/oci/test.env deleted file mode 100644 index 6e1f61b59ea..00000000000 --- a/pkg/e2e/fixtures/publish/oci/test.env +++ /dev/null @@ -1 +0,0 @@ -HELLO=WORLD \ No newline at end of file diff --git a/pkg/e2e/fixtures/publish/publish-sensitive.env b/pkg/e2e/fixtures/publish/publish-sensitive.env deleted file mode 100644 index 6ced3351048..00000000000 --- a/pkg/e2e/fixtures/publish/publish-sensitive.env +++ /dev/null @@ -1 +0,0 @@ -GITHUB_TOKEN=ghp_1234567890abcdefghijklmnopqrstuvwxyz diff --git a/pkg/e2e/fixtures/publish/publish.env b/pkg/e2e/fixtures/publish/publish.env deleted file mode 100644 index 62eddb614c6..00000000000 --- a/pkg/e2e/fixtures/publish/publish.env +++ /dev/null @@ -1,2 +0,0 @@ -FOO=bar -QUIX= \ No newline at end of file diff --git a/pkg/e2e/fixtures/publish/secret.txt b/pkg/e2e/fixtures/publish/secret.txt deleted file mode 100644 index 5df0a6eeac3..00000000000 --- a/pkg/e2e/fixtures/publish/secret.txt +++ /dev/null @@ -1,3 +0,0 @@ ------BEGIN DSA PRIVATE KEY----- -wxyz+ABC= ------END DSA PRIVATE KEY----- \ No newline at end of file diff --git a/pkg/e2e/fixtures/recreate-volumes/bind.yaml b/pkg/e2e/fixtures/recreate-volumes/bind.yaml deleted file mode 100644 index a67244ca530..00000000000 --- a/pkg/e2e/fixtures/recreate-volumes/bind.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - app: - image: alpine - volumes: - - .:/my_vol diff --git a/pkg/e2e/fixtures/recreate-volumes/compose.yaml b/pkg/e2e/fixtures/recreate-volumes/compose.yaml deleted file mode 100644 index e0e40c721b7..00000000000 --- a/pkg/e2e/fixtures/recreate-volumes/compose.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - app: - image: alpine - volumes: - - my_vol:/my_vol - -volumes: - my_vol: - labels: - foo: bar diff --git a/pkg/e2e/fixtures/recreate-volumes/compose2.yaml b/pkg/e2e/fixtures/recreate-volumes/compose2.yaml deleted file mode 100644 index 96a073f0516..00000000000 --- a/pkg/e2e/fixtures/recreate-volumes/compose2.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - app: - image: alpine - volumes: - - my_vol:/my_vol - -volumes: - my_vol: - labels: - foo: zot diff --git a/pkg/e2e/fixtures/resources/compose.yaml b/pkg/e2e/fixtures/resources/compose.yaml deleted file mode 100644 index 4823688803e..00000000000 --- a/pkg/e2e/fixtures/resources/compose.yaml +++ /dev/null @@ -1,5 +0,0 @@ -volumes: - my_vol: {} - -networks: - my_net: {} \ No newline at end of file diff --git a/pkg/e2e/fixtures/restart-test/compose-depends-on.yaml b/pkg/e2e/fixtures/restart-test/compose-depends-on.yaml deleted file mode 100644 index 092d862f698..00000000000 --- a/pkg/e2e/fixtures/restart-test/compose-depends-on.yaml +++ /dev/null @@ -1,27 +0,0 @@ -services: - with-restart: - image: nginx:alpine - init: true - command: tail -f /dev/null - stop_signal: SIGTERM - depends_on: - nginx: {condition: service_healthy, restart: true} - - no-restart: - image: nginx:alpine - init: true - command: tail -f /dev/null - stop_signal: SIGTERM - depends_on: - nginx: { condition: service_healthy } - - nginx: - image: nginx:alpine - labels: - TEST: ${LABEL:-test} - stop_signal: SIGTERM - healthcheck: - test: "echo | nc -w 5 localhost:80" - interval: 2s - timeout: 1s - retries: 10 diff --git a/pkg/e2e/fixtures/restart-test/compose.yaml b/pkg/e2e/fixtures/restart-test/compose.yaml deleted file mode 100644 index 92c28d3a706..00000000000 --- a/pkg/e2e/fixtures/restart-test/compose.yaml +++ /dev/null @@ -1,12 +0,0 @@ -services: - restart: - image: alpine - init: true - command: ash -c "if [[ -f /tmp/restart.lock ]] ; then sleep infinity; else touch /tmp/restart.lock; fi" - - test: - profiles: - - test - image: alpine - init: true - command: ash -c "if [[ -f /tmp/restart.lock ]] ; then sleep infinity; else touch /tmp/restart.lock; fi" diff --git a/pkg/e2e/fixtures/run-test/build-once-nested.yaml b/pkg/e2e/fixtures/run-test/build-once-nested.yaml deleted file mode 100644 index 4972db5a7bb..00000000000 --- a/pkg/e2e/fixtures/run-test/build-once-nested.yaml +++ /dev/null @@ -1,32 +0,0 @@ -services: - # Database service with build - db: - pull_policy: build - build: - dockerfile_inline: | - FROM alpine - RUN echo "DB built at $(date)" > /db-build.txt - CMD sleep 3600 - - # API service that depends on db - api: - pull_policy: build - build: - dockerfile_inline: | - FROM alpine - RUN echo "API built at $(date)" > /api-build.txt - CMD sleep 3600 - depends_on: - - db - - # App service that depends on api (which depends on db) - app: - pull_policy: build - build: - dockerfile_inline: | - FROM alpine - RUN echo "App built at $(date)" > /app-build.txt - CMD echo "App running" - depends_on: - - api - diff --git a/pkg/e2e/fixtures/run-test/build-once-no-deps.yaml b/pkg/e2e/fixtures/run-test/build-once-no-deps.yaml deleted file mode 100644 index 36f4258b380..00000000000 --- a/pkg/e2e/fixtures/run-test/build-once-no-deps.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - # Simple service with no dependencies - simple: - pull_policy: build - build: - dockerfile_inline: | - FROM alpine - RUN echo "Simple built at $(date)" > /build.txt - CMD echo "Simple service" - diff --git a/pkg/e2e/fixtures/run-test/build-once.yaml b/pkg/e2e/fixtures/run-test/build-once.yaml deleted file mode 100644 index 7a6f84dbc42..00000000000 --- a/pkg/e2e/fixtures/run-test/build-once.yaml +++ /dev/null @@ -1,18 +0,0 @@ -services: - # Service with pull_policy: build to ensure it always rebuilds - # This is the key to testing the bug - without the fix, this would build twice - nginx: - pull_policy: build - build: - dockerfile_inline: | - FROM alpine - RUN echo "Nginx built at $(date)" > /build-time.txt - CMD sleep 3600 - - # Service that depends on nginx - curl: - image: alpine - depends_on: - - nginx - command: echo "curl service" - diff --git a/pkg/e2e/fixtures/run-test/compose.yaml b/pkg/e2e/fixtures/run-test/compose.yaml deleted file mode 100644 index aef80119039..00000000000 --- a/pkg/e2e/fixtures/run-test/compose.yaml +++ /dev/null @@ -1,31 +0,0 @@ -services: - back: - image: alpine - command: echo "Hello there!!" - depends_on: - - db - networks: - - backnet - db: - image: nginx:alpine - networks: - - backnet - volumes: - - data:/test - front: - image: nginx:alpine - networks: - - frontnet - build: - build: - dockerfile_inline: "FROM base" - additional_contexts: - base: "service:build_base" - build_base: - build: - dockerfile_inline: "FROM alpine" -networks: - frontnet: - backnet: -volumes: - data: diff --git a/pkg/e2e/fixtures/run-test/deps.yaml b/pkg/e2e/fixtures/run-test/deps.yaml deleted file mode 100644 index 6e0e394a32f..00000000000 --- a/pkg/e2e/fixtures/run-test/deps.yaml +++ /dev/null @@ -1,13 +0,0 @@ -services: - service_a: - image: bash - command: echo "a" - depends_on: - - shared_dep - service_b: - image: bash - command: echo "b" - depends_on: - - shared_dep - shared_dep: - image: bash \ No newline at end of file diff --git a/pkg/e2e/fixtures/run-test/orphan.yaml b/pkg/e2e/fixtures/run-test/orphan.yaml deleted file mode 100644 index b059acc9759..00000000000 --- a/pkg/e2e/fixtures/run-test/orphan.yaml +++ /dev/null @@ -1,4 +0,0 @@ -services: - simple: - image: alpine - command: echo "Hi there!!" diff --git a/pkg/e2e/fixtures/run-test/piped-test.yaml b/pkg/e2e/fixtures/run-test/piped-test.yaml deleted file mode 100644 index 247bd923ace..00000000000 --- a/pkg/e2e/fixtures/run-test/piped-test.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - piped-test: - image: alpine - command: cat - # Service that will receive piped input and echo it back - tty-test: - image: alpine - command: sh -c "if [ -t 0 ]; then echo 'TTY detected'; else echo 'No TTY detected'; fi" - # Service to test TTY detection \ No newline at end of file diff --git a/pkg/e2e/fixtures/run-test/ports.yaml b/pkg/e2e/fixtures/run-test/ports.yaml deleted file mode 100644 index f6f93aa10e1..00000000000 --- a/pkg/e2e/fixtures/run-test/ports.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - back: - image: alpine - ports: - - 8082:80 diff --git a/pkg/e2e/fixtures/run-test/pull.yaml b/pkg/e2e/fixtures/run-test/pull.yaml deleted file mode 100644 index 223fecd50d7..00000000000 --- a/pkg/e2e/fixtures/run-test/pull.yaml +++ /dev/null @@ -1,4 +0,0 @@ -services: - backend: - image: nginx - command: nginx -t \ No newline at end of file diff --git a/pkg/e2e/fixtures/run-test/quiet-pull.yaml b/pkg/e2e/fixtures/run-test/quiet-pull.yaml deleted file mode 100644 index 922676363f5..00000000000 --- a/pkg/e2e/fixtures/run-test/quiet-pull.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - backend: - image: hello-world \ No newline at end of file diff --git a/pkg/e2e/fixtures/run-test/run.env b/pkg/e2e/fixtures/run-test/run.env deleted file mode 100644 index 6ac867af71b..00000000000 --- a/pkg/e2e/fixtures/run-test/run.env +++ /dev/null @@ -1 +0,0 @@ -FOO=BAR diff --git a/pkg/e2e/fixtures/scale/Dockerfile b/pkg/e2e/fixtures/scale/Dockerfile deleted file mode 100644 index 7f341f9525b..00000000000 --- a/pkg/e2e/fixtures/scale/Dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM nginx:alpine -ARG FOO -LABEL FOO=$FOO \ No newline at end of file diff --git a/pkg/e2e/fixtures/scale/build.yaml b/pkg/e2e/fixtures/scale/build.yaml deleted file mode 100644 index cd109c7a849..00000000000 --- a/pkg/e2e/fixtures/scale/build.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - test: - build: . diff --git a/pkg/e2e/fixtures/scale/compose.yaml b/pkg/e2e/fixtures/scale/compose.yaml deleted file mode 100644 index 619630876b1..00000000000 --- a/pkg/e2e/fixtures/scale/compose.yaml +++ /dev/null @@ -1,17 +0,0 @@ -services: - back: - image: nginx:alpine - depends_on: - - db - db: - image: nginx:alpine - environment: - - MAYBE - front: - image: nginx:alpine - deploy: - replicas: 2 - dbadmin: - image: nginx:alpine - deploy: - replicas: 0 \ No newline at end of file diff --git a/pkg/e2e/fixtures/sentences/compose.yaml b/pkg/e2e/fixtures/sentences/compose.yaml deleted file mode 100644 index 3cabccab16d..00000000000 --- a/pkg/e2e/fixtures/sentences/compose.yaml +++ /dev/null @@ -1,19 +0,0 @@ -services: - db: - image: gtardif/sentences-db - init: true - words: - image: gtardif/sentences-api - init: true - ports: - - "95:8080" - web: - image: gtardif/sentences-web - init: true - ports: - - "90:80" - labels: - - "my-label=test" - healthcheck: - test: ["CMD", "curl", "-f", "http://localhost:80/"] - interval: 2s diff --git a/pkg/e2e/fixtures/simple-build-test/compose-interpolate.yaml b/pkg/e2e/fixtures/simple-build-test/compose-interpolate.yaml deleted file mode 100644 index 57d092a9909..00000000000 --- a/pkg/e2e/fixtures/simple-build-test/compose-interpolate.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - nginx: - build: - context: nginx-build - dockerfile: ${MYVAR} diff --git a/pkg/e2e/fixtures/simple-build-test/compose.yaml b/pkg/e2e/fixtures/simple-build-test/compose.yaml deleted file mode 100644 index 7e72f1c2081..00000000000 --- a/pkg/e2e/fixtures/simple-build-test/compose.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - nginx: - build: - context: nginx-build - dockerfile: Dockerfile diff --git a/pkg/e2e/fixtures/simple-build-test/nginx-build/Dockerfile b/pkg/e2e/fixtures/simple-build-test/nginx-build/Dockerfile deleted file mode 100644 index dd79c0e4a31..00000000000 --- a/pkg/e2e/fixtures/simple-build-test/nginx-build/Dockerfile +++ /dev/null @@ -1,19 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM nginx:alpine - -ARG FOO -LABEL FOO=$FOO -COPY static /usr/share/nginx/html diff --git a/pkg/e2e/fixtures/simple-build-test/nginx-build/static/index.html b/pkg/e2e/fixtures/simple-build-test/nginx-build/static/index.html deleted file mode 100644 index 63159b9e91b..00000000000 --- a/pkg/e2e/fixtures/simple-build-test/nginx-build/static/index.html +++ /dev/null @@ -1,10 +0,0 @@ - - - - - Docker Nginx - - -

Hello from Nginx container

- - diff --git a/pkg/e2e/fixtures/simple-composefile/compose.yaml b/pkg/e2e/fixtures/simple-composefile/compose.yaml deleted file mode 100644 index 8b0b49ea156..00000000000 --- a/pkg/e2e/fixtures/simple-composefile/compose.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - simple: - image: alpine - command: top - another: - image: alpine - command: top diff --git a/pkg/e2e/fixtures/simple-composefile/id.yaml b/pkg/e2e/fixtures/simple-composefile/id.yaml deleted file mode 100644 index 67ac13f7308..00000000000 --- a/pkg/e2e/fixtures/simple-composefile/id.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - test: - image: ${ID:?ID variable must be set} diff --git a/pkg/e2e/fixtures/start-fail/compose.yaml b/pkg/e2e/fixtures/start-fail/compose.yaml deleted file mode 100644 index 4c88576c872..00000000000 --- a/pkg/e2e/fixtures/start-fail/compose.yaml +++ /dev/null @@ -1,16 +0,0 @@ -services: - fail: - image: alpine - init: true - command: sleep infinity - healthcheck: - test: "false" - interval: 1s - retries: 3 - depends: - image: alpine - init: true - command: sleep infinity - depends_on: - fail: - condition: service_healthy diff --git a/pkg/e2e/fixtures/start-fail/start-depends_on-long-lived.yaml b/pkg/e2e/fixtures/start-fail/start-depends_on-long-lived.yaml deleted file mode 100644 index a3c920e0f32..00000000000 --- a/pkg/e2e/fixtures/start-fail/start-depends_on-long-lived.yaml +++ /dev/null @@ -1,14 +0,0 @@ -services: - safe: - image: 'alpine' - init: true - command: ['/bin/sh', '-c', 'sleep infinity'] # never exiting - failure: - image: 'alpine' - init: true - command: ['/bin/sh', '-c', 'sleep 1 ; echo "exiting with error" ; exit 42'] - test: - image: 'alpine' - init: true - command: ['/bin/sh', '-c', 'sleep 99999 ; echo "tests are OK"'] # very long job - depends_on: [safe] diff --git a/pkg/e2e/fixtures/start-stop/compose.yaml b/pkg/e2e/fixtures/start-stop/compose.yaml deleted file mode 100644 index 15f69b2e305..00000000000 --- a/pkg/e2e/fixtures/start-stop/compose.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - simple: - image: nginx:alpine - another: - image: nginx:alpine diff --git a/pkg/e2e/fixtures/start-stop/other.yaml b/pkg/e2e/fixtures/start-stop/other.yaml deleted file mode 100644 index 58782726184..00000000000 --- a/pkg/e2e/fixtures/start-stop/other.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - a-different-one: - image: nginx:alpine - and-another-one: - image: nginx:alpine diff --git a/pkg/e2e/fixtures/start-stop/start-stop-deps.yaml b/pkg/e2e/fixtures/start-stop/start-stop-deps.yaml deleted file mode 100644 index fb1f7fad702..00000000000 --- a/pkg/e2e/fixtures/start-stop/start-stop-deps.yaml +++ /dev/null @@ -1,17 +0,0 @@ -services: - another_2: - image: nginx:alpine - another: - image: nginx:alpine - depends_on: - - another_2 - dep_2: - image: nginx:alpine - dep_1: - image: nginx:alpine - depends_on: - - dep_2 - desired: - image: nginx:alpine - depends_on: - - dep_1 diff --git a/pkg/e2e/fixtures/start_interval/compose.yaml b/pkg/e2e/fixtures/start_interval/compose.yaml deleted file mode 100644 index c78a9f43a05..00000000000 --- a/pkg/e2e/fixtures/start_interval/compose.yaml +++ /dev/null @@ -1,15 +0,0 @@ -services: - test: - image: "nginx" - healthcheck: - interval: 30s - start_period: 10s - start_interval: 1s - test: "/bin/true" - - error: - image: "nginx" - healthcheck: - interval: 30s - start_interval: 1s - test: "/bin/true" \ No newline at end of file diff --git a/pkg/e2e/fixtures/stdout-stderr/compose.yaml b/pkg/e2e/fixtures/stdout-stderr/compose.yaml deleted file mode 100644 index 53a44b4552c..00000000000 --- a/pkg/e2e/fixtures/stdout-stderr/compose.yaml +++ /dev/null @@ -1,7 +0,0 @@ -services: - stderr: - image: alpine - init: true - command: /bin/ash /log_to_stderr.sh - volumes: - - ./log_to_stderr.sh:/log_to_stderr.sh diff --git a/pkg/e2e/fixtures/stdout-stderr/log_to_stderr.sh b/pkg/e2e/fixtures/stdout-stderr/log_to_stderr.sh deleted file mode 100755 index f015ca89bea..00000000000 --- a/pkg/e2e/fixtures/stdout-stderr/log_to_stderr.sh +++ /dev/null @@ -1,16 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - ->&2 echo "log to stderr" -echo "log to stdout" diff --git a/pkg/e2e/fixtures/stop/compose.yaml b/pkg/e2e/fixtures/stop/compose.yaml deleted file mode 100644 index f81462ae321..00000000000 --- a/pkg/e2e/fixtures/stop/compose.yaml +++ /dev/null @@ -1,9 +0,0 @@ -services: - service1: - image: alpine - command: /bin/true - service2: - image: alpine - command: ping -c 2 localhost - pre_stop: - - command: echo "stop hook running..." diff --git a/pkg/e2e/fixtures/switch-volumes/compose.yaml b/pkg/e2e/fixtures/switch-volumes/compose.yaml deleted file mode 100644 index 9da0dcba175..00000000000 --- a/pkg/e2e/fixtures/switch-volumes/compose.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - app: - image: alpine - volumes: - - my_vol:/my_vol - -volumes: - my_vol: - external: true - name: test_external_volume diff --git a/pkg/e2e/fixtures/switch-volumes/compose2.yaml b/pkg/e2e/fixtures/switch-volumes/compose2.yaml deleted file mode 100644 index 6d52097f925..00000000000 --- a/pkg/e2e/fixtures/switch-volumes/compose2.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - app: - image: alpine - volumes: - - my_vol:/my_vol - -volumes: - my_vol: - external: true - name: test_external_volume_2 diff --git a/pkg/e2e/fixtures/ups-deps-stop/compose.yaml b/pkg/e2e/fixtures/ups-deps-stop/compose.yaml deleted file mode 100644 index c99087f65a7..00000000000 --- a/pkg/e2e/fixtures/ups-deps-stop/compose.yaml +++ /dev/null @@ -1,11 +0,0 @@ -services: - dependency: - image: alpine - init: true - command: /bin/sh -c 'while true; do echo "hello dependency"; sleep 1; done' - - app: - depends_on: ['dependency'] - image: alpine - init: true - command: /bin/sh -c 'while true; do echo "hello app"; sleep 1; done' diff --git a/pkg/e2e/fixtures/ups-deps-stop/orphan.yaml b/pkg/e2e/fixtures/ups-deps-stop/orphan.yaml deleted file mode 100644 index 69e50e39ca7..00000000000 --- a/pkg/e2e/fixtures/ups-deps-stop/orphan.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - orphan: - image: alpine - init: true - command: /bin/sh -c 'while true; do echo "hello orphan"; sleep 1; done' diff --git a/pkg/e2e/fixtures/volume-test/compose.yaml b/pkg/e2e/fixtures/volume-test/compose.yaml deleted file mode 100644 index 7567da42ef8..00000000000 --- a/pkg/e2e/fixtures/volume-test/compose.yaml +++ /dev/null @@ -1,33 +0,0 @@ -services: - nginx: - build: nginx-build - volumes: - - ./static:/usr/share/nginx/html - ports: - - 8090:80 - - nginx2: - build: nginx-build - volumes: - - staticVol:/usr/share/nginx/html:ro - - /usr/src/app/node_modules - - otherVol:/usr/share/nginx/test - ports: - - 9090:80 - configs: - - myconfig - secrets: - - mysecret - -volumes: - staticVol: - otherVol: - name: myVolume - -configs: - myconfig: - file: ./static/index.html - -secrets: - mysecret: - file: ./static/index.html \ No newline at end of file diff --git a/pkg/e2e/fixtures/volume-test/nginx-build/Dockerfile b/pkg/e2e/fixtures/volume-test/nginx-build/Dockerfile deleted file mode 100644 index a05029ea412..00000000000 --- a/pkg/e2e/fixtures/volume-test/nginx-build/Dockerfile +++ /dev/null @@ -1,15 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM nginx:alpine diff --git a/pkg/e2e/fixtures/volume-test/static/index.html b/pkg/e2e/fixtures/volume-test/static/index.html deleted file mode 100644 index 63159b9e91b..00000000000 --- a/pkg/e2e/fixtures/volume-test/static/index.html +++ /dev/null @@ -1,10 +0,0 @@ - - - - - Docker Nginx - - -

Hello from Nginx container

- - diff --git a/pkg/e2e/fixtures/volumes/compose.yaml b/pkg/e2e/fixtures/volumes/compose.yaml deleted file mode 100644 index 4aad0482b5d..00000000000 --- a/pkg/e2e/fixtures/volumes/compose.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - with_image: - image: alpine - command: "ls -al /mnt/image" - volumes: - - type: image - source: nginx:alpine - target: /mnt/image - image: - subpath: usr/share/nginx/html/ \ No newline at end of file diff --git a/pkg/e2e/fixtures/wait/compose.yaml b/pkg/e2e/fixtures/wait/compose.yaml deleted file mode 100644 index 1a001e6fa87..00000000000 --- a/pkg/e2e/fixtures/wait/compose.yaml +++ /dev/null @@ -1,11 +0,0 @@ -services: - faster: - image: alpine - command: sleep 2 - slower: - image: alpine - command: sleep 5 - infinity: - image: alpine - command: sleep infinity - diff --git a/pkg/e2e/fixtures/watch/compose.yaml b/pkg/e2e/fixtures/watch/compose.yaml deleted file mode 100644 index 7e5e0d28bf4..00000000000 --- a/pkg/e2e/fixtures/watch/compose.yaml +++ /dev/null @@ -1,43 +0,0 @@ -x-dev: &x-dev - watch: - - action: sync - path: ./data - target: /app/data - ignore: - - '*.foo' - - ./ignored - - action: sync+restart - path: ./config - target: /app/config - -services: - alpine: - build: - dockerfile_inline: |- - FROM alpine - RUN mkdir -p /app/data - RUN mkdir -p /app/config - init: true - command: sleep infinity - develop: *x-dev - busybox: - build: - dockerfile_inline: |- - FROM busybox - RUN mkdir -p /app/data - RUN mkdir -p /app/config - init: true - command: sleep infinity - develop: *x-dev - debian: - build: - dockerfile_inline: |- - FROM debian - RUN mkdir -p /app/data - RUN mkdir -p /app/config - init: true - command: sleep infinity - volumes: - - ./dat:/app/dat - - ./data-logs:/app/data-logs - develop: *x-dev diff --git a/pkg/e2e/fixtures/watch/config/file.config b/pkg/e2e/fixtures/watch/config/file.config deleted file mode 100644 index 227b0b6ed7f..00000000000 --- a/pkg/e2e/fixtures/watch/config/file.config +++ /dev/null @@ -1 +0,0 @@ -This is a config file diff --git a/pkg/e2e/fixtures/watch/dat/meow.dat b/pkg/e2e/fixtures/watch/dat/meow.dat deleted file mode 100644 index 0dd3d19a01d..00000000000 --- a/pkg/e2e/fixtures/watch/dat/meow.dat +++ /dev/null @@ -1 +0,0 @@ -i am a wannabe cat diff --git a/pkg/e2e/fixtures/watch/data-logs/server.log b/pkg/e2e/fixtures/watch/data-logs/server.log deleted file mode 100644 index b6b65a681d9..00000000000 --- a/pkg/e2e/fixtures/watch/data-logs/server.log +++ /dev/null @@ -1 +0,0 @@ -[INFO] Server started successfully on port 8080 diff --git a/pkg/e2e/fixtures/watch/data/hello.txt b/pkg/e2e/fixtures/watch/data/hello.txt deleted file mode 100644 index 95d09f2b101..00000000000 --- a/pkg/e2e/fixtures/watch/data/hello.txt +++ /dev/null @@ -1 +0,0 @@ -hello world \ No newline at end of file diff --git a/pkg/e2e/fixtures/watch/exec.yaml b/pkg/e2e/fixtures/watch/exec.yaml deleted file mode 100644 index 9d232ac76b3..00000000000 --- a/pkg/e2e/fixtures/watch/exec.yaml +++ /dev/null @@ -1,15 +0,0 @@ -services: - test: - build: - dockerfile_inline: FROM alpine - command: ping localhost - volumes: - - /data - develop: - watch: - - path: . - target: /data - initial_sync: true - action: sync+exec - exec: - command: echo "SUCCESS" \ No newline at end of file diff --git a/pkg/e2e/fixtures/watch/include.yaml b/pkg/e2e/fixtures/watch/include.yaml deleted file mode 100644 index ccd9d45042b..00000000000 --- a/pkg/e2e/fixtures/watch/include.yaml +++ /dev/null @@ -1,12 +0,0 @@ -services: - a: - build: - dockerfile_inline: | - FROM nginx - RUN mkdir /data/ - develop: - watch: - - path: . - include: A.* - target: /data/ - action: sync diff --git a/pkg/e2e/fixtures/watch/rebuild.yaml b/pkg/e2e/fixtures/watch/rebuild.yaml deleted file mode 100644 index 561659a6ca4..00000000000 --- a/pkg/e2e/fixtures/watch/rebuild.yaml +++ /dev/null @@ -1,31 +0,0 @@ -services: - a: - build: - dockerfile_inline: | - FROM nginx - RUN mkdir /data - COPY test /data/a - develop: - watch: - - path: test - action: rebuild - b: - build: - dockerfile_inline: | - FROM nginx - RUN mkdir /data - COPY test /data/b - develop: - watch: - - path: test - action: rebuild - c: - build: - dockerfile_inline: | - FROM nginx - RUN mkdir /data - COPY test /data/c - develop: - watch: - - path: test - action: rebuild diff --git a/pkg/e2e/fixtures/watch/with-external-network.yaml b/pkg/e2e/fixtures/watch/with-external-network.yaml deleted file mode 100644 index e9c948920c0..00000000000 --- a/pkg/e2e/fixtures/watch/with-external-network.yaml +++ /dev/null @@ -1,19 +0,0 @@ - -services: - ext-alpine: - build: - dockerfile_inline: |- - FROM alpine - init: true - command: sleep infinity - develop: - watch: - - action: rebuild - path: .env - networks: - - external_network_test - -networks: - external_network_test: - name: e2e-watch-external_network_test - external: true diff --git a/pkg/e2e/fixtures/watch/x-initialSync.yaml b/pkg/e2e/fixtures/watch/x-initialSync.yaml deleted file mode 100644 index 6a954bd6aee..00000000000 --- a/pkg/e2e/fixtures/watch/x-initialSync.yaml +++ /dev/null @@ -1,15 +0,0 @@ -services: - test: - build: - dockerfile_inline: FROM alpine - command: ping localhost - volumes: - - /data - develop: - watch: - - path: . - target: /data - action: sync+exec - exec: - command: echo "SUCCESS" - x-initialSync: true \ No newline at end of file diff --git a/pkg/e2e/fixtures/wrong-composefile/build-error.yml b/pkg/e2e/fixtures/wrong-composefile/build-error.yml deleted file mode 100644 index ea13a847bf1..00000000000 --- a/pkg/e2e/fixtures/wrong-composefile/build-error.yml +++ /dev/null @@ -1,3 +0,0 @@ -services: - simple: - build: service1 diff --git a/pkg/e2e/fixtures/wrong-composefile/compose.yaml b/pkg/e2e/fixtures/wrong-composefile/compose.yaml deleted file mode 100644 index b1c12a69211..00000000000 --- a/pkg/e2e/fixtures/wrong-composefile/compose.yaml +++ /dev/null @@ -1,4 +0,0 @@ -services: - simple: - image: nginx:alpine - wrongField: test diff --git a/pkg/e2e/fixtures/wrong-composefile/service1/Dockerfile b/pkg/e2e/fixtures/wrong-composefile/service1/Dockerfile deleted file mode 100644 index b57af61969d..00000000000 --- a/pkg/e2e/fixtures/wrong-composefile/service1/Dockerfile +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2020 Docker Compose CLI authors - -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at - -# http://www.apache.org/licenses/LICENSE-2.0 - -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -FROM nginx - -WRONG DOCKERFILE diff --git a/pkg/e2e/fixtures/wrong-composefile/unknown-image.yml b/pkg/e2e/fixtures/wrong-composefile/unknown-image.yml deleted file mode 100644 index 3000c46863b..00000000000 --- a/pkg/e2e/fixtures/wrong-composefile/unknown-image.yml +++ /dev/null @@ -1,3 +0,0 @@ -services: - simple: - image: unknownimage diff --git a/pkg/e2e/framework.go b/pkg/e2e/framework.go deleted file mode 100644 index a4624d1de8d..00000000000 --- a/pkg/e2e/framework.go +++ /dev/null @@ -1,520 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "encoding/json" - "errors" - "fmt" - "io" - "io/fs" - "net/http" - "os" - "path/filepath" - "runtime" - "strings" - "testing" - "time" - - cp "github.com/otiai10/copy" - "github.com/stretchr/testify/require" - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" - "gotest.tools/v3/poll" - - "github.com/docker/compose/v5/cmd/compose" -) - -var ( - // DockerExecutableName is the OS dependent Docker CLI binary name - DockerExecutableName = "docker" - - // DockerComposeExecutableName is the OS dependent Docker CLI binary name - DockerComposeExecutableName = "docker-" + compose.PluginName - - // DockerScanExecutableName is the OS dependent Docker Scan plugin binary name - DockerScanExecutableName = "docker-scan" - - // DockerBuildxExecutableName is the Os dependent Buildx plugin binary name - DockerBuildxExecutableName = "docker-buildx" - - // DockerModelExecutableName is the Os dependent Docker-Model plugin binary name - DockerModelExecutableName = "docker-model" - - // WindowsExecutableSuffix is the Windows executable suffix - WindowsExecutableSuffix = ".exe" -) - -func init() { - if runtime.GOOS == "windows" { - DockerExecutableName += WindowsExecutableSuffix - DockerComposeExecutableName += WindowsExecutableSuffix - DockerScanExecutableName += WindowsExecutableSuffix - DockerBuildxExecutableName += WindowsExecutableSuffix - } -} - -// CLI is used to wrap the CLI for end to end testing -type CLI struct { - // ConfigDir for Docker configuration (set as DOCKER_CONFIG) - ConfigDir string - - // HomeDir for tools that look for user files (set as HOME) - HomeDir string - - // env overrides to apply to every invoked command - // - // To populate, use WithEnv when creating a CLI instance. - env []string -} - -// CLIOption to customize behavior for all commands for a CLI instance. -type CLIOption func(c *CLI) - -// NewParallelCLI marks the parent test as parallel and returns a CLI instance -// suitable for usage across child tests. -func NewParallelCLI(t *testing.T, opts ...CLIOption) *CLI { - t.Helper() - t.Parallel() - return NewCLI(t, opts...) -} - -// NewCLI creates a CLI instance for running E2E tests. -func NewCLI(t testing.TB, opts ...CLIOption) *CLI { - t.Helper() - - configDir := t.TempDir() - copyLocalConfig(t, configDir) - initializePlugins(t, configDir) - initializeContextDir(t, configDir) - - c := &CLI{ - ConfigDir: configDir, - HomeDir: t.TempDir(), - } - - for _, opt := range opts { - opt(c) - } - c.RunDockerComposeCmdNoCheck(t, "version") - return c -} - -// WithEnv sets environment variables that will be passed to commands. -func WithEnv(env ...string) CLIOption { - return func(c *CLI) { - c.env = append(c.env, env...) - } -} - -func copyLocalConfig(t testing.TB, configDir string) { - t.Helper() - - // copy local config.json if exists - localConfig := filepath.Join(os.Getenv("HOME"), ".docker", "config.json") - // if no config present just continue - if _, err := os.Stat(localConfig); err != nil { - // copy the local config.json to the test config dir - CopyFile(t, localConfig, filepath.Join(configDir, "config.json")) - } -} - -// initializePlugins copies the necessary plugin files to the temporary config -// directory for the test. -func initializePlugins(t testing.TB, configDir string) { - t.Cleanup(func() { - if t.Failed() { - if conf, err := os.ReadFile(filepath.Join(configDir, "config.json")); err == nil { - t.Logf("Config: %s\n", string(conf)) - } - t.Log("Contents of config dir:") - for _, p := range dirContents(configDir) { - t.Logf(" - %s", p) - } - } - }) - - require.NoError(t, os.MkdirAll(filepath.Join(configDir, "cli-plugins"), 0o755), - "Failed to create cli-plugins directory") - composePlugin, err := findExecutable(DockerComposeExecutableName) - if errors.Is(err, fs.ErrNotExist) { - t.Logf("WARNING: docker-compose cli-plugin not found") - } - - if err == nil { - CopyFile(t, composePlugin, filepath.Join(configDir, "cli-plugins", DockerComposeExecutableName)) - buildxPlugin, err := findPluginExecutable(DockerBuildxExecutableName) - if err != nil { - t.Logf("WARNING: docker-buildx cli-plugin not found, using default buildx installation.") - } else { - CopyFile(t, buildxPlugin, filepath.Join(configDir, "cli-plugins", DockerBuildxExecutableName)) - } - // We don't need a functional scan plugin, but a valid plugin binary - CopyFile(t, composePlugin, filepath.Join(configDir, "cli-plugins", DockerScanExecutableName)) - - modelPlugin, err := findPluginExecutable(DockerModelExecutableName) - if err != nil { - t.Logf("WARNING: docker-model cli-plugin not found") - } else { - CopyFile(t, modelPlugin, filepath.Join(configDir, "cli-plugins", DockerModelExecutableName)) - } - } -} - -func initializeContextDir(t testing.TB, configDir string) { - dockerUserDir := ".docker/contexts" - userDir, err := os.UserHomeDir() - require.NoError(t, err, "Failed to get user home directory") - userContextsDir := filepath.Join(userDir, dockerUserDir) - if checkExists(userContextsDir) { - dstContexts := filepath.Join(configDir, "contexts") - require.NoError(t, cp.Copy(userContextsDir, dstContexts), "Failed to copy contexts directory") - } -} - -func checkExists(path string) bool { - _, err := os.Stat(path) - return err == nil -} - -func dirContents(dir string) []string { - var res []string - _ = filepath.Walk(dir, func(path string, info os.FileInfo, err error) error { - res = append(res, path) - return nil - }) - return res -} - -func findExecutable(executableName string) (string, error) { - bin := os.Getenv("COMPOSE_E2E_BIN_PATH") - if bin == "" { - _, filename, _, _ := runtime.Caller(0) - buildPath := filepath.Join(filepath.Dir(filename), "..", "..", "bin", "build") - var err error - bin, err = filepath.Abs(filepath.Join(buildPath, executableName)) - if err != nil { - return "", err - } - } - - if _, err := os.Stat(bin); err == nil { - return bin, nil - } - return "", fmt.Errorf("looking for %q: %w", bin, fs.ErrNotExist) -} - -func findPluginExecutable(pluginExecutableName string) (string, error) { - dockerUserDir := ".docker/cli-plugins" - userDir, err := os.UserHomeDir() - if err != nil { - return "", err - } - candidates := []string{ - filepath.Join(userDir, dockerUserDir), - "/usr/local/lib/docker/cli-plugins", - "/usr/local/libexec/docker/cli-plugins", - "/usr/lib/docker/cli-plugins", - "/usr/libexec/docker/cli-plugins", - } - for _, path := range candidates { - bin, err := filepath.Abs(filepath.Join(path, pluginExecutableName)) - if err != nil { - return "", err - } - if _, err := os.Stat(bin); err == nil { - return bin, nil - } - } - - return "", fmt.Errorf("plugin not found %s: %w", pluginExecutableName, os.ErrNotExist) -} - -// CopyFile copies a file from a sourceFile to a destinationFile setting permissions to 0755 -func CopyFile(t testing.TB, sourceFile string, destinationFile string) { - t.Helper() - - src, err := os.Open(sourceFile) - require.NoError(t, err, "Failed to open source file: %s") - //nolint:errcheck - defer src.Close() - - dst, err := os.OpenFile(destinationFile, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0o755) - require.NoError(t, err, "Failed to open destination file: %s", destinationFile) - //nolint:errcheck - defer dst.Close() - - _, err = io.Copy(dst, src) - require.NoError(t, err, "Failed to copy file: %s", sourceFile) -} - -// BaseEnvironment provides the minimal environment variables used across all -// Docker / Compose commands. -func (c *CLI) BaseEnvironment() []string { - env := []string{ - "HOME=" + c.HomeDir, - "USER=" + os.Getenv("USER"), - "DOCKER_CONFIG=" + c.ConfigDir, - "KUBECONFIG=invalid", - "PATH=" + os.Getenv("PATH"), - } - dockerContextEnv, ok := os.LookupEnv("DOCKER_CONTEXT") - if ok { - env = append(env, "DOCKER_CONTEXT="+dockerContextEnv) - } - - if coverdir, ok := os.LookupEnv("GOCOVERDIR"); ok { - _, filename, _, _ := runtime.Caller(0) - root := filepath.Join(filepath.Dir(filename), "..", "..") - coverdir = filepath.Join(root, coverdir) - env = append(env, fmt.Sprintf("GOCOVERDIR=%s", coverdir)) - } - return env -} - -// NewCmd creates a cmd object configured with the test environment set -func (c *CLI) NewCmd(command string, args ...string) icmd.Cmd { - return icmd.Cmd{ - Command: append([]string{command}, args...), - Env: append(c.BaseEnvironment(), c.env...), - } -} - -// NewCmdWithEnv creates a cmd object configured with the test environment set with additional env vars -func (c *CLI) NewCmdWithEnv(envvars []string, command string, args ...string) icmd.Cmd { - // base env -> CLI overrides -> cmd overrides - cmdEnv := append(c.BaseEnvironment(), c.env...) - cmdEnv = append(cmdEnv, envvars...) - return icmd.Cmd{ - Command: append([]string{command}, args...), - Env: cmdEnv, - } -} - -// MetricsSocket get the path where test metrics will be sent -func (c *CLI) MetricsSocket() string { - return filepath.Join(c.ConfigDir, "docker-cli.sock") -} - -// NewDockerCmd creates a docker cmd without running it -func (c *CLI) NewDockerCmd(t testing.TB, args ...string) icmd.Cmd { - t.Helper() - for _, arg := range args { - if arg == compose.PluginName { - t.Fatal("This test called 'RunDockerCmd' for 'compose'. Please prefer 'RunDockerComposeCmd' to be able to test as a plugin and standalone") - } - } - return c.NewCmd(DockerExecutableName, args...) -} - -// RunDockerOrExitError runs a docker command and returns a result -func (c *CLI) RunDockerOrExitError(t testing.TB, args ...string) *icmd.Result { - t.Helper() - t.Logf("\t[%s] docker %s\n", t.Name(), strings.Join(args, " ")) - return icmd.RunCmd(c.NewDockerCmd(t, args...)) -} - -// RunCmd runs a command, expects no error and returns a result -func (c *CLI) RunCmd(t testing.TB, args ...string) *icmd.Result { - t.Helper() - t.Logf("\t[%s] %s\n", t.Name(), strings.Join(args, " ")) - assert.Assert(t, len(args) >= 1, "require at least one command in parameters") - res := icmd.RunCmd(c.NewCmd(args[0], args[1:]...)) - res.Assert(t, icmd.Success) - return res -} - -// RunCmdInDir runs a command in a given dir, expects no error and returns a result -func (c *CLI) RunCmdInDir(t testing.TB, dir string, args ...string) *icmd.Result { - t.Helper() - t.Logf("\t[%s] %s\n", t.Name(), strings.Join(args, " ")) - assert.Assert(t, len(args) >= 1, "require at least one command in parameters") - cmd := c.NewCmd(args[0], args[1:]...) - cmd.Dir = dir - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Success) - return res -} - -// RunDockerCmd runs a docker command, expects no error and returns a result -func (c *CLI) RunDockerCmd(t testing.TB, args ...string) *icmd.Result { - t.Helper() - res := c.RunDockerOrExitError(t, args...) - res.Assert(t, icmd.Success) - return res -} - -// RunDockerComposeCmd runs a docker compose command, expects no error and returns a result -func (c *CLI) RunDockerComposeCmd(t testing.TB, args ...string) *icmd.Result { - t.Helper() - res := c.RunDockerComposeCmdNoCheck(t, args...) - res.Assert(t, icmd.Success) - return res -} - -// RunDockerComposeCmdNoCheck runs a docker compose command, don't presume of any expectation and returns a result -func (c *CLI) RunDockerComposeCmdNoCheck(t testing.TB, args ...string) *icmd.Result { - t.Helper() - cmd := c.NewDockerComposeCmd(t, args...) - cmd.Stdout = os.Stdout - t.Logf("Running command: %s", strings.Join(cmd.Command, " ")) - return icmd.RunCmd(cmd) -} - -// NewDockerComposeCmd creates a command object for Compose, either in plugin -// or standalone mode (based on build tags). -func (c *CLI) NewDockerComposeCmd(t testing.TB, args ...string) icmd.Cmd { - t.Helper() - if composeStandaloneMode { - return c.NewCmd(ComposeStandalonePath(t), args...) - } - args = append([]string{"compose"}, args...) - return c.NewCmd(DockerExecutableName, args...) -} - -// ComposeStandalonePath returns the path to the locally-built Compose -// standalone binary from the repo. -// -// This function will fail the test immediately if invoked when not running -// in standalone test mode. -func ComposeStandalonePath(t testing.TB) string { - t.Helper() - if !composeStandaloneMode { - require.Fail(t, "Not running in standalone mode") - } - composeBinary, err := findExecutable(DockerComposeExecutableName) - require.NoError(t, err, "Could not find standalone Compose binary (%q)", - DockerComposeExecutableName) - return composeBinary -} - -// StdoutContains returns a predicate on command result expecting a string in stdout -func StdoutContains(expected string) func(*icmd.Result) bool { - return func(res *icmd.Result) bool { - return strings.Contains(res.Stdout(), expected) - } -} - -func IsHealthy(service string) func(res *icmd.Result) bool { - return func(res *icmd.Result) bool { - type state struct { - Name string `json:"name"` - Health string `json:"health"` - } - - decoder := json.NewDecoder(strings.NewReader(res.Stdout())) - for decoder.More() { - ps := state{} - err := decoder.Decode(&ps) - if err != nil { - return false - } - if ps.Name == service && ps.Health == "healthy" { - return true - } - } - return false - } -} - -// WaitForCmdResult try to execute a cmd until resulting output matches given predicate -func (c *CLI) WaitForCmdResult( - t testing.TB, - command icmd.Cmd, - predicate func(*icmd.Result) bool, - timeout time.Duration, - delay time.Duration, -) { - t.Helper() - assert.Assert(t, timeout.Nanoseconds() > delay.Nanoseconds(), "timeout must be greater than delay") - var res *icmd.Result - checkStopped := func(logt poll.LogT) poll.Result { - fmt.Printf("\t[%s] %s\n", t.Name(), strings.Join(command.Command, " ")) - res = icmd.RunCmd(command) - if !predicate(res) { - return poll.Continue("Cmd output did not match requirement: %q", res.Combined()) - } - return poll.Success() - } - poll.WaitOn(t, checkStopped, poll.WithDelay(delay), poll.WithTimeout(timeout)) -} - -// WaitForCondition wait for predicate to execute to true -func (c *CLI) WaitForCondition( - t testing.TB, - predicate func() (bool, string), - timeout time.Duration, - delay time.Duration, -) { - t.Helper() - checkStopped := func(logt poll.LogT) poll.Result { - pass, description := predicate() - if !pass { - return poll.Continue("Condition not met: %q", description) - } - return poll.Success() - } - poll.WaitOn(t, checkStopped, poll.WithDelay(delay), poll.WithTimeout(timeout)) -} - -// Lines split output into lines -func Lines(output string) []string { - return strings.Split(strings.TrimSpace(output), "\n") -} - -// HTTPGetWithRetry performs an HTTP GET on an `endpoint`, using retryDelay also as a request timeout. -// In the case of an error or the response status is not the expected one, it retries the same request, -// returning the response body as a string (empty if we could not reach it) -func HTTPGetWithRetry( - t testing.TB, - endpoint string, - expectedStatus int, - retryDelay time.Duration, - timeout time.Duration, -) string { - t.Helper() - var ( - r *http.Response - err error - ) - client := &http.Client{ - Timeout: retryDelay, - } - fmt.Printf("\t[%s] GET %s\n", t.Name(), endpoint) - checkUp := func(t poll.LogT) poll.Result { - r, err = client.Get(endpoint) - if err != nil { - return poll.Continue("reaching %q: Error %s", endpoint, err.Error()) - } - if r.StatusCode == expectedStatus { - return poll.Success() - } - return poll.Continue("reaching %q: %d != %d", endpoint, r.StatusCode, expectedStatus) - } - poll.WaitOn(t, checkUp, poll.WithDelay(retryDelay), poll.WithTimeout(timeout)) - if r != nil { - b, err := io.ReadAll(r.Body) - assert.NilError(t, err) - return string(b) - } - return "" -} - -func (c *CLI) cleanupWithDown(t testing.TB, project string, args ...string) { - t.Helper() - c.RunDockerComposeCmd(t, append([]string{"-p", project, "down", "-v", "--remove-orphans"}, args...)...) -} diff --git a/pkg/e2e/healthcheck_test.go b/pkg/e2e/healthcheck_test.go deleted file mode 100644 index 227d835a8d3..00000000000 --- a/pkg/e2e/healthcheck_test.go +++ /dev/null @@ -1,55 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - "time" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestStartInterval(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-start-interval" - - t.Cleanup(func() { - c.cleanupWithDown(t, projectName) - }) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/start_interval/compose.yaml", "--project-name", projectName, "up", "--wait", "-d", "error") - res.Assert(t, icmd.Expected{ExitCode: 1, Err: "healthcheck.start_interval requires healthcheck.start_period to be set"}) - - timeout := time.After(30 * time.Second) - done := make(chan bool) - go func() { - //nolint:nolintlint,testifylint // helper asserts inside goroutine; acceptable in this e2e test - res := c.RunDockerComposeCmd(t, "-f", "fixtures/start_interval/compose.yaml", "--project-name", projectName, "up", "--wait", "-d", "test") - out := res.Combined() - assert.Assert(t, strings.Contains(out, "Healthy"), out) - done <- true - }() - - select { - case <-timeout: - t.Fatal("test did not finish in time") - case <-done: - break - } -} diff --git a/pkg/e2e/hooks_test.go b/pkg/e2e/hooks_test.go deleted file mode 100644 index b77500c6bf6..00000000000 --- a/pkg/e2e/hooks_test.go +++ /dev/null @@ -1,109 +0,0 @@ -/* -Copyright 2023 Docker Compose CLI authors - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package e2e - -import ( - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestPostStartHookInError(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "hooks-post-start-failure" - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "-v", "--remove-orphans", "-t", "0") - }) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/hooks/poststart/compose-error.yaml", "--project-name", projectName, "up", "-d") - res.Assert(t, icmd.Expected{ExitCode: 1}) - assert.Assert(t, strings.Contains(res.Combined(), "test hook exited with status 127"), res.Combined()) -} - -func TestPostStartHookSuccess(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "hooks-post-start-success" - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "-v", "--remove-orphans", "-t", "0") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/hooks/poststart/compose-success.yaml", "--project-name", projectName, "up", "-d") - res.Assert(t, icmd.Expected{ExitCode: 0}) -} - -func TestPreStopHookSuccess(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "hooks-pre-stop-success" - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/hooks/prestop/compose-success.yaml", "--project-name", projectName, "down", "-v", "--remove-orphans", "-t", "0") - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/hooks/prestop/compose-success.yaml", "--project-name", projectName, "up", "-d") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - res = c.RunDockerComposeCmd(t, "-f", "fixtures/hooks/prestop/compose-success.yaml", "--project-name", projectName, "down", "-v", "--remove-orphans", "-t", "0") - res.Assert(t, icmd.Expected{ExitCode: 0}) -} - -func TestPreStopHookInError(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "hooks-pre-stop-failure" - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "-f", "fixtures/hooks/prestop/compose-success.yaml", "--project-name", projectName, "down", "-v", "--remove-orphans", "-t", "0") - }) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/hooks/prestop/compose-error.yaml", "--project-name", projectName, "up", "-d") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - res = c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/hooks/prestop/compose-error.yaml", "--project-name", projectName, "down", "-v", "--remove-orphans", "-t", "0") - res.Assert(t, icmd.Expected{ExitCode: 1}) - assert.Assert(t, strings.Contains(res.Combined(), "sample hook exited with status 127")) -} - -func TestPreStopHookSuccessWithPreviousStop(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "hooks-pre-stop-success-with-previous-stop" - - t.Cleanup(func() { - res := c.RunDockerComposeCmd(t, "-f", "fixtures/hooks/compose.yaml", "--project-name", projectName, "down", "-v", "--remove-orphans", "-t", "0") - res.Assert(t, icmd.Expected{ExitCode: 0}) - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/hooks/compose.yaml", "--project-name", projectName, "up", "-d") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - res = c.RunDockerComposeCmd(t, "-f", "fixtures/hooks/compose.yaml", "--project-name", projectName, "stop", "sample") - res.Assert(t, icmd.Expected{ExitCode: 0}) -} - -func TestPostStartAndPreStopHook(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "hooks-post-start-and-pre-stop" - - t.Cleanup(func() { - res := c.RunDockerComposeCmd(t, "-f", "fixtures/hooks/compose.yaml", "--project-name", projectName, "down", "-v", "--remove-orphans", "-t", "0") - res.Assert(t, icmd.Expected{ExitCode: 0}) - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/hooks/compose.yaml", "--project-name", projectName, "up", "-d") - res.Assert(t, icmd.Expected{ExitCode: 0}) -} diff --git a/pkg/e2e/ipc_test.go b/pkg/e2e/ipc_test.go deleted file mode 100644 index 7a46192a601..00000000000 --- a/pkg/e2e/ipc_test.go +++ /dev/null @@ -1,64 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "strings" - "testing" - - "gotest.tools/v3/icmd" -) - -func TestIPC(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "ipc_e2e" - var cid string - t.Run("create ipc mode container", func(t *testing.T) { - res := c.RunDockerCmd(t, "run", "-d", "--rm", "--ipc=shareable", "--name", "ipc_mode_container", "alpine", - "top") - cid = strings.Trim(res.Stdout(), "\n") - }) - - t.Run("up", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/ipc-test/compose.yaml", "--project-name", projectName, "up", "-d") - }) - - t.Run("check running project", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-p", projectName, "ps") - res.Assert(t, icmd.Expected{Out: `shareable`}) - }) - - t.Run("check ipcmode in container inspect", func(t *testing.T) { - res := c.RunDockerCmd(t, "inspect", projectName+"-shareable-1") - res.Assert(t, icmd.Expected{Out: `"IpcMode": "shareable",`}) - - res = c.RunDockerCmd(t, "inspect", projectName+"-service-1") - res.Assert(t, icmd.Expected{Out: `"IpcMode": "container:`}) - - res = c.RunDockerCmd(t, "inspect", projectName+"-container-1") - res.Assert(t, icmd.Expected{Out: fmt.Sprintf(`"IpcMode": "container:%s",`, cid)}) - }) - - t.Run("down", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - t.Run("remove ipc mode container", func(t *testing.T) { - _ = c.RunDockerCmd(t, "rm", "-f", "ipc_mode_container") - }) -} diff --git a/pkg/e2e/logs_test.go b/pkg/e2e/logs_test.go deleted file mode 100644 index 6250bee8c0f..00000000000 --- a/pkg/e2e/logs_test.go +++ /dev/null @@ -1,131 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "io" - "os" - "path/filepath" - "strings" - "testing" - "time" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" - "gotest.tools/v3/poll" -) - -func TestLocalComposeLogs(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "compose-e2e-logs" - - t.Run("up", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/logs-test/compose.yaml", "--project-name", projectName, "up", "-d") - }) - - t.Run("logs", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "logs") - res.Assert(t, icmd.Expected{Out: `PING localhost`}) - res.Assert(t, icmd.Expected{Out: `hello`}) - }) - - t.Run("logs ping", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "logs", "ping") - res.Assert(t, icmd.Expected{Out: `PING localhost`}) - assert.Assert(t, !strings.Contains(res.Stdout(), "hello")) - }) - - t.Run("logs hello", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "logs", "hello", "ping") - res.Assert(t, icmd.Expected{Out: `PING localhost`}) - res.Assert(t, icmd.Expected{Out: `hello`}) - }) - - t.Run("logs hello index", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "logs", "--index", "2", "hello") - - // docker-compose logs hello - // logs-test-hello-2 | hello - // logs-test-hello-1 | hello - t.Log(res.Stdout()) - assert.Assert(t, !strings.Contains(res.Stdout(), "hello-1")) - assert.Assert(t, strings.Contains(res.Stdout(), "hello-2")) - }) - - t.Run("down", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) -} - -func TestLocalComposeLogsFollow(t *testing.T) { - c := NewCLI(t, WithEnv("REPEAT=20")) - const projectName = "compose-e2e-logs" - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/logs-test/compose.yaml", "--project-name", projectName, "up", "-d", "ping") - - cmd := c.NewDockerComposeCmd(t, "--project-name", projectName, "logs", "-f") - res := icmd.StartCmd(cmd) - t.Cleanup(func() { - _ = res.Cmd.Process.Kill() - }) - - poll.WaitOn(t, expectOutput(res, "ping-1 "), poll.WithDelay(100*time.Millisecond), poll.WithTimeout(1*time.Second)) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/logs-test/compose.yaml", "--project-name", projectName, "up", "-d") - - poll.WaitOn(t, expectOutput(res, "hello-1 "), poll.WithDelay(100*time.Millisecond), poll.WithTimeout(1*time.Second)) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/logs-test/compose.yaml", "--project-name", projectName, "up", "-d", "--scale", "ping=2", "ping") - - poll.WaitOn(t, expectOutput(res, "ping-2 "), poll.WithDelay(100*time.Millisecond), poll.WithTimeout(20*time.Second)) -} - -func TestLocalComposeLargeLogs(t *testing.T) { - const projectName = "compose-e2e-large_logs" - file := filepath.Join(t.TempDir(), "large.txt") - c := NewCLI(t, WithEnv("FILE="+file)) - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - f, err := os.Create(file) - assert.NilError(t, err) - for i := 0; i < 300_000; i++ { - _, err := io.WriteString(f, fmt.Sprintf("This is line %d in a laaaarge text file\n", i)) - assert.NilError(t, err) - } - assert.NilError(t, f.Close()) - - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/logs-test/cat.yaml", "--project-name", projectName, "up", "--abort-on-container-exit", "--menu=false") - cmd.Stdout = io.Discard - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{Out: "test-1 exited with code 0"}) -} - -func expectOutput(res *icmd.Result, expected string) func(t poll.LogT) poll.Result { - return func(t poll.LogT) poll.Result { - if strings.Contains(res.Stdout(), expected) { - return poll.Success() - } - return poll.Continue("condition not met") - } -} diff --git a/pkg/e2e/main_test.go b/pkg/e2e/main_test.go deleted file mode 100644 index 415987bc71b..00000000000 --- a/pkg/e2e/main_test.go +++ /dev/null @@ -1,27 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "os" - "testing" -) - -func TestMain(m *testing.M) { - exitCode := m.Run() - os.Exit(exitCode) -} diff --git a/pkg/e2e/model_test.go b/pkg/e2e/model_test.go deleted file mode 100644 index f30d7c5bb0c..00000000000 --- a/pkg/e2e/model_test.go +++ /dev/null @@ -1,29 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" -) - -func TestComposeModel(t *testing.T) { - t.Skip("waiting for docker-model release") - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "model-test") - - c.RunDockerComposeCmd(t, "-f", "./fixtures/model/compose.yaml", "run", "test", "sh", "-c", "curl ${FOO_URL}") -} diff --git a/pkg/e2e/networks_test.go b/pkg/e2e/networks_test.go deleted file mode 100644 index f556681e664..00000000000 --- a/pkg/e2e/networks_test.go +++ /dev/null @@ -1,220 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "net/http" - "strings" - "testing" - "time" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestNetworks(t *testing.T) { - // fixture is shared with TestNetworkModes and is not safe to run concurrently - const projectName = "network-e2e" - c := NewCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME="+projectName, - "COMPOSE_FILE=./fixtures/network-test/compose.yaml", - )) - - c.RunDockerComposeCmd(t, "down", "-t0", "-v") - - c.RunDockerComposeCmd(t, "up", "-d") - - res := c.RunDockerComposeCmd(t, "ps") - res.Assert(t, icmd.Expected{Out: `web`}) - - endpoint := "http://localhost:80" - output := HTTPGetWithRetry(t, endpoint+"/words/noun", http.StatusOK, 2*time.Second, 20*time.Second) - assert.Assert(t, strings.Contains(output, `"word":`)) - - res = c.RunDockerCmd(t, "network", "ls") - res.Assert(t, icmd.Expected{Out: projectName + "_dbnet"}) - res.Assert(t, icmd.Expected{Out: "microservices"}) - - res = c.RunDockerComposeCmd(t, "port", "words", "8080") - res.Assert(t, icmd.Expected{Out: `0.0.0.0:8080`}) - - c.RunDockerComposeCmd(t, "down", "-t0", "-v") - res = c.RunDockerCmd(t, "network", "ls") - assert.Assert(t, !strings.Contains(res.Combined(), projectName), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "microservices"), res.Combined()) -} - -func TestNetworkAliases(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "network_alias_e2e" - defer c.cleanupWithDown(t, projectName) - - t.Run("up", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/network-alias/compose.yaml", "--project-name", projectName, "up", - "-d") - }) - - t.Run("curl alias", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/network-alias/compose.yaml", "--project-name", projectName, - "exec", "-T", "container1", "curl", "http://alias-of-container2/") - assert.Assert(t, strings.Contains(res.Stdout(), "Welcome to nginx!"), res.Stdout()) - }) - - t.Run("curl links", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/network-alias/compose.yaml", "--project-name", projectName, - "exec", "-T", "container1", "curl", "http://container/") - assert.Assert(t, strings.Contains(res.Stdout(), "Welcome to nginx!"), res.Stdout()) - }) - - t.Run("down", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) -} - -func TestNetworkLinks(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "network_link_e2e" - - t.Run("up", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/network-links/compose.yaml", "--project-name", projectName, "up", - "-d") - }) - - t.Run("curl links in default bridge network", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/network-links/compose.yaml", "--project-name", projectName, - "exec", "-T", "container2", "curl", "http://container1/") - assert.Assert(t, strings.Contains(res.Stdout(), "Welcome to nginx!"), res.Stdout()) - }) - - t.Run("down", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) -} - -func TestIPAMConfig(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "ipam_e2e" - - t.Run("ensure we do not reuse previous networks", func(t *testing.T) { - c.RunDockerOrExitError(t, "network", "rm", projectName+"_default") - }) - - t.Run("up", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/ipam/compose.yaml", "--project-name", projectName, "up", "-d") - }) - - t.Run("ensure service get fixed IP assigned", func(t *testing.T) { - res := c.RunDockerCmd(t, "inspect", projectName+"-foo-1", "-f", - fmt.Sprintf(`{{ $network := index .NetworkSettings.Networks "%s_default" }}{{ $network.IPAMConfig.IPv4Address }}`, projectName)) - res.Assert(t, icmd.Expected{Out: "10.1.0.100"}) - }) - - t.Run("down", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) -} - -func TestNetworkModes(t *testing.T) { - // fixture is shared with TestNetworks and is not safe to run concurrently - c := NewCLI(t) - - const projectName = "network_mode_service_run" - defer c.cleanupWithDown(t, projectName) - - t.Run("run with service mode dependency", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/network-test/compose.yaml", "--project-name", projectName, "run", "-T", "mydb", "echo", "success") - res.Assert(t, icmd.Expected{Out: "success"}) - }) -} - -func TestNetworkConfigChanged(t *testing.T) { - t.Skip("unstable") - // fixture is shared with TestNetworks and is not safe to run concurrently - c := NewCLI(t) - const projectName = "network_config_change" - - c.RunDockerComposeCmd(t, "-f", "./fixtures/network-test/compose.subnet.yaml", "--project-name", projectName, "up", "-d") - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "exec", "test", "hostname", "-i") - res.Assert(t, icmd.Expected{Out: "172.99.0."}) - res.Combined() - - cmd := c.NewCmdWithEnv([]string{"SUBNET=192.168.0.0/16"}, - "docker", "compose", "-f", "./fixtures/network-test/compose.subnet.yaml", "--project-name", projectName, "up", "-d") - res = icmd.RunCmd(cmd) - res.Assert(t, icmd.Success) - out := res.Combined() - fmt.Println(out) - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "exec", "test", "hostname", "-i") - res.Assert(t, icmd.Expected{Out: "192.168.0."}) -} - -func TestMacAddress(t *testing.T) { - c := NewCLI(t) - const projectName = "network_mac_address" - c.RunDockerComposeCmd(t, "-f", "./fixtures/network-test/mac_address.yaml", "--project-name", projectName, "up", "-d") - t.Cleanup(func() { - c.cleanupWithDown(t, projectName) - }) - res := c.RunDockerCmd(t, "inspect", fmt.Sprintf("%s-test-1", projectName), "-f", "{{ (index .NetworkSettings.Networks \"network_mac_address_default\" ).MacAddress }}") - res.Assert(t, icmd.Expected{Out: "00:e0:84:35:d0:e8"}) -} - -func TestInterfaceName(t *testing.T) { - c := NewCLI(t) - - version := c.RunDockerCmd(t, "version", "-f", "{{.Server.Version}}") - major, _, found := strings.Cut(version.Combined(), ".") - assert.Assert(t, found) - if major == "26" || major == "27" { - t.Skip("Skipping test due to docker version < 28") - } - - const projectName = "network_interface_name" - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/network-interface-name/compose.yaml", "--project-name", projectName, "run", "test") - t.Cleanup(func() { - c.cleanupWithDown(t, projectName) - }) - res.Assert(t, icmd.Expected{Out: "foobar@"}) -} - -func TestNetworkRecreate(t *testing.T) { - c := NewCLI(t) - const projectName = "network_recreate" - t.Cleanup(func() { - c.cleanupWithDown(t, projectName) - }) - c.RunDockerComposeCmd(t, "-f", "./fixtures/network-recreate/compose.yaml", "--project-name", projectName, "up", "-d") - - c = NewCLI(t, WithEnv("FOO=bar")) - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/network-recreate/compose.yaml", "--project-name", projectName, "--progress=plain", "up", "-d") - err := res.Stderr() - fmt.Println(err) - hasStopped := strings.Contains(err, "Stopped") - hasResumed := strings.Contains(err, "Started") || strings.Contains(err, "Recreated") - if !hasStopped || !hasResumed { - t.Fatalf("unexpected output, missing expected events, stderr: %s", err) - } -} diff --git a/pkg/e2e/noDeps_test.go b/pkg/e2e/noDeps_test.go deleted file mode 100644 index 85f3872f7e9..00000000000 --- a/pkg/e2e/noDeps_test.go +++ /dev/null @@ -1,60 +0,0 @@ -//go:build !windows - -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "testing" - - "gotest.tools/v3/icmd" -) - -func TestNoDepsVolumeFrom(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-no-deps-volume-from" - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - c.RunDockerComposeCmd(t, "-f", "fixtures/no-deps/volume-from.yaml", "--project-name", projectName, "up", "-d") - - c.RunDockerComposeCmd(t, "-f", "fixtures/no-deps/volume-from.yaml", "--project-name", projectName, "up", "--no-deps", "-d", "app") - - c.RunDockerCmd(t, "rm", "-f", fmt.Sprintf("%s-db-1", projectName)) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/no-deps/volume-from.yaml", "--project-name", projectName, "up", "--no-deps", "-d", "app") - res.Assert(t, icmd.Expected{ExitCode: 1, Err: "cannot share volume with service db: container missing"}) -} - -func TestNoDepsNetworkMode(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-no-deps-network-mode" - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - c.RunDockerComposeCmd(t, "-f", "fixtures/no-deps/network-mode.yaml", "--project-name", projectName, "up", "-d") - - c.RunDockerComposeCmd(t, "-f", "fixtures/no-deps/network-mode.yaml", "--project-name", projectName, "up", "--no-deps", "-d", "app") - - c.RunDockerCmd(t, "rm", "-f", fmt.Sprintf("%s-db-1", projectName)) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/no-deps/network-mode.yaml", "--project-name", projectName, "up", "--no-deps", "-d", "app") - res.Assert(t, icmd.Expected{ExitCode: 1, Err: "cannot share network namespace with service db: container missing"}) -} diff --git a/pkg/e2e/orphans_test.go b/pkg/e2e/orphans_test.go deleted file mode 100644 index e721e7a540b..00000000000 --- a/pkg/e2e/orphans_test.go +++ /dev/null @@ -1,40 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - - "gotest.tools/v3/assert" -) - -func TestRemoveOrphans(t *testing.T) { - c := NewCLI(t) - - const projectName = "compose-e2e-orphans" - defer c.cleanupWithDown(t, projectName) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/orphans/compose.yaml", "-p", projectName, "run", "orphan") - res := c.RunDockerComposeCmd(t, "-p", projectName, "ps", "--all") - assert.Check(t, strings.Contains(res.Combined(), "compose-e2e-orphans-orphan-run-")) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/orphans/compose.yaml", "-p", projectName, "up", "-d") - - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps", "--all") - assert.Check(t, !strings.Contains(res.Combined(), "compose-e2e-orphans-orphan-run-")) -} diff --git a/pkg/e2e/pause_test.go b/pkg/e2e/pause_test.go deleted file mode 100644 index 200301a2415..00000000000 --- a/pkg/e2e/pause_test.go +++ /dev/null @@ -1,160 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "encoding/json" - "errors" - "fmt" - "net" - "net/http" - "os" - "testing" - "time" - - "github.com/stretchr/testify/require" - "gotest.tools/v3/icmd" -) - -func TestPause(t *testing.T) { - if _, ok := os.LookupEnv("CI"); ok { - t.Skip("Skipping test on CI... flaky") - } - cli := NewParallelCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=e2e-pause", - "COMPOSE_FILE=./fixtures/pause/compose.yaml")) - - cleanup := func() { - cli.RunDockerComposeCmd(t, "down", "-v", "--remove-orphans", "-t", "0") - } - cleanup() - t.Cleanup(cleanup) - - // launch both services and verify that they are accessible - cli.RunDockerComposeCmd(t, "up", "-d") - urls := map[string]string{ - "a": urlForService(t, cli, "a", 80), - "b": urlForService(t, cli, "b", 80), - } - for _, url := range urls { - HTTPGetWithRetry(t, url, http.StatusOK, 50*time.Millisecond, 20*time.Second) - } - - // pause a and verify that it can no longer be hit but b still can - cli.RunDockerComposeCmd(t, "pause", "a") - httpClient := http.Client{Timeout: 250 * time.Millisecond} - resp, err := httpClient.Get(urls["a"]) - if resp != nil { - _ = resp.Body.Close() - } - require.Error(t, err, "a should no longer respond") - var netErr net.Error - errors.As(err, &netErr) - require.True(t, netErr.Timeout(), "Error should have indicated a timeout") - HTTPGetWithRetry(t, urls["b"], http.StatusOK, 50*time.Millisecond, 5*time.Second) - - // unpause a and verify that both containers work again - cli.RunDockerComposeCmd(t, "unpause", "a") - for _, url := range urls { - HTTPGetWithRetry(t, url, http.StatusOK, 50*time.Millisecond, 5*time.Second) - } -} - -func TestPauseServiceNotRunning(t *testing.T) { - cli := NewParallelCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=e2e-pause-svc-not-running", - "COMPOSE_FILE=./fixtures/pause/compose.yaml")) - - cleanup := func() { - cli.RunDockerComposeCmd(t, "down", "-v", "--remove-orphans", "-t", "0") - } - cleanup() - t.Cleanup(cleanup) - - // pause a and verify that it can no longer be hit but b still can - res := cli.RunDockerComposeCmdNoCheck(t, "pause", "a") - - // TODO: `docker pause` errors in this case, should Compose be consistent? - res.Assert(t, icmd.Expected{ExitCode: 0}) -} - -func TestPauseServiceAlreadyPaused(t *testing.T) { - cli := NewParallelCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=e2e-pause-svc-already-paused", - "COMPOSE_FILE=./fixtures/pause/compose.yaml")) - - cleanup := func() { - cli.RunDockerComposeCmd(t, "down", "-v", "--remove-orphans", "-t", "0") - } - cleanup() - t.Cleanup(cleanup) - - // launch a and wait for it to come up - cli.RunDockerComposeCmd(t, "up", "--menu=false", "--wait", "a") - HTTPGetWithRetry(t, urlForService(t, cli, "a", 80), http.StatusOK, 50*time.Millisecond, 10*time.Second) - - // pause a twice - first time should pass, second time fail - cli.RunDockerComposeCmd(t, "pause", "a") - res := cli.RunDockerComposeCmdNoCheck(t, "pause", "a") - res.Assert(t, icmd.Expected{ExitCode: 1, Err: "already paused"}) -} - -func TestPauseServiceDoesNotExist(t *testing.T) { - cli := NewParallelCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=e2e-pause-svc-not-exist", - "COMPOSE_FILE=./fixtures/pause/compose.yaml")) - - cleanup := func() { - cli.RunDockerComposeCmd(t, "down", "-v", "--remove-orphans", "-t", "0") - } - cleanup() - t.Cleanup(cleanup) - - // pause a and verify that it can no longer be hit but b still can - res := cli.RunDockerComposeCmdNoCheck(t, "pause", "does_not_exist") - // TODO: `compose down does_not_exist` and similar error, this should too - res.Assert(t, icmd.Expected{ExitCode: 0}) -} - -func urlForService(t testing.TB, cli *CLI, service string, targetPort int) string { - t.Helper() - return fmt.Sprintf( - "http://localhost:%d", - publishedPortForService(t, cli, service, targetPort), - ) -} - -func publishedPortForService(t testing.TB, cli *CLI, service string, targetPort int) int { - t.Helper() - res := cli.RunDockerComposeCmd(t, "ps", "--format=json", service) - var svc struct { - Publishers []struct { - TargetPort int - PublishedPort int - } - } - require.NoError(t, json.Unmarshal([]byte(res.Stdout()), &svc), - "Failed to parse `%s` output", res.Cmd.String()) - for _, pp := range svc.Publishers { - if pp.TargetPort == targetPort { - return pp.PublishedPort - } - } - require.Failf(t, "No published port for target port", - "Target port: %d\nService: %s", targetPort, res.Combined()) - return -1 -} diff --git a/pkg/e2e/profiles_test.go b/pkg/e2e/profiles_test.go deleted file mode 100644 index dffc209d00e..00000000000 --- a/pkg/e2e/profiles_test.go +++ /dev/null @@ -1,207 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -const ( - profiledService = "profiled-service" - regularService = "regular-service" -) - -func TestExplicitProfileUsage(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "compose-e2e-explicit-profiles" - const profileName = "test-profile" - - t.Run("compose up with profile", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "-p", projectName, "--profile", profileName, "up", "-d") - res.Assert(t, icmd.Expected{ExitCode: 0}) - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps") - res.Assert(t, icmd.Expected{Out: regularService}) - res.Assert(t, icmd.Expected{Out: profiledService}) - }) - - t.Run("compose stop with profile", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "-p", projectName, "--profile", profileName, "stop") - res.Assert(t, icmd.Expected{ExitCode: 0}) - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps", "--status", "running") - assert.Assert(t, !strings.Contains(res.Combined(), regularService)) - assert.Assert(t, !strings.Contains(res.Combined(), profiledService)) - }) - - t.Run("compose start with profile", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "-p", projectName, "--profile", profileName, "start") - res.Assert(t, icmd.Expected{ExitCode: 0}) - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps", "--status", "running") - res.Assert(t, icmd.Expected{Out: regularService}) - res.Assert(t, icmd.Expected{Out: profiledService}) - }) - - t.Run("compose restart with profile", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "-p", projectName, "--profile", profileName, "restart") - res.Assert(t, icmd.Expected{ExitCode: 0}) - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps", "--status", "running") - res.Assert(t, icmd.Expected{Out: regularService}) - res.Assert(t, icmd.Expected{Out: profiledService}) - }) - - t.Run("down", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - t.Run("check containers after down", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps") - assert.Assert(t, !strings.Contains(res.Combined(), projectName), res.Combined()) - }) -} - -func TestNoProfileUsage(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "compose-e2e-no-profiles" - - t.Run("compose up without profile", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "-p", projectName, "up", "-d") - res.Assert(t, icmd.Expected{ExitCode: 0}) - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps") - res.Assert(t, icmd.Expected{Out: regularService}) - assert.Assert(t, !strings.Contains(res.Combined(), profiledService)) - }) - - t.Run("compose stop without profile", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "-p", projectName, "stop") - res.Assert(t, icmd.Expected{ExitCode: 0}) - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps", "--status", "running") - assert.Assert(t, !strings.Contains(res.Combined(), regularService)) - assert.Assert(t, !strings.Contains(res.Combined(), profiledService)) - }) - - t.Run("compose start without profile", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "-p", projectName, "start") - res.Assert(t, icmd.Expected{ExitCode: 0}) - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps", "--status", "running") - res.Assert(t, icmd.Expected{Out: regularService}) - assert.Assert(t, !strings.Contains(res.Combined(), profiledService)) - }) - - t.Run("compose restart without profile", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "-p", projectName, "restart") - res.Assert(t, icmd.Expected{ExitCode: 0}) - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps", "--status", "running") - res.Assert(t, icmd.Expected{Out: regularService}) - assert.Assert(t, !strings.Contains(res.Combined(), profiledService)) - }) - - t.Run("down", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - t.Run("check containers after down", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps") - assert.Assert(t, !strings.Contains(res.Combined(), projectName), res.Combined()) - }) -} - -func TestActiveProfileViaTargetedService(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "compose-e2e-via-target-service-profiles" - const profileName = "test-profile" - - t.Run("compose up with service name", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "-p", projectName, "up", profiledService, "-d") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps") - assert.Assert(t, !strings.Contains(res.Combined(), regularService)) - res.Assert(t, icmd.Expected{Out: profiledService}) - - res = c.RunDockerComposeCmd(t, "-p", projectName, "--profile", profileName, "ps") - assert.Assert(t, !strings.Contains(res.Combined(), regularService)) - res.Assert(t, icmd.Expected{Out: profiledService}) - }) - - t.Run("compose stop with service name", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "-p", projectName, "stop", profiledService) - res.Assert(t, icmd.Expected{ExitCode: 0}) - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps", "--status", "running") - assert.Assert(t, !strings.Contains(res.Combined(), regularService)) - assert.Assert(t, !strings.Contains(res.Combined(), profiledService)) - }) - - t.Run("compose start with service name", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "-p", projectName, "start", profiledService) - res.Assert(t, icmd.Expected{ExitCode: 0}) - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps", "--status", "running") - assert.Assert(t, !strings.Contains(res.Combined(), regularService)) - res.Assert(t, icmd.Expected{Out: profiledService}) - }) - - t.Run("compose restart with service name", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "-p", projectName, "restart") - res.Assert(t, icmd.Expected{ExitCode: 0}) - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps", "--status", "running") - assert.Assert(t, !strings.Contains(res.Combined(), regularService)) - res.Assert(t, icmd.Expected{Out: profiledService}) - }) - - t.Run("down", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - t.Run("check containers after down", func(t *testing.T) { - res := c.RunDockerCmd(t, "ps") - assert.Assert(t, !strings.Contains(res.Combined(), projectName), res.Combined()) - }) -} - -func TestDotEnvProfileUsage(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "compose-e2e-dotenv-profiles" - const profileName = "test-profile" - - t.Cleanup(func() { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - t.Run("compose up with profile", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/compose.yaml", - "--env-file", "./fixtures/profiles/test-profile.env", - "-p", projectName, "--profile", profileName, "up", "-d") - res.Assert(t, icmd.Expected{ExitCode: 0}) - res = c.RunDockerComposeCmd(t, "-p", projectName, "ps") - res.Assert(t, icmd.Expected{Out: regularService}) - res.Assert(t, icmd.Expected{Out: profiledService}) - }) -} diff --git a/pkg/e2e/providers_test.go b/pkg/e2e/providers_test.go deleted file mode 100644 index b026f1f1434..00000000000 --- a/pkg/e2e/providers_test.go +++ /dev/null @@ -1,64 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "bufio" - "fmt" - "os" - "path/filepath" - "slices" - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestDependsOnMultipleProviders(t *testing.T) { - provider, err := findExecutable("example-provider") - assert.NilError(t, err) - - path := fmt.Sprintf("%s%s%s", os.Getenv("PATH"), string(os.PathListSeparator), filepath.Dir(provider)) - c := NewParallelCLI(t, WithEnv("PATH="+path)) - const projectName = "depends-on-multiple-providers" - t.Cleanup(func() { - c.cleanupWithDown(t, projectName) - }) - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/providers/depends-on-multiple-providers.yaml", "--project-name", projectName, "up") - res.Assert(t, icmd.Success) - env := getEnv(res.Combined(), false) - assert.Check(t, slices.Contains(env, "PROVIDER1_URL=https://magic.cloud/provider1"), env) - assert.Check(t, slices.Contains(env, "PROVIDER2_URL=https://magic.cloud/provider2"), env) -} - -func getEnv(out string, run bool) []string { - var env []string - scanner := bufio.NewScanner(strings.NewReader(out)) - for scanner.Scan() { - line := scanner.Text() - if !run && strings.HasPrefix(line, "test-1 | ") { - env = append(env, line[10:]) - } - if run && strings.Contains(line, "=") && len(strings.Split(line, "=")) == 2 { - env = append(env, line) - } - } - slices.Sort(env) - return env -} diff --git a/pkg/e2e/ps_test.go b/pkg/e2e/ps_test.go deleted file mode 100644 index 77a49accb75..00000000000 --- a/pkg/e2e/ps_test.go +++ /dev/null @@ -1,132 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "encoding/json" - "strings" - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - "gotest.tools/v3/icmd" - - "github.com/docker/compose/v5/pkg/api" -) - -func TestPs(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-ps" - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/ps-test/compose.yaml", "--project-name", projectName, "up", "-d") - require.NoError(t, res.Error) - t.Cleanup(func() { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - assert.Contains(t, res.Combined(), "Container e2e-ps-busybox-1 Started", res.Combined()) - - t.Run("table", func(t *testing.T) { - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/ps-test/compose.yaml", "--project-name", projectName, "ps") - lines := strings.Split(res.Stdout(), "\n") - assert.Len(t, lines, 4) - count := 0 - for _, line := range lines[1:3] { - if strings.Contains(line, "e2e-ps-busybox-1") { - assert.Contains(t, line, "127.0.0.1:8001->8000/tcp") - count++ - } - if strings.Contains(line, "e2e-ps-nginx-1") { - assert.Contains(t, line, "80/tcp, 443/tcp, 8080/tcp") - count++ - } - } - assert.Equal(t, 2, count, "Did not match both services:\n"+res.Combined()) - }) - - t.Run("json", func(t *testing.T) { - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/ps-test/compose.yaml", "--project-name", projectName, "ps", - "--format", "json") - type element struct { - Name string - Project string - Publishers api.PortPublishers - } - var output []element - out := res.Stdout() - dec := json.NewDecoder(strings.NewReader(out)) - for dec.More() { - var s element - require.NoError(t, dec.Decode(&s), "Failed to unmarshal ps JSON output") - output = append(output, s) - } - - count := 0 - assert.Len(t, output, 2) - for _, service := range output { - assert.Equal(t, projectName, service.Project) - publishers := service.Publishers - if service.Name == "e2e-ps-busybox-1" { - assert.Len(t, publishers, 1) - assert.Equal(t, api.PortPublishers{ - { - URL: "127.0.0.1", - TargetPort: 8000, - PublishedPort: 8001, - Protocol: "tcp", - }, - }, publishers) - count++ - } - if service.Name == "e2e-ps-nginx-1" { - assert.Len(t, publishers, 3) - assert.Equal(t, api.PortPublishers{ - {TargetPort: 80, Protocol: "tcp"}, - {TargetPort: 443, Protocol: "tcp"}, - {TargetPort: 8080, Protocol: "tcp"}, - }, publishers) - - count++ - } - } - assert.Equal(t, 2, count, "Did not match both services:\n"+res.Combined()) - }) - - t.Run("ps --all", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "stop") - require.NoError(t, res.Error) - - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/ps-test/compose.yaml", "--project-name", projectName, "ps") - lines := strings.Split(res.Stdout(), "\n") - assert.Len(t, lines, 2) - - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/ps-test/compose.yaml", "--project-name", projectName, "ps", "--all") - lines = strings.Split(res.Stdout(), "\n") - assert.Len(t, lines, 4) - }) - - t.Run("ps unknown", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "stop") - require.NoError(t, res.Error) - - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/ps-test/compose.yaml", "--project-name", projectName, "ps", "nginx") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmdNoCheck(t, "-f", "./fixtures/ps-test/compose.yaml", "--project-name", projectName, "ps", "unknown") - res.Assert(t, icmd.Expected{ExitCode: 1, Err: "no such service: unknown"}) - }) -} diff --git a/pkg/e2e/publish_test.go b/pkg/e2e/publish_test.go deleted file mode 100644 index b5488df601c..00000000000 --- a/pkg/e2e/publish_test.go +++ /dev/null @@ -1,164 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestPublishChecks(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "compose-e2e-explicit-profiles" - - t.Run("publish error env_file", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "-f", "./fixtures/publish/compose-env-file.yml", - "-p", projectName, "publish", "test/test") - res.Assert(t, icmd.Expected{ExitCode: 1, Err: `service "serviceA" has env_file declared. -To avoid leaking sensitive data,`}) - }) - - t.Run("publish multiple errors env_file and environment", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "-f", "./fixtures/publish/compose-multi-env-config.yml", - "-p", projectName, "publish", "test/test") - // we don't in which order the services will be loaded, so we can't predict the order of the error messages - assert.Assert(t, strings.Contains(res.Combined(), `service "serviceB" has env_file declared.`), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), `To avoid leaking sensitive data, you must either explicitly allow the sending of environment variables by using the --with-env flag, -or remove sensitive data from your Compose configuration -`), res.Combined()) - }) - - t.Run("publish success environment", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/publish/compose-environment.yml", - "-p", projectName, "publish", "test/test", "--with-env", "-y", "--dry-run") - assert.Assert(t, strings.Contains(res.Combined(), "test/test publishing"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "test/test published"), res.Combined()) - }) - - t.Run("publish success env_file", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/publish/compose-env-file.yml", - "-p", projectName, "publish", "test/test", "--with-env", "-y", "--dry-run") - assert.Assert(t, strings.Contains(res.Combined(), "test/test publishing"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "test/test published"), res.Combined()) - }) - - t.Run("publish with extends", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/publish/compose-with-extends.yml", - "-p", projectName, "publish", "test/test", "--dry-run") - assert.Assert(t, strings.Contains(res.Combined(), "test/test published"), res.Combined()) - }) - - t.Run("refuse to publish with bind mount", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/publish/compose-bind-mount.yml", - "-p", projectName, "publish", "test/test", "--dry-run") - cmd.Stdin = strings.NewReader("n\n") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ExitCode: 0}) - out := res.Combined() - assert.Assert(t, strings.Contains(out, "you are about to publish bind mounts declaration within your OCI artifact."), out) - assert.Assert(t, strings.Contains(out, "e2e/fixtures/publish:/user-data"), out) - assert.Assert(t, strings.Contains(out, "Are you ok to publish these bind mount declarations?"), out) - assert.Assert(t, !strings.Contains(out, "serviceA published"), out) - }) - - t.Run("publish with bind mount", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/publish/compose-bind-mount.yml", - "-p", projectName, "publish", "test/test", "--dry-run") - cmd.Stdin = strings.NewReader("y\n") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ExitCode: 0}) - assert.Assert(t, strings.Contains(res.Combined(), "you are about to publish bind mounts declaration within your OCI artifact."), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "Are you ok to publish these bind mount declarations?"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "e2e/fixtures/publish:/user-data"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "test/test published"), res.Combined()) - }) - - t.Run("refuse to publish with build section only", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "-f", "./fixtures/publish/compose-build-only.yml", - "-p", projectName, "publish", "test/test", "--with-env", "-y", "--dry-run") - res.Assert(t, icmd.Expected{ExitCode: 1}) - assert.Assert(t, strings.Contains(res.Combined(), "your Compose stack cannot be published as it only contains a build section for service(s):"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "serviceA"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "serviceB"), res.Combined()) - }) - - t.Run("refuse to publish with local include", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "-f", "./fixtures/publish/compose-local-include.yml", - "-p", projectName, "publish", "test/test", "--dry-run") - res.Assert(t, icmd.Expected{ExitCode: 1, Err: "cannot publish compose file with local includes"}) - }) - - t.Run("detect sensitive data", func(t *testing.T) { - cmd := c.NewDockerComposeCmd(t, "-f", "./fixtures/publish/compose-sensitive.yml", - "-p", projectName, "publish", "test/test", "--with-env", "--dry-run") - cmd.Stdin = strings.NewReader("n\n") - res := icmd.RunCmd(cmd) - res.Assert(t, icmd.Expected{ExitCode: 0}) - - output := res.Combined() - assert.Assert(t, strings.Contains(output, "you are about to publish sensitive data within your OCI artifact.\n"), output) - assert.Assert(t, strings.Contains(output, "please double check that you are not leaking sensitive data"), output) - assert.Assert(t, strings.Contains(output, "AWS Client ID\n\"services.serviceA.environment.AWS_ACCESS_KEY_ID\": A3TX1234567890ABCDEF"), output) - assert.Assert(t, strings.Contains(output, "AWS Secret Key\n\"services.serviceA.environment.AWS_SECRET_ACCESS_KEY\": aws\"12345+67890/abcdefghijklm+NOPQRSTUVWXYZ+\""), output) - assert.Assert(t, strings.Contains(output, "Github authentication\n\"GITHUB_TOKEN\": ghp_1234567890abcdefghijklmnopqrstuvwxyz"), output) - assert.Assert(t, strings.Contains(output, "JSON Web Token\n\"\": eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9."+ - "eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw"), output) - assert.Assert(t, strings.Contains(output, "Private Key\n\"\": -----BEGIN DSA PRIVATE KEY-----\nwxyz+ABC=\n-----END DSA PRIVATE KEY-----"), output) - }) -} - -func TestPublish(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "compose-e2e-publish" - const registryName = projectName + "-registry" - c.RunDockerCmd(t, "run", "--name", registryName, "-P", "-d", "registry:3") - port := c.RunDockerCmd(t, "inspect", "--format", `{{ (index (index .NetworkSettings.Ports "5000/tcp") 0).HostPort }}`, registryName).Stdout() - registry := "localhost:" + strings.TrimSpace(port) - t.Cleanup(func() { - c.RunDockerCmd(t, "rm", "--force", registryName) - }) - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/publish/oci/compose.yaml", "-f", "./fixtures/publish/oci/compose-override.yaml", - "-p", projectName, "publish", "--with-env", "--yes", "--insecure-registry", registry+"/test:test") - res.Assert(t, icmd.Expected{ExitCode: 0}) - - // docker exec -it compose-e2e-publish-registry tree /var/lib/registry/docker/registry/v2/ - - cmd := c.NewDockerComposeCmd(t, "--verbose", "--project-name=oci", - "--insecure-registry", registry, - "-f", fmt.Sprintf("oci://%s/test:test", registry), "config") - res = icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "XDG_CACHE_HOME="+t.TempDir()) - }) - res.Assert(t, icmd.Expected{ExitCode: 0}) - assert.Equal(t, res.Stdout(), `name: oci -services: - app: - environment: - HELLO: WORLD - image: alpine - networks: - default: null -networks: - default: - name: oci_default -`) -} diff --git a/pkg/e2e/pull_test.go b/pkg/e2e/pull_test.go deleted file mode 100644 index 799bdbb2fc7..00000000000 --- a/pkg/e2e/pull_test.go +++ /dev/null @@ -1,76 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestComposePull(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("Verify image pulled", func(t *testing.T) { - // cleanup existing images - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/compose-pull/simple", "down", "--rmi", "all") - - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/compose-pull/simple", "pull") - output := res.Combined() - - assert.Assert(t, strings.Contains(output, "Image alpine:3.14 Pulled")) - assert.Assert(t, strings.Contains(output, "Image alpine:3.15 Pulled")) - - // verify default policy is 'always' for pull command - res = c.RunDockerComposeCmd(t, "--project-directory", "fixtures/compose-pull/simple", "pull") - output = res.Combined() - - assert.Assert(t, strings.Contains(output, "Image alpine:3.14 Pulled")) - assert.Assert(t, strings.Contains(output, "Image alpine:3.15 Pulled")) - }) - - t.Run("Verify skipped pull if image is already present locally", func(t *testing.T) { - // make sure the required image is present - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/compose-pull/image-present-locally", "pull") - - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/compose-pull/image-present-locally", "pull") - output := res.Combined() - - assert.Assert(t, strings.Contains(output, "alpine:3.13.12 Skipped Image is already present locally")) - // image with :latest tag gets pulled regardless if pull_policy: missing or if_not_present - assert.Assert(t, strings.Contains(output, "alpine:latest Pulled")) - }) - - t.Run("Verify skipped no image to be pulled", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/compose-pull/no-image-name-given", "pull") - output := res.Combined() - - assert.Assert(t, strings.Contains(output, "Skipped No image to be pulled")) - }) - - t.Run("Verify pull failure", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/compose-pull/unknown-image", "pull") - res.Assert(t, icmd.Expected{ExitCode: 1, Err: "pull access denied for does_not_exists"}) - }) - - t.Run("Verify ignore pull failure", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/compose-pull/unknown-image", "pull", "--ignore-pull-failures") - res.Assert(t, icmd.Expected{Err: "Some service image(s) must be built from source by running:"}) - }) -} diff --git a/pkg/e2e/recreate_no_deps_test.go b/pkg/e2e/recreate_no_deps_test.go deleted file mode 100644 index 2b32e0d5bc3..00000000000 --- a/pkg/e2e/recreate_no_deps_test.go +++ /dev/null @@ -1,39 +0,0 @@ -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" - - "gotest.tools/v3/icmd" -) - -func TestRecreateWithNoDeps(t *testing.T) { - c := NewParallelCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=recreate-no-deps", - )) - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/dependencies/recreate-no-deps.yaml", "up", "-d") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/dependencies/recreate-no-deps.yaml", "up", "-d", "--force-recreate", "--no-deps", "my-service") - res.Assert(t, icmd.Success) - - RequireServiceState(t, c, "my-service", "running") - - c.RunDockerComposeCmd(t, "down") -} diff --git a/pkg/e2e/restart_test.go b/pkg/e2e/restart_test.go deleted file mode 100644 index 8b81e228a18..00000000000 --- a/pkg/e2e/restart_test.go +++ /dev/null @@ -1,115 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "strings" - "testing" - "time" - - testify "github.com/stretchr/testify/assert" - "gotest.tools/v3/assert" -) - -func assertServiceStatus(t *testing.T, projectName, service, status string, ps string) { - // match output with random spaces like: - // e2e-start-stop-db-1 alpine:latest "echo hello" db 1 minutes ago Exited (0) 1 minutes ago - regx := fmt.Sprintf("%s-%s-1.+%s\\s+.+%s.+", projectName, service, service, status) - testify.Regexp(t, regx, ps) -} - -func TestRestart(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-restart" - - t.Run("Up a project", func(t *testing.T) { - // This is just to ensure the containers do NOT exist - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/restart-test/compose.yaml", "--project-name", projectName, "up", "-d") - assert.Assert(t, strings.Contains(res.Combined(), "Container e2e-restart-restart-1 Started"), res.Combined()) - - c.WaitForCmdResult(t, c.NewDockerComposeCmd(t, "--project-name", projectName, "ps", "-a", "--format", - "json"), - StdoutContains(`"State":"exited"`), 10*time.Second, 1*time.Second) - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "ps", "-a") - assertServiceStatus(t, projectName, "restart", "Exited", res.Stdout()) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/restart-test/compose.yaml", "--project-name", projectName, "restart") - - // Give the same time but it must NOT exit - time.Sleep(time.Second) - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "ps") - assertServiceStatus(t, projectName, "restart", "Up", res.Stdout()) - - // Clean up - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) -} - -func TestRestartWithDependencies(t *testing.T) { - c := NewCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=e2e-restart-deps", - )) - baseService := "nginx" - depWithRestart := "with-restart" - depNoRestart := "no-restart" - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "down", "--remove-orphans") - }) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/restart-test/compose-depends-on.yaml", "up", "-d") - - res := c.RunDockerComposeCmd(t, "restart", baseService) - out := res.Combined() - assert.Assert(t, strings.Contains(out, fmt.Sprintf("Container e2e-restart-deps-%s-1 Restarting", baseService)), out) - assert.Assert(t, strings.Contains(out, fmt.Sprintf("Container e2e-restart-deps-%s-1 Healthy", baseService)), out) - assert.Assert(t, strings.Contains(out, fmt.Sprintf("Container e2e-restart-deps-%s-1 Started", depWithRestart)), out) - assert.Assert(t, !strings.Contains(out, depNoRestart), out) - - c = NewParallelCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=e2e-restart-deps", - "LABEL=recreate", - )) - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/restart-test/compose-depends-on.yaml", "up", "-d") - out = res.Combined() - assert.Assert(t, strings.Contains(out, fmt.Sprintf("Container e2e-restart-deps-%s-1 Stopped", depWithRestart)), out) - assert.Assert(t, strings.Contains(out, fmt.Sprintf("Container e2e-restart-deps-%s-1 Recreated", baseService)), out) - assert.Assert(t, strings.Contains(out, fmt.Sprintf("Container e2e-restart-deps-%s-1 Healthy", baseService)), out) - assert.Assert(t, strings.Contains(out, fmt.Sprintf("Container e2e-restart-deps-%s-1 Started", depWithRestart)), out) - assert.Assert(t, strings.Contains(out, fmt.Sprintf("Container e2e-restart-deps-%s-1 Running", depNoRestart)), out) -} - -func TestRestartWithProfiles(t *testing.T) { - c := NewParallelCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=e2e-restart-profiles", - )) - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "down", "--remove-orphans") - }) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/restart-test/compose.yaml", "--profile", "test", "up", "-d") - - res := c.RunDockerComposeCmd(t, "restart", "test") - fmt.Println(res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "Container e2e-restart-profiles-test-1 Started"), res.Combined()) -} diff --git a/pkg/e2e/scale_test.go b/pkg/e2e/scale_test.go deleted file mode 100644 index 0e16c7afb3e..00000000000 --- a/pkg/e2e/scale_test.go +++ /dev/null @@ -1,217 +0,0 @@ -/* -Copyright 2020 Docker Compose CLI authors - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package e2e - -import ( - "fmt" - "strings" - "testing" - - testify "github.com/stretchr/testify/assert" - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -const NO_STATE_TO_CHECK = "" - -func TestScaleBasicCases(t *testing.T) { - c := NewCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=scale-basic-tests")) - - reset := func() { - c.RunDockerComposeCmd(t, "down", "--rmi", "all") - } - t.Cleanup(reset) - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "up", "-d") - res.Assert(t, icmd.Success) - - t.Log("scale up one service") - res = c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "scale", "dbadmin=2") - out := res.Combined() - checkServiceContainer(t, out, "scale-basic-tests-dbadmin", "Started", 2) - - t.Log("scale up 2 services") - res = c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "scale", "front=3", "back=2") - out = res.Combined() - checkServiceContainer(t, out, "scale-basic-tests-front", "Running", 2) - checkServiceContainer(t, out, "scale-basic-tests-front", "Started", 1) - checkServiceContainer(t, out, "scale-basic-tests-back", "Running", 1) - checkServiceContainer(t, out, "scale-basic-tests-back", "Started", 1) - - t.Log("scale down one service") - res = c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "scale", "dbadmin=1") - out = res.Combined() - checkServiceContainer(t, out, "scale-basic-tests-dbadmin", "Running", 1) - - t.Log("scale to 0 a service") - res = c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "scale", "dbadmin=0") - assert.Check(t, res.Stdout() == "", res.Stdout()) - - t.Log("scale down 2 services") - res = c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "scale", "front=2", "back=1") - out = res.Combined() - checkServiceContainer(t, out, "scale-basic-tests-front", "Running", 2) - assert.Check(t, !strings.Contains(out, "Container scale-basic-tests-front-3 Running"), res.Combined()) - checkServiceContainer(t, out, "scale-basic-tests-back", "Running", 1) -} - -func TestScaleWithDepsCases(t *testing.T) { - c := NewCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=scale-deps-tests")) - - reset := func() { - c.RunDockerComposeCmd(t, "down", "--rmi", "all") - } - t.Cleanup(reset) - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "up", "-d", "--scale", "db=2") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "ps") - checkServiceContainer(t, res.Combined(), "scale-deps-tests-db", NO_STATE_TO_CHECK, 2) - - t.Log("scale up 1 service with --no-deps") - _ = c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "scale", "--no-deps", "back=2") - res = c.RunDockerComposeCmd(t, "ps") - checkServiceContainer(t, res.Combined(), "scale-deps-tests-back", NO_STATE_TO_CHECK, 2) - checkServiceContainer(t, res.Combined(), "scale-deps-tests-db", NO_STATE_TO_CHECK, 2) - - t.Log("scale up 1 service without --no-deps") - _ = c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "scale", "back=2") - res = c.RunDockerComposeCmd(t, "ps") - checkServiceContainer(t, res.Combined(), "scale-deps-tests-back", NO_STATE_TO_CHECK, 2) - checkServiceContainer(t, res.Combined(), "scale-deps-tests-db", NO_STATE_TO_CHECK, 1) -} - -func TestScaleUpAndDownPreserveContainerNumber(t *testing.T) { - const projectName = "scale-up-down-test" - - c := NewCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME="+projectName)) - - reset := func() { - c.RunDockerComposeCmd(t, "down", "--rmi", "all") - } - t.Cleanup(reset) - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "up", "-d", "--scale", "db=2", "db") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "ps", "--format", "{{.Name}}", "db") - res.Assert(t, icmd.Success) - assert.Equal(t, strings.TrimSpace(res.Stdout()), projectName+"-db-1\n"+projectName+"-db-2") - - t.Log("scale down removes replica #2") - res = c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "up", "-d", "--scale", "db=1", "db") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "ps", "--format", "{{.Name}}", "db") - res.Assert(t, icmd.Success) - assert.Equal(t, strings.TrimSpace(res.Stdout()), projectName+"-db-1") - - t.Log("scale up restores replica #2") - res = c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "up", "-d", "--scale", "db=2", "db") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "ps", "--format", "{{.Name}}", "db") - res.Assert(t, icmd.Success) - assert.Equal(t, strings.TrimSpace(res.Stdout()), projectName+"-db-1\n"+projectName+"-db-2") -} - -func TestScaleDownRemovesObsolete(t *testing.T) { - const projectName = "scale-down-obsolete-test" - c := NewCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME="+projectName)) - - reset := func() { - c.RunDockerComposeCmd(t, "down", "--rmi", "all") - } - t.Cleanup(reset) - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/scale", "up", "-d", "db") - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "ps", "--format", "{{.Name}}", "db") - res.Assert(t, icmd.Success) - assert.Equal(t, strings.TrimSpace(res.Stdout()), projectName+"-db-1") - - cmd := c.NewDockerComposeCmd(t, "--project-directory", "fixtures/scale", "up", "-d", "--scale", "db=2", "db") - res = icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "MAYBE=value") - }) - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "ps", "--format", "{{.Name}}", "db") - res.Assert(t, icmd.Success) - assert.Equal(t, strings.TrimSpace(res.Stdout()), projectName+"-db-1\n"+projectName+"-db-2") - - t.Log("scale down removes obsolete replica #1") - cmd = c.NewDockerComposeCmd(t, "--project-directory", "fixtures/scale", "up", "-d", "--scale", "db=1", "db") - res = icmd.RunCmd(cmd, func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "MAYBE=value") - }) - res.Assert(t, icmd.Success) - - res = c.RunDockerComposeCmd(t, "ps", "--format", "{{.Name}}", "db") - res.Assert(t, icmd.Success) - assert.Equal(t, strings.TrimSpace(res.Stdout()), projectName+"-db-1") -} - -func checkServiceContainer(t *testing.T, stdout, containerName, containerState string, count int) { - found := 0 - lines := strings.SplitSeq(stdout, "\n") - for line := range lines { - if strings.Contains(line, containerName) && strings.Contains(line, containerState) { - found++ - } - } - if found == count { - return - } - errMessage := fmt.Sprintf("expected %d but found %d instance(s) of container %s in stoud", count, found, containerName) - if containerState != "" { - errMessage += fmt.Sprintf(" with expected state %s", containerState) - } - testify.Fail(t, errMessage, stdout) -} - -func TestScaleDownNoRecreate(t *testing.T) { - const projectName = "scale-down-recreated-test" - c := NewCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME="+projectName)) - - reset := func() { - c.RunDockerComposeCmd(t, "down", "--rmi", "all") - } - t.Cleanup(reset) - c.RunDockerComposeCmd(t, "-f", "fixtures/scale/build.yaml", "build", "--build-arg", "FOO=test") - c.RunDockerComposeCmd(t, "-f", "fixtures/scale/build.yaml", "up", "-d", "--scale", "test=2") - - c.RunDockerComposeCmd(t, "-f", "fixtures/scale/build.yaml", "build", "--build-arg", "FOO=updated") - c.RunDockerComposeCmd(t, "-f", "fixtures/scale/build.yaml", "up", "-d", "--scale", "test=4", "--no-recreate") - - res := c.RunDockerComposeCmd(t, "ps", "--format", "{{.Name}}", "test") - res.Assert(t, icmd.Success) - assert.Check(t, strings.Contains(res.Stdout(), "scale-down-recreated-test-test-1")) - assert.Check(t, strings.Contains(res.Stdout(), "scale-down-recreated-test-test-2")) - assert.Check(t, strings.Contains(res.Stdout(), "scale-down-recreated-test-test-3")) - assert.Check(t, strings.Contains(res.Stdout(), "scale-down-recreated-test-test-4")) - - t.Log("scale down removes obsolete replica #1 and #2") - c.NewDockerComposeCmd(t, "--project-directory", "fixtures/scale", "up", "-d", "--scale", "test=2") - - res = c.RunDockerComposeCmd(t, "ps", "--format", "{{.Name}}", "test") - res.Assert(t, icmd.Success) - assert.Check(t, strings.Contains(res.Stdout(), "scale-down-recreated-test-test-3")) - assert.Check(t, strings.Contains(res.Stdout(), "scale-down-recreated-test-test-4")) -} diff --git a/pkg/e2e/secrets_test.go b/pkg/e2e/secrets_test.go deleted file mode 100644 index 3e3895112a3..00000000000 --- a/pkg/e2e/secrets_test.go +++ /dev/null @@ -1,53 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "testing" - - "gotest.tools/v3/icmd" -) - -func TestSecretFromEnv(t *testing.T) { - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "env-secret") - - t.Run("compose run", func(t *testing.T) { - res := icmd.RunCmd(c.NewDockerComposeCmd(t, "-f", "./fixtures/env-secret/compose.yaml", "run", "foo"), - func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "SECRET=BAR") - }) - res.Assert(t, icmd.Expected{Out: "BAR"}) - }) - t.Run("secret uid", func(t *testing.T) { - res := icmd.RunCmd(c.NewDockerComposeCmd(t, "-f", "./fixtures/env-secret/compose.yaml", "run", "foo", "ls", "-al", "/var/run/secrets/bar"), - func(cmd *icmd.Cmd) { - cmd.Env = append(cmd.Env, "SECRET=BAR") - }) - res.Assert(t, icmd.Expected{Out: "-r--r----- 1 1005 1005"}) - }) -} - -func TestSecretFromInclude(t *testing.T) { - c := NewParallelCLI(t) - defer c.cleanupWithDown(t, "env-secret-include") - - t.Run("compose run", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/env-secret/compose.yaml", "run", "included") - res.Assert(t, icmd.Expected{Out: "this-is-secret"}) - }) -} diff --git a/pkg/e2e/start_stop_test.go b/pkg/e2e/start_stop_test.go deleted file mode 100644 index 19f07b71960..00000000000 --- a/pkg/e2e/start_stop_test.go +++ /dev/null @@ -1,284 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "strings" - "testing" - - testify "github.com/stretchr/testify/assert" - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestStartStop(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-start-stop-no-dependencies" - - getProjectRegx := func(status string) string { - // match output with random spaces like: - // e2e-start-stop running(3) - return fmt.Sprintf("%s\\s+%s\$%d\$", projectName, status, 2) - } - - t.Run("Up a project", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/start-stop/compose.yaml", "--project-name", projectName, "up", - "-d") - assert.Assert(t, strings.Contains(res.Combined(), "Container e2e-start-stop-no-dependencies-simple-1 Started"), res.Combined()) - - res = c.RunDockerComposeCmd(t, "ls", "--all") - testify.Regexp(t, getProjectRegx("running"), res.Stdout()) - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "ps") - assertServiceStatus(t, projectName, "simple", "Up", res.Stdout()) - assertServiceStatus(t, projectName, "another", "Up", res.Stdout()) - }) - - t.Run("stop project", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/start-stop/compose.yaml", "--project-name", projectName, "stop") - - res := c.RunDockerComposeCmd(t, "ls") - assert.Assert(t, !strings.Contains(res.Combined(), "e2e-start-stop-no-dependencies"), res.Combined()) - - res = c.RunDockerComposeCmd(t, "ls", "--all") - testify.Regexp(t, getProjectRegx("exited"), res.Stdout()) - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "ps") - assert.Assert(t, !strings.Contains(res.Combined(), "e2e-start-stop-no-dependencies-words-1"), res.Combined()) - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "ps", "--all") - assertServiceStatus(t, projectName, "simple", "Exited", res.Stdout()) - assertServiceStatus(t, projectName, "another", "Exited", res.Stdout()) - }) - - t.Run("start project", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/start-stop/compose.yaml", "--project-name", projectName, "start") - - res := c.RunDockerComposeCmd(t, "ls") - testify.Regexp(t, getProjectRegx("running"), res.Stdout()) - }) - - t.Run("down", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) -} - -func TestStartStopWithDependencies(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-start-stop-with-dependencies" - - defer c.RunDockerComposeCmd(t, "--project-name", projectName, "rm", "-fsv") - - t.Run("Up", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/dependencies/compose.yaml", "--project-name", projectName, - "up", "-d") - assert.Assert(t, strings.Contains(res.Combined(), "Container e2e-start-stop-with-dependencies-foo-1 Started"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "Container e2e-start-stop-with-dependencies-bar-1 Started"), res.Combined()) - }) - - t.Run("stop foo", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "stop", "foo") - - assert.Assert(t, strings.Contains(res.Combined(), "Container e2e-start-stop-with-dependencies-foo-1 Stopped"), res.Combined()) - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "ps", "--status", "running") - assert.Assert(t, strings.Contains(res.Combined(), "e2e-start-stop-with-dependencies-bar-1"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "e2e-start-stop-with-dependencies-foo-1"), res.Combined()) - }) - - t.Run("start foo", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "stop") - assert.Assert(t, strings.Contains(res.Combined(), "Container e2e-start-stop-with-dependencies-bar-1 Stopped"), res.Combined()) - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "start", "foo") - out := res.Combined() - assert.Assert(t, strings.Contains(out, "Container e2e-start-stop-with-dependencies-bar-1 Started"), out) - assert.Assert(t, strings.Contains(out, "Container e2e-start-stop-with-dependencies-foo-1 Started"), out) - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "ps", "--status", "running") - out = res.Combined() - assert.Assert(t, strings.Contains(out, "e2e-start-stop-with-dependencies-bar-1"), out) - assert.Assert(t, strings.Contains(out, "e2e-start-stop-with-dependencies-foo-1"), out) - }) - - t.Run("Up no-deps links", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/links/compose.yaml", "--project-name", projectName, "up", - "--no-deps", "-d", "foo") - assert.Assert(t, strings.Contains(res.Combined(), "Container e2e-start-stop-with-dependencies-foo-1 Started"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "Container e2e-start-stop-with-dependencies-bar-1 Started"), res.Combined()) - }) - - t.Run("down", func(t *testing.T) { - _ = c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) -} - -func TestStartStopWithOneOffs(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-start-stop-with-oneoffs" - - t.Run("Up", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/dependencies/compose.yaml", "--project-name", projectName, - "up", "-d") - assert.Assert(t, strings.Contains(res.Combined(), "Container e2e-start-stop-with-oneoffs-foo-1 Started"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "Container e2e-start-stop-with-oneoffs-bar-1 Started"), res.Combined()) - }) - - t.Run("run one-off", func(t *testing.T) { - c.RunDockerComposeCmd(t, "-f", "./fixtures/dependencies/compose.yaml", "--project-name", projectName, "run", "-d", "bar", "sleep", "infinity") - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "ps", "-a") - assert.Assert(t, strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-foo-1"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-bar-1"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-bar-run"), res.Combined()) - }) - - t.Run("stop (not one-off containers)", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "stop") - assert.Assert(t, strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-foo-1"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-bar-1"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "e2e_start_stop_with_oneoffs-bar-run"), res.Combined()) - - res = c.RunDockerComposeCmd(t, "--project-name", projectName, "ps", "-a", "--status", "running") - assert.Assert(t, strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-bar-run"), res.Combined()) - }) - - t.Run("start (not one-off containers)", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "start") - assert.Assert(t, strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-foo-1"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-bar-1"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-bar-run"), res.Combined()) - }) - - t.Run("restart (not one-off containers)", func(t *testing.T) { - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "restart") - assert.Assert(t, strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-foo-1"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-bar-1"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-bar-run"), res.Combined()) - }) - - t.Run("down", func(t *testing.T) { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--remove-orphans") - - res := c.RunDockerComposeCmd(t, "--project-name", projectName, "ps", "-a", "--status", "running") - assert.Assert(t, !strings.Contains(res.Combined(), "e2e-start-stop-with-oneoffs-bar"), res.Combined()) - }) -} - -func TestStartAlreadyRunning(t *testing.T) { - cli := NewParallelCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=e2e-start-stop-svc-already-running", - "COMPOSE_FILE=./fixtures/start-stop/compose.yaml")) - t.Cleanup(func() { - cli.RunDockerComposeCmd(t, "down", "--remove-orphans", "-v", "-t", "0") - }) - - cli.RunDockerComposeCmd(t, "up", "-d", "--wait") - - res := cli.RunDockerComposeCmd(t, "start", "simple") - assert.Equal(t, res.Stdout(), "", "No output should have been written to stdout") -} - -func TestStopAlreadyStopped(t *testing.T) { - cli := NewParallelCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=e2e-start-stop-svc-already-stopped", - "COMPOSE_FILE=./fixtures/start-stop/compose.yaml")) - t.Cleanup(func() { - cli.RunDockerComposeCmd(t, "down", "--remove-orphans", "-v", "-t", "0") - }) - - cli.RunDockerComposeCmd(t, "up", "-d", "--wait") - - // stop the container - cli.RunDockerComposeCmd(t, "stop", "simple") - - // attempt to stop it again - res := cli.RunDockerComposeCmdNoCheck(t, "stop", "simple") - // TODO: for consistency, this should NOT write any output because the - // container is already stopped - res.Assert(t, icmd.Expected{ - ExitCode: 0, - Err: "Container e2e-start-stop-svc-already-stopped-simple-1 Stopped", - }) -} - -func TestStartStopMultipleServices(t *testing.T) { - cli := NewParallelCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=e2e-start-stop-svc-multiple", - "COMPOSE_FILE=./fixtures/start-stop/compose.yaml")) - t.Cleanup(func() { - cli.RunDockerComposeCmd(t, "down", "--remove-orphans", "-v", "-t", "0") - }) - - cli.RunDockerComposeCmd(t, "up", "-d", "--wait") - - res := cli.RunDockerComposeCmd(t, "stop", "simple", "another") - services := []string{"simple", "another"} - for _, svc := range services { - stopMsg := fmt.Sprintf("Container e2e-start-stop-svc-multiple-%s-1 Stopped", svc) - assert.Assert(t, strings.Contains(res.Stderr(), stopMsg), - fmt.Sprintf("Missing stop message for %s\n%s", svc, res.Combined())) - } - - res = cli.RunDockerComposeCmd(t, "start", "simple", "another") - for _, svc := range services { - startMsg := fmt.Sprintf("Container e2e-start-stop-svc-multiple-%s-1 Started", svc) - assert.Assert(t, strings.Contains(res.Stderr(), startMsg), - fmt.Sprintf("Missing start message for %s\n%s", svc, res.Combined())) - } -} - -func TestStartSingleServiceAndDependency(t *testing.T) { - cli := NewParallelCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=e2e-start-single-deps", - "COMPOSE_FILE=./fixtures/start-stop/start-stop-deps.yaml")) - t.Cleanup(func() { - cli.RunDockerComposeCmd(t, "down", "--remove-orphans", "-v", "-t", "0") - }) - - cli.RunDockerComposeCmd(t, "create", "desired") - - res := cli.RunDockerComposeCmd(t, "start", "desired") - desiredServices := []string{"desired", "dep_1", "dep_2"} - for _, s := range desiredServices { - startMsg := fmt.Sprintf("Container e2e-start-single-deps-%s-1 Started", s) - assert.Assert(t, strings.Contains(res.Combined(), startMsg), - fmt.Sprintf("Missing start message for service: %s\n%s", s, res.Combined())) - } - undesiredServices := []string{"another", "another_2"} - for _, s := range undesiredServices { - assert.Assert(t, !strings.Contains(res.Combined(), s), - fmt.Sprintf("Shouldn't have message for service: %s\n%s", s, res.Combined())) - } -} - -func TestStartStopMultipleFiles(t *testing.T) { - cli := NewParallelCLI(t, WithEnv("COMPOSE_PROJECT_NAME=e2e-start-stop-svc-multiple-files")) - t.Cleanup(func() { - cli.RunDockerComposeCmd(t, "-p", "e2e-start-stop-svc-multiple-files", "down", "--remove-orphans") - }) - - cli.RunDockerComposeCmd(t, "-f", "./fixtures/start-stop/compose.yaml", "up", "-d") - cli.RunDockerComposeCmd(t, "-f", "./fixtures/start-stop/other.yaml", "up", "-d") - - res := cli.RunDockerComposeCmd(t, "-f", "./fixtures/start-stop/compose.yaml", "stop") - assert.Assert(t, strings.Contains(res.Combined(), "Container e2e-start-stop-svc-multiple-files-simple-1 Stopped"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), "Container e2e-start-stop-svc-multiple-files-another-1 Stopped"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "Container e2e-start-stop-svc-multiple-files-a-different-one-1 Stopped"), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), "Container e2e-start-stop-svc-multiple-files-and-another-one-1 Stopped"), res.Combined()) -} diff --git a/pkg/e2e/up_test.go b/pkg/e2e/up_test.go deleted file mode 100644 index d34f2061e25..00000000000 --- a/pkg/e2e/up_test.go +++ /dev/null @@ -1,225 +0,0 @@ -//go:build !windows - -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "context" - "errors" - "fmt" - "os/exec" - "strings" - "syscall" - "testing" - "time" - - "github.com/stretchr/testify/require" - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" - - "github.com/docker/compose/v5/pkg/utils" -) - -func TestUpServiceUnhealthy(t *testing.T) { - c := NewParallelCLI(t) - const projectName = "e2e-start-fail" - - res := c.RunDockerComposeCmdNoCheck(t, "-f", "fixtures/start-fail/compose.yaml", "--project-name", projectName, "up", "-d") - res.Assert(t, icmd.Expected{ExitCode: 1, Err: `container e2e-start-fail-fail-1 is unhealthy`}) - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") -} - -func TestUpDependenciesNotStopped(t *testing.T) { - c := NewParallelCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME=up-deps-stop", - )) - - reset := func() { - c.RunDockerComposeCmdNoCheck(t, "down", "-t=0", "--remove-orphans", "-v") - } - reset() - t.Cleanup(reset) - - t.Log("Launching orphan container (background)") - c.RunDockerComposeCmd(t, - "-f=./fixtures/ups-deps-stop/orphan.yaml", - "up", - "--wait", - "--detach", - "orphan", - ) - RequireServiceState(t, c, "orphan", "running") - - t.Log("Launching app container with implicit dependency") - upOut := &utils.SafeBuffer{} - testCmd := c.NewDockerComposeCmd(t, - "-f=./fixtures/ups-deps-stop/compose.yaml", - "up", - "--menu=false", - "app", - ) - - ctx, cancel := context.WithTimeout(t.Context(), 15*time.Second) - t.Cleanup(cancel) - - cmd, err := StartWithNewGroupID(ctx, testCmd, upOut, nil) - assert.NilError(t, err, "Failed to run compose up") - - t.Log("Waiting for containers to be in running state") - upOut.RequireEventuallyContains(t, "hello app") - RequireServiceState(t, c, "app", "running") - RequireServiceState(t, c, "dependency", "running") - - t.Log("Simulating Ctrl-C") - require.NoError(t, syscall.Kill(-cmd.Process.Pid, syscall.SIGINT), - "Failed to send SIGINT to compose up process") - - t.Log("Waiting for `compose up` to exit") - err = cmd.Wait() - if err != nil { - var exitErr *exec.ExitError - errors.As(err, &exitErr) - if exitErr.ExitCode() == -1 { - t.Fatalf("`compose up` was killed: %v", err) - } - require.Equal(t, 130, exitErr.ExitCode()) - } - - RequireServiceState(t, c, "app", "exited") - // dependency should still be running - RequireServiceState(t, c, "dependency", "running") - RequireServiceState(t, c, "orphan", "running") -} - -func TestUpWithBuildDependencies(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("up with service using image build by an another service", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "built-image-dependency") - - res := c.RunDockerComposeCmd(t, "--project-directory", "fixtures/dependencies", - "-f", "fixtures/dependencies/service-image-depends-on.yaml", "up", "-d") - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/dependencies", - "-f", "fixtures/dependencies/service-image-depends-on.yaml", "down", "--rmi", "all") - }) - - res.Assert(t, icmd.Success) - }) -} - -func TestUpWithDependencyExit(t *testing.T) { - c := NewParallelCLI(t) - - t.Run("up with dependency to exit before being healthy", func(t *testing.T) { - res := c.RunDockerComposeCmdNoCheck(t, "--project-directory", "fixtures/dependencies", - "-f", "fixtures/dependencies/dependency-exit.yaml", "up", "-d") - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", "dependencies", "down") - }) - - res.Assert(t, icmd.Expected{ExitCode: 1, Err: "dependency failed to start: container dependencies-db-1 exited (1)"}) - }) -} - -func TestScaleDoesntRecreate(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-scale" - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - c.RunDockerComposeCmd(t, "-f", "fixtures/simple-composefile/compose.yaml", "--project-name", projectName, "up", "-d") - - res := c.RunDockerComposeCmd(t, "-f", "fixtures/simple-composefile/compose.yaml", "--project-name", projectName, "up", "--scale", "simple=2", "-d") - assert.Check(t, !strings.Contains(res.Combined(), "Recreated")) -} - -func TestUpWithDependencyNotRequired(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-dependency-not-required" - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - }) - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/dependencies/deps-not-required.yaml", "--project-name", projectName, - "--profile", "not-required", "up", "-d") - assert.Assert(t, strings.Contains(res.Combined(), "foo"), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), " optional dependency \"bar\" failed to start"), res.Combined()) -} - -func TestUpWithAllResources(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-all-resources" - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "-v") - }) - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/resources/compose.yaml", "--all-resources", "--project-name", projectName, "up") - assert.Assert(t, strings.Contains(res.Combined(), fmt.Sprintf(`Volume %s_my_vol Created`, projectName)), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), fmt.Sprintf(`Network %s_my_net Created`, projectName)), res.Combined()) -} - -func TestUpProfile(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-up-profile" - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "--profile", "test", "down", "-v") - }) - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/profiles/docker-compose.yaml", "--project-name", projectName, "up", "foo") - assert.Assert(t, strings.Contains(res.Combined(), `Container db_c Created`), res.Combined()) - assert.Assert(t, strings.Contains(res.Combined(), `Container foo_c Created`), res.Combined()) - assert.Assert(t, !strings.Contains(res.Combined(), `Container bar_c Created`), res.Combined()) -} - -func TestUpImageID(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-up-image-id" - - digest := strings.TrimSpace(c.RunDockerCmd(t, "image", "inspect", "alpine", "-f", "{{ .ID }}").Stdout()) - _, id, _ := strings.Cut(digest, ":") - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "-v") - }) - - c = NewCLI(t, WithEnv(fmt.Sprintf("ID=%s", id))) - c.RunDockerComposeCmd(t, "-f", "./fixtures/simple-composefile/id.yaml", "--project-name", projectName, "up") -} - -func TestUpStopWithLogsMixed(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-stop-logs" - - t.Cleanup(func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "-v") - }) - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/stop/compose.yaml", "--project-name", projectName, "up", "--abort-on-container-exit") - // assert we still get service2 logs after service 1 Stopped event - res.Assert(t, icmd.Expected{ - Err: "Container compose-e2e-stop-logs-service1-1 Stopped", - }) - // assert we get stop hook logs - res.Assert(t, icmd.Expected{Out: "service2-1 -> | stop hook running...\nservice2-1 | 64 bytes"}) -} diff --git a/pkg/e2e/volumes_test.go b/pkg/e2e/volumes_test.go deleted file mode 100644 index d3e5787fa02..00000000000 --- a/pkg/e2e/volumes_test.go +++ /dev/null @@ -1,236 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "fmt" - "net/http" - "path/filepath" - "runtime" - "strings" - "testing" - "time" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestLocalComposeVolume(t *testing.T) { - c := NewParallelCLI(t) - - const projectName = "compose-e2e-volume" - - t.Run("up with build and no image name, volume", func(t *testing.T) { - // ensure local test run does not reuse previously build image - c.RunDockerOrExitError(t, "rmi", "compose-e2e-volume-nginx") - c.RunDockerOrExitError(t, "volume", "rm", projectName+"-staticVol") - c.RunDockerOrExitError(t, "volume", "rm", "myvolume") - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/volume-test", "--project-name", projectName, "up", - "-d") - }) - - t.Run("access bind mount data", func(t *testing.T) { - output := HTTPGetWithRetry(t, "http://localhost:8090", http.StatusOK, 2*time.Second, 20*time.Second) - assert.Assert(t, strings.Contains(output, "Hello from Nginx container")) - }) - - t.Run("check container volume specs", func(t *testing.T) { - res := c.RunDockerCmd(t, "inspect", "compose-e2e-volume-nginx2-1", "--format", "{{ json .Mounts }}") - output := res.Stdout() - assert.Assert(t, strings.Contains(output, `"Destination":"/usr/src/app/node_modules","Driver":"local","Mode":"z","RW":true,"Propagation":""`), output) - assert.Assert(t, strings.Contains(output, `"Destination":"/myconfig","Mode":"","RW":false,"Propagation":"rprivate"`), output) - }) - - t.Run("check config content", func(t *testing.T) { - output := c.RunDockerCmd(t, "exec", "compose-e2e-volume-nginx2-1", "cat", "/myconfig").Stdout() - assert.Assert(t, strings.Contains(output, `Hello from Nginx container`), output) - }) - - t.Run("check secrets content", func(t *testing.T) { - output := c.RunDockerCmd(t, "exec", "compose-e2e-volume-nginx2-1", "cat", "/run/secrets/mysecret").Stdout() - assert.Assert(t, strings.Contains(output, `Hello from Nginx container`), output) - }) - - t.Run("check container bind-mounts specs", func(t *testing.T) { - res := c.RunDockerCmd(t, "inspect", "compose-e2e-volume-nginx-1", "--format", "{{ json .Mounts }}") - output := res.Stdout() - assert.Assert(t, strings.Contains(output, `"Type":"bind"`)) - assert.Assert(t, strings.Contains(output, `"Destination":"/usr/share/nginx/html"`)) - }) - - t.Run("should inherit anonymous volumes", func(t *testing.T) { - c.RunDockerOrExitError(t, "exec", "compose-e2e-volume-nginx2-1", "touch", "/usr/src/app/node_modules/test") - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/volume-test", "--project-name", projectName, "up", "--force-recreate", "-d") - c.RunDockerOrExitError(t, "exec", "compose-e2e-volume-nginx2-1", "ls", "/usr/src/app/node_modules/test") - }) - - t.Run("should renew anonymous volumes", func(t *testing.T) { - c.RunDockerOrExitError(t, "exec", "compose-e2e-volume-nginx2-1", "touch", "/usr/src/app/node_modules/test") - c.RunDockerComposeCmd(t, "--project-directory", "fixtures/volume-test", "--project-name", projectName, "up", "--force-recreate", "--renew-anon-volumes", "-d") - c.RunDockerOrExitError(t, "exec", "compose-e2e-volume-nginx2-1", "ls", "/usr/src/app/node_modules/test") - }) - - t.Run("cleanup volume project", func(t *testing.T) { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--volumes") - ls := c.RunDockerCmd(t, "volume", "ls").Stdout() - assert.Assert(t, !strings.Contains(ls, projectName+"-staticVol")) - assert.Assert(t, !strings.Contains(ls, "myvolume")) - }) -} - -func TestProjectVolumeBind(t *testing.T) { - if composeStandaloneMode { - t.Skip() - } - c := NewParallelCLI(t) - const projectName = "compose-e2e-project-volume-bind" - - t.Run("up on project volume with bind specification", func(t *testing.T) { - if runtime.GOOS == "windows" { - t.Skip("Running on Windows. Skipping...") - } - tmpDir := t.TempDir() - - c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - - c.RunDockerOrExitError(t, "volume", "rm", "-f", projectName+"_project-data").Assert(t, icmd.Success) - cmd := c.NewCmdWithEnv([]string{"TEST_DIR=" + tmpDir}, - "docker", "compose", "--project-directory", "fixtures/project-volume-bind-test", "--project-name", projectName, "up", "-d") - icmd.RunCmd(cmd).Assert(t, icmd.Success) - defer c.RunDockerComposeCmd(t, "--project-name", projectName, "down") - - c.RunCmd(t, "sh", "-c", "echo SUCCESS > "+filepath.Join(tmpDir, "resultfile")).Assert(t, icmd.Success) - - ret := c.RunDockerOrExitError(t, "exec", "frontend", "bash", "-c", "cat /data/resultfile").Assert(t, icmd.Success) - assert.Assert(t, strings.Contains(ret.Stdout(), "SUCCESS")) - }) -} - -func TestUpSwitchVolumes(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-switch-volumes" - t.Cleanup(func() { - c.cleanupWithDown(t, projectName) - c.RunDockerCmd(t, "volume", "rm", "-f", "test_external_volume") - c.RunDockerCmd(t, "volume", "rm", "-f", "test_external_volume_2") - }) - - c.RunDockerCmd(t, "volume", "create", "test_external_volume") - c.RunDockerCmd(t, "volume", "create", "test_external_volume_2") - - c.RunDockerComposeCmd(t, "-f", "./fixtures/switch-volumes/compose.yaml", "--project-name", projectName, "up", "-d") - - res := c.RunDockerCmd(t, "inspect", fmt.Sprintf("%s-app-1", projectName), "-f", "{{ (index .Mounts 0).Name }}") - res.Assert(t, icmd.Expected{Out: "test_external_volume"}) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/switch-volumes/compose2.yaml", "--project-name", projectName, "up", "-d") - res = c.RunDockerCmd(t, "inspect", fmt.Sprintf("%s-app-1", projectName), "-f", "{{ (index .Mounts 0).Name }}") - res.Assert(t, icmd.Expected{Out: "test_external_volume_2"}) -} - -func TestUpRecreateVolumes(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-recreate-volumes" - t.Cleanup(func() { - c.cleanupWithDown(t, projectName) - }) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/recreate-volumes/compose.yaml", "--project-name", projectName, "up", "-d") - - res := c.RunDockerCmd(t, "volume", "inspect", fmt.Sprintf("%s_my_vol", projectName), "-f", "{{ index .Labels \"foo\" }}") - res.Assert(t, icmd.Expected{Out: "bar"}) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/recreate-volumes/compose2.yaml", "--project-name", projectName, "up", "-d", "-y") - res = c.RunDockerCmd(t, "volume", "inspect", fmt.Sprintf("%s_my_vol", projectName), "-f", "{{ index .Labels \"foo\" }}") - res.Assert(t, icmd.Expected{Out: "zot"}) -} - -func TestUpRecreateVolumes_IgnoreBinds(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-recreate-volumes" - t.Cleanup(func() { - c.cleanupWithDown(t, projectName) - }) - - c.RunDockerComposeCmd(t, "-f", "./fixtures/recreate-volumes/bind.yaml", "--project-name", projectName, "up", "-d") - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/recreate-volumes/bind.yaml", "--project-name", projectName, "up", "-d") - assert.Check(t, !strings.Contains(res.Combined(), "Recreated")) -} - -func TestImageVolume(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-image-volume" - t.Cleanup(func() { - c.cleanupWithDown(t, projectName) - }) - - version := c.RunDockerCmd(t, "version", "-f", "{{.Server.Version}}") - major, _, found := strings.Cut(version.Combined(), ".") - assert.Assert(t, found) - if major == "26" || major == "27" { - t.Skip("Skipping test due to docker version < 28") - } - - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/volumes/compose.yaml", "--project-name", projectName, "up", "with_image") - out := res.Combined() - assert.Check(t, strings.Contains(out, "index.html")) -} - -func TestImageVolumeRecreateOnRebuild(t *testing.T) { - c := NewCLI(t) - const projectName = "compose-e2e-image-volume-recreate" - t.Cleanup(func() { - c.cleanupWithDown(t, projectName) - c.RunDockerOrExitError(t, "rmi", "-f", "image-volume-source") - }) - - version := c.RunDockerCmd(t, "version", "-f", "{{.Server.Version}}") - major, _, found := strings.Cut(version.Combined(), ".") - assert.Assert(t, found) - if major == "26" || major == "27" { - t.Skip("Skipping test due to docker version < 28") - } - - // First build and run with initial content - c.RunDockerComposeCmd(t, "-f", "./fixtures/image-volume-recreate/compose.yaml", - "--project-name", projectName, "build", "--build-arg", "CONTENT=foo") - res := c.RunDockerComposeCmd(t, "-f", "./fixtures/image-volume-recreate/compose.yaml", - "--project-name", projectName, "up", "-d") - assert.Check(t, !strings.Contains(res.Combined(), "error")) - - // Check initial content - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/image-volume-recreate/compose.yaml", - "--project-name", projectName, "logs", "consumer") - assert.Check(t, strings.Contains(res.Combined(), "foo"), "Expected 'foo' in output, got: %s", res.Combined()) - - // Rebuild source image with different content - c.RunDockerComposeCmd(t, "-f", "./fixtures/image-volume-recreate/compose.yaml", - "--project-name", projectName, "build", "--build-arg", "CONTENT=bar") - - // Run up again - consumer should be recreated because source image changed - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/image-volume-recreate/compose.yaml", - "--project-name", projectName, "up", "-d") - // The consumer container should be recreated - assert.Check(t, strings.Contains(res.Combined(), "Recreate") || strings.Contains(res.Combined(), "Created"), - "Expected container to be recreated, got: %s", res.Combined()) - - // Check updated content - res = c.RunDockerComposeCmd(t, "-f", "./fixtures/image-volume-recreate/compose.yaml", - "--project-name", projectName, "logs", "consumer") - assert.Check(t, strings.Contains(res.Combined(), "bar"), "Expected 'bar' in output after rebuild, got: %s", res.Combined()) -} diff --git a/pkg/e2e/wait_test.go b/pkg/e2e/wait_test.go deleted file mode 100644 index 171027fd9c6..00000000000 --- a/pkg/e2e/wait_test.go +++ /dev/null @@ -1,106 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "strings" - "testing" - "time" - - "gotest.tools/v3/assert" - "gotest.tools/v3/icmd" -) - -func TestWaitOnFaster(t *testing.T) { - const projectName = "e2e-wait-faster" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/wait/compose.yaml", "--project-name", projectName, "up", "-d") - c.RunDockerComposeCmd(t, "--project-name", projectName, "wait", "faster") -} - -func TestWaitOnSlower(t *testing.T) { - const projectName = "e2e-wait-slower" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/wait/compose.yaml", "--project-name", projectName, "up", "-d") - c.RunDockerComposeCmd(t, "--project-name", projectName, "wait", "slower") -} - -func TestWaitOnInfinity(t *testing.T) { - const projectName = "e2e-wait-infinity" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/wait/compose.yaml", "--project-name", projectName, "up", "-d") - - cmd := c.NewDockerComposeCmd(t, "--project-name", projectName, "wait", "infinity") - r := icmd.StartCmd(cmd) - assert.NilError(t, r.Error) - t.Cleanup(func() { - if r.Cmd.Process != nil { - _ = r.Cmd.Process.Kill() - } - }) - - finished := make(chan struct{}) - ticker := time.NewTicker(7 * time.Second) - go func() { - _ = r.Cmd.Wait() - finished <- struct{}{} - }() - - select { - case <-finished: - t.Fatal("wait infinity should not finish") - case <-ticker.C: - } -} - -func TestWaitAndDrop(t *testing.T) { - const projectName = "e2e-wait-and-drop" - c := NewParallelCLI(t) - - cleanup := func() { - c.RunDockerComposeCmd(t, "--project-name", projectName, "down", "--timeout=0", "--remove-orphans") - } - t.Cleanup(cleanup) - cleanup() - - c.RunDockerComposeCmd(t, "-f", "./fixtures/wait/compose.yaml", "--project-name", projectName, "up", "-d") - c.RunDockerComposeCmd(t, "--project-name", projectName, "wait", "--down-project", "faster") - - res := c.RunDockerCmd(t, "ps", "--all") - assert.Assert(t, !strings.Contains(res.Combined(), projectName), res.Combined()) -} diff --git a/pkg/e2e/watch_test.go b/pkg/e2e/watch_test.go deleted file mode 100644 index 360fe5210e3..00000000000 --- a/pkg/e2e/watch_test.go +++ /dev/null @@ -1,431 +0,0 @@ -/* - Copyright 2023 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package e2e - -import ( - "bytes" - "crypto/rand" - "fmt" - "os" - "path/filepath" - "strings" - "sync/atomic" - "testing" - "time" - - "github.com/stretchr/testify/require" - "gotest.tools/v3/assert" - "gotest.tools/v3/assert/cmp" - "gotest.tools/v3/icmd" - "gotest.tools/v3/poll" -) - -func TestWatch(t *testing.T) { - services := []string{"alpine", "busybox", "debian"} - for _, svcName := range services { - t.Run(svcName, func(t *testing.T) { - t.Helper() - doTest(t, svcName) - }) - } -} - -func TestRebuildOnDotEnvWithExternalNetwork(t *testing.T) { - const projectName = "test_rebuild_on_dotenv_with_external_network" - const svcName = "ext-alpine" - containerName := strings.Join([]string{projectName, svcName, "1"}, "-") - const networkName = "e2e-watch-external_network_test" - const dotEnvFilepath = "./fixtures/watch/.env" - - c := NewCLI(t, WithEnv( - "COMPOSE_PROJECT_NAME="+projectName, - "COMPOSE_FILE=./fixtures/watch/with-external-network.yaml", - )) - - cleanup := func() { - c.RunDockerComposeCmdNoCheck(t, "down", "--remove-orphans", "--volumes", "--rmi=local") - c.RunDockerOrExitError(t, "network", "rm", networkName) - os.Remove(dotEnvFilepath) //nolint:errcheck - } - cleanup() - - t.Log("create network that is referenced by the container we're testing") - c.RunDockerCmd(t, "network", "create", networkName) - res := c.RunDockerCmd(t, "network", "ls") - assert.Assert(t, !strings.Contains(res.Combined(), projectName), res.Combined()) - - t.Log("create a dotenv file that will be used to trigger the rebuild") - err := os.WriteFile(dotEnvFilepath, []byte("HELLO=WORLD"), 0o666) - assert.NilError(t, err) - _, err = os.ReadFile(dotEnvFilepath) - assert.NilError(t, err) - - // TODO: refactor this duplicated code into frameworks? Maybe? - t.Log("starting docker compose watch") - cmd := c.NewDockerComposeCmd(t, "--verbose", "watch", svcName) - // stream output since watch runs in the background - cmd.Stdout = os.Stdout - cmd.Stderr = os.Stderr - r := icmd.StartCmd(cmd) - require.NoError(t, r.Error) - var testComplete atomic.Bool - go func() { - // if the process exits abnormally before the test is done, fail the test - if err := r.Cmd.Wait(); err != nil && !t.Failed() && !testComplete.Load() { - assert.Check(t, cmp.Nil(err)) - } - }() - - t.Log("wait for watch to start watching") - c.WaitForCondition(t, func() (bool, string) { - out := r.String() - return strings.Contains(out, "Watch enabled"), "watch not started" - }, 30*time.Second, 1*time.Second) - - pn := c.RunDockerCmd(t, "inspect", containerName, "-f", "{{ .HostConfig.NetworkMode }}") - assert.Equal(t, strings.TrimSpace(pn.Stdout()), networkName) - - t.Log("create a dotenv file that will be used to trigger the rebuild") - err = os.WriteFile(dotEnvFilepath, []byte("HELLO=WORLD\nTEST=REBUILD"), 0o666) - assert.NilError(t, err) - _, err = os.ReadFile(dotEnvFilepath) - assert.NilError(t, err) - - // NOTE: are there any other ways to check if the container has been rebuilt? - t.Log("check if the container has been rebuild") - c.WaitForCondition(t, func() (bool, string) { - out := r.String() - if strings.Count(out, "batch complete") != 1 { - return false, fmt.Sprintf("container %s was not rebuilt", containerName) - } - return true, fmt.Sprintf("container %s was rebuilt", containerName) - }, 30*time.Second, 1*time.Second) - - pn2 := c.RunDockerCmd(t, "inspect", containerName, "-f", "{{ .HostConfig.NetworkMode }}") - assert.Equal(t, strings.TrimSpace(pn2.Stdout()), networkName) - - assert.Check(t, !strings.Contains(r.Combined(), "Application failed to start after update")) - - t.Cleanup(cleanup) - t.Cleanup(func() { - // IMPORTANT: watch doesn't exit on its own, don't leak processes! - if r.Cmd.Process != nil { - t.Logf("Killing watch process: pid[%d]", r.Cmd.Process.Pid) - _ = r.Cmd.Process.Kill() - } - }) - testComplete.Store(true) -} - -// NOTE: these tests all share a single Compose file but are safe to run -// concurrently (though that's not recommended). -func doTest(t *testing.T, svcName string) { - tmpdir := t.TempDir() - dataDir := filepath.Join(tmpdir, "data") - configDir := filepath.Join(tmpdir, "config") - - writeTestFile := func(name, contents, sourceDir string) { - t.Helper() - dest := filepath.Join(sourceDir, name) - require.NoError(t, os.MkdirAll(filepath.Dir(dest), 0o700)) - t.Logf("writing %q to %q", contents, dest) - require.NoError(t, os.WriteFile(dest, []byte(contents+"\n"), 0o600)) - } - writeDataFile := func(name, contents string) { - writeTestFile(name, contents, dataDir) - } - - composeFilePath := filepath.Join(tmpdir, "compose.yaml") - CopyFile(t, filepath.Join("fixtures", "watch", "compose.yaml"), composeFilePath) - - projName := "e2e-watch-" + svcName - env := []string{ - "COMPOSE_FILE=" + composeFilePath, - "COMPOSE_PROJECT_NAME=" + projName, - } - - cli := NewCLI(t, WithEnv(env...)) - - // important that --rmi is used to prune the images and ensure that watch builds on launch - defer cli.cleanupWithDown(t, projName, "--rmi=local") - - cmd := cli.NewDockerComposeCmd(t, "--verbose", "watch", svcName) - // stream output since watch runs in the background - cmd.Stdout = os.Stdout - cmd.Stderr = os.Stderr - r := icmd.StartCmd(cmd) - require.NoError(t, r.Error) - t.Cleanup(func() { - // IMPORTANT: watch doesn't exit on its own, don't leak processes! - if r.Cmd.Process != nil { - t.Logf("Killing watch process: pid[%d]", r.Cmd.Process.Pid) - _ = r.Cmd.Process.Kill() - } - }) - var testComplete atomic.Bool - go func() { - // if the process exits abnormally before the test is done, fail the test - if err := r.Cmd.Wait(); err != nil && !t.Failed() && !testComplete.Load() { - assert.Check(t, cmp.Nil(err)) - } - }() - - require.NoError(t, os.Mkdir(dataDir, 0o700)) - - checkFileContents := func(path string, contents string) poll.Check { - return func(pollLog poll.LogT) poll.Result { - if r.Cmd.ProcessState != nil { - return poll.Error(fmt.Errorf("watch process exited early: %s", r.Cmd.ProcessState)) - } - res := icmd.RunCmd(cli.NewDockerComposeCmd(t, "exec", svcName, "cat", path)) - if strings.Contains(res.Stdout(), contents) { - return poll.Success() - } - return poll.Continue("%v", res.Combined()) - } - } - - waitForFlush := func() { - b := make([]byte, 32) - _, _ = rand.Read(b) - sentinelVal := fmt.Sprintf("%x", b) - writeDataFile("wait.txt", sentinelVal) - poll.WaitOn(t, checkFileContents("/app/data/wait.txt", sentinelVal)) - } - - t.Logf("Writing to a file until Compose watch is up and running") - poll.WaitOn(t, func(t poll.LogT) poll.Result { - writeDataFile("hello.txt", "hello world") - return checkFileContents("/app/data/hello.txt", "hello world")(t) - }, poll.WithDelay(time.Second)) - - t.Logf("Modifying file contents") - writeDataFile("hello.txt", "hello watch") - poll.WaitOn(t, checkFileContents("/app/data/hello.txt", "hello watch")) - - t.Logf("Deleting file") - require.NoError(t, os.Remove(filepath.Join(dataDir, "hello.txt"))) - waitForFlush() - cli.RunDockerComposeCmdNoCheck(t, "exec", svcName, "stat", "/app/data/hello.txt"). - Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "No such file or directory", - }) - - t.Logf("Writing to ignored paths") - writeDataFile("data.foo", "ignored") - writeDataFile(filepath.Join("ignored", "hello.txt"), "ignored") - waitForFlush() - cli.RunDockerComposeCmdNoCheck(t, "exec", svcName, "stat", "/app/data/data.foo"). - Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "No such file or directory", - }) - cli.RunDockerComposeCmdNoCheck(t, "exec", svcName, "stat", "/app/data/ignored"). - Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "No such file or directory", - }) - - t.Logf("Creating subdirectory") - require.NoError(t, os.Mkdir(filepath.Join(dataDir, "subdir"), 0o700)) - waitForFlush() - cli.RunDockerComposeCmd(t, "exec", svcName, "stat", "/app/data/subdir") - - t.Logf("Writing to file in subdirectory") - writeDataFile(filepath.Join("subdir", "file.txt"), "a") - poll.WaitOn(t, checkFileContents("/app/data/subdir/file.txt", "a")) - - t.Logf("Writing to file multiple times") - writeDataFile(filepath.Join("subdir", "file.txt"), "x") - writeDataFile(filepath.Join("subdir", "file.txt"), "y") - writeDataFile(filepath.Join("subdir", "file.txt"), "z") - poll.WaitOn(t, checkFileContents("/app/data/subdir/file.txt", "z")) - writeDataFile(filepath.Join("subdir", "file.txt"), "z") - writeDataFile(filepath.Join("subdir", "file.txt"), "y") - writeDataFile(filepath.Join("subdir", "file.txt"), "x") - poll.WaitOn(t, checkFileContents("/app/data/subdir/file.txt", "x")) - - t.Logf("Deleting directory") - require.NoError(t, os.RemoveAll(filepath.Join(dataDir, "subdir"))) - waitForFlush() - cli.RunDockerComposeCmdNoCheck(t, "exec", svcName, "stat", "/app/data/subdir"). - Assert(t, icmd.Expected{ - ExitCode: 1, - Err: "No such file or directory", - }) - - t.Logf("Sync and restart use case") - require.NoError(t, os.Mkdir(configDir, 0o700)) - writeTestFile("file.config", "This is an updated config file", configDir) - checkRestart := func(state string) poll.Check { - return func(pollLog poll.LogT) poll.Result { - if strings.Contains(r.Combined(), state) { - return poll.Success() - } - return poll.Continue("%v", r.Combined()) - } - } - poll.WaitOn(t, checkRestart(fmt.Sprintf("service(s) [%q] restarted", svcName))) - poll.WaitOn(t, checkFileContents("/app/config/file.config", "This is an updated config file")) - - testComplete.Store(true) -} - -func TestWatchExec(t *testing.T) { - c := NewCLI(t) - const projectName = "test_watch_exec" - - defer c.cleanupWithDown(t, projectName) - - tmpdir := t.TempDir() - composeFilePath := filepath.Join(tmpdir, "compose.yaml") - CopyFile(t, filepath.Join("fixtures", "watch", "exec.yaml"), composeFilePath) - cmd := c.NewDockerComposeCmd(t, "-p", projectName, "-f", composeFilePath, "up", "--watch") - buffer := bytes.NewBuffer(nil) - cmd.Stdout = buffer - watch := icmd.StartCmd(cmd) - - poll.WaitOn(t, func(l poll.LogT) poll.Result { - out := buffer.String() - if strings.Contains(out, "64 bytes from") { - return poll.Success() - } - return poll.Continue("%v", watch.Stdout()) - }) - - t.Logf("Create new file") - - testFile := filepath.Join(tmpdir, "test") - require.NoError(t, os.WriteFile(testFile, []byte("test\n"), 0o600)) - - poll.WaitOn(t, func(l poll.LogT) poll.Result { - out := buffer.String() - if strings.Contains(out, "SUCCESS") { - return poll.Success() - } - return poll.Continue("%v", out) - }) - c.RunDockerComposeCmdNoCheck(t, "-p", projectName, "kill", "-s", "9") -} - -func TestWatchMultiServices(t *testing.T) { - c := NewCLI(t) - const projectName = "test_watch_rebuild" - - defer c.cleanupWithDown(t, projectName) - - tmpdir := t.TempDir() - composeFilePath := filepath.Join(tmpdir, "compose.yaml") - CopyFile(t, filepath.Join("fixtures", "watch", "rebuild.yaml"), composeFilePath) - - testFile := filepath.Join(tmpdir, "test") - require.NoError(t, os.WriteFile(testFile, []byte("test"), 0o600)) - - cmd := c.NewDockerComposeCmd(t, "-p", projectName, "-f", composeFilePath, "up", "--watch") - buffer := bytes.NewBuffer(nil) - cmd.Stdout = buffer - watch := icmd.StartCmd(cmd) - - poll.WaitOn(t, func(l poll.LogT) poll.Result { - if strings.Contains(watch.Stdout(), "Attaching to ") { - return poll.Success() - } - return poll.Continue("%v", watch.Stdout()) - }) - - waitRebuild := func(service string, expected string) { - poll.WaitOn(t, func(l poll.LogT) poll.Result { - cat := c.RunDockerComposeCmdNoCheck(t, "-p", projectName, "exec", service, "cat", "/data/"+service) - if strings.Contains(cat.Stdout(), expected) { - return poll.Success() - } - return poll.Continue("%v", cat.Combined()) - }) - } - waitRebuild("a", "test") - waitRebuild("b", "test") - waitRebuild("c", "test") - - require.NoError(t, os.WriteFile(testFile, []byte("updated"), 0o600)) - waitRebuild("a", "updated") - waitRebuild("b", "updated") - waitRebuild("c", "updated") - - c.RunDockerComposeCmdNoCheck(t, "-p", projectName, "kill", "-s", "9") -} - -func TestWatchIncludes(t *testing.T) { - c := NewCLI(t) - const projectName = "test_watch_includes" - - defer c.cleanupWithDown(t, projectName) - - tmpdir := t.TempDir() - composeFilePath := filepath.Join(tmpdir, "compose.yaml") - CopyFile(t, filepath.Join("fixtures", "watch", "include.yaml"), composeFilePath) - - cmd := c.NewDockerComposeCmd(t, "-p", projectName, "-f", composeFilePath, "up", "--watch") - buffer := bytes.NewBuffer(nil) - cmd.Stdout = buffer - watch := icmd.StartCmd(cmd) - - poll.WaitOn(t, func(l poll.LogT) poll.Result { - if strings.Contains(watch.Stdout(), "Attaching to ") { - return poll.Success() - } - return poll.Continue("%v", watch.Stdout()) - }) - - require.NoError(t, os.WriteFile(filepath.Join(tmpdir, "B.test"), []byte("test"), 0o600)) - require.NoError(t, os.WriteFile(filepath.Join(tmpdir, "A.test"), []byte("test"), 0o600)) - - poll.WaitOn(t, func(l poll.LogT) poll.Result { - cat := c.RunDockerComposeCmdNoCheck(t, "-p", projectName, "exec", "a", "ls", "/data/") - if strings.Contains(cat.Stdout(), "A.test") { - assert.Check(t, !strings.Contains(cat.Stdout(), "B.test")) - return poll.Success() - } - return poll.Continue("%v", cat.Combined()) - }) - - c.RunDockerComposeCmdNoCheck(t, "-p", projectName, "kill", "-s", "9") -} - -func TestCheckWarningXInitialSyn(t *testing.T) { - c := NewCLI(t) - const projectName = "test_watch_warn_initial_syn" - - defer c.cleanupWithDown(t, projectName) - - tmpdir := t.TempDir() - composeFilePath := filepath.Join(tmpdir, "compose.yaml") - CopyFile(t, filepath.Join("fixtures", "watch", "x-initialSync.yaml"), composeFilePath) - cmd := c.NewDockerComposeCmd(t, "-p", projectName, "-f", composeFilePath, "--verbose", "up", "--watch") - buffer := bytes.NewBuffer(nil) - cmd.Stdout = buffer - watch := icmd.StartCmd(cmd) - - poll.WaitOn(t, func(l poll.LogT) poll.Result { - if strings.Contains(watch.Combined(), "x-initialSync is DEPRECATED, please use the official `initial_sync` attribute") { - return poll.Success() - } - return poll.Continue("%v", watch.Stdout()) - }) - - c.RunDockerComposeCmdNoCheck(t, "-p", projectName, "kill", "-s", "9") -} diff --git a/pkg/mocks/mock_docker_api.go b/pkg/mocks/mock_docker_api.go deleted file mode 100644 index 4a6ebaaccf4..00000000000 --- a/pkg/mocks/mock_docker_api.go +++ /dev/null @@ -1,1871 +0,0 @@ -// Code generated by MockGen. DO NOT EDIT. -// Source: github.com/docker/docker/client (interfaces: APIClient) -// -// Generated by this command: -// -// mockgen -destination pkg/mocks/mock_docker_api.go -package mocks github.com/docker/docker/client APIClient -// - -// Package mocks is a generated GoMock package. -package mocks - -import ( - context "context" - io "io" - net "net" - http "net/http" - reflect "reflect" - - types "github.com/docker/docker/api/types" - build "github.com/docker/docker/api/types/build" - checkpoint "github.com/docker/docker/api/types/checkpoint" - common "github.com/docker/docker/api/types/common" - container "github.com/docker/docker/api/types/container" - events "github.com/docker/docker/api/types/events" - filters "github.com/docker/docker/api/types/filters" - image "github.com/docker/docker/api/types/image" - network "github.com/docker/docker/api/types/network" - registry "github.com/docker/docker/api/types/registry" - swarm "github.com/docker/docker/api/types/swarm" - system "github.com/docker/docker/api/types/system" - volume "github.com/docker/docker/api/types/volume" - client "github.com/docker/docker/client" - v1 "github.com/opencontainers/image-spec/specs-go/v1" - gomock "go.uber.org/mock/gomock" -) - -// MockAPIClient is a mock of APIClient interface. -type MockAPIClient struct { - ctrl *gomock.Controller - recorder *MockAPIClientMockRecorder -} - -// MockAPIClientMockRecorder is the mock recorder for MockAPIClient. -type MockAPIClientMockRecorder struct { - mock *MockAPIClient -} - -// NewMockAPIClient creates a new mock instance. -func NewMockAPIClient(ctrl *gomock.Controller) *MockAPIClient { - mock := &MockAPIClient{ctrl: ctrl} - mock.recorder = &MockAPIClientMockRecorder{mock} - return mock -} - -// EXPECT returns an object that allows the caller to indicate expected use. -func (m *MockAPIClient) EXPECT() *MockAPIClientMockRecorder { - return m.recorder -} - -// BuildCachePrune mocks base method. -func (m *MockAPIClient) BuildCachePrune(arg0 context.Context, arg1 build.CachePruneOptions) (*build.CachePruneReport, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "BuildCachePrune", arg0, arg1) - ret0, _ := ret[0].(*build.CachePruneReport) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// BuildCachePrune indicates an expected call of BuildCachePrune. -func (mr *MockAPIClientMockRecorder) BuildCachePrune(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "BuildCachePrune", reflect.TypeOf((*MockAPIClient)(nil).BuildCachePrune), arg0, arg1) -} - -// BuildCancel mocks base method. -func (m *MockAPIClient) BuildCancel(arg0 context.Context, arg1 string) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "BuildCancel", arg0, arg1) - ret0, _ := ret[0].(error) - return ret0 -} - -// BuildCancel indicates an expected call of BuildCancel. -func (mr *MockAPIClientMockRecorder) BuildCancel(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "BuildCancel", reflect.TypeOf((*MockAPIClient)(nil).BuildCancel), arg0, arg1) -} - -// CheckpointCreate mocks base method. -func (m *MockAPIClient) CheckpointCreate(arg0 context.Context, arg1 string, arg2 checkpoint.CreateOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "CheckpointCreate", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// CheckpointCreate indicates an expected call of CheckpointCreate. -func (mr *MockAPIClientMockRecorder) CheckpointCreate(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CheckpointCreate", reflect.TypeOf((*MockAPIClient)(nil).CheckpointCreate), arg0, arg1, arg2) -} - -// CheckpointDelete mocks base method. -func (m *MockAPIClient) CheckpointDelete(arg0 context.Context, arg1 string, arg2 checkpoint.DeleteOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "CheckpointDelete", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// CheckpointDelete indicates an expected call of CheckpointDelete. -func (mr *MockAPIClientMockRecorder) CheckpointDelete(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CheckpointDelete", reflect.TypeOf((*MockAPIClient)(nil).CheckpointDelete), arg0, arg1, arg2) -} - -// CheckpointList mocks base method. -func (m *MockAPIClient) CheckpointList(arg0 context.Context, arg1 string, arg2 checkpoint.ListOptions) ([]checkpoint.Summary, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "CheckpointList", arg0, arg1, arg2) - ret0, _ := ret[0].([]checkpoint.Summary) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// CheckpointList indicates an expected call of CheckpointList. -func (mr *MockAPIClientMockRecorder) CheckpointList(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CheckpointList", reflect.TypeOf((*MockAPIClient)(nil).CheckpointList), arg0, arg1, arg2) -} - -// ClientVersion mocks base method. -func (m *MockAPIClient) ClientVersion() string { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ClientVersion") - ret0, _ := ret[0].(string) - return ret0 -} - -// ClientVersion indicates an expected call of ClientVersion. -func (mr *MockAPIClientMockRecorder) ClientVersion() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ClientVersion", reflect.TypeOf((*MockAPIClient)(nil).ClientVersion)) -} - -// Close mocks base method. -func (m *MockAPIClient) Close() error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Close") - ret0, _ := ret[0].(error) - return ret0 -} - -// Close indicates an expected call of Close. -func (mr *MockAPIClientMockRecorder) Close() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Close", reflect.TypeOf((*MockAPIClient)(nil).Close)) -} - -// ConfigCreate mocks base method. -func (m *MockAPIClient) ConfigCreate(arg0 context.Context, arg1 swarm.ConfigSpec) (swarm.ConfigCreateResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ConfigCreate", arg0, arg1) - ret0, _ := ret[0].(swarm.ConfigCreateResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ConfigCreate indicates an expected call of ConfigCreate. -func (mr *MockAPIClientMockRecorder) ConfigCreate(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ConfigCreate", reflect.TypeOf((*MockAPIClient)(nil).ConfigCreate), arg0, arg1) -} - -// ConfigInspectWithRaw mocks base method. -func (m *MockAPIClient) ConfigInspectWithRaw(arg0 context.Context, arg1 string) (swarm.Config, []byte, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ConfigInspectWithRaw", arg0, arg1) - ret0, _ := ret[0].(swarm.Config) - ret1, _ := ret[1].([]byte) - ret2, _ := ret[2].(error) - return ret0, ret1, ret2 -} - -// ConfigInspectWithRaw indicates an expected call of ConfigInspectWithRaw. -func (mr *MockAPIClientMockRecorder) ConfigInspectWithRaw(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ConfigInspectWithRaw", reflect.TypeOf((*MockAPIClient)(nil).ConfigInspectWithRaw), arg0, arg1) -} - -// ConfigList mocks base method. -func (m *MockAPIClient) ConfigList(arg0 context.Context, arg1 swarm.ConfigListOptions) ([]swarm.Config, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ConfigList", arg0, arg1) - ret0, _ := ret[0].([]swarm.Config) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ConfigList indicates an expected call of ConfigList. -func (mr *MockAPIClientMockRecorder) ConfigList(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ConfigList", reflect.TypeOf((*MockAPIClient)(nil).ConfigList), arg0, arg1) -} - -// ConfigRemove mocks base method. -func (m *MockAPIClient) ConfigRemove(arg0 context.Context, arg1 string) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ConfigRemove", arg0, arg1) - ret0, _ := ret[0].(error) - return ret0 -} - -// ConfigRemove indicates an expected call of ConfigRemove. -func (mr *MockAPIClientMockRecorder) ConfigRemove(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ConfigRemove", reflect.TypeOf((*MockAPIClient)(nil).ConfigRemove), arg0, arg1) -} - -// ConfigUpdate mocks base method. -func (m *MockAPIClient) ConfigUpdate(arg0 context.Context, arg1 string, arg2 swarm.Version, arg3 swarm.ConfigSpec) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ConfigUpdate", arg0, arg1, arg2, arg3) - ret0, _ := ret[0].(error) - return ret0 -} - -// ConfigUpdate indicates an expected call of ConfigUpdate. -func (mr *MockAPIClientMockRecorder) ConfigUpdate(arg0, arg1, arg2, arg3 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ConfigUpdate", reflect.TypeOf((*MockAPIClient)(nil).ConfigUpdate), arg0, arg1, arg2, arg3) -} - -// ContainerAttach mocks base method. -func (m *MockAPIClient) ContainerAttach(arg0 context.Context, arg1 string, arg2 container.AttachOptions) (types.HijackedResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerAttach", arg0, arg1, arg2) - ret0, _ := ret[0].(types.HijackedResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerAttach indicates an expected call of ContainerAttach. -func (mr *MockAPIClientMockRecorder) ContainerAttach(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerAttach", reflect.TypeOf((*MockAPIClient)(nil).ContainerAttach), arg0, arg1, arg2) -} - -// ContainerCommit mocks base method. -func (m *MockAPIClient) ContainerCommit(arg0 context.Context, arg1 string, arg2 container.CommitOptions) (common.IDResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerCommit", arg0, arg1, arg2) - ret0, _ := ret[0].(common.IDResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerCommit indicates an expected call of ContainerCommit. -func (mr *MockAPIClientMockRecorder) ContainerCommit(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerCommit", reflect.TypeOf((*MockAPIClient)(nil).ContainerCommit), arg0, arg1, arg2) -} - -// ContainerCreate mocks base method. -func (m *MockAPIClient) ContainerCreate(arg0 context.Context, arg1 *container.Config, arg2 *container.HostConfig, arg3 *network.NetworkingConfig, arg4 *v1.Platform, arg5 string) (container.CreateResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerCreate", arg0, arg1, arg2, arg3, arg4, arg5) - ret0, _ := ret[0].(container.CreateResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerCreate indicates an expected call of ContainerCreate. -func (mr *MockAPIClientMockRecorder) ContainerCreate(arg0, arg1, arg2, arg3, arg4, arg5 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerCreate", reflect.TypeOf((*MockAPIClient)(nil).ContainerCreate), arg0, arg1, arg2, arg3, arg4, arg5) -} - -// ContainerDiff mocks base method. -func (m *MockAPIClient) ContainerDiff(arg0 context.Context, arg1 string) ([]container.FilesystemChange, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerDiff", arg0, arg1) - ret0, _ := ret[0].([]container.FilesystemChange) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerDiff indicates an expected call of ContainerDiff. -func (mr *MockAPIClientMockRecorder) ContainerDiff(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerDiff", reflect.TypeOf((*MockAPIClient)(nil).ContainerDiff), arg0, arg1) -} - -// ContainerExecAttach mocks base method. -func (m *MockAPIClient) ContainerExecAttach(arg0 context.Context, arg1 string, arg2 container.ExecStartOptions) (types.HijackedResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerExecAttach", arg0, arg1, arg2) - ret0, _ := ret[0].(types.HijackedResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerExecAttach indicates an expected call of ContainerExecAttach. -func (mr *MockAPIClientMockRecorder) ContainerExecAttach(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerExecAttach", reflect.TypeOf((*MockAPIClient)(nil).ContainerExecAttach), arg0, arg1, arg2) -} - -// ContainerExecCreate mocks base method. -func (m *MockAPIClient) ContainerExecCreate(arg0 context.Context, arg1 string, arg2 container.ExecOptions) (common.IDResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerExecCreate", arg0, arg1, arg2) - ret0, _ := ret[0].(common.IDResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerExecCreate indicates an expected call of ContainerExecCreate. -func (mr *MockAPIClientMockRecorder) ContainerExecCreate(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerExecCreate", reflect.TypeOf((*MockAPIClient)(nil).ContainerExecCreate), arg0, arg1, arg2) -} - -// ContainerExecInspect mocks base method. -func (m *MockAPIClient) ContainerExecInspect(arg0 context.Context, arg1 string) (container.ExecInspect, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerExecInspect", arg0, arg1) - ret0, _ := ret[0].(container.ExecInspect) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerExecInspect indicates an expected call of ContainerExecInspect. -func (mr *MockAPIClientMockRecorder) ContainerExecInspect(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerExecInspect", reflect.TypeOf((*MockAPIClient)(nil).ContainerExecInspect), arg0, arg1) -} - -// ContainerExecResize mocks base method. -func (m *MockAPIClient) ContainerExecResize(arg0 context.Context, arg1 string, arg2 container.ResizeOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerExecResize", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// ContainerExecResize indicates an expected call of ContainerExecResize. -func (mr *MockAPIClientMockRecorder) ContainerExecResize(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerExecResize", reflect.TypeOf((*MockAPIClient)(nil).ContainerExecResize), arg0, arg1, arg2) -} - -// ContainerExecStart mocks base method. -func (m *MockAPIClient) ContainerExecStart(arg0 context.Context, arg1 string, arg2 container.ExecStartOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerExecStart", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// ContainerExecStart indicates an expected call of ContainerExecStart. -func (mr *MockAPIClientMockRecorder) ContainerExecStart(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerExecStart", reflect.TypeOf((*MockAPIClient)(nil).ContainerExecStart), arg0, arg1, arg2) -} - -// ContainerExport mocks base method. -func (m *MockAPIClient) ContainerExport(arg0 context.Context, arg1 string) (io.ReadCloser, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerExport", arg0, arg1) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerExport indicates an expected call of ContainerExport. -func (mr *MockAPIClientMockRecorder) ContainerExport(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerExport", reflect.TypeOf((*MockAPIClient)(nil).ContainerExport), arg0, arg1) -} - -// ContainerInspect mocks base method. -func (m *MockAPIClient) ContainerInspect(arg0 context.Context, arg1 string) (container.InspectResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerInspect", arg0, arg1) - ret0, _ := ret[0].(container.InspectResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerInspect indicates an expected call of ContainerInspect. -func (mr *MockAPIClientMockRecorder) ContainerInspect(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerInspect", reflect.TypeOf((*MockAPIClient)(nil).ContainerInspect), arg0, arg1) -} - -// ContainerInspectWithRaw mocks base method. -func (m *MockAPIClient) ContainerInspectWithRaw(arg0 context.Context, arg1 string, arg2 bool) (container.InspectResponse, []byte, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerInspectWithRaw", arg0, arg1, arg2) - ret0, _ := ret[0].(container.InspectResponse) - ret1, _ := ret[1].([]byte) - ret2, _ := ret[2].(error) - return ret0, ret1, ret2 -} - -// ContainerInspectWithRaw indicates an expected call of ContainerInspectWithRaw. -func (mr *MockAPIClientMockRecorder) ContainerInspectWithRaw(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerInspectWithRaw", reflect.TypeOf((*MockAPIClient)(nil).ContainerInspectWithRaw), arg0, arg1, arg2) -} - -// ContainerKill mocks base method. -func (m *MockAPIClient) ContainerKill(arg0 context.Context, arg1, arg2 string) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerKill", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// ContainerKill indicates an expected call of ContainerKill. -func (mr *MockAPIClientMockRecorder) ContainerKill(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerKill", reflect.TypeOf((*MockAPIClient)(nil).ContainerKill), arg0, arg1, arg2) -} - -// ContainerList mocks base method. -func (m *MockAPIClient) ContainerList(arg0 context.Context, arg1 container.ListOptions) ([]container.Summary, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerList", arg0, arg1) - ret0, _ := ret[0].([]container.Summary) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerList indicates an expected call of ContainerList. -func (mr *MockAPIClientMockRecorder) ContainerList(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerList", reflect.TypeOf((*MockAPIClient)(nil).ContainerList), arg0, arg1) -} - -// ContainerLogs mocks base method. -func (m *MockAPIClient) ContainerLogs(arg0 context.Context, arg1 string, arg2 container.LogsOptions) (io.ReadCloser, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerLogs", arg0, arg1, arg2) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerLogs indicates an expected call of ContainerLogs. -func (mr *MockAPIClientMockRecorder) ContainerLogs(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerLogs", reflect.TypeOf((*MockAPIClient)(nil).ContainerLogs), arg0, arg1, arg2) -} - -// ContainerPause mocks base method. -func (m *MockAPIClient) ContainerPause(arg0 context.Context, arg1 string) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerPause", arg0, arg1) - ret0, _ := ret[0].(error) - return ret0 -} - -// ContainerPause indicates an expected call of ContainerPause. -func (mr *MockAPIClientMockRecorder) ContainerPause(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerPause", reflect.TypeOf((*MockAPIClient)(nil).ContainerPause), arg0, arg1) -} - -// ContainerRemove mocks base method. -func (m *MockAPIClient) ContainerRemove(arg0 context.Context, arg1 string, arg2 container.RemoveOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerRemove", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// ContainerRemove indicates an expected call of ContainerRemove. -func (mr *MockAPIClientMockRecorder) ContainerRemove(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerRemove", reflect.TypeOf((*MockAPIClient)(nil).ContainerRemove), arg0, arg1, arg2) -} - -// ContainerRename mocks base method. -func (m *MockAPIClient) ContainerRename(arg0 context.Context, arg1, arg2 string) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerRename", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// ContainerRename indicates an expected call of ContainerRename. -func (mr *MockAPIClientMockRecorder) ContainerRename(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerRename", reflect.TypeOf((*MockAPIClient)(nil).ContainerRename), arg0, arg1, arg2) -} - -// ContainerResize mocks base method. -func (m *MockAPIClient) ContainerResize(arg0 context.Context, arg1 string, arg2 container.ResizeOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerResize", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// ContainerResize indicates an expected call of ContainerResize. -func (mr *MockAPIClientMockRecorder) ContainerResize(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerResize", reflect.TypeOf((*MockAPIClient)(nil).ContainerResize), arg0, arg1, arg2) -} - -// ContainerRestart mocks base method. -func (m *MockAPIClient) ContainerRestart(arg0 context.Context, arg1 string, arg2 container.StopOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerRestart", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// ContainerRestart indicates an expected call of ContainerRestart. -func (mr *MockAPIClientMockRecorder) ContainerRestart(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerRestart", reflect.TypeOf((*MockAPIClient)(nil).ContainerRestart), arg0, arg1, arg2) -} - -// ContainerStart mocks base method. -func (m *MockAPIClient) ContainerStart(arg0 context.Context, arg1 string, arg2 container.StartOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerStart", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// ContainerStart indicates an expected call of ContainerStart. -func (mr *MockAPIClientMockRecorder) ContainerStart(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerStart", reflect.TypeOf((*MockAPIClient)(nil).ContainerStart), arg0, arg1, arg2) -} - -// ContainerStatPath mocks base method. -func (m *MockAPIClient) ContainerStatPath(arg0 context.Context, arg1, arg2 string) (container.PathStat, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerStatPath", arg0, arg1, arg2) - ret0, _ := ret[0].(container.PathStat) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerStatPath indicates an expected call of ContainerStatPath. -func (mr *MockAPIClientMockRecorder) ContainerStatPath(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerStatPath", reflect.TypeOf((*MockAPIClient)(nil).ContainerStatPath), arg0, arg1, arg2) -} - -// ContainerStats mocks base method. -func (m *MockAPIClient) ContainerStats(arg0 context.Context, arg1 string, arg2 bool) (container.StatsResponseReader, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerStats", arg0, arg1, arg2) - ret0, _ := ret[0].(container.StatsResponseReader) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerStats indicates an expected call of ContainerStats. -func (mr *MockAPIClientMockRecorder) ContainerStats(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerStats", reflect.TypeOf((*MockAPIClient)(nil).ContainerStats), arg0, arg1, arg2) -} - -// ContainerStatsOneShot mocks base method. -func (m *MockAPIClient) ContainerStatsOneShot(arg0 context.Context, arg1 string) (container.StatsResponseReader, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerStatsOneShot", arg0, arg1) - ret0, _ := ret[0].(container.StatsResponseReader) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerStatsOneShot indicates an expected call of ContainerStatsOneShot. -func (mr *MockAPIClientMockRecorder) ContainerStatsOneShot(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerStatsOneShot", reflect.TypeOf((*MockAPIClient)(nil).ContainerStatsOneShot), arg0, arg1) -} - -// ContainerStop mocks base method. -func (m *MockAPIClient) ContainerStop(arg0 context.Context, arg1 string, arg2 container.StopOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerStop", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// ContainerStop indicates an expected call of ContainerStop. -func (mr *MockAPIClientMockRecorder) ContainerStop(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerStop", reflect.TypeOf((*MockAPIClient)(nil).ContainerStop), arg0, arg1, arg2) -} - -// ContainerTop mocks base method. -func (m *MockAPIClient) ContainerTop(arg0 context.Context, arg1 string, arg2 []string) (container.TopResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerTop", arg0, arg1, arg2) - ret0, _ := ret[0].(container.TopResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerTop indicates an expected call of ContainerTop. -func (mr *MockAPIClientMockRecorder) ContainerTop(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerTop", reflect.TypeOf((*MockAPIClient)(nil).ContainerTop), arg0, arg1, arg2) -} - -// ContainerUnpause mocks base method. -func (m *MockAPIClient) ContainerUnpause(arg0 context.Context, arg1 string) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerUnpause", arg0, arg1) - ret0, _ := ret[0].(error) - return ret0 -} - -// ContainerUnpause indicates an expected call of ContainerUnpause. -func (mr *MockAPIClientMockRecorder) ContainerUnpause(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerUnpause", reflect.TypeOf((*MockAPIClient)(nil).ContainerUnpause), arg0, arg1) -} - -// ContainerUpdate mocks base method. -func (m *MockAPIClient) ContainerUpdate(arg0 context.Context, arg1 string, arg2 container.UpdateConfig) (container.UpdateResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerUpdate", arg0, arg1, arg2) - ret0, _ := ret[0].(container.UpdateResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainerUpdate indicates an expected call of ContainerUpdate. -func (mr *MockAPIClientMockRecorder) ContainerUpdate(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerUpdate", reflect.TypeOf((*MockAPIClient)(nil).ContainerUpdate), arg0, arg1, arg2) -} - -// ContainerWait mocks base method. -func (m *MockAPIClient) ContainerWait(arg0 context.Context, arg1 string, arg2 container.WaitCondition) (<-chan container.WaitResponse, <-chan error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainerWait", arg0, arg1, arg2) - ret0, _ := ret[0].(<-chan container.WaitResponse) - ret1, _ := ret[1].(<-chan error) - return ret0, ret1 -} - -// ContainerWait indicates an expected call of ContainerWait. -func (mr *MockAPIClientMockRecorder) ContainerWait(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainerWait", reflect.TypeOf((*MockAPIClient)(nil).ContainerWait), arg0, arg1, arg2) -} - -// ContainersPrune mocks base method. -func (m *MockAPIClient) ContainersPrune(arg0 context.Context, arg1 filters.Args) (container.PruneReport, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContainersPrune", arg0, arg1) - ret0, _ := ret[0].(container.PruneReport) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ContainersPrune indicates an expected call of ContainersPrune. -func (mr *MockAPIClientMockRecorder) ContainersPrune(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContainersPrune", reflect.TypeOf((*MockAPIClient)(nil).ContainersPrune), arg0, arg1) -} - -// CopyFromContainer mocks base method. -func (m *MockAPIClient) CopyFromContainer(arg0 context.Context, arg1, arg2 string) (io.ReadCloser, container.PathStat, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "CopyFromContainer", arg0, arg1, arg2) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(container.PathStat) - ret2, _ := ret[2].(error) - return ret0, ret1, ret2 -} - -// CopyFromContainer indicates an expected call of CopyFromContainer. -func (mr *MockAPIClientMockRecorder) CopyFromContainer(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CopyFromContainer", reflect.TypeOf((*MockAPIClient)(nil).CopyFromContainer), arg0, arg1, arg2) -} - -// CopyToContainer mocks base method. -func (m *MockAPIClient) CopyToContainer(arg0 context.Context, arg1, arg2 string, arg3 io.Reader, arg4 container.CopyToContainerOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "CopyToContainer", arg0, arg1, arg2, arg3, arg4) - ret0, _ := ret[0].(error) - return ret0 -} - -// CopyToContainer indicates an expected call of CopyToContainer. -func (mr *MockAPIClientMockRecorder) CopyToContainer(arg0, arg1, arg2, arg3, arg4 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CopyToContainer", reflect.TypeOf((*MockAPIClient)(nil).CopyToContainer), arg0, arg1, arg2, arg3, arg4) -} - -// DaemonHost mocks base method. -func (m *MockAPIClient) DaemonHost() string { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "DaemonHost") - ret0, _ := ret[0].(string) - return ret0 -} - -// DaemonHost indicates an expected call of DaemonHost. -func (mr *MockAPIClientMockRecorder) DaemonHost() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "DaemonHost", reflect.TypeOf((*MockAPIClient)(nil).DaemonHost)) -} - -// DialHijack mocks base method. -func (m *MockAPIClient) DialHijack(arg0 context.Context, arg1, arg2 string, arg3 map[string][]string) (net.Conn, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "DialHijack", arg0, arg1, arg2, arg3) - ret0, _ := ret[0].(net.Conn) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// DialHijack indicates an expected call of DialHijack. -func (mr *MockAPIClientMockRecorder) DialHijack(arg0, arg1, arg2, arg3 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "DialHijack", reflect.TypeOf((*MockAPIClient)(nil).DialHijack), arg0, arg1, arg2, arg3) -} - -// Dialer mocks base method. -func (m *MockAPIClient) Dialer() func(context.Context) (net.Conn, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Dialer") - ret0, _ := ret[0].(func(context.Context) (net.Conn, error)) - return ret0 -} - -// Dialer indicates an expected call of Dialer. -func (mr *MockAPIClientMockRecorder) Dialer() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Dialer", reflect.TypeOf((*MockAPIClient)(nil).Dialer)) -} - -// DiskUsage mocks base method. -func (m *MockAPIClient) DiskUsage(arg0 context.Context, arg1 types.DiskUsageOptions) (types.DiskUsage, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "DiskUsage", arg0, arg1) - ret0, _ := ret[0].(types.DiskUsage) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// DiskUsage indicates an expected call of DiskUsage. -func (mr *MockAPIClientMockRecorder) DiskUsage(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "DiskUsage", reflect.TypeOf((*MockAPIClient)(nil).DiskUsage), arg0, arg1) -} - -// DistributionInspect mocks base method. -func (m *MockAPIClient) DistributionInspect(arg0 context.Context, arg1, arg2 string) (registry.DistributionInspect, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "DistributionInspect", arg0, arg1, arg2) - ret0, _ := ret[0].(registry.DistributionInspect) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// DistributionInspect indicates an expected call of DistributionInspect. -func (mr *MockAPIClientMockRecorder) DistributionInspect(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "DistributionInspect", reflect.TypeOf((*MockAPIClient)(nil).DistributionInspect), arg0, arg1, arg2) -} - -// Events mocks base method. -func (m *MockAPIClient) Events(arg0 context.Context, arg1 events.ListOptions) (<-chan events.Message, <-chan error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Events", arg0, arg1) - ret0, _ := ret[0].(<-chan events.Message) - ret1, _ := ret[1].(<-chan error) - return ret0, ret1 -} - -// Events indicates an expected call of Events. -func (mr *MockAPIClientMockRecorder) Events(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Events", reflect.TypeOf((*MockAPIClient)(nil).Events), arg0, arg1) -} - -// HTTPClient mocks base method. -func (m *MockAPIClient) HTTPClient() *http.Client { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "HTTPClient") - ret0, _ := ret[0].(*http.Client) - return ret0 -} - -// HTTPClient indicates an expected call of HTTPClient. -func (mr *MockAPIClientMockRecorder) HTTPClient() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "HTTPClient", reflect.TypeOf((*MockAPIClient)(nil).HTTPClient)) -} - -// ImageBuild mocks base method. -func (m *MockAPIClient) ImageBuild(arg0 context.Context, arg1 io.Reader, arg2 build.ImageBuildOptions) (build.ImageBuildResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ImageBuild", arg0, arg1, arg2) - ret0, _ := ret[0].(build.ImageBuildResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImageBuild indicates an expected call of ImageBuild. -func (mr *MockAPIClientMockRecorder) ImageBuild(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImageBuild", reflect.TypeOf((*MockAPIClient)(nil).ImageBuild), arg0, arg1, arg2) -} - -// ImageCreate mocks base method. -func (m *MockAPIClient) ImageCreate(arg0 context.Context, arg1 string, arg2 image.CreateOptions) (io.ReadCloser, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ImageCreate", arg0, arg1, arg2) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImageCreate indicates an expected call of ImageCreate. -func (mr *MockAPIClientMockRecorder) ImageCreate(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImageCreate", reflect.TypeOf((*MockAPIClient)(nil).ImageCreate), arg0, arg1, arg2) -} - -// ImageHistory mocks base method. -func (m *MockAPIClient) ImageHistory(arg0 context.Context, arg1 string, arg2 ...client.ImageHistoryOption) ([]image.HistoryResponseItem, error) { - m.ctrl.T.Helper() - varargs := []any{arg0, arg1} - for _, a := range arg2 { - varargs = append(varargs, a) - } - ret := m.ctrl.Call(m, "ImageHistory", varargs...) - ret0, _ := ret[0].([]image.HistoryResponseItem) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImageHistory indicates an expected call of ImageHistory. -func (mr *MockAPIClientMockRecorder) ImageHistory(arg0, arg1 any, arg2 ...any) *gomock.Call { - mr.mock.ctrl.T.Helper() - varargs := append([]any{arg0, arg1}, arg2...) - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImageHistory", reflect.TypeOf((*MockAPIClient)(nil).ImageHistory), varargs...) -} - -// ImageImport mocks base method. -func (m *MockAPIClient) ImageImport(arg0 context.Context, arg1 image.ImportSource, arg2 string, arg3 image.ImportOptions) (io.ReadCloser, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ImageImport", arg0, arg1, arg2, arg3) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImageImport indicates an expected call of ImageImport. -func (mr *MockAPIClientMockRecorder) ImageImport(arg0, arg1, arg2, arg3 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImageImport", reflect.TypeOf((*MockAPIClient)(nil).ImageImport), arg0, arg1, arg2, arg3) -} - -// ImageInspect mocks base method. -func (m *MockAPIClient) ImageInspect(arg0 context.Context, arg1 string, arg2 ...client.ImageInspectOption) (image.InspectResponse, error) { - m.ctrl.T.Helper() - varargs := []any{arg0, arg1} - for _, a := range arg2 { - varargs = append(varargs, a) - } - ret := m.ctrl.Call(m, "ImageInspect", varargs...) - ret0, _ := ret[0].(image.InspectResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImageInspect indicates an expected call of ImageInspect. -func (mr *MockAPIClientMockRecorder) ImageInspect(arg0, arg1 any, arg2 ...any) *gomock.Call { - mr.mock.ctrl.T.Helper() - varargs := append([]any{arg0, arg1}, arg2...) - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImageInspect", reflect.TypeOf((*MockAPIClient)(nil).ImageInspect), varargs...) -} - -// ImageInspectWithRaw mocks base method. -func (m *MockAPIClient) ImageInspectWithRaw(arg0 context.Context, arg1 string) (image.InspectResponse, []byte, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ImageInspectWithRaw", arg0, arg1) - ret0, _ := ret[0].(image.InspectResponse) - ret1, _ := ret[1].([]byte) - ret2, _ := ret[2].(error) - return ret0, ret1, ret2 -} - -// ImageInspectWithRaw indicates an expected call of ImageInspectWithRaw. -func (mr *MockAPIClientMockRecorder) ImageInspectWithRaw(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImageInspectWithRaw", reflect.TypeOf((*MockAPIClient)(nil).ImageInspectWithRaw), arg0, arg1) -} - -// ImageList mocks base method. -func (m *MockAPIClient) ImageList(arg0 context.Context, arg1 image.ListOptions) ([]image.Summary, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ImageList", arg0, arg1) - ret0, _ := ret[0].([]image.Summary) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImageList indicates an expected call of ImageList. -func (mr *MockAPIClientMockRecorder) ImageList(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImageList", reflect.TypeOf((*MockAPIClient)(nil).ImageList), arg0, arg1) -} - -// ImageLoad mocks base method. -func (m *MockAPIClient) ImageLoad(arg0 context.Context, arg1 io.Reader, arg2 ...client.ImageLoadOption) (image.LoadResponse, error) { - m.ctrl.T.Helper() - varargs := []any{arg0, arg1} - for _, a := range arg2 { - varargs = append(varargs, a) - } - ret := m.ctrl.Call(m, "ImageLoad", varargs...) - ret0, _ := ret[0].(image.LoadResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImageLoad indicates an expected call of ImageLoad. -func (mr *MockAPIClientMockRecorder) ImageLoad(arg0, arg1 any, arg2 ...any) *gomock.Call { - mr.mock.ctrl.T.Helper() - varargs := append([]any{arg0, arg1}, arg2...) - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImageLoad", reflect.TypeOf((*MockAPIClient)(nil).ImageLoad), varargs...) -} - -// ImagePull mocks base method. -func (m *MockAPIClient) ImagePull(arg0 context.Context, arg1 string, arg2 image.PullOptions) (io.ReadCloser, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ImagePull", arg0, arg1, arg2) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImagePull indicates an expected call of ImagePull. -func (mr *MockAPIClientMockRecorder) ImagePull(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImagePull", reflect.TypeOf((*MockAPIClient)(nil).ImagePull), arg0, arg1, arg2) -} - -// ImagePush mocks base method. -func (m *MockAPIClient) ImagePush(arg0 context.Context, arg1 string, arg2 image.PushOptions) (io.ReadCloser, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ImagePush", arg0, arg1, arg2) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImagePush indicates an expected call of ImagePush. -func (mr *MockAPIClientMockRecorder) ImagePush(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImagePush", reflect.TypeOf((*MockAPIClient)(nil).ImagePush), arg0, arg1, arg2) -} - -// ImageRemove mocks base method. -func (m *MockAPIClient) ImageRemove(arg0 context.Context, arg1 string, arg2 image.RemoveOptions) ([]image.DeleteResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ImageRemove", arg0, arg1, arg2) - ret0, _ := ret[0].([]image.DeleteResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImageRemove indicates an expected call of ImageRemove. -func (mr *MockAPIClientMockRecorder) ImageRemove(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImageRemove", reflect.TypeOf((*MockAPIClient)(nil).ImageRemove), arg0, arg1, arg2) -} - -// ImageSave mocks base method. -func (m *MockAPIClient) ImageSave(arg0 context.Context, arg1 []string, arg2 ...client.ImageSaveOption) (io.ReadCloser, error) { - m.ctrl.T.Helper() - varargs := []any{arg0, arg1} - for _, a := range arg2 { - varargs = append(varargs, a) - } - ret := m.ctrl.Call(m, "ImageSave", varargs...) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImageSave indicates an expected call of ImageSave. -func (mr *MockAPIClientMockRecorder) ImageSave(arg0, arg1 any, arg2 ...any) *gomock.Call { - mr.mock.ctrl.T.Helper() - varargs := append([]any{arg0, arg1}, arg2...) - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImageSave", reflect.TypeOf((*MockAPIClient)(nil).ImageSave), varargs...) -} - -// ImageSearch mocks base method. -func (m *MockAPIClient) ImageSearch(arg0 context.Context, arg1 string, arg2 registry.SearchOptions) ([]registry.SearchResult, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ImageSearch", arg0, arg1, arg2) - ret0, _ := ret[0].([]registry.SearchResult) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImageSearch indicates an expected call of ImageSearch. -func (mr *MockAPIClientMockRecorder) ImageSearch(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImageSearch", reflect.TypeOf((*MockAPIClient)(nil).ImageSearch), arg0, arg1, arg2) -} - -// ImageTag mocks base method. -func (m *MockAPIClient) ImageTag(arg0 context.Context, arg1, arg2 string) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ImageTag", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// ImageTag indicates an expected call of ImageTag. -func (mr *MockAPIClientMockRecorder) ImageTag(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImageTag", reflect.TypeOf((*MockAPIClient)(nil).ImageTag), arg0, arg1, arg2) -} - -// ImagesPrune mocks base method. -func (m *MockAPIClient) ImagesPrune(arg0 context.Context, arg1 filters.Args) (image.PruneReport, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ImagesPrune", arg0, arg1) - ret0, _ := ret[0].(image.PruneReport) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ImagesPrune indicates an expected call of ImagesPrune. -func (mr *MockAPIClientMockRecorder) ImagesPrune(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ImagesPrune", reflect.TypeOf((*MockAPIClient)(nil).ImagesPrune), arg0, arg1) -} - -// Info mocks base method. -func (m *MockAPIClient) Info(arg0 context.Context) (system.Info, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Info", arg0) - ret0, _ := ret[0].(system.Info) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Info indicates an expected call of Info. -func (mr *MockAPIClientMockRecorder) Info(arg0 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Info", reflect.TypeOf((*MockAPIClient)(nil).Info), arg0) -} - -// NegotiateAPIVersion mocks base method. -func (m *MockAPIClient) NegotiateAPIVersion(arg0 context.Context) { - m.ctrl.T.Helper() - m.ctrl.Call(m, "NegotiateAPIVersion", arg0) -} - -// NegotiateAPIVersion indicates an expected call of NegotiateAPIVersion. -func (mr *MockAPIClientMockRecorder) NegotiateAPIVersion(arg0 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NegotiateAPIVersion", reflect.TypeOf((*MockAPIClient)(nil).NegotiateAPIVersion), arg0) -} - -// NegotiateAPIVersionPing mocks base method. -func (m *MockAPIClient) NegotiateAPIVersionPing(arg0 types.Ping) { - m.ctrl.T.Helper() - m.ctrl.Call(m, "NegotiateAPIVersionPing", arg0) -} - -// NegotiateAPIVersionPing indicates an expected call of NegotiateAPIVersionPing. -func (mr *MockAPIClientMockRecorder) NegotiateAPIVersionPing(arg0 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NegotiateAPIVersionPing", reflect.TypeOf((*MockAPIClient)(nil).NegotiateAPIVersionPing), arg0) -} - -// NetworkConnect mocks base method. -func (m *MockAPIClient) NetworkConnect(arg0 context.Context, arg1, arg2 string, arg3 *network.EndpointSettings) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "NetworkConnect", arg0, arg1, arg2, arg3) - ret0, _ := ret[0].(error) - return ret0 -} - -// NetworkConnect indicates an expected call of NetworkConnect. -func (mr *MockAPIClientMockRecorder) NetworkConnect(arg0, arg1, arg2, arg3 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NetworkConnect", reflect.TypeOf((*MockAPIClient)(nil).NetworkConnect), arg0, arg1, arg2, arg3) -} - -// NetworkCreate mocks base method. -func (m *MockAPIClient) NetworkCreate(arg0 context.Context, arg1 string, arg2 network.CreateOptions) (network.CreateResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "NetworkCreate", arg0, arg1, arg2) - ret0, _ := ret[0].(network.CreateResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// NetworkCreate indicates an expected call of NetworkCreate. -func (mr *MockAPIClientMockRecorder) NetworkCreate(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NetworkCreate", reflect.TypeOf((*MockAPIClient)(nil).NetworkCreate), arg0, arg1, arg2) -} - -// NetworkDisconnect mocks base method. -func (m *MockAPIClient) NetworkDisconnect(arg0 context.Context, arg1, arg2 string, arg3 bool) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "NetworkDisconnect", arg0, arg1, arg2, arg3) - ret0, _ := ret[0].(error) - return ret0 -} - -// NetworkDisconnect indicates an expected call of NetworkDisconnect. -func (mr *MockAPIClientMockRecorder) NetworkDisconnect(arg0, arg1, arg2, arg3 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NetworkDisconnect", reflect.TypeOf((*MockAPIClient)(nil).NetworkDisconnect), arg0, arg1, arg2, arg3) -} - -// NetworkInspect mocks base method. -func (m *MockAPIClient) NetworkInspect(arg0 context.Context, arg1 string, arg2 network.InspectOptions) (network.Inspect, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "NetworkInspect", arg0, arg1, arg2) - ret0, _ := ret[0].(network.Inspect) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// NetworkInspect indicates an expected call of NetworkInspect. -func (mr *MockAPIClientMockRecorder) NetworkInspect(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NetworkInspect", reflect.TypeOf((*MockAPIClient)(nil).NetworkInspect), arg0, arg1, arg2) -} - -// NetworkInspectWithRaw mocks base method. -func (m *MockAPIClient) NetworkInspectWithRaw(arg0 context.Context, arg1 string, arg2 network.InspectOptions) (network.Inspect, []byte, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "NetworkInspectWithRaw", arg0, arg1, arg2) - ret0, _ := ret[0].(network.Inspect) - ret1, _ := ret[1].([]byte) - ret2, _ := ret[2].(error) - return ret0, ret1, ret2 -} - -// NetworkInspectWithRaw indicates an expected call of NetworkInspectWithRaw. -func (mr *MockAPIClientMockRecorder) NetworkInspectWithRaw(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NetworkInspectWithRaw", reflect.TypeOf((*MockAPIClient)(nil).NetworkInspectWithRaw), arg0, arg1, arg2) -} - -// NetworkList mocks base method. -func (m *MockAPIClient) NetworkList(arg0 context.Context, arg1 network.ListOptions) ([]network.Inspect, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "NetworkList", arg0, arg1) - ret0, _ := ret[0].([]network.Inspect) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// NetworkList indicates an expected call of NetworkList. -func (mr *MockAPIClientMockRecorder) NetworkList(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NetworkList", reflect.TypeOf((*MockAPIClient)(nil).NetworkList), arg0, arg1) -} - -// NetworkRemove mocks base method. -func (m *MockAPIClient) NetworkRemove(arg0 context.Context, arg1 string) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "NetworkRemove", arg0, arg1) - ret0, _ := ret[0].(error) - return ret0 -} - -// NetworkRemove indicates an expected call of NetworkRemove. -func (mr *MockAPIClientMockRecorder) NetworkRemove(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NetworkRemove", reflect.TypeOf((*MockAPIClient)(nil).NetworkRemove), arg0, arg1) -} - -// NetworksPrune mocks base method. -func (m *MockAPIClient) NetworksPrune(arg0 context.Context, arg1 filters.Args) (network.PruneReport, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "NetworksPrune", arg0, arg1) - ret0, _ := ret[0].(network.PruneReport) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// NetworksPrune indicates an expected call of NetworksPrune. -func (mr *MockAPIClientMockRecorder) NetworksPrune(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NetworksPrune", reflect.TypeOf((*MockAPIClient)(nil).NetworksPrune), arg0, arg1) -} - -// NodeInspectWithRaw mocks base method. -func (m *MockAPIClient) NodeInspectWithRaw(arg0 context.Context, arg1 string) (swarm.Node, []byte, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "NodeInspectWithRaw", arg0, arg1) - ret0, _ := ret[0].(swarm.Node) - ret1, _ := ret[1].([]byte) - ret2, _ := ret[2].(error) - return ret0, ret1, ret2 -} - -// NodeInspectWithRaw indicates an expected call of NodeInspectWithRaw. -func (mr *MockAPIClientMockRecorder) NodeInspectWithRaw(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NodeInspectWithRaw", reflect.TypeOf((*MockAPIClient)(nil).NodeInspectWithRaw), arg0, arg1) -} - -// NodeList mocks base method. -func (m *MockAPIClient) NodeList(arg0 context.Context, arg1 swarm.NodeListOptions) ([]swarm.Node, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "NodeList", arg0, arg1) - ret0, _ := ret[0].([]swarm.Node) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// NodeList indicates an expected call of NodeList. -func (mr *MockAPIClientMockRecorder) NodeList(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NodeList", reflect.TypeOf((*MockAPIClient)(nil).NodeList), arg0, arg1) -} - -// NodeRemove mocks base method. -func (m *MockAPIClient) NodeRemove(arg0 context.Context, arg1 string, arg2 swarm.NodeRemoveOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "NodeRemove", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// NodeRemove indicates an expected call of NodeRemove. -func (mr *MockAPIClientMockRecorder) NodeRemove(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NodeRemove", reflect.TypeOf((*MockAPIClient)(nil).NodeRemove), arg0, arg1, arg2) -} - -// NodeUpdate mocks base method. -func (m *MockAPIClient) NodeUpdate(arg0 context.Context, arg1 string, arg2 swarm.Version, arg3 swarm.NodeSpec) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "NodeUpdate", arg0, arg1, arg2, arg3) - ret0, _ := ret[0].(error) - return ret0 -} - -// NodeUpdate indicates an expected call of NodeUpdate. -func (mr *MockAPIClientMockRecorder) NodeUpdate(arg0, arg1, arg2, arg3 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NodeUpdate", reflect.TypeOf((*MockAPIClient)(nil).NodeUpdate), arg0, arg1, arg2, arg3) -} - -// Ping mocks base method. -func (m *MockAPIClient) Ping(arg0 context.Context) (types.Ping, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Ping", arg0) - ret0, _ := ret[0].(types.Ping) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Ping indicates an expected call of Ping. -func (mr *MockAPIClientMockRecorder) Ping(arg0 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Ping", reflect.TypeOf((*MockAPIClient)(nil).Ping), arg0) -} - -// PluginCreate mocks base method. -func (m *MockAPIClient) PluginCreate(arg0 context.Context, arg1 io.Reader, arg2 types.PluginCreateOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "PluginCreate", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// PluginCreate indicates an expected call of PluginCreate. -func (mr *MockAPIClientMockRecorder) PluginCreate(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "PluginCreate", reflect.TypeOf((*MockAPIClient)(nil).PluginCreate), arg0, arg1, arg2) -} - -// PluginDisable mocks base method. -func (m *MockAPIClient) PluginDisable(arg0 context.Context, arg1 string, arg2 types.PluginDisableOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "PluginDisable", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// PluginDisable indicates an expected call of PluginDisable. -func (mr *MockAPIClientMockRecorder) PluginDisable(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "PluginDisable", reflect.TypeOf((*MockAPIClient)(nil).PluginDisable), arg0, arg1, arg2) -} - -// PluginEnable mocks base method. -func (m *MockAPIClient) PluginEnable(arg0 context.Context, arg1 string, arg2 types.PluginEnableOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "PluginEnable", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// PluginEnable indicates an expected call of PluginEnable. -func (mr *MockAPIClientMockRecorder) PluginEnable(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "PluginEnable", reflect.TypeOf((*MockAPIClient)(nil).PluginEnable), arg0, arg1, arg2) -} - -// PluginInspectWithRaw mocks base method. -func (m *MockAPIClient) PluginInspectWithRaw(arg0 context.Context, arg1 string) (*types.Plugin, []byte, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "PluginInspectWithRaw", arg0, arg1) - ret0, _ := ret[0].(*types.Plugin) - ret1, _ := ret[1].([]byte) - ret2, _ := ret[2].(error) - return ret0, ret1, ret2 -} - -// PluginInspectWithRaw indicates an expected call of PluginInspectWithRaw. -func (mr *MockAPIClientMockRecorder) PluginInspectWithRaw(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "PluginInspectWithRaw", reflect.TypeOf((*MockAPIClient)(nil).PluginInspectWithRaw), arg0, arg1) -} - -// PluginInstall mocks base method. -func (m *MockAPIClient) PluginInstall(arg0 context.Context, arg1 string, arg2 types.PluginInstallOptions) (io.ReadCloser, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "PluginInstall", arg0, arg1, arg2) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// PluginInstall indicates an expected call of PluginInstall. -func (mr *MockAPIClientMockRecorder) PluginInstall(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "PluginInstall", reflect.TypeOf((*MockAPIClient)(nil).PluginInstall), arg0, arg1, arg2) -} - -// PluginList mocks base method. -func (m *MockAPIClient) PluginList(arg0 context.Context, arg1 filters.Args) (types.PluginsListResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "PluginList", arg0, arg1) - ret0, _ := ret[0].(types.PluginsListResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// PluginList indicates an expected call of PluginList. -func (mr *MockAPIClientMockRecorder) PluginList(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "PluginList", reflect.TypeOf((*MockAPIClient)(nil).PluginList), arg0, arg1) -} - -// PluginPush mocks base method. -func (m *MockAPIClient) PluginPush(arg0 context.Context, arg1, arg2 string) (io.ReadCloser, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "PluginPush", arg0, arg1, arg2) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// PluginPush indicates an expected call of PluginPush. -func (mr *MockAPIClientMockRecorder) PluginPush(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "PluginPush", reflect.TypeOf((*MockAPIClient)(nil).PluginPush), arg0, arg1, arg2) -} - -// PluginRemove mocks base method. -func (m *MockAPIClient) PluginRemove(arg0 context.Context, arg1 string, arg2 types.PluginRemoveOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "PluginRemove", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// PluginRemove indicates an expected call of PluginRemove. -func (mr *MockAPIClientMockRecorder) PluginRemove(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "PluginRemove", reflect.TypeOf((*MockAPIClient)(nil).PluginRemove), arg0, arg1, arg2) -} - -// PluginSet mocks base method. -func (m *MockAPIClient) PluginSet(arg0 context.Context, arg1 string, arg2 []string) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "PluginSet", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// PluginSet indicates an expected call of PluginSet. -func (mr *MockAPIClientMockRecorder) PluginSet(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "PluginSet", reflect.TypeOf((*MockAPIClient)(nil).PluginSet), arg0, arg1, arg2) -} - -// PluginUpgrade mocks base method. -func (m *MockAPIClient) PluginUpgrade(arg0 context.Context, arg1 string, arg2 types.PluginInstallOptions) (io.ReadCloser, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "PluginUpgrade", arg0, arg1, arg2) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// PluginUpgrade indicates an expected call of PluginUpgrade. -func (mr *MockAPIClientMockRecorder) PluginUpgrade(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "PluginUpgrade", reflect.TypeOf((*MockAPIClient)(nil).PluginUpgrade), arg0, arg1, arg2) -} - -// RegistryLogin mocks base method. -func (m *MockAPIClient) RegistryLogin(arg0 context.Context, arg1 registry.AuthConfig) (registry.AuthenticateOKBody, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "RegistryLogin", arg0, arg1) - ret0, _ := ret[0].(registry.AuthenticateOKBody) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// RegistryLogin indicates an expected call of RegistryLogin. -func (mr *MockAPIClientMockRecorder) RegistryLogin(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "RegistryLogin", reflect.TypeOf((*MockAPIClient)(nil).RegistryLogin), arg0, arg1) -} - -// SecretCreate mocks base method. -func (m *MockAPIClient) SecretCreate(arg0 context.Context, arg1 swarm.SecretSpec) (swarm.SecretCreateResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "SecretCreate", arg0, arg1) - ret0, _ := ret[0].(swarm.SecretCreateResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// SecretCreate indicates an expected call of SecretCreate. -func (mr *MockAPIClientMockRecorder) SecretCreate(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SecretCreate", reflect.TypeOf((*MockAPIClient)(nil).SecretCreate), arg0, arg1) -} - -// SecretInspectWithRaw mocks base method. -func (m *MockAPIClient) SecretInspectWithRaw(arg0 context.Context, arg1 string) (swarm.Secret, []byte, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "SecretInspectWithRaw", arg0, arg1) - ret0, _ := ret[0].(swarm.Secret) - ret1, _ := ret[1].([]byte) - ret2, _ := ret[2].(error) - return ret0, ret1, ret2 -} - -// SecretInspectWithRaw indicates an expected call of SecretInspectWithRaw. -func (mr *MockAPIClientMockRecorder) SecretInspectWithRaw(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SecretInspectWithRaw", reflect.TypeOf((*MockAPIClient)(nil).SecretInspectWithRaw), arg0, arg1) -} - -// SecretList mocks base method. -func (m *MockAPIClient) SecretList(arg0 context.Context, arg1 swarm.SecretListOptions) ([]swarm.Secret, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "SecretList", arg0, arg1) - ret0, _ := ret[0].([]swarm.Secret) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// SecretList indicates an expected call of SecretList. -func (mr *MockAPIClientMockRecorder) SecretList(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SecretList", reflect.TypeOf((*MockAPIClient)(nil).SecretList), arg0, arg1) -} - -// SecretRemove mocks base method. -func (m *MockAPIClient) SecretRemove(arg0 context.Context, arg1 string) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "SecretRemove", arg0, arg1) - ret0, _ := ret[0].(error) - return ret0 -} - -// SecretRemove indicates an expected call of SecretRemove. -func (mr *MockAPIClientMockRecorder) SecretRemove(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SecretRemove", reflect.TypeOf((*MockAPIClient)(nil).SecretRemove), arg0, arg1) -} - -// SecretUpdate mocks base method. -func (m *MockAPIClient) SecretUpdate(arg0 context.Context, arg1 string, arg2 swarm.Version, arg3 swarm.SecretSpec) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "SecretUpdate", arg0, arg1, arg2, arg3) - ret0, _ := ret[0].(error) - return ret0 -} - -// SecretUpdate indicates an expected call of SecretUpdate. -func (mr *MockAPIClientMockRecorder) SecretUpdate(arg0, arg1, arg2, arg3 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SecretUpdate", reflect.TypeOf((*MockAPIClient)(nil).SecretUpdate), arg0, arg1, arg2, arg3) -} - -// ServerVersion mocks base method. -func (m *MockAPIClient) ServerVersion(arg0 context.Context) (types.Version, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ServerVersion", arg0) - ret0, _ := ret[0].(types.Version) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ServerVersion indicates an expected call of ServerVersion. -func (mr *MockAPIClientMockRecorder) ServerVersion(arg0 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ServerVersion", reflect.TypeOf((*MockAPIClient)(nil).ServerVersion), arg0) -} - -// ServiceCreate mocks base method. -func (m *MockAPIClient) ServiceCreate(arg0 context.Context, arg1 swarm.ServiceSpec, arg2 swarm.ServiceCreateOptions) (swarm.ServiceCreateResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ServiceCreate", arg0, arg1, arg2) - ret0, _ := ret[0].(swarm.ServiceCreateResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ServiceCreate indicates an expected call of ServiceCreate. -func (mr *MockAPIClientMockRecorder) ServiceCreate(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ServiceCreate", reflect.TypeOf((*MockAPIClient)(nil).ServiceCreate), arg0, arg1, arg2) -} - -// ServiceInspectWithRaw mocks base method. -func (m *MockAPIClient) ServiceInspectWithRaw(arg0 context.Context, arg1 string, arg2 swarm.ServiceInspectOptions) (swarm.Service, []byte, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ServiceInspectWithRaw", arg0, arg1, arg2) - ret0, _ := ret[0].(swarm.Service) - ret1, _ := ret[1].([]byte) - ret2, _ := ret[2].(error) - return ret0, ret1, ret2 -} - -// ServiceInspectWithRaw indicates an expected call of ServiceInspectWithRaw. -func (mr *MockAPIClientMockRecorder) ServiceInspectWithRaw(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ServiceInspectWithRaw", reflect.TypeOf((*MockAPIClient)(nil).ServiceInspectWithRaw), arg0, arg1, arg2) -} - -// ServiceList mocks base method. -func (m *MockAPIClient) ServiceList(arg0 context.Context, arg1 swarm.ServiceListOptions) ([]swarm.Service, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ServiceList", arg0, arg1) - ret0, _ := ret[0].([]swarm.Service) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ServiceList indicates an expected call of ServiceList. -func (mr *MockAPIClientMockRecorder) ServiceList(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ServiceList", reflect.TypeOf((*MockAPIClient)(nil).ServiceList), arg0, arg1) -} - -// ServiceLogs mocks base method. -func (m *MockAPIClient) ServiceLogs(arg0 context.Context, arg1 string, arg2 container.LogsOptions) (io.ReadCloser, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ServiceLogs", arg0, arg1, arg2) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ServiceLogs indicates an expected call of ServiceLogs. -func (mr *MockAPIClientMockRecorder) ServiceLogs(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ServiceLogs", reflect.TypeOf((*MockAPIClient)(nil).ServiceLogs), arg0, arg1, arg2) -} - -// ServiceRemove mocks base method. -func (m *MockAPIClient) ServiceRemove(arg0 context.Context, arg1 string) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ServiceRemove", arg0, arg1) - ret0, _ := ret[0].(error) - return ret0 -} - -// ServiceRemove indicates an expected call of ServiceRemove. -func (mr *MockAPIClientMockRecorder) ServiceRemove(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ServiceRemove", reflect.TypeOf((*MockAPIClient)(nil).ServiceRemove), arg0, arg1) -} - -// ServiceUpdate mocks base method. -func (m *MockAPIClient) ServiceUpdate(arg0 context.Context, arg1 string, arg2 swarm.Version, arg3 swarm.ServiceSpec, arg4 swarm.ServiceUpdateOptions) (swarm.ServiceUpdateResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ServiceUpdate", arg0, arg1, arg2, arg3, arg4) - ret0, _ := ret[0].(swarm.ServiceUpdateResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// ServiceUpdate indicates an expected call of ServiceUpdate. -func (mr *MockAPIClientMockRecorder) ServiceUpdate(arg0, arg1, arg2, arg3, arg4 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ServiceUpdate", reflect.TypeOf((*MockAPIClient)(nil).ServiceUpdate), arg0, arg1, arg2, arg3, arg4) -} - -// SwarmGetUnlockKey mocks base method. -func (m *MockAPIClient) SwarmGetUnlockKey(arg0 context.Context) (swarm.UnlockKeyResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "SwarmGetUnlockKey", arg0) - ret0, _ := ret[0].(swarm.UnlockKeyResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// SwarmGetUnlockKey indicates an expected call of SwarmGetUnlockKey. -func (mr *MockAPIClientMockRecorder) SwarmGetUnlockKey(arg0 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SwarmGetUnlockKey", reflect.TypeOf((*MockAPIClient)(nil).SwarmGetUnlockKey), arg0) -} - -// SwarmInit mocks base method. -func (m *MockAPIClient) SwarmInit(arg0 context.Context, arg1 swarm.InitRequest) (string, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "SwarmInit", arg0, arg1) - ret0, _ := ret[0].(string) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// SwarmInit indicates an expected call of SwarmInit. -func (mr *MockAPIClientMockRecorder) SwarmInit(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SwarmInit", reflect.TypeOf((*MockAPIClient)(nil).SwarmInit), arg0, arg1) -} - -// SwarmInspect mocks base method. -func (m *MockAPIClient) SwarmInspect(arg0 context.Context) (swarm.Swarm, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "SwarmInspect", arg0) - ret0, _ := ret[0].(swarm.Swarm) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// SwarmInspect indicates an expected call of SwarmInspect. -func (mr *MockAPIClientMockRecorder) SwarmInspect(arg0 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SwarmInspect", reflect.TypeOf((*MockAPIClient)(nil).SwarmInspect), arg0) -} - -// SwarmJoin mocks base method. -func (m *MockAPIClient) SwarmJoin(arg0 context.Context, arg1 swarm.JoinRequest) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "SwarmJoin", arg0, arg1) - ret0, _ := ret[0].(error) - return ret0 -} - -// SwarmJoin indicates an expected call of SwarmJoin. -func (mr *MockAPIClientMockRecorder) SwarmJoin(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SwarmJoin", reflect.TypeOf((*MockAPIClient)(nil).SwarmJoin), arg0, arg1) -} - -// SwarmLeave mocks base method. -func (m *MockAPIClient) SwarmLeave(arg0 context.Context, arg1 bool) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "SwarmLeave", arg0, arg1) - ret0, _ := ret[0].(error) - return ret0 -} - -// SwarmLeave indicates an expected call of SwarmLeave. -func (mr *MockAPIClientMockRecorder) SwarmLeave(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SwarmLeave", reflect.TypeOf((*MockAPIClient)(nil).SwarmLeave), arg0, arg1) -} - -// SwarmUnlock mocks base method. -func (m *MockAPIClient) SwarmUnlock(arg0 context.Context, arg1 swarm.UnlockRequest) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "SwarmUnlock", arg0, arg1) - ret0, _ := ret[0].(error) - return ret0 -} - -// SwarmUnlock indicates an expected call of SwarmUnlock. -func (mr *MockAPIClientMockRecorder) SwarmUnlock(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SwarmUnlock", reflect.TypeOf((*MockAPIClient)(nil).SwarmUnlock), arg0, arg1) -} - -// SwarmUpdate mocks base method. -func (m *MockAPIClient) SwarmUpdate(arg0 context.Context, arg1 swarm.Version, arg2 swarm.Spec, arg3 swarm.UpdateFlags) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "SwarmUpdate", arg0, arg1, arg2, arg3) - ret0, _ := ret[0].(error) - return ret0 -} - -// SwarmUpdate indicates an expected call of SwarmUpdate. -func (mr *MockAPIClientMockRecorder) SwarmUpdate(arg0, arg1, arg2, arg3 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SwarmUpdate", reflect.TypeOf((*MockAPIClient)(nil).SwarmUpdate), arg0, arg1, arg2, arg3) -} - -// TaskInspectWithRaw mocks base method. -func (m *MockAPIClient) TaskInspectWithRaw(arg0 context.Context, arg1 string) (swarm.Task, []byte, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "TaskInspectWithRaw", arg0, arg1) - ret0, _ := ret[0].(swarm.Task) - ret1, _ := ret[1].([]byte) - ret2, _ := ret[2].(error) - return ret0, ret1, ret2 -} - -// TaskInspectWithRaw indicates an expected call of TaskInspectWithRaw. -func (mr *MockAPIClientMockRecorder) TaskInspectWithRaw(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "TaskInspectWithRaw", reflect.TypeOf((*MockAPIClient)(nil).TaskInspectWithRaw), arg0, arg1) -} - -// TaskList mocks base method. -func (m *MockAPIClient) TaskList(arg0 context.Context, arg1 swarm.TaskListOptions) ([]swarm.Task, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "TaskList", arg0, arg1) - ret0, _ := ret[0].([]swarm.Task) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// TaskList indicates an expected call of TaskList. -func (mr *MockAPIClientMockRecorder) TaskList(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "TaskList", reflect.TypeOf((*MockAPIClient)(nil).TaskList), arg0, arg1) -} - -// TaskLogs mocks base method. -func (m *MockAPIClient) TaskLogs(arg0 context.Context, arg1 string, arg2 container.LogsOptions) (io.ReadCloser, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "TaskLogs", arg0, arg1, arg2) - ret0, _ := ret[0].(io.ReadCloser) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// TaskLogs indicates an expected call of TaskLogs. -func (mr *MockAPIClientMockRecorder) TaskLogs(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "TaskLogs", reflect.TypeOf((*MockAPIClient)(nil).TaskLogs), arg0, arg1, arg2) -} - -// VolumeCreate mocks base method. -func (m *MockAPIClient) VolumeCreate(arg0 context.Context, arg1 volume.CreateOptions) (volume.Volume, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "VolumeCreate", arg0, arg1) - ret0, _ := ret[0].(volume.Volume) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// VolumeCreate indicates an expected call of VolumeCreate. -func (mr *MockAPIClientMockRecorder) VolumeCreate(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "VolumeCreate", reflect.TypeOf((*MockAPIClient)(nil).VolumeCreate), arg0, arg1) -} - -// VolumeInspect mocks base method. -func (m *MockAPIClient) VolumeInspect(arg0 context.Context, arg1 string) (volume.Volume, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "VolumeInspect", arg0, arg1) - ret0, _ := ret[0].(volume.Volume) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// VolumeInspect indicates an expected call of VolumeInspect. -func (mr *MockAPIClientMockRecorder) VolumeInspect(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "VolumeInspect", reflect.TypeOf((*MockAPIClient)(nil).VolumeInspect), arg0, arg1) -} - -// VolumeInspectWithRaw mocks base method. -func (m *MockAPIClient) VolumeInspectWithRaw(arg0 context.Context, arg1 string) (volume.Volume, []byte, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "VolumeInspectWithRaw", arg0, arg1) - ret0, _ := ret[0].(volume.Volume) - ret1, _ := ret[1].([]byte) - ret2, _ := ret[2].(error) - return ret0, ret1, ret2 -} - -// VolumeInspectWithRaw indicates an expected call of VolumeInspectWithRaw. -func (mr *MockAPIClientMockRecorder) VolumeInspectWithRaw(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "VolumeInspectWithRaw", reflect.TypeOf((*MockAPIClient)(nil).VolumeInspectWithRaw), arg0, arg1) -} - -// VolumeList mocks base method. -func (m *MockAPIClient) VolumeList(arg0 context.Context, arg1 volume.ListOptions) (volume.ListResponse, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "VolumeList", arg0, arg1) - ret0, _ := ret[0].(volume.ListResponse) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// VolumeList indicates an expected call of VolumeList. -func (mr *MockAPIClientMockRecorder) VolumeList(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "VolumeList", reflect.TypeOf((*MockAPIClient)(nil).VolumeList), arg0, arg1) -} - -// VolumeRemove mocks base method. -func (m *MockAPIClient) VolumeRemove(arg0 context.Context, arg1 string, arg2 bool) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "VolumeRemove", arg0, arg1, arg2) - ret0, _ := ret[0].(error) - return ret0 -} - -// VolumeRemove indicates an expected call of VolumeRemove. -func (mr *MockAPIClientMockRecorder) VolumeRemove(arg0, arg1, arg2 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "VolumeRemove", reflect.TypeOf((*MockAPIClient)(nil).VolumeRemove), arg0, arg1, arg2) -} - -// VolumeUpdate mocks base method. -func (m *MockAPIClient) VolumeUpdate(arg0 context.Context, arg1 string, arg2 swarm.Version, arg3 volume.UpdateOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "VolumeUpdate", arg0, arg1, arg2, arg3) - ret0, _ := ret[0].(error) - return ret0 -} - -// VolumeUpdate indicates an expected call of VolumeUpdate. -func (mr *MockAPIClientMockRecorder) VolumeUpdate(arg0, arg1, arg2, arg3 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "VolumeUpdate", reflect.TypeOf((*MockAPIClient)(nil).VolumeUpdate), arg0, arg1, arg2, arg3) -} - -// VolumesPrune mocks base method. -func (m *MockAPIClient) VolumesPrune(arg0 context.Context, arg1 filters.Args) (volume.PruneReport, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "VolumesPrune", arg0, arg1) - ret0, _ := ret[0].(volume.PruneReport) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// VolumesPrune indicates an expected call of VolumesPrune. -func (mr *MockAPIClientMockRecorder) VolumesPrune(arg0, arg1 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "VolumesPrune", reflect.TypeOf((*MockAPIClient)(nil).VolumesPrune), arg0, arg1) -} diff --git a/pkg/mocks/mock_docker_cli.go b/pkg/mocks/mock_docker_cli.go deleted file mode 100644 index 663c57ee928..00000000000 --- a/pkg/mocks/mock_docker_cli.go +++ /dev/null @@ -1,275 +0,0 @@ -// Code generated by MockGen. DO NOT EDIT. -// Source: github.com/docker/cli/cli/command (interfaces: Cli) -// -// Generated by this command: -// -// mockgen -destination pkg/mocks/mock_docker_cli.go -package mocks github.com/docker/cli/cli/command Cli -// - -// Package mocks is a generated GoMock package. -package mocks - -import ( - reflect "reflect" - - command "github.com/docker/cli/cli/command" - configfile "github.com/docker/cli/cli/config/configfile" - docker "github.com/docker/cli/cli/context/docker" - store "github.com/docker/cli/cli/context/store" - streams "github.com/docker/cli/cli/streams" - client "github.com/docker/docker/client" - metric "go.opentelemetry.io/otel/metric" - resource "go.opentelemetry.io/otel/sdk/resource" - trace "go.opentelemetry.io/otel/trace" - gomock "go.uber.org/mock/gomock" -) - -// MockCli is a mock of Cli interface. -type MockCli struct { - ctrl *gomock.Controller - recorder *MockCliMockRecorder -} - -// MockCliMockRecorder is the mock recorder for MockCli. -type MockCliMockRecorder struct { - mock *MockCli -} - -// NewMockCli creates a new mock instance. -func NewMockCli(ctrl *gomock.Controller) *MockCli { - mock := &MockCli{ctrl: ctrl} - mock.recorder = &MockCliMockRecorder{mock} - return mock -} - -// EXPECT returns an object that allows the caller to indicate expected use. -func (m *MockCli) EXPECT() *MockCliMockRecorder { - return m.recorder -} - -// Apply mocks base method. -func (m *MockCli) Apply(arg0 ...command.CLIOption) error { - m.ctrl.T.Helper() - varargs := []any{} - for _, a := range arg0 { - varargs = append(varargs, a) - } - ret := m.ctrl.Call(m, "Apply", varargs...) - ret0, _ := ret[0].(error) - return ret0 -} - -// Apply indicates an expected call of Apply. -func (mr *MockCliMockRecorder) Apply(arg0 ...any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Apply", reflect.TypeOf((*MockCli)(nil).Apply), arg0...) -} - -// BuildKitEnabled mocks base method. -func (m *MockCli) BuildKitEnabled() (bool, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "BuildKitEnabled") - ret0, _ := ret[0].(bool) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// BuildKitEnabled indicates an expected call of BuildKitEnabled. -func (mr *MockCliMockRecorder) BuildKitEnabled() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "BuildKitEnabled", reflect.TypeOf((*MockCli)(nil).BuildKitEnabled)) -} - -// Client mocks base method. -func (m *MockCli) Client() client.APIClient { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Client") - ret0, _ := ret[0].(client.APIClient) - return ret0 -} - -// Client indicates an expected call of Client. -func (mr *MockCliMockRecorder) Client() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Client", reflect.TypeOf((*MockCli)(nil).Client)) -} - -// ConfigFile mocks base method. -func (m *MockCli) ConfigFile() *configfile.ConfigFile { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ConfigFile") - ret0, _ := ret[0].(*configfile.ConfigFile) - return ret0 -} - -// ConfigFile indicates an expected call of ConfigFile. -func (mr *MockCliMockRecorder) ConfigFile() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ConfigFile", reflect.TypeOf((*MockCli)(nil).ConfigFile)) -} - -// ContextStore mocks base method. -func (m *MockCli) ContextStore() store.Store { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ContextStore") - ret0, _ := ret[0].(store.Store) - return ret0 -} - -// ContextStore indicates an expected call of ContextStore. -func (mr *MockCliMockRecorder) ContextStore() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ContextStore", reflect.TypeOf((*MockCli)(nil).ContextStore)) -} - -// CurrentContext mocks base method. -func (m *MockCli) CurrentContext() string { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "CurrentContext") - ret0, _ := ret[0].(string) - return ret0 -} - -// CurrentContext indicates an expected call of CurrentContext. -func (mr *MockCliMockRecorder) CurrentContext() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CurrentContext", reflect.TypeOf((*MockCli)(nil).CurrentContext)) -} - -// CurrentVersion mocks base method. -func (m *MockCli) CurrentVersion() string { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "CurrentVersion") - ret0, _ := ret[0].(string) - return ret0 -} - -// CurrentVersion indicates an expected call of CurrentVersion. -func (mr *MockCliMockRecorder) CurrentVersion() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CurrentVersion", reflect.TypeOf((*MockCli)(nil).CurrentVersion)) -} - -// DockerEndpoint mocks base method. -func (m *MockCli) DockerEndpoint() docker.Endpoint { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "DockerEndpoint") - ret0, _ := ret[0].(docker.Endpoint) - return ret0 -} - -// DockerEndpoint indicates an expected call of DockerEndpoint. -func (mr *MockCliMockRecorder) DockerEndpoint() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "DockerEndpoint", reflect.TypeOf((*MockCli)(nil).DockerEndpoint)) -} - -// Err mocks base method. -func (m *MockCli) Err() *streams.Out { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Err") - ret0, _ := ret[0].(*streams.Out) - return ret0 -} - -// Err indicates an expected call of Err. -func (mr *MockCliMockRecorder) Err() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Err", reflect.TypeOf((*MockCli)(nil).Err)) -} - -// In mocks base method. -func (m *MockCli) In() *streams.In { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "In") - ret0, _ := ret[0].(*streams.In) - return ret0 -} - -// In indicates an expected call of In. -func (mr *MockCliMockRecorder) In() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "In", reflect.TypeOf((*MockCli)(nil).In)) -} - -// MeterProvider mocks base method. -func (m *MockCli) MeterProvider() metric.MeterProvider { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "MeterProvider") - ret0, _ := ret[0].(metric.MeterProvider) - return ret0 -} - -// MeterProvider indicates an expected call of MeterProvider. -func (mr *MockCliMockRecorder) MeterProvider() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "MeterProvider", reflect.TypeOf((*MockCli)(nil).MeterProvider)) -} - -// Out mocks base method. -func (m *MockCli) Out() *streams.Out { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Out") - ret0, _ := ret[0].(*streams.Out) - return ret0 -} - -// Out indicates an expected call of Out. -func (mr *MockCliMockRecorder) Out() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Out", reflect.TypeOf((*MockCli)(nil).Out)) -} - -// Resource mocks base method. -func (m *MockCli) Resource() *resource.Resource { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Resource") - ret0, _ := ret[0].(*resource.Resource) - return ret0 -} - -// Resource indicates an expected call of Resource. -func (mr *MockCliMockRecorder) Resource() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Resource", reflect.TypeOf((*MockCli)(nil).Resource)) -} - -// ServerInfo mocks base method. -func (m *MockCli) ServerInfo() command.ServerInfo { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "ServerInfo") - ret0, _ := ret[0].(command.ServerInfo) - return ret0 -} - -// ServerInfo indicates an expected call of ServerInfo. -func (mr *MockCliMockRecorder) ServerInfo() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ServerInfo", reflect.TypeOf((*MockCli)(nil).ServerInfo)) -} - -// SetIn mocks base method. -func (m *MockCli) SetIn(arg0 *streams.In) { - m.ctrl.T.Helper() - m.ctrl.Call(m, "SetIn", arg0) -} - -// SetIn indicates an expected call of SetIn. -func (mr *MockCliMockRecorder) SetIn(arg0 any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "SetIn", reflect.TypeOf((*MockCli)(nil).SetIn), arg0) -} - -// TracerProvider mocks base method. -func (m *MockCli) TracerProvider() trace.TracerProvider { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "TracerProvider") - ret0, _ := ret[0].(trace.TracerProvider) - return ret0 -} - -// TracerProvider indicates an expected call of TracerProvider. -func (mr *MockCliMockRecorder) TracerProvider() *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "TracerProvider", reflect.TypeOf((*MockCli)(nil).TracerProvider)) -} diff --git a/pkg/mocks/mock_docker_compose_api.go b/pkg/mocks/mock_docker_compose_api.go deleted file mode 100644 index db6ddb92ec4..00000000000 --- a/pkg/mocks/mock_docker_compose_api.go +++ /dev/null @@ -1,590 +0,0 @@ -// Code generated by MockGen. DO NOT EDIT. -// Source: ./pkg/api/api.go -// -// Generated by this command: -// -// mockgen -destination pkg/mocks/mock_docker_compose_api.go -package mocks -source=./pkg/api/api.go Service -// - -// Package mocks is a generated GoMock package. -package mocks - -import ( - context "context" - reflect "reflect" - - types "github.com/compose-spec/compose-go/v2/types" - api "github.com/docker/compose/v5/pkg/api" - gomock "go.uber.org/mock/gomock" -) - -// MockCompose is a mock of Compose interface. -type MockCompose struct { - ctrl *gomock.Controller - recorder *MockComposeMockRecorder -} - -// MockComposeMockRecorder is the mock recorder for MockCompose. -type MockComposeMockRecorder struct { - mock *MockCompose -} - -// NewMockCompose creates a new mock instance. -func NewMockCompose(ctrl *gomock.Controller) *MockCompose { - mock := &MockCompose{ctrl: ctrl} - mock.recorder = &MockComposeMockRecorder{mock} - return mock -} - -// EXPECT returns an object that allows the caller to indicate expected use. -func (m *MockCompose) EXPECT() *MockComposeMockRecorder { - return m.recorder -} - -// Attach mocks base method. -func (m *MockCompose) Attach(ctx context.Context, projectName string, options api.AttachOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Attach", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Attach indicates an expected call of Attach. -func (mr *MockComposeMockRecorder) Attach(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Attach", reflect.TypeOf((*MockCompose)(nil).Attach), ctx, projectName, options) -} - -// Build mocks base method. -func (m *MockCompose) Build(ctx context.Context, project *types.Project, options api.BuildOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Build", ctx, project, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Build indicates an expected call of Build. -func (mr *MockComposeMockRecorder) Build(ctx, project, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Build", reflect.TypeOf((*MockCompose)(nil).Build), ctx, project, options) -} - -// Commit mocks base method. -func (m *MockCompose) Commit(ctx context.Context, projectName string, options api.CommitOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Commit", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Commit indicates an expected call of Commit. -func (mr *MockComposeMockRecorder) Commit(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Commit", reflect.TypeOf((*MockCompose)(nil).Commit), ctx, projectName, options) -} - -// Copy mocks base method. -func (m *MockCompose) Copy(ctx context.Context, projectName string, options api.CopyOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Copy", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Copy indicates an expected call of Copy. -func (mr *MockComposeMockRecorder) Copy(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Copy", reflect.TypeOf((*MockCompose)(nil).Copy), ctx, projectName, options) -} - -// Create mocks base method. -func (m *MockCompose) Create(ctx context.Context, project *types.Project, options api.CreateOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Create", ctx, project, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Create indicates an expected call of Create. -func (mr *MockComposeMockRecorder) Create(ctx, project, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Create", reflect.TypeOf((*MockCompose)(nil).Create), ctx, project, options) -} - -// Down mocks base method. -func (m *MockCompose) Down(ctx context.Context, projectName string, options api.DownOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Down", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Down indicates an expected call of Down. -func (mr *MockComposeMockRecorder) Down(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Down", reflect.TypeOf((*MockCompose)(nil).Down), ctx, projectName, options) -} - -// Events mocks base method. -func (m *MockCompose) Events(ctx context.Context, projectName string, options api.EventsOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Events", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Events indicates an expected call of Events. -func (mr *MockComposeMockRecorder) Events(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Events", reflect.TypeOf((*MockCompose)(nil).Events), ctx, projectName, options) -} - -// Exec mocks base method. -func (m *MockCompose) Exec(ctx context.Context, projectName string, options api.RunOptions) (int, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Exec", ctx, projectName, options) - ret0, _ := ret[0].(int) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Exec indicates an expected call of Exec. -func (mr *MockComposeMockRecorder) Exec(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Exec", reflect.TypeOf((*MockCompose)(nil).Exec), ctx, projectName, options) -} - -// Export mocks base method. -func (m *MockCompose) Export(ctx context.Context, projectName string, options api.ExportOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Export", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Export indicates an expected call of Export. -func (mr *MockComposeMockRecorder) Export(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Export", reflect.TypeOf((*MockCompose)(nil).Export), ctx, projectName, options) -} - -// Generate mocks base method. -func (m *MockCompose) Generate(ctx context.Context, options api.GenerateOptions) (*types.Project, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Generate", ctx, options) - ret0, _ := ret[0].(*types.Project) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Generate indicates an expected call of Generate. -func (mr *MockComposeMockRecorder) Generate(ctx, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Generate", reflect.TypeOf((*MockCompose)(nil).Generate), ctx, options) -} - -// Images mocks base method. -func (m *MockCompose) Images(ctx context.Context, projectName string, options api.ImagesOptions) (map[string]api.ImageSummary, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Images", ctx, projectName, options) - ret0, _ := ret[0].(map[string]api.ImageSummary) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Images indicates an expected call of Images. -func (mr *MockComposeMockRecorder) Images(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Images", reflect.TypeOf((*MockCompose)(nil).Images), ctx, projectName, options) -} - -// Kill mocks base method. -func (m *MockCompose) Kill(ctx context.Context, projectName string, options api.KillOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Kill", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Kill indicates an expected call of Kill. -func (mr *MockComposeMockRecorder) Kill(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Kill", reflect.TypeOf((*MockCompose)(nil).Kill), ctx, projectName, options) -} - -// List mocks base method. -func (m *MockCompose) List(ctx context.Context, options api.ListOptions) ([]api.Stack, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "List", ctx, options) - ret0, _ := ret[0].([]api.Stack) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// List indicates an expected call of List. -func (mr *MockComposeMockRecorder) List(ctx, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "List", reflect.TypeOf((*MockCompose)(nil).List), ctx, options) -} - -// LoadProject mocks base method. -func (m *MockCompose) LoadProject(ctx context.Context, options api.ProjectLoadOptions) (*types.Project, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "LoadProject", ctx, options) - ret0, _ := ret[0].(*types.Project) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// LoadProject indicates an expected call of LoadProject. -func (mr *MockComposeMockRecorder) LoadProject(ctx, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "LoadProject", reflect.TypeOf((*MockCompose)(nil).LoadProject), ctx, options) -} - -// Logs mocks base method. -func (m *MockCompose) Logs(ctx context.Context, projectName string, consumer api.LogConsumer, options api.LogOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Logs", ctx, projectName, consumer, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Logs indicates an expected call of Logs. -func (mr *MockComposeMockRecorder) Logs(ctx, projectName, consumer, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Logs", reflect.TypeOf((*MockCompose)(nil).Logs), ctx, projectName, consumer, options) -} - -// Pause mocks base method. -func (m *MockCompose) Pause(ctx context.Context, projectName string, options api.PauseOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Pause", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Pause indicates an expected call of Pause. -func (mr *MockComposeMockRecorder) Pause(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Pause", reflect.TypeOf((*MockCompose)(nil).Pause), ctx, projectName, options) -} - -// Port mocks base method. -func (m *MockCompose) Port(ctx context.Context, projectName, service string, port uint16, options api.PortOptions) (string, int, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Port", ctx, projectName, service, port, options) - ret0, _ := ret[0].(string) - ret1, _ := ret[1].(int) - ret2, _ := ret[2].(error) - return ret0, ret1, ret2 -} - -// Port indicates an expected call of Port. -func (mr *MockComposeMockRecorder) Port(ctx, projectName, service, port, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Port", reflect.TypeOf((*MockCompose)(nil).Port), ctx, projectName, service, port, options) -} - -// Ps mocks base method. -func (m *MockCompose) Ps(ctx context.Context, projectName string, options api.PsOptions) ([]api.ContainerSummary, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Ps", ctx, projectName, options) - ret0, _ := ret[0].([]api.ContainerSummary) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Ps indicates an expected call of Ps. -func (mr *MockComposeMockRecorder) Ps(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Ps", reflect.TypeOf((*MockCompose)(nil).Ps), ctx, projectName, options) -} - -// Publish mocks base method. -func (m *MockCompose) Publish(ctx context.Context, project *types.Project, repository string, options api.PublishOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Publish", ctx, project, repository, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Publish indicates an expected call of Publish. -func (mr *MockComposeMockRecorder) Publish(ctx, project, repository, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Publish", reflect.TypeOf((*MockCompose)(nil).Publish), ctx, project, repository, options) -} - -// Pull mocks base method. -func (m *MockCompose) Pull(ctx context.Context, project *types.Project, options api.PullOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Pull", ctx, project, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Pull indicates an expected call of Pull. -func (mr *MockComposeMockRecorder) Pull(ctx, project, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Pull", reflect.TypeOf((*MockCompose)(nil).Pull), ctx, project, options) -} - -// Push mocks base method. -func (m *MockCompose) Push(ctx context.Context, project *types.Project, options api.PushOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Push", ctx, project, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Push indicates an expected call of Push. -func (mr *MockComposeMockRecorder) Push(ctx, project, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Push", reflect.TypeOf((*MockCompose)(nil).Push), ctx, project, options) -} - -// Remove mocks base method. -func (m *MockCompose) Remove(ctx context.Context, projectName string, options api.RemoveOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Remove", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Remove indicates an expected call of Remove. -func (mr *MockComposeMockRecorder) Remove(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Remove", reflect.TypeOf((*MockCompose)(nil).Remove), ctx, projectName, options) -} - -// Restart mocks base method. -func (m *MockCompose) Restart(ctx context.Context, projectName string, options api.RestartOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Restart", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Restart indicates an expected call of Restart. -func (mr *MockComposeMockRecorder) Restart(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Restart", reflect.TypeOf((*MockCompose)(nil).Restart), ctx, projectName, options) -} - -// RunOneOffContainer mocks base method. -func (m *MockCompose) RunOneOffContainer(ctx context.Context, project *types.Project, opts api.RunOptions) (int, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "RunOneOffContainer", ctx, project, opts) - ret0, _ := ret[0].(int) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// RunOneOffContainer indicates an expected call of RunOneOffContainer. -func (mr *MockComposeMockRecorder) RunOneOffContainer(ctx, project, opts any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "RunOneOffContainer", reflect.TypeOf((*MockCompose)(nil).RunOneOffContainer), ctx, project, opts) -} - -// Scale mocks base method. -func (m *MockCompose) Scale(ctx context.Context, project *types.Project, options api.ScaleOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Scale", ctx, project, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Scale indicates an expected call of Scale. -func (mr *MockComposeMockRecorder) Scale(ctx, project, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Scale", reflect.TypeOf((*MockCompose)(nil).Scale), ctx, project, options) -} - -// Start mocks base method. -func (m *MockCompose) Start(ctx context.Context, projectName string, options api.StartOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Start", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Start indicates an expected call of Start. -func (mr *MockComposeMockRecorder) Start(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Start", reflect.TypeOf((*MockCompose)(nil).Start), ctx, projectName, options) -} - -// Stop mocks base method. -func (m *MockCompose) Stop(ctx context.Context, projectName string, options api.StopOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Stop", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Stop indicates an expected call of Stop. -func (mr *MockComposeMockRecorder) Stop(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Stop", reflect.TypeOf((*MockCompose)(nil).Stop), ctx, projectName, options) -} - -// Top mocks base method. -func (m *MockCompose) Top(ctx context.Context, projectName string, services []string) ([]api.ContainerProcSummary, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Top", ctx, projectName, services) - ret0, _ := ret[0].([]api.ContainerProcSummary) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Top indicates an expected call of Top. -func (mr *MockComposeMockRecorder) Top(ctx, projectName, services any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Top", reflect.TypeOf((*MockCompose)(nil).Top), ctx, projectName, services) -} - -// UnPause mocks base method. -func (m *MockCompose) UnPause(ctx context.Context, projectName string, options api.PauseOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "UnPause", ctx, projectName, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// UnPause indicates an expected call of UnPause. -func (mr *MockComposeMockRecorder) UnPause(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "UnPause", reflect.TypeOf((*MockCompose)(nil).UnPause), ctx, projectName, options) -} - -// Up mocks base method. -func (m *MockCompose) Up(ctx context.Context, project *types.Project, options api.UpOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Up", ctx, project, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Up indicates an expected call of Up. -func (mr *MockComposeMockRecorder) Up(ctx, project, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Up", reflect.TypeOf((*MockCompose)(nil).Up), ctx, project, options) -} - -// Viz mocks base method. -func (m *MockCompose) Viz(ctx context.Context, project *types.Project, options api.VizOptions) (string, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Viz", ctx, project, options) - ret0, _ := ret[0].(string) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Viz indicates an expected call of Viz. -func (mr *MockComposeMockRecorder) Viz(ctx, project, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Viz", reflect.TypeOf((*MockCompose)(nil).Viz), ctx, project, options) -} - -// Volumes mocks base method. -func (m *MockCompose) Volumes(ctx context.Context, project string, options api.VolumesOptions) ([]api.VolumesSummary, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Volumes", ctx, project, options) - ret0, _ := ret[0].([]api.VolumesSummary) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Volumes indicates an expected call of Volumes. -func (mr *MockComposeMockRecorder) Volumes(ctx, project, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Volumes", reflect.TypeOf((*MockCompose)(nil).Volumes), ctx, project, options) -} - -// Wait mocks base method. -func (m *MockCompose) Wait(ctx context.Context, projectName string, options api.WaitOptions) (int64, error) { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Wait", ctx, projectName, options) - ret0, _ := ret[0].(int64) - ret1, _ := ret[1].(error) - return ret0, ret1 -} - -// Wait indicates an expected call of Wait. -func (mr *MockComposeMockRecorder) Wait(ctx, projectName, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Wait", reflect.TypeOf((*MockCompose)(nil).Wait), ctx, projectName, options) -} - -// Watch mocks base method. -func (m *MockCompose) Watch(ctx context.Context, project *types.Project, options api.WatchOptions) error { - m.ctrl.T.Helper() - ret := m.ctrl.Call(m, "Watch", ctx, project, options) - ret0, _ := ret[0].(error) - return ret0 -} - -// Watch indicates an expected call of Watch. -func (mr *MockComposeMockRecorder) Watch(ctx, project, options any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Watch", reflect.TypeOf((*MockCompose)(nil).Watch), ctx, project, options) -} - -// MockLogConsumer is a mock of LogConsumer interface. -type MockLogConsumer struct { - ctrl *gomock.Controller - recorder *MockLogConsumerMockRecorder -} - -// MockLogConsumerMockRecorder is the mock recorder for MockLogConsumer. -type MockLogConsumerMockRecorder struct { - mock *MockLogConsumer -} - -// NewMockLogConsumer creates a new mock instance. -func NewMockLogConsumer(ctrl *gomock.Controller) *MockLogConsumer { - mock := &MockLogConsumer{ctrl: ctrl} - mock.recorder = &MockLogConsumerMockRecorder{mock} - return mock -} - -// EXPECT returns an object that allows the caller to indicate expected use. -func (m *MockLogConsumer) EXPECT() *MockLogConsumerMockRecorder { - return m.recorder -} - -// Err mocks base method. -func (m *MockLogConsumer) Err(containerName, message string) { - m.ctrl.T.Helper() - m.ctrl.Call(m, "Err", containerName, message) -} - -// Err indicates an expected call of Err. -func (mr *MockLogConsumerMockRecorder) Err(containerName, message any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Err", reflect.TypeOf((*MockLogConsumer)(nil).Err), containerName, message) -} - -// Log mocks base method. -func (m *MockLogConsumer) Log(containerName, message string) { - m.ctrl.T.Helper() - m.ctrl.Call(m, "Log", containerName, message) -} - -// Log indicates an expected call of Log. -func (mr *MockLogConsumerMockRecorder) Log(containerName, message any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Log", reflect.TypeOf((*MockLogConsumer)(nil).Log), containerName, message) -} - -// Status mocks base method. -func (m *MockLogConsumer) Status(container, msg string) { - m.ctrl.T.Helper() - m.ctrl.Call(m, "Status", container, msg) -} - -// Status indicates an expected call of Status. -func (mr *MockLogConsumerMockRecorder) Status(container, msg any) *gomock.Call { - mr.mock.ctrl.T.Helper() - return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Status", reflect.TypeOf((*MockLogConsumer)(nil).Status), container, msg) -} diff --git a/pkg/remote/cache.go b/pkg/remote/cache.go deleted file mode 100644 index a0a6d03194e..00000000000 --- a/pkg/remote/cache.go +++ /dev/null @@ -1,36 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package remote - -import ( - "os" - "path/filepath" -) - -func cacheDir() (string, error) { - cache, ok := os.LookupEnv("XDG_CACHE_HOME") - if ok { - return filepath.Join(cache, "docker-compose"), nil - } - - path, err := osDependentCacheDir() - if err != nil { - return "", err - } - err = os.MkdirAll(path, 0o700) - return path, err -} diff --git a/pkg/remote/cache_darwin.go b/pkg/remote/cache_darwin.go deleted file mode 100644 index 7830e8ade45..00000000000 --- a/pkg/remote/cache_darwin.go +++ /dev/null @@ -1,34 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package remote - -import ( - "os" - "path/filepath" -) - -// Based on https://github.com/adrg/xdg -// Licensed under MIT License (MIT) -// Copyright (c) 2014 Adrian-George Bostan - -func osDependentCacheDir() (string, error) { - home, err := os.UserHomeDir() - if err != nil { - return "", err - } - return filepath.Join(home, "Library", "Caches", "docker-compose"), nil -} diff --git a/pkg/remote/cache_unix.go b/pkg/remote/cache_unix.go deleted file mode 100644 index 9887dc7bb4e..00000000000 --- a/pkg/remote/cache_unix.go +++ /dev/null @@ -1,36 +0,0 @@ -//go:build linux || openbsd || freebsd - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package remote - -import ( - "os" - "path/filepath" -) - -// Based on https://github.com/adrg/xdg -// Licensed under MIT License (MIT) -// Copyright (c) 2014 Adrian-George Bostan - -func osDependentCacheDir() (string, error) { - home, err := os.UserHomeDir() - if err != nil { - return "", err - } - return filepath.Join(home, ".cache", "docker-compose"), nil -} diff --git a/pkg/remote/cache_windows.go b/pkg/remote/cache_windows.go deleted file mode 100644 index 5bc7a2f1d92..00000000000 --- a/pkg/remote/cache_windows.go +++ /dev/null @@ -1,49 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package remote - -import ( - "os" - "path/filepath" - - "golang.org/x/sys/windows" -) - -// Based on https://github.com/adrg/xdg -// Licensed under MIT License (MIT) -// Copyright (c) 2014 Adrian-George Bostan - -func osDependentCacheDir() (string, error) { - flags := []uint32{windows.KF_FLAG_DEFAULT, windows.KF_FLAG_DEFAULT_PATH} - for _, flag := range flags { - p, _ := windows.KnownFolderPath(windows.FOLDERID_LocalAppData, flag|windows.KF_FLAG_DONT_VERIFY) - if p != "" { - return filepath.Join(p, "cache", "docker-compose"), nil - } - } - - appData, ok := os.LookupEnv("LOCALAPPDATA") - if ok { - return filepath.Join(appData, "cache", "docker-compose"), nil - } - - home, err := os.UserHomeDir() - if err != nil { - return "", err - } - return filepath.Join(home, "AppData", "Local", "cache", "docker-compose"), nil -} diff --git a/pkg/remote/git.go b/pkg/remote/git.go deleted file mode 100644 index 689f08dd903..00000000000 --- a/pkg/remote/git.go +++ /dev/null @@ -1,267 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package remote - -import ( - "bufio" - "bytes" - "context" - "fmt" - "os" - "os/exec" - "path/filepath" - "regexp" - "strconv" - "strings" - - "github.com/compose-spec/compose-go/v2/cli" - "github.com/compose-spec/compose-go/v2/loader" - "github.com/compose-spec/compose-go/v2/types" - "github.com/docker/cli/cli/command" - gitutil "github.com/moby/buildkit/frontend/dockerfile/dfgitutil" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/pkg/api" -) - -const GIT_REMOTE_ENABLED = "COMPOSE_EXPERIMENTAL_GIT_REMOTE" - -func gitRemoteLoaderEnabled() (bool, error) { - if v := os.Getenv(GIT_REMOTE_ENABLED); v != "" { - enabled, err := strconv.ParseBool(v) - if err != nil { - return false, fmt.Errorf("COMPOSE_EXPERIMENTAL_GIT_REMOTE environment variable expects boolean value: %w", err) - } - return enabled, err - } - return true, nil -} - -func NewGitRemoteLoader(dockerCli command.Cli, offline bool) loader.ResourceLoader { - return gitRemoteLoader{ - dockerCli: dockerCli, - offline: offline, - known: map[string]string{}, - } -} - -type gitRemoteLoader struct { - dockerCli command.Cli - offline bool - known map[string]string -} - -func (g gitRemoteLoader) Accept(path string) bool { - _, _, err := gitutil.ParseGitRef(path) - return err == nil -} - -var commitSHA = regexp.MustCompile(`^[a-f0-9]{40}$`) - -func (g gitRemoteLoader) Load(ctx context.Context, path string) (string, error) { - enabled, err := gitRemoteLoaderEnabled() - if err != nil { - return "", err - } - if !enabled { - return "", fmt.Errorf("git remote resource is disabled by %q", GIT_REMOTE_ENABLED) - } - - ref, _, err := gitutil.ParseGitRef(path) - if err != nil { - return "", err - } - - local, ok := g.known[path] - if !ok { - if ref.Ref == "" { - ref.Ref = "HEAD" // default branch - } - - err = g.resolveGitRef(ctx, path, ref) - if err != nil { - return "", err - } - - cache, err := cacheDir() - if err != nil { - return "", fmt.Errorf("initializing remote resource cache: %w", err) - } - - local = filepath.Join(cache, ref.Ref) - if _, err := os.Stat(local); os.IsNotExist(err) { - if g.offline { - return "", nil - } - err = g.checkout(ctx, local, ref) - if err != nil { - return "", err - } - } - g.known[path] = local - } - if ref.SubDir != "" { - if err := validateGitSubDir(local, ref.SubDir); err != nil { - return "", err - } - local = filepath.Join(local, ref.SubDir) - } - stat, err := os.Stat(local) - if err != nil { - return "", err - } - if stat.IsDir() { - local, err = findFile(cli.DefaultFileNames, local) - } - return local, err -} - -func (g gitRemoteLoader) Dir(path string) string { - return g.known[path] -} - -// validateGitSubDir ensures a subdirectory path is contained within the base directory -// and doesn't escape via path traversal. Unlike validatePathInBase for OCI artifacts, -// this allows nested directories but prevents traversal outside the base. -func validateGitSubDir(base, subDir string) error { - cleanSubDir := filepath.Clean(subDir) - - if filepath.IsAbs(cleanSubDir) { - return fmt.Errorf("git subdirectory must be relative, got: %s", subDir) - } - - if cleanSubDir == ".." || strings.HasPrefix(cleanSubDir, "../") || strings.HasPrefix(cleanSubDir, "..\\") { - return fmt.Errorf("git subdirectory path traversal detected: %s", subDir) - } - - if len(cleanSubDir) >= 2 && cleanSubDir[1] == ':' { - return fmt.Errorf("git subdirectory must be relative, got: %s", subDir) - } - - targetPath := filepath.Join(base, cleanSubDir) - cleanBase := filepath.Clean(base) - cleanTarget := filepath.Clean(targetPath) - - // Ensure the target starts with the base path - relPath, err := filepath.Rel(cleanBase, cleanTarget) - if err != nil { - return fmt.Errorf("invalid git subdirectory path: %w", err) - } - - if relPath == ".." || strings.HasPrefix(relPath, "../") || strings.HasPrefix(relPath, "..\\") { - return fmt.Errorf("git subdirectory escapes base directory: %s", subDir) - } - - return nil -} - -func (g gitRemoteLoader) resolveGitRef(ctx context.Context, path string, ref *gitutil.GitRef) error { - if !commitSHA.MatchString(ref.Ref) { - cmd := exec.CommandContext(ctx, "git", "ls-remote", "--exit-code", ref.Remote, ref.Ref) - cmd.Env = g.gitCommandEnv() - out, err := cmd.CombinedOutput() - if err != nil { - if cmd.ProcessState.ExitCode() == 2 { - return fmt.Errorf("repository does not contain ref %s, output: %q: %w", path, string(out), err) - } - return fmt.Errorf("failed to access repository at %s:\n %s", ref.Remote, out) - } - if len(out) < 40 { - return fmt.Errorf("unexpected git command output: %q", string(out)) - } - sha := string(out[:40]) - if !commitSHA.MatchString(sha) { - return fmt.Errorf("invalid commit sha %q", sha) - } - ref.Ref = sha - } - return nil -} - -func (g gitRemoteLoader) checkout(ctx context.Context, path string, ref *gitutil.GitRef) error { - err := os.MkdirAll(path, 0o700) - if err != nil { - return err - } - err = exec.CommandContext(ctx, "git", "init", path).Run() - if err != nil { - return err - } - - cmd := exec.CommandContext(ctx, "git", "remote", "add", "origin", ref.Remote) - cmd.Dir = path - err = cmd.Run() - if err != nil { - return err - } - - cmd = exec.CommandContext(ctx, "git", "fetch", "--depth=1", "origin", ref.Ref) - cmd.Env = g.gitCommandEnv() - cmd.Dir = path - - err = g.run(cmd) - if err != nil { - return err - } - - cmd = exec.CommandContext(ctx, "git", "checkout", ref.Ref) - cmd.Dir = path - err = cmd.Run() - if err != nil { - return err - } - return nil -} - -func (g gitRemoteLoader) run(cmd *exec.Cmd) error { - if logrus.IsLevelEnabled(logrus.DebugLevel) { - output, err := cmd.CombinedOutput() - scanner := bufio.NewScanner(bytes.NewBuffer(output)) - for scanner.Scan() { - line := scanner.Text() - logrus.Debug(line) - } - return err - } - return cmd.Run() -} - -func (g gitRemoteLoader) gitCommandEnv() []string { - env := types.NewMapping(os.Environ()) - if env["GIT_TERMINAL_PROMPT"] == "" { - // Disable prompting for passwords by Git until user explicitly asks for it. - env["GIT_TERMINAL_PROMPT"] = "0" - } - if env["GIT_SSH"] == "" && env["GIT_SSH_COMMAND"] == "" { - // Disable any ssh connection pooling by Git and do not attempt to prompt the user. - env["GIT_SSH_COMMAND"] = "ssh -o ControlMaster=no -o BatchMode=yes" - } - v := env.Values() - return v -} - -func findFile(names []string, pwd string) (string, error) { - for _, n := range names { - f := filepath.Join(pwd, n) - if fi, err := os.Stat(f); err == nil && !fi.IsDir() { - return f, nil - } - } - return "", api.ErrNotFound -} - -var _ loader.ResourceLoader = gitRemoteLoader{} diff --git a/pkg/remote/git_test.go b/pkg/remote/git_test.go deleted file mode 100644 index f78bcc25dbb..00000000000 --- a/pkg/remote/git_test.go +++ /dev/null @@ -1,175 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package remote - -import ( - "testing" - - "gotest.tools/v3/assert" -) - -func TestValidateGitSubDir(t *testing.T) { - base := "/tmp/cache/compose/abc123def456" - - tests := []struct { - name string - subDir string - wantErr bool - }{ - { - name: "valid simple directory", - subDir: "examples", - wantErr: false, - }, - { - name: "valid nested directory", - subDir: "examples/nginx", - wantErr: false, - }, - { - name: "valid deeply nested directory", - subDir: "examples/web/frontend/config", - wantErr: false, - }, - { - name: "valid current directory", - subDir: ".", - wantErr: false, - }, - { - name: "valid directory with redundant separators", - subDir: "examples//nginx", - wantErr: false, - }, - { - name: "valid directory with dots in name", - subDir: "examples/nginx.conf.d", - wantErr: false, - }, - { - name: "path traversal - parent directory", - subDir: "..", - wantErr: true, - }, - { - name: "path traversal - multiple parent directories", - subDir: "../../../etc/passwd", - wantErr: true, - }, - { - name: "path traversal - deeply nested escape", - subDir: "../../../../../../../tmp/pwned", - wantErr: true, - }, - { - name: "path traversal - mixed with valid path", - subDir: "examples/../../etc/passwd", - wantErr: true, - }, - { - name: "path traversal - at the end", - subDir: "examples/..", - wantErr: false, // This resolves to "." which is the current directory, safe - }, - { - name: "path traversal - in the middle", - subDir: "examples/../../../etc/passwd", - wantErr: true, - }, - { - name: "path traversal - windows style", - subDir: "..\\..\\..\\windows\\system32", - wantErr: true, - }, - { - name: "absolute unix path", - subDir: "/etc/passwd", - wantErr: true, - }, - { - name: "absolute windows path", - subDir: "C:\\windows\\system32\\config\\sam", - wantErr: true, - }, - { - name: "absolute path with home directory", - subDir: "/home/user/.ssh/id_rsa", - wantErr: true, - }, - { - name: "normalized path that would escape", - subDir: "./../../etc/passwd", - wantErr: true, - }, - { - name: "directory name with three dots", - subDir: ".../config", - wantErr: false, - }, - { - name: "directory name with four dots", - subDir: "..../config", - wantErr: false, - }, - { - name: "directory name with five dots", - subDir: "...../etc/passwd", - wantErr: false, // ".....'' is a valid directory name, not path traversal - }, - { - name: "directory name starting with two dots and letter", - subDir: "..foo/bar", - wantErr: false, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - err := validateGitSubDir(base, tt.subDir) - if (err != nil) != tt.wantErr { - t.Errorf("validateGitSubDir(%q, %q) error = %v, wantErr %v", - base, tt.subDir, err, tt.wantErr) - } - }) - } -} - -// TestValidateGitSubDirSecurityScenarios tests specific security scenarios -func TestValidateGitSubDirSecurityScenarios(t *testing.T) { - base := "/var/cache/docker-compose/git/1234567890abcdef" - - // Test the exact vulnerability scenario from the issue - t.Run("CVE scenario - /tmp traversal", func(t *testing.T) { - maliciousPath := "../../../../../../../tmp/pwned" - err := validateGitSubDir(base, maliciousPath) - assert.ErrorContains(t, err, "path traversal") - }) - - // Test variations of the attack - t.Run("CVE scenario - /etc traversal", func(t *testing.T) { - maliciousPath := "../../../../../../../../etc/passwd" - err := validateGitSubDir(base, maliciousPath) - assert.ErrorContains(t, err, "path traversal") - }) - - // Test that legitimate nested paths still work - t.Run("legitimate nested path", func(t *testing.T) { - validPath := "examples/docker-compose/nginx/config" - err := validateGitSubDir(base, validPath) - assert.NilError(t, err) - }) -} diff --git a/pkg/remote/oci.go b/pkg/remote/oci.go deleted file mode 100644 index e64570d2ab0..00000000000 --- a/pkg/remote/oci.go +++ /dev/null @@ -1,262 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package remote - -import ( - "context" - "encoding/json" - "fmt" - "os" - "path/filepath" - "strconv" - "strings" - - "github.com/compose-spec/compose-go/v2/loader" - "github.com/containerd/containerd/v2/core/images" - "github.com/containerd/containerd/v2/core/remotes" - "github.com/distribution/reference" - "github.com/docker/cli/cli/command" - spec "github.com/opencontainers/image-spec/specs-go/v1" - - "github.com/docker/compose/v5/internal/oci" - "github.com/docker/compose/v5/pkg/api" -) - -const ( - OCI_REMOTE_ENABLED = "COMPOSE_EXPERIMENTAL_OCI_REMOTE" - OciPrefix = "oci://" -) - -// validatePathInBase ensures a file path is contained within the base directory, -// as OCI artifacts resources must all live within the same folder. -func validatePathInBase(base, unsafePath string) error { - // Reject paths with path separators regardless of OS - if strings.ContainsAny(unsafePath, "\\/") { - return fmt.Errorf("invalid OCI artifact") - } - - // Join the base with the untrusted path - targetPath := filepath.Join(base, unsafePath) - - // Get the directory of the target path - targetDir := filepath.Dir(targetPath) - - // Clean both paths to resolve any .. or . components - cleanBase := filepath.Clean(base) - cleanTargetDir := filepath.Clean(targetDir) - - // Check if the target directory is the same as base directory - if cleanTargetDir != cleanBase { - return fmt.Errorf("invalid OCI artifact") - } - - return nil -} - -func ociRemoteLoaderEnabled() (bool, error) { - if v := os.Getenv(OCI_REMOTE_ENABLED); v != "" { - enabled, err := strconv.ParseBool(v) - if err != nil { - return false, fmt.Errorf("COMPOSE_EXPERIMENTAL_OCI_REMOTE environment variable expects boolean value: %w", err) - } - return enabled, err - } - return true, nil -} - -func NewOCIRemoteLoader(dockerCli command.Cli, offline bool, options api.OCIOptions) loader.ResourceLoader { - return ociRemoteLoader{ - dockerCli: dockerCli, - offline: offline, - known: map[string]string{}, - insecureRegistries: options.InsecureRegistries, - } -} - -type ociRemoteLoader struct { - dockerCli command.Cli - offline bool - known map[string]string - insecureRegistries []string -} - -func (g ociRemoteLoader) Accept(path string) bool { - return strings.HasPrefix(path, OciPrefix) -} - -//nolint:gocyclo -func (g ociRemoteLoader) Load(ctx context.Context, path string) (string, error) { - enabled, err := ociRemoteLoaderEnabled() - if err != nil { - return "", err - } - if !enabled { - return "", fmt.Errorf("OCI remote resource is disabled by %q", OCI_REMOTE_ENABLED) - } - - if g.offline { - return "", nil - } - - local, ok := g.known[path] - if !ok { - ref, err := reference.ParseDockerRef(path[len(OciPrefix):]) - if err != nil { - return "", err - } - - resolver := oci.NewResolver(g.dockerCli.ConfigFile(), g.insecureRegistries...) - - descriptor, content, err := oci.Get(ctx, resolver, ref) - if err != nil { - return "", fmt.Errorf("failed to pull OCI resource %q: %w", ref, err) - } - - cache, err := cacheDir() - if err != nil { - return "", fmt.Errorf("initializing remote resource cache: %w", err) - } - - local = filepath.Join(cache, descriptor.Digest.Hex()) - if _, err = os.Stat(local); os.IsNotExist(err) { - - // a Compose application bundle is published as image index - if images.IsIndexType(descriptor.MediaType) { - var index spec.Index - err = json.Unmarshal(content, &index) - if err != nil { - return "", err - } - found := false - for _, manifest := range index.Manifests { - if manifest.ArtifactType != oci.ComposeProjectArtifactType { - continue - } - found = true - digested, err := reference.WithDigest(ref, manifest.Digest) - if err != nil { - return "", err - } - descriptor, content, err = oci.Get(ctx, resolver, digested) - if err != nil { - return "", fmt.Errorf("failed to pull OCI resource %q: %w", ref, err) - } - } - if !found { - return "", fmt.Errorf("OCI index %s doesn't refer to compose artifacts", ref) - } - } - - var manifest spec.Manifest - err = json.Unmarshal(content, &manifest) - if err != nil { - return "", err - } - - err = g.pullComposeFiles(ctx, local, manifest, ref, resolver) - if err != nil { - // we need to clean up the directory to be sure we won't let empty files present - _ = os.RemoveAll(local) - return "", err - } - } - g.known[path] = local - } - return filepath.Join(local, "compose.yaml"), nil -} - -func (g ociRemoteLoader) Dir(path string) string { - return g.known[path] -} - -func (g ociRemoteLoader) pullComposeFiles(ctx context.Context, local string, manifest spec.Manifest, ref reference.Named, resolver remotes.Resolver) error { - err := os.MkdirAll(local, 0o700) - if err != nil { - return err - } - if (manifest.ArtifactType != "" && manifest.ArtifactType != oci.ComposeProjectArtifactType) || - (manifest.ArtifactType == "" && manifest.Config.MediaType != oci.ComposeEmptyConfigMediaType) { - return fmt.Errorf("%s is not a compose project OCI artifact, but %s", ref.String(), manifest.ArtifactType) - } - - for i, layer := range manifest.Layers { - digested, err := reference.WithDigest(ref, layer.Digest) - if err != nil { - return err - } - - _, content, err := oci.Get(ctx, resolver, digested) - if err != nil { - return err - } - - switch layer.MediaType { - case oci.ComposeYAMLMediaType: - if err := writeComposeFile(layer, i, local, content); err != nil { - return err - } - case oci.ComposeEnvFileMediaType: - if err := writeEnvFile(layer, local, content); err != nil { - return err - } - case oci.ComposeEmptyConfigMediaType: - } - } - return nil -} - -func writeComposeFile(layer spec.Descriptor, i int, local string, content []byte) error { - file := "compose.yaml" - if _, ok := layer.Annotations["com.docker.compose.extends"]; ok { - file = layer.Annotations["com.docker.compose.file"] - if err := validatePathInBase(local, file); err != nil { - return err - } - } - f, err := os.OpenFile(filepath.Join(local, file), os.O_RDWR|os.O_CREATE|os.O_APPEND, 0o600) - if err != nil { - return err - } - defer func() { _ = f.Close() }() - if _, ok := layer.Annotations["com.docker.compose.file"]; i > 0 && ok { - _, err := f.Write([]byte("\n---\n")) - if err != nil { - return err - } - } - _, err = f.Write(content) - return err -} - -func writeEnvFile(layer spec.Descriptor, local string, content []byte) error { - envfilePath, ok := layer.Annotations["com.docker.compose.envfile"] - if !ok { - return fmt.Errorf("missing annotation com.docker.compose.envfile in layer %q", layer.Digest) - } - if err := validatePathInBase(local, envfilePath); err != nil { - return err - } - otherFile, err := os.Create(filepath.Join(local, envfilePath)) - if err != nil { - return err - } - defer func() { _ = otherFile.Close() }() - _, err = otherFile.Write(content) - return err -} - -var _ loader.ResourceLoader = ociRemoteLoader{} diff --git a/pkg/remote/oci_test.go b/pkg/remote/oci_test.go deleted file mode 100644 index 28a4dbd4847..00000000000 --- a/pkg/remote/oci_test.go +++ /dev/null @@ -1,139 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package remote - -import ( - "path/filepath" - "testing" - - spec "github.com/opencontainers/image-spec/specs-go/v1" - "gotest.tools/v3/assert" -) - -func TestValidatePathInBase(t *testing.T) { - base := "/tmp/cache/compose" - - tests := []struct { - name string - unsafePath string - wantErr bool - }{ - { - name: "valid simple filename", - unsafePath: "compose.yaml", - wantErr: false, - }, - { - name: "valid hashed filename", - unsafePath: "f8f9ede3d201ec37d5a5e3a77bbadab79af26035e53135e19571f50d541d390c.yaml", - wantErr: false, - }, - { - name: "valid env file", - unsafePath: ".env", - wantErr: false, - }, - { - name: "valid env file with suffix", - unsafePath: ".env.prod", - wantErr: false, - }, - { - name: "unix path traversal", - unsafePath: "../../../etc/passwd", - wantErr: true, - }, - { - name: "windows path traversal", - unsafePath: "..\\..\\..\\windows\\system32\\config\\sam", - wantErr: true, - }, - { - name: "subdirectory unix", - unsafePath: "config/base.yaml", - wantErr: true, - }, - { - name: "subdirectory windows", - unsafePath: "config\\base.yaml", - wantErr: true, - }, - { - name: "absolute unix path", - unsafePath: "/etc/passwd", - wantErr: true, - }, - { - name: "absolute windows path", - unsafePath: "C:\\windows\\system32\\config\\sam", - wantErr: true, - }, - { - name: "parent reference only", - unsafePath: "..", - wantErr: true, - }, - { - name: "mixed separators", - unsafePath: "config/sub\\file.yaml", - wantErr: true, - }, - { - name: "filename with spaces", - unsafePath: "my file.yaml", - wantErr: false, - }, - { - name: "filename with special chars", - unsafePath: "file-name_v1.2.3.yaml", - wantErr: false, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - err := validatePathInBase(base, tt.unsafePath) - if (err != nil) != tt.wantErr { - targetPath := filepath.Join(base, tt.unsafePath) - targetDir := filepath.Dir(targetPath) - t.Errorf("validatePathInBase(%q, %q) error = %v, wantErr %v\ntargetDir=%q base=%q", - base, tt.unsafePath, err, tt.wantErr, targetDir, base) - } - }) - } -} - -func TestWriteComposeFileWithExtendsPathTraversal(t *testing.T) { - tmpDir := t.TempDir() - - // Create a layer with com.docker.compose.extends=true and a path traversal attempt - layer := spec.Descriptor{ - MediaType: "application/vnd.docker.compose.file.v1+yaml", - Digest: "sha256:test123", - Size: 100, - Annotations: map[string]string{ - "com.docker.compose.extends": "true", - "com.docker.compose.file": "../other", - }, - } - - content := []byte("services:\n test:\n image: nginx\n") - - // writeComposeFile should return an error due to path traversal - err := writeComposeFile(layer, 0, tmpDir, content) - assert.Error(t, err, "invalid OCI artifact") -} diff --git a/pkg/utils/durationutils.go b/pkg/utils/durationutils.go deleted file mode 100644 index 98ab3c91615..00000000000 --- a/pkg/utils/durationutils.go +++ /dev/null @@ -1,27 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package utils - -import "time" - -func DurationSecondToInt(d *time.Duration) *int { - if d == nil { - return nil - } - timeout := int(d.Seconds()) - return &timeout -} diff --git a/pkg/utils/safebuffer.go b/pkg/utils/safebuffer.go deleted file mode 100644 index 0545c463ce6..00000000000 --- a/pkg/utils/safebuffer.go +++ /dev/null @@ -1,78 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package utils - -import ( - "bytes" - "strings" - "sync" - "testing" - "time" - - "github.com/stretchr/testify/require" -) - -// SafeBuffer is a thread safe version of bytes.Buffer -type SafeBuffer struct { - m sync.RWMutex - b bytes.Buffer -} - -// Read is a thread safe version of bytes.Buffer::Read -func (b *SafeBuffer) Read(p []byte) (n int, err error) { - b.m.RLock() - defer b.m.RUnlock() - return b.b.Read(p) -} - -// Write is a thread safe version of bytes.Buffer::Write -func (b *SafeBuffer) Write(p []byte) (n int, err error) { - b.m.Lock() - defer b.m.Unlock() - return b.b.Write(p) -} - -// String is a thread safe version of bytes.Buffer::String -func (b *SafeBuffer) String() string { - b.m.RLock() - defer b.m.RUnlock() - return b.b.String() -} - -// Bytes is a thread safe version of bytes.Buffer::Bytes -func (b *SafeBuffer) Bytes() []byte { - b.m.RLock() - defer b.m.RUnlock() - return b.b.Bytes() -} - -// RequireEventuallyContains is a thread safe eventual checker for the buffer content -func (b *SafeBuffer) RequireEventuallyContains(t testing.TB, v string) { - t.Helper() - var bufContents strings.Builder - require.Eventuallyf(t, func() bool { - b.m.Lock() - defer b.m.Unlock() - if _, err := b.b.WriteTo(&bufContents); err != nil { - require.FailNowf(t, "Failed to copy from buffer", - "Error: %v", err) - } - return strings.Contains(bufContents.String(), v) - }, 2*time.Second, 20*time.Millisecond, - "Buffer did not contain %q\n============\n%s\n============", - v, &bufContents) -} diff --git a/pkg/utils/set.go b/pkg/utils/set.go deleted file mode 100644 index 5a092d7c266..00000000000 --- a/pkg/utils/set.go +++ /dev/null @@ -1,93 +0,0 @@ -/* - - Copyright 2020 Docker Compose CLI authors - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package utils - -type Set[T comparable] map[T]struct{} - -func NewSet[T comparable](v ...T) Set[T] { - if len(v) == 0 { - return make(Set[T]) - } - - out := make(Set[T], len(v)) - for i := range v { - out.Add(v[i]) - } - return out -} - -func (s Set[T]) Has(v T) bool { - _, ok := s[v] - return ok -} - -func (s Set[T]) Add(v T) { - s[v] = struct{}{} -} - -func (s Set[T]) AddAll(v ...T) { - for _, e := range v { - s[e] = struct{}{} - } -} - -func (s Set[T]) Remove(v T) bool { - _, ok := s[v] - if ok { - delete(s, v) - } - return ok -} - -func (s Set[T]) Clear() { - for v := range s { - delete(s, v) - } -} - -func (s Set[T]) Elements() []T { - elements := make([]T, 0, len(s)) - for v := range s { - elements = append(elements, v) - } - return elements -} - -func (s Set[T]) RemoveAll(elements ...T) { - for _, e := range elements { - s.Remove(e) - } -} - -func (s Set[T]) Diff(other Set[T]) Set[T] { - out := make(Set[T]) - for k := range s { - if _, ok := other[k]; !ok { - out[k] = struct{}{} - } - } - return out -} - -func (s Set[T]) Union(other Set[T]) Set[T] { - out := make(Set[T]) - for k := range s { - out[k] = struct{}{} - } - for k := range other { - out[k] = struct{}{} - } - return out -} diff --git a/pkg/utils/set_test.go b/pkg/utils/set_test.go deleted file mode 100644 index 5bdd6cca3f9..00000000000 --- a/pkg/utils/set_test.go +++ /dev/null @@ -1,41 +0,0 @@ -/* - Copyright 2022 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package utils - -import ( - "testing" - - "github.com/stretchr/testify/require" -) - -func TestSet_Has(t *testing.T) { - x := NewSet[string]("value") - require.True(t, x.Has("value")) - require.False(t, x.Has("VALUE")) -} - -func TestSet_Diff(t *testing.T) { - a := NewSet[int](1, 2) - b := NewSet[int](2, 3) - require.ElementsMatch(t, []int{1}, a.Diff(b).Elements()) - require.ElementsMatch(t, []int{3}, b.Diff(a).Elements()) -} - -func TestSet_Union(t *testing.T) { - a := NewSet[int](1, 2) - b := NewSet[int](2, 3) - require.ElementsMatch(t, []int{1, 2, 3}, a.Union(b).Elements()) - require.ElementsMatch(t, []int{1, 2, 3}, b.Union(a).Elements()) -} diff --git a/pkg/utils/stringutils.go b/pkg/utils/stringutils.go deleted file mode 100644 index 7135e91781c..00000000000 --- a/pkg/utils/stringutils.go +++ /dev/null @@ -1,32 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package utils - -import ( - "strconv" - "strings" -) - -// StringToBool converts a string to a boolean ignoring errors -func StringToBool(s string) bool { - s = strings.ToLower(strings.TrimSpace(s)) - if s == "y" { - return true - } - b, _ := strconv.ParseBool(s) - return b -} diff --git a/pkg/utils/writer.go b/pkg/utils/writer.go deleted file mode 100644 index 1b4c8ca14a9..00000000000 --- a/pkg/utils/writer.go +++ /dev/null @@ -1,62 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package utils - -import ( - "bytes" - "io" -) - -// GetWriter creates an io.Writer that will actually split by line and format by LogConsumer -func GetWriter(consumer func(string)) io.WriteCloser { - return &splitWriter{ - buffer: bytes.Buffer{}, - consumer: consumer, - } -} - -type splitWriter struct { - buffer bytes.Buffer - consumer func(string) -} - -// Write implements io.Writer. joins all input, splits on the separator and yields each chunk -func (s *splitWriter) Write(b []byte) (int, error) { - n, err := s.buffer.Write(b) - if err != nil { - return n, err - } - for { - b = s.buffer.Bytes() - index := bytes.Index(b, []byte{'\n'}) - if index < 0 { - break - } - line := s.buffer.Next(index + 1) - s.consumer(string(line[:len(line)-1])) - } - return n, nil -} - -func (s *splitWriter) Close() error { - b := s.buffer.Bytes() - if len(b) == 0 { - return nil - } - s.consumer(string(b)) - return nil -} diff --git a/pkg/utils/writer_test.go b/pkg/utils/writer_test.go deleted file mode 100644 index bb7aed03752..00000000000 --- a/pkg/utils/writer_test.go +++ /dev/null @@ -1,39 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package utils - -import ( - "testing" - - "gotest.tools/v3/assert" -) - -//nolint:errcheck -func TestSplitWriter(t *testing.T) { - var lines []string - w := GetWriter(func(line string) { - lines = append(lines, line) - }) - w.Write([]byte("h")) - w.Write([]byte("e")) - w.Write([]byte("l")) - w.Write([]byte("l")) - w.Write([]byte("o")) - w.Write([]byte("\n")) - w.Write([]byte("world!\n")) - assert.DeepEqual(t, lines, []string{"hello", "world!"}) -} diff --git a/pkg/watch/debounce.go b/pkg/watch/debounce.go deleted file mode 100644 index d3cddb7e636..00000000000 --- a/pkg/watch/debounce.go +++ /dev/null @@ -1,74 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "context" - "time" - - "github.com/jonboulle/clockwork" - "github.com/sirupsen/logrus" - - "github.com/docker/compose/v5/pkg/utils" -) - -const QuietPeriod = 500 * time.Millisecond - -// BatchDebounceEvents groups identical file events within a sliding time window and writes the results to the returned -// channel. -// -// The returned channel is closed when the debouncer is stopped via context cancellation or by closing the input channel. -func BatchDebounceEvents(ctx context.Context, clock clockwork.Clock, input <-chan FileEvent) <-chan []FileEvent { - out := make(chan []FileEvent) - go func() { - defer close(out) - seen := utils.Set[FileEvent]{} - flushEvents := func() { - if len(seen) == 0 { - return - } - logrus.Debugf("flush: %d events %s", len(seen), seen) - - events := make([]FileEvent, 0, len(seen)) - for e := range seen { - events = append(events, e) - } - out <- events - seen = utils.Set[FileEvent]{} - } - - t := clock.NewTicker(QuietPeriod) - defer t.Stop() - for { - select { - case <-ctx.Done(): - return - case <-t.Chan(): - flushEvents() - case e, ok := <-input: - if !ok { - // input channel was closed - flushEvents() - return - } - if _, ok := seen[e]; !ok { - seen.Add(e) - } - t.Reset(QuietPeriod) - } - } - }() - return out -} diff --git a/pkg/watch/debounce_test.go b/pkg/watch/debounce_test.go deleted file mode 100644 index 39029f845c2..00000000000 --- a/pkg/watch/debounce_test.go +++ /dev/null @@ -1,66 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "context" - "slices" - "testing" - "time" - - "github.com/jonboulle/clockwork" - "gotest.tools/v3/assert" -) - -func Test_BatchDebounceEvents(t *testing.T) { - ch := make(chan FileEvent) - clock := clockwork.NewFakeClock() - ctx, stop := context.WithCancel(t.Context()) - t.Cleanup(stop) - - eventBatchCh := BatchDebounceEvents(ctx, clock, ch) - for i := 0; i < 100; i++ { - path := "/a" - if i%2 == 0 { - path = "/b" - } - - ch <- FileEvent(path) - } - // we sent 100 events + the debouncer - err := clock.BlockUntilContext(ctx, 101) - assert.NilError(t, err) - clock.Advance(QuietPeriod) - select { - case batch := <-eventBatchCh: - slices.Sort(batch) - assert.Equal(t, len(batch), 2) - assert.Equal(t, batch[0], FileEvent("/a")) - assert.Equal(t, batch[1], FileEvent("/b")) - case <-time.After(50 * time.Millisecond): - t.Fatal("timed out waiting for events") - } - err = clock.BlockUntilContext(ctx, 1) - assert.NilError(t, err) - clock.Advance(QuietPeriod) - - // there should only be a single batch - select { - case batch := <-eventBatchCh: - t.Fatalf("unexpected events: %v", batch) - case <-time.After(50 * time.Millisecond): - // channel is empty - } -} diff --git a/pkg/watch/dockerignore.go b/pkg/watch/dockerignore.go deleted file mode 100644 index ce7d57fbfee..00000000000 --- a/pkg/watch/dockerignore.go +++ /dev/null @@ -1,171 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "fmt" - "io" - "os" - "path/filepath" - "slices" - "strings" - - "github.com/compose-spec/compose-go/v2/types" - "github.com/moby/patternmatcher" - "github.com/moby/patternmatcher/ignorefile" - - "github.com/docker/compose/v5/internal/paths" -) - -type dockerPathMatcher struct { - repoRoot string - matcher *patternmatcher.PatternMatcher -} - -func (i dockerPathMatcher) Matches(f string) (bool, error) { - if !filepath.IsAbs(f) { - f = filepath.Join(i.repoRoot, f) - } - return i.matcher.MatchesOrParentMatches(f) -} - -func (i dockerPathMatcher) MatchesEntireDir(f string) (bool, error) { - matches, err := i.Matches(f) - if !matches || err != nil { - return matches, err - } - - // We match the dir, but we might exclude files underneath it. - if i.matcher.Exclusions() { - for _, pattern := range i.matcher.Patterns() { - if !pattern.Exclusion() { - continue - } - if paths.IsChild(f, pattern.String()) { - // Found an exclusion match -- we don't match this whole dir - return false, nil - } - } - return true, nil - } - return true, nil -} - -func LoadDockerIgnore(build *types.BuildConfig) (PathMatcher, error) { - if build == nil { - return EmptyMatcher{}, nil - } - repoRoot := build.Context - absRoot, err := filepath.Abs(repoRoot) - if err != nil { - return nil, err - } - - // first try Dockerfile-specific ignore-file - f, err := os.Open(filepath.Join(repoRoot, build.Dockerfile+".dockerignore")) - if os.IsNotExist(err) { - // defaults to a global .dockerignore - f, err = os.Open(filepath.Join(repoRoot, ".dockerignore")) - if os.IsNotExist(err) { - return NewDockerPatternMatcher(repoRoot, nil) - } - } - if err != nil { - return nil, err - } - defer func() { _ = f.Close() }() - - patterns, err := readDockerignorePatterns(f) - if err != nil { - return nil, err - } - - return NewDockerPatternMatcher(absRoot, patterns) -} - -// Make all the patterns use absolute paths. -func absPatterns(absRoot string, patterns []string) []string { - absPatterns := make([]string, 0, len(patterns)) - for _, p := range patterns { - // The pattern parsing here is loosely adapted from fileutils' NewPatternMatcher - p = strings.TrimSpace(p) - if p == "" { - continue - } - p = filepath.Clean(p) - - pPath := p - isExclusion := false - if p[0] == '!' { - pPath = p[1:] - isExclusion = true - } - - if !filepath.IsAbs(pPath) { - pPath = filepath.Join(absRoot, pPath) - } - absPattern := pPath - if isExclusion { - absPattern = fmt.Sprintf("!%s", pPath) - } - absPatterns = append(absPatterns, absPattern) - } - return absPatterns -} - -func NewDockerPatternMatcher(repoRoot string, patterns []string) (*dockerPathMatcher, error) { - absRoot, err := filepath.Abs(repoRoot) - if err != nil { - return nil, err - } - - // Check if "*" is present in patterns - hasAllPattern := slices.Contains(patterns, "*") - if hasAllPattern { - // Remove all non-exclusion patterns (those that don't start with '!') - patterns = slices.DeleteFunc(patterns, func(p string) bool { - return p != "" && p[0] != '!' // Only keep exclusion patterns - }) - } - - pm, err := patternmatcher.New(absPatterns(absRoot, patterns)) - if err != nil { - return nil, err - } - - return &dockerPathMatcher{ - repoRoot: absRoot, - matcher: pm, - }, nil -} - -func readDockerignorePatterns(r io.Reader) ([]string, error) { - patterns, err := ignorefile.ReadAll(r) - if err != nil { - return nil, fmt.Errorf("error reading .dockerignore: %w", err) - } - return patterns, nil -} - -func DockerIgnoreTesterFromContents(repoRoot string, contents string) (*dockerPathMatcher, error) { - patterns, err := ignorefile.ReadAll(strings.NewReader(contents)) - if err != nil { - return nil, fmt.Errorf("error reading .dockerignore: %w", err) - } - - return NewDockerPatternMatcher(repoRoot, patterns) -} diff --git a/pkg/watch/dockerignore_test.go b/pkg/watch/dockerignore_test.go deleted file mode 100644 index 6e88a857476..00000000000 --- a/pkg/watch/dockerignore_test.go +++ /dev/null @@ -1,108 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "testing" -) - -func TestNewDockerPatternMatcher(t *testing.T) { - tests := []struct { - name string - repoRoot string - patterns []string - expectedErr bool - expectedRoot string - expectedPat []string - }{ - { - name: "Basic patterns without wildcard", - repoRoot: "/repo", - patterns: []string{"dir1/", "file.txt"}, - expectedErr: false, - expectedRoot: "/repo", - expectedPat: []string{"/repo/dir1", "/repo/file.txt"}, - }, - { - name: "Patterns with exclusion", - repoRoot: "/repo", - patterns: []string{"dir1/", "!file.txt"}, - expectedErr: false, - expectedRoot: "/repo", - expectedPat: []string{"/repo/dir1", "!/repo/file.txt"}, - }, - { - name: "Wildcard with exclusion", - repoRoot: "/repo", - patterns: []string{"*", "!file.txt"}, - expectedErr: false, - expectedRoot: "/repo", - expectedPat: []string{"!/repo/file.txt"}, - }, - { - name: "No patterns", - repoRoot: "/repo", - patterns: []string{}, - expectedErr: false, - expectedRoot: "/repo", - expectedPat: nil, - }, - { - name: "Only exclusion pattern", - repoRoot: "/repo", - patterns: []string{"!file.txt"}, - expectedErr: false, - expectedRoot: "/repo", - expectedPat: []string{"!/repo/file.txt"}, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - // Call the function with the test data - matcher, err := NewDockerPatternMatcher(tt.repoRoot, tt.patterns) - - // Check if we expect an error - if (err != nil) != tt.expectedErr { - t.Fatalf("expected error: %v, got: %v", tt.expectedErr, err) - } - - // If no error is expected, check the output - if !tt.expectedErr { - if matcher.repoRoot != tt.expectedRoot { - t.Errorf("expected root: %v, got: %v", tt.expectedRoot, matcher.repoRoot) - } - - // Compare patterns - actualPatterns := matcher.matcher.Patterns() - if len(actualPatterns) != len(tt.expectedPat) { - t.Errorf("expected patterns length: %v, got: %v", len(tt.expectedPat), len(actualPatterns)) - } - - for i, expectedPat := range tt.expectedPat { - actualPatternStr := actualPatterns[i].String() - if actualPatterns[i].Exclusion() { - actualPatternStr = "!" + actualPatternStr - } - if actualPatternStr != expectedPat { - t.Errorf("expected pattern: %v, got: %v", expectedPat, actualPatterns[i]) - } - } - } - }) - } -} diff --git a/pkg/watch/ephemeral.go b/pkg/watch/ephemeral.go deleted file mode 100644 index 77589a9413d..00000000000 --- a/pkg/watch/ephemeral.go +++ /dev/null @@ -1,59 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -// EphemeralPathMatcher filters out spurious changes that we don't want to -// rebuild on, like IDE temp/lock files. -// -// This isn't an ideal solution. In an ideal world, the user would put -// everything to ignore in their tiltignore/dockerignore files. This is a -// stop-gap so they don't have a terrible experience if those files aren't -// there or aren't in the right places. -// -// NOTE: The underlying `patternmatcher` is NOT always Goroutine-safe, so -// this is not a singleton; we create an instance for each watcher currently. -func EphemeralPathMatcher() PathMatcher { - golandPatterns := []string{"**/*___jb_old___", "**/*___jb_tmp___", "**/.idea/**"} - emacsPatterns := []string{"**/.#*", "**/#*#"} - // if .swp is taken (presumably because multiple vims are running in that dir), - // vim will go with .swo, .swn, etc, and then even .svz, .svy! - // https://github.com/vim/vim/blob/ea781459b9617aa47335061fcc78403495260315/src/memline.c#L5076 - // ignoring .sw? seems dangerous, since things like .swf or .swi exist, but ignoring the first few - // seems safe and should catch most cases - vimPatterns := []string{"**/4913", "**/*~", "**/.*.swp", "**/.*.swx", "**/.*.swo", "**/.*.swn"} - // kate (the default text editor for KDE) uses a file similar to Vim's .swp - // files, but it doesn't have the "incrementing" character problem mentioned - // above - katePatterns := []string{"**/.*.kate-swp"} - // go stdlib creates tmpfiles to determine umask for setting permissions - // during file creation; they are then immediately deleted - // https://github.com/golang/go/blob/0b5218cf4e3e5c17344ea113af346e8e0836f6c4/src/cmd/go/internal/work/exec.go#L1764 - goPatterns := []string{"**/*-go-tmp-umask"} - - var allPatterns []string - allPatterns = append(allPatterns, golandPatterns...) - allPatterns = append(allPatterns, emacsPatterns...) - allPatterns = append(allPatterns, vimPatterns...) - allPatterns = append(allPatterns, katePatterns...) - allPatterns = append(allPatterns, goPatterns...) - - matcher, err := NewDockerPatternMatcher("/", allPatterns) - if err != nil { - panic(err) - } - return matcher -} diff --git a/pkg/watch/ephemeral_test.go b/pkg/watch/ephemeral_test.go deleted file mode 100644 index 9f7b81819be..00000000000 --- a/pkg/watch/ephemeral_test.go +++ /dev/null @@ -1,48 +0,0 @@ -/* -Copyright 2023 Docker Compose CLI authors - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package watch_test - -import ( - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - - "github.com/docker/compose/v5/pkg/watch" -) - -func TestEphemeralPathMatcher(t *testing.T) { - ignored := []string{ - ".file.txt.swp", - "/path/file.txt~", - "/home/moby/proj/.idea/modules.xml", - ".#file.txt", - "#file.txt#", - "/dir/.file.txt.kate-swp", - "/go/app/1234-go-tmp-umask", - } - matcher := watch.EphemeralPathMatcher() - for _, p := range ignored { - ok, err := matcher.Matches(p) - require.NoErrorf(t, err, "Matching %s", p) - assert.Truef(t, ok, "Path %s should have matched", p) - } - - const includedPath = "normal.txt" - ok, err := matcher.Matches(includedPath) - require.NoErrorf(t, err, "Matching %s", includedPath) - assert.Falsef(t, ok, "Path %s should NOT have matched", includedPath) -} diff --git a/pkg/watch/notify.go b/pkg/watch/notify.go deleted file mode 100644 index d63f5caf28b..00000000000 --- a/pkg/watch/notify.go +++ /dev/null @@ -1,136 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "expvar" - "fmt" - "os" - "path/filepath" - "strconv" -) - -var numberOfWatches = expvar.NewInt("watch.naive.numberOfWatches") - -type FileEvent string - -func NewFileEvent(p string) FileEvent { - if !filepath.IsAbs(p) { - panic(fmt.Sprintf("NewFileEvent only accepts absolute paths. Actual: %s", p)) - } - return FileEvent(p) -} - -type Notify interface { - // Start watching the paths set at init time - Start() error - - // Stop watching and close all channels - Close() error - - // A channel to read off incoming file changes - Events() chan FileEvent - - // A channel to read off show-stopping errors - Errors() chan error -} - -// When we specify directories to watch, we often want to -// ignore some subset of the files under those directories. -// -// For example: -// - Watch /src/repo, but ignore /src/repo/.git -// - Watch /src/repo, but ignore everything in /src/repo/bazel-bin except /src/repo/bazel-bin/app-binary -// -// The PathMatcher interface helps us manage these ignores. -type PathMatcher interface { - Matches(file string) (bool, error) - - // If this matches the entire dir, we can often optimize filetree walks a bit. - MatchesEntireDir(file string) (bool, error) -} - -// AnyMatcher is a PathMatcher to match any path -type AnyMatcher struct{} - -func (AnyMatcher) Matches(f string) (bool, error) { return true, nil } -func (AnyMatcher) MatchesEntireDir(f string) (bool, error) { return true, nil } - -var _ PathMatcher = AnyMatcher{} - -// EmptyMatcher is a PathMatcher to match no path -type EmptyMatcher struct{} - -func (EmptyMatcher) Matches(f string) (bool, error) { return false, nil } -func (EmptyMatcher) MatchesEntireDir(f string) (bool, error) { return false, nil } - -var _ PathMatcher = EmptyMatcher{} - -func NewWatcher(paths []string) (Notify, error) { - return newWatcher(paths) -} - -const WindowsBufferSizeEnvVar = "COMPOSE_WATCH_WINDOWS_BUFFER_SIZE" - -const defaultBufferSize int = 65536 - -func DesiredWindowsBufferSize() int { - envVar := os.Getenv(WindowsBufferSizeEnvVar) - if envVar != "" { - size, err := strconv.Atoi(envVar) - if err == nil { - return size - } - } - return defaultBufferSize -} - -type CompositePathMatcher struct { - Matchers []PathMatcher -} - -func NewCompositeMatcher(matchers ...PathMatcher) PathMatcher { - if len(matchers) == 0 { - return EmptyMatcher{} - } - return CompositePathMatcher{Matchers: matchers} -} - -func (c CompositePathMatcher) Matches(f string) (bool, error) { - for _, t := range c.Matchers { - ret, err := t.Matches(f) - if err != nil { - return false, err - } - if ret { - return true, nil - } - } - return false, nil -} - -func (c CompositePathMatcher) MatchesEntireDir(f string) (bool, error) { - for _, t := range c.Matchers { - matches, err := t.MatchesEntireDir(f) - if matches || err != nil { - return matches, err - } - } - return false, nil -} - -var _ PathMatcher = CompositePathMatcher{} diff --git a/pkg/watch/notify_test.go b/pkg/watch/notify_test.go deleted file mode 100644 index be52ced3a3f..00000000000 --- a/pkg/watch/notify_test.go +++ /dev/null @@ -1,657 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "bytes" - "context" - "fmt" - "os" - "path/filepath" - "runtime" - "strings" - "testing" - "time" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -// Each implementation of the notify interface should have the same basic -// behavior. - -func TestWindowsBufferSize(t *testing.T) { - t.Run("empty value", func(t *testing.T) { - t.Setenv(WindowsBufferSizeEnvVar, "") - assert.Equal(t, defaultBufferSize, DesiredWindowsBufferSize()) - }) - - t.Run("invalid value", func(t *testing.T) { - t.Setenv(WindowsBufferSizeEnvVar, "a") - assert.Equal(t, defaultBufferSize, DesiredWindowsBufferSize()) - }) - - t.Run("valid value", func(t *testing.T) { - t.Setenv(WindowsBufferSizeEnvVar, "10") - assert.Equal(t, 10, DesiredWindowsBufferSize()) - }) -} - -func TestNoEvents(t *testing.T) { - f := newNotifyFixture(t) - f.assertEvents() -} - -func TestNoWatches(t *testing.T) { - f := newNotifyFixture(t) - f.paths = nil - f.rebuildWatcher() - f.assertEvents() -} - -func TestEventOrdering(t *testing.T) { - if runtime.GOOS == "windows" { - // https://qualapps.blogspot.com/2010/05/understanding-readdirectorychangesw_19.html - t.Skip("Windows doesn't make great guarantees about duplicate/out-of-order events") - return - } - f := newNotifyFixture(t) - - count := 8 - dirs := make([]string, count) - for i := range dirs { - dir := f.TempDir("watched") - dirs[i] = dir - f.watch(dir) - } - - f.fsync() - f.events = nil - - var expected []string - for i, dir := range dirs { - base := fmt.Sprintf("%d.txt", i) - p := filepath.Join(dir, base) - err := os.WriteFile(p, []byte(base), os.FileMode(0o777)) - if err != nil { - t.Fatal(err) - } - expected = append(expected, filepath.Join(dir, base)) - } - - f.assertEvents(expected...) -} - -// Simulate a git branch switch that creates a bunch -// of directories, creates files in them, then deletes -// them all quickly. Make sure there are no errors. -func TestGitBranchSwitch(t *testing.T) { - f := newNotifyFixture(t) - - count := 10 - dirs := make([]string, count) - for i := range dirs { - dir := f.TempDir("watched") - dirs[i] = dir - f.watch(dir) - } - - f.fsync() - f.events = nil - - // consume all the events in the background - ctx, cancel := context.WithCancel(t.Context()) - done := f.consumeEventsInBackground(ctx) - - for i, dir := range dirs { - for j := 0; j < count; j++ { - base := fmt.Sprintf("x/y/dir-%d/x.txt", j) - p := filepath.Join(dir, base) - f.WriteFile(p, "contents") - } - - if i != 0 { - err := os.RemoveAll(dir) - require.NoError(t, err) - } - } - - cancel() - err := <-done - if err != nil { - t.Fatal(err) - } - - f.fsync() - f.events = nil - - // Make sure the watch on the first dir still works. - dir := dirs[0] - path := filepath.Join(dir, "change") - - f.WriteFile(path, "hello\n") - f.fsync() - - f.assertEvents(path) - - // Make sure there are no errors in the out stream - assert.Empty(t, f.out.String()) -} - -func TestWatchesAreRecursive(t *testing.T) { - f := newNotifyFixture(t) - - root := f.TempDir("root") - - // add a sub directory - subPath := filepath.Join(root, "sub") - f.MkdirAll(subPath) - - // watch parent - f.watch(root) - - f.fsync() - f.events = nil - // change sub directory - changeFilePath := filepath.Join(subPath, "change") - f.WriteFile(changeFilePath, "change") - - f.assertEvents(changeFilePath) -} - -func TestNewDirectoriesAreRecursivelyWatched(t *testing.T) { - f := newNotifyFixture(t) - - root := f.TempDir("root") - - // watch parent - f.watch(root) - f.fsync() - f.events = nil - - // add a sub directory - subPath := filepath.Join(root, "sub") - f.MkdirAll(subPath) - - // change something inside sub directory - changeFilePath := filepath.Join(subPath, "change") - file, err := os.OpenFile(changeFilePath, os.O_RDONLY|os.O_CREATE, 0o666) - if err != nil { - t.Fatal(err) - } - _ = file.Close() - f.assertEvents(subPath, changeFilePath) -} - -func TestWatchNonExistentPath(t *testing.T) { - f := newNotifyFixture(t) - - root := f.TempDir("root") - path := filepath.Join(root, "change") - - f.watch(path) - f.fsync() - - d1 := "hello\ngo\n" - f.WriteFile(path, d1) - f.assertEvents(path) -} - -func TestWatchNonExistentPathDoesNotFireSiblingEvent(t *testing.T) { - f := newNotifyFixture(t) - - root := f.TempDir("root") - watchedFile := filepath.Join(root, "a.txt") - unwatchedSibling := filepath.Join(root, "b.txt") - - f.watch(watchedFile) - f.fsync() - - d1 := "hello\ngo\n" - f.WriteFile(unwatchedSibling, d1) - f.assertEvents() -} - -func TestRemove(t *testing.T) { - f := newNotifyFixture(t) - - root := f.TempDir("root") - path := filepath.Join(root, "change") - - d1 := "hello\ngo\n" - f.WriteFile(path, d1) - - f.watch(path) - f.fsync() - f.events = nil - err := os.Remove(path) - if err != nil { - t.Fatal(err) - } - f.assertEvents(path) -} - -func TestRemoveAndAddBack(t *testing.T) { - f := newNotifyFixture(t) - - path := filepath.Join(f.paths[0], "change") - - d1 := []byte("hello\ngo\n") - err := os.WriteFile(path, d1, 0o644) - if err != nil { - t.Fatal(err) - } - f.watch(path) - f.assertEvents(path) - - err = os.Remove(path) - if err != nil { - t.Fatal(err) - } - - f.assertEvents(path) - f.events = nil - - err = os.WriteFile(path, d1, 0o644) - if err != nil { - t.Fatal(err) - } - - f.assertEvents(path) -} - -func TestSingleFile(t *testing.T) { - f := newNotifyFixture(t) - - root := f.TempDir("root") - path := filepath.Join(root, "change") - - d1 := "hello\ngo\n" - f.WriteFile(path, d1) - - f.watch(path) - f.fsync() - - d2 := []byte("hello\nworld\n") - err := os.WriteFile(path, d2, 0o644) - if err != nil { - t.Fatal(err) - } - f.assertEvents(path) -} - -func TestWriteBrokenLink(t *testing.T) { - if runtime.GOOS == "windows" { - t.Skip("no user-space symlinks on windows") - } - f := newNotifyFixture(t) - - link := filepath.Join(f.paths[0], "brokenLink") - missingFile := filepath.Join(f.paths[0], "missingFile") - err := os.Symlink(missingFile, link) - if err != nil { - t.Fatal(err) - } - - f.assertEvents(link) -} - -func TestWriteGoodLink(t *testing.T) { - if runtime.GOOS == "windows" { - t.Skip("no user-space symlinks on windows") - } - f := newNotifyFixture(t) - - goodFile := filepath.Join(f.paths[0], "goodFile") - err := os.WriteFile(goodFile, []byte("hello"), 0o644) - if err != nil { - t.Fatal(err) - } - - link := filepath.Join(f.paths[0], "goodFileSymlink") - err = os.Symlink(goodFile, link) - if err != nil { - t.Fatal(err) - } - - f.assertEvents(goodFile, link) -} - -func TestWatchBrokenLink(t *testing.T) { - if runtime.GOOS == "windows" { - t.Skip("no user-space symlinks on windows") - } - f := newNotifyFixture(t) - - newRoot, err := NewDir(t.Name()) - if err != nil { - t.Fatal(err) - } - defer func() { - err := newRoot.TearDown() - if err != nil { - fmt.Printf("error tearing down temp dir: %v\n", err) - } - }() - - link := filepath.Join(newRoot.Path(), "brokenLink") - missingFile := filepath.Join(newRoot.Path(), "missingFile") - err = os.Symlink(missingFile, link) - if err != nil { - t.Fatal(err) - } - - f.watch(newRoot.Path()) - err = os.Remove(link) - require.NoError(t, err) - f.assertEvents(link) -} - -func TestMoveAndReplace(t *testing.T) { - f := newNotifyFixture(t) - - root := f.TempDir("root") - file := filepath.Join(root, "myfile") - f.WriteFile(file, "hello") - - f.watch(file) - tmpFile := filepath.Join(root, ".myfile.swp") - f.WriteFile(tmpFile, "world") - - err := os.Rename(tmpFile, file) - if err != nil { - t.Fatal(err) - } - - f.assertEvents(file) -} - -func TestWatchBothDirAndFile(t *testing.T) { - f := newNotifyFixture(t) - - dir := f.JoinPath("foo") - fileA := f.JoinPath("foo", "a") - fileB := f.JoinPath("foo", "b") - f.WriteFile(fileA, "a") - f.WriteFile(fileB, "b") - - f.watch(fileA) - f.watch(dir) - f.fsync() - f.events = nil - - f.WriteFile(fileB, "b-new") - f.assertEvents(fileB) -} - -func TestWatchNonexistentFileInNonexistentDirectoryCreatedSimultaneously(t *testing.T) { - f := newNotifyFixture(t) - - root := f.JoinPath("root") - err := os.Mkdir(root, 0o777) - if err != nil { - t.Fatal(err) - } - file := f.JoinPath("root", "parent", "a") - - f.watch(file) - f.fsync() - f.events = nil - f.WriteFile(file, "hello") - f.assertEvents(file) -} - -func TestWatchNonexistentDirectory(t *testing.T) { - f := newNotifyFixture(t) - - root := f.JoinPath("root") - err := os.Mkdir(root, 0o777) - if err != nil { - t.Fatal(err) - } - parent := f.JoinPath("parent") - file := f.JoinPath("parent", "a") - - f.watch(parent) - f.fsync() - f.events = nil - - err = os.Mkdir(parent, 0o777) - if err != nil { - t.Fatal(err) - } - - // for directories that were the root of an Add, we don't report creation, cf. watcher_darwin.go - f.assertEvents() - - f.events = nil - f.WriteFile(file, "hello") - - f.assertEvents(file) -} - -func TestWatchNonexistentFileInNonexistentDirectory(t *testing.T) { - f := newNotifyFixture(t) - - root := f.JoinPath("root") - err := os.Mkdir(root, 0o777) - if err != nil { - t.Fatal(err) - } - parent := f.JoinPath("parent") - file := f.JoinPath("parent", "a") - - f.watch(file) - f.assertEvents() - - err = os.Mkdir(parent, 0o777) - if err != nil { - t.Fatal(err) - } - - f.assertEvents() - f.WriteFile(file, "hello") - f.assertEvents(file) -} - -func TestWatchCountInnerFile(t *testing.T) { - f := newNotifyFixture(t) - - root := f.paths[0] - a := f.JoinPath(root, "a") - b := f.JoinPath(a, "b") - file := f.JoinPath(b, "bigFile") - f.WriteFile(file, "hello") - f.assertEvents(a, b, file) - - expectedWatches := 3 - if isRecursiveWatcher() { - expectedWatches = 1 - } - assert.Equal(t, expectedWatches, int(numberOfWatches.Value())) -} - -func isRecursiveWatcher() bool { - return runtime.GOOS == "darwin" || runtime.GOOS == "windows" -} - -type notifyFixture struct { - ctx context.Context - cancel func() - out *bytes.Buffer - *TempDirFixture - notify Notify - paths []string - events []FileEvent -} - -func newNotifyFixture(t *testing.T) *notifyFixture { - out := bytes.NewBuffer(nil) - ctx, cancel := context.WithCancel(t.Context()) - nf := ¬ifyFixture{ - ctx: ctx, - cancel: cancel, - TempDirFixture: NewTempDirFixture(t), - paths: []string{}, - out: out, - } - nf.watch(nf.TempDir("watched")) - t.Cleanup(nf.tearDown) - return nf -} - -func (f *notifyFixture) watch(path string) { - f.paths = append(f.paths, path) - f.rebuildWatcher() -} - -func (f *notifyFixture) rebuildWatcher() { - // sync any outstanding events and close the old watcher - if f.notify != nil { - f.fsync() - f.closeWatcher() - } - - // create a new watcher - notify, err := NewWatcher(f.paths) - if err != nil { - f.T().Fatal(err) - } - f.notify = notify - err = f.notify.Start() - if err != nil { - f.T().Fatal(err) - } -} - -func (f *notifyFixture) assertEvents(expected ...string) { - f.fsync() - if runtime.GOOS == "windows" { - // NOTE(nick): It's unclear to me why an extra fsync() helps - // here, but it makes the I/O way more predictable. - f.fsync() - } - - if len(f.events) != len(expected) { - f.T().Fatalf("Got %d events (expected %d): %v %v", len(f.events), len(expected), f.events, expected) - } - - for i, actual := range f.events { - e := FileEvent(expected[i]) - if actual != e { - f.T().Fatalf("Got event %v (expected %v)", actual, e) - } - } -} - -func (f *notifyFixture) consumeEventsInBackground(ctx context.Context) chan error { - done := make(chan error) - go func() { - for { - select { - case <-f.ctx.Done(): - close(done) - return - case <-ctx.Done(): - close(done) - return - case err := <-f.notify.Errors(): - done <- err - close(done) - return - case <-f.notify.Events(): - } - } - }() - return done -} - -func (f *notifyFixture) fsync() { - f.fsyncWithRetryCount(3) -} - -func (f *notifyFixture) fsyncWithRetryCount(retryCount int) { - if len(f.paths) == 0 { - return - } - - syncPathBase := fmt.Sprintf("sync-%d.txt", time.Now().UnixNano()) - syncPath := filepath.Join(f.paths[0], syncPathBase) - anySyncPath := filepath.Join(f.paths[0], "sync-") - timeout := time.After(250 * time.Second) - - f.WriteFile(syncPath, time.Now().String()) - -F: - for { - select { - case <-f.ctx.Done(): - return - case err := <-f.notify.Errors(): - f.T().Fatal(err) - - case event := <-f.notify.Events(): - if strings.Contains(string(event), syncPath) { - break F - } - if strings.Contains(string(event), anySyncPath) { - continue - } - - // Don't bother tracking duplicate changes to the same path - // for testing. - if len(f.events) > 0 && f.events[len(f.events)-1] == event { - continue - } - - f.events = append(f.events, event) - - case <-timeout: - if retryCount <= 0 { - f.T().Fatalf("fsync: timeout") - } else { - f.fsyncWithRetryCount(retryCount - 1) - } - return - } - } -} - -func (f *notifyFixture) closeWatcher() { - notify := f.notify - err := notify.Close() - if err != nil { - f.T().Fatal(err) - } - - // drain channels from watcher - go func() { - for range notify.Events() { - } - }() - - go func() { - for range notify.Errors() { - } - }() -} - -func (f *notifyFixture) tearDown() { - f.cancel() - f.closeWatcher() - numberOfWatches.Set(0) -} diff --git a/pkg/watch/paths.go b/pkg/watch/paths.go deleted file mode 100644 index c0c893cd995..00000000000 --- a/pkg/watch/paths.go +++ /dev/null @@ -1,41 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "fmt" - "os" - "path/filepath" -) - -func greatestExistingAncestor(path string) (string, error) { - if path == string(filepath.Separator) || - path == fmt.Sprintf("%s%s", filepath.VolumeName(path), string(filepath.Separator)) { - return "", fmt.Errorf("cannot watch root directory") - } - - _, err := os.Stat(path) - if err != nil && !os.IsNotExist(err) { - return "", fmt.Errorf("os.Stat(%q): %w", path, err) - } - - if os.IsNotExist(err) { - return greatestExistingAncestor(filepath.Dir(path)) - } - - return path, nil -} diff --git a/pkg/watch/paths_test.go b/pkg/watch/paths_test.go deleted file mode 100644 index 72b707e5163..00000000000 --- a/pkg/watch/paths_test.go +++ /dev/null @@ -1,44 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "runtime" - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -func TestGreatestExistingAncestor(t *testing.T) { - f := NewTempDirFixture(t) - - p, err := greatestExistingAncestor(f.Path()) - require.NoError(t, err) - assert.Equal(t, f.Path(), p) - - p, err = greatestExistingAncestor(f.JoinPath("missing")) - require.NoError(t, err) - assert.Equal(t, f.Path(), p) - - missingTopLevel := "/missingDir/a/b/c" - if runtime.GOOS == "windows" { - missingTopLevel = "C:\\missingDir\\a\\b\\c" - } - _, err = greatestExistingAncestor(missingTopLevel) - assert.Contains(t, err.Error(), "cannot watch root directory") -} diff --git a/pkg/watch/temp.go b/pkg/watch/temp.go deleted file mode 100644 index 011f547c1b4..00000000000 --- a/pkg/watch/temp.go +++ /dev/null @@ -1,88 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "os" - "path/filepath" -) - -// TempDir holds a temp directory and allows easy access to new temp directories. -type TempDir struct { - dir string -} - -// NewDir creates a new TempDir in the default location (typically $TMPDIR) -func NewDir(prefix string) (*TempDir, error) { - return NewDirAtRoot("", prefix) -} - -// NewDirAtRoot creates a new TempDir at the given root. -func NewDirAtRoot(root, prefix string) (*TempDir, error) { - tmpDir, err := os.MkdirTemp(root, prefix) - if err != nil { - return nil, err - } - - realTmpDir, err := filepath.EvalSymlinks(tmpDir) - if err != nil { - return nil, err - } - - return &TempDir{dir: realTmpDir}, nil -} - -// NewDirAtSlashTmp creates a new TempDir at /tmp -func NewDirAtSlashTmp(prefix string) (*TempDir, error) { - fullyResolvedPath, err := filepath.EvalSymlinks("/tmp") - if err != nil { - return nil, err - } - return NewDirAtRoot(fullyResolvedPath, prefix) -} - -// d.NewDir creates a new TempDir under d -func (d *TempDir) NewDir(prefix string) (*TempDir, error) { - d2, err := os.MkdirTemp(d.dir, prefix) - if err != nil { - return nil, err - } - return &TempDir{d2}, nil -} - -func (d *TempDir) NewDeterministicDir(name string) (*TempDir, error) { - d2 := filepath.Join(d.dir, name) - err := os.Mkdir(d2, 0o700) - if os.IsExist(err) { - return nil, err - } else if err != nil { - return nil, err - } - return &TempDir{d2}, nil -} - -func (d *TempDir) TearDown() error { - return os.RemoveAll(d.dir) -} - -func (d *TempDir) Path() string { - return d.dir -} - -// Possible extensions: -// temp file -// named directories or files (e.g., we know we want one git repo for our object, but it should be temporary) diff --git a/pkg/watch/temp_dir_fixture.go b/pkg/watch/temp_dir_fixture.go deleted file mode 100644 index a0855e875c8..00000000000 --- a/pkg/watch/temp_dir_fixture.go +++ /dev/null @@ -1,199 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "os" - "path/filepath" - "regexp" - "runtime" - "strings" - "testing" -) - -type TempDirFixture struct { - t testing.TB - dir *TempDir - oldDir string -} - -// everything not listed in this character class will get replaced by _, so that it's a safe filename -var sanitizeForFilenameRe = regexp.MustCompile("[^a-zA-Z0-9.]") - -func SanitizeFileName(name string) string { - return sanitizeForFilenameRe.ReplaceAllString(name, "_") -} - -func NewTempDirFixture(t testing.TB) *TempDirFixture { - dir, err := NewDir(SanitizeFileName(t.Name())) - if err != nil { - t.Fatalf("Error making temp dir: %v", err) - } - - ret := &TempDirFixture{ - t: t, - dir: dir, - } - - t.Cleanup(ret.tearDown) - - return ret -} - -func (f *TempDirFixture) T() testing.TB { - return f.t -} - -func (f *TempDirFixture) Path() string { - return f.dir.Path() -} - -func (f *TempDirFixture) Chdir() { - cwd, err := os.Getwd() - if err != nil { - f.t.Fatal(err) - } - - f.oldDir = cwd - - err = os.Chdir(f.Path()) - if err != nil { - f.t.Fatal(err) - } -} - -func (f *TempDirFixture) JoinPath(path ...string) string { - p := []string{} - isAbs := len(path) > 0 && filepath.IsAbs(path[0]) - if isAbs { - if !strings.HasPrefix(path[0], f.Path()) { - f.t.Fatalf("Path outside fixture tempdir are forbidden: %s", path[0]) - } - } else { - p = append(p, f.Path()) - } - - p = append(p, path...) - return filepath.Join(p...) -} - -func (f *TempDirFixture) JoinPaths(paths []string) []string { - joined := make([]string, len(paths)) - for i, p := range paths { - joined[i] = f.JoinPath(p) - } - return joined -} - -// Returns the full path to the file written. -func (f *TempDirFixture) WriteFile(path string, contents string) string { - fullPath := f.JoinPath(path) - base := filepath.Dir(fullPath) - err := os.MkdirAll(base, os.FileMode(0o777)) - if err != nil { - f.t.Fatal(err) - } - err = os.WriteFile(fullPath, []byte(contents), os.FileMode(0o777)) - if err != nil { - f.t.Fatal(err) - } - return fullPath -} - -// Returns the full path to the file written. -func (f *TempDirFixture) CopyFile(originalPath, newPath string) { - contents, err := os.ReadFile(originalPath) - if err != nil { - f.t.Fatal(err) - } - f.WriteFile(newPath, string(contents)) -} - -// Read the file. -func (f *TempDirFixture) ReadFile(path string) string { - fullPath := f.JoinPath(path) - contents, err := os.ReadFile(fullPath) - if err != nil { - f.t.Fatal(err) - } - return string(contents) -} - -func (f *TempDirFixture) WriteSymlink(linkContents, destPath string) { - fullDestPath := f.JoinPath(destPath) - err := os.MkdirAll(filepath.Dir(fullDestPath), os.FileMode(0o777)) - if err != nil { - f.t.Fatal(err) - } - err = os.Symlink(linkContents, fullDestPath) - if err != nil { - f.t.Fatal(err) - } -} - -func (f *TempDirFixture) MkdirAll(path string) { - fullPath := f.JoinPath(path) - err := os.MkdirAll(fullPath, os.FileMode(0o777)) - if err != nil { - f.t.Fatal(err) - } -} - -func (f *TempDirFixture) TouchFiles(paths []string) { - for _, p := range paths { - f.WriteFile(p, "") - } -} - -func (f *TempDirFixture) Rm(pathInRepo string) { - fullPath := f.JoinPath(pathInRepo) - err := os.RemoveAll(fullPath) - if err != nil { - f.t.Fatal(err) - } -} - -func (f *TempDirFixture) NewFile(prefix string) (*os.File, error) { - return os.CreateTemp(f.dir.Path(), prefix) -} - -func (f *TempDirFixture) TempDir(prefix string) string { - name, err := os.MkdirTemp(f.dir.Path(), prefix) - if err != nil { - f.t.Fatal(err) - } - return name -} - -func (f *TempDirFixture) tearDown() { - if f.oldDir != "" { - err := os.Chdir(f.oldDir) - if err != nil { - f.t.Fatal(err) - } - } - - err := f.dir.TearDown() - if err != nil && runtime.GOOS == "windows" && - (strings.Contains(err.Error(), "The process cannot access the file") || - strings.Contains(err.Error(), "Access is denied")) { - // NOTE(nick): I'm not convinced that this is a real problem. - // I think it might just be clean up of file notification I/O. - } else if err != nil { - f.t.Fatal(err) - } -} diff --git a/pkg/watch/watcher_darwin.go b/pkg/watch/watcher_darwin.go deleted file mode 100644 index a63612aedff..00000000000 --- a/pkg/watch/watcher_darwin.go +++ /dev/null @@ -1,144 +0,0 @@ -//go:build fsnotify - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "fmt" - "os" - "path/filepath" - "sync" - "time" - - "github.com/fsnotify/fsevents" - - pathutil "github.com/docker/compose/v5/internal/paths" -) - -// A file watcher optimized for Darwin. -// Uses FSEvents to avoid the terrible perf characteristics of kqueue. Requires CGO -type fseventNotify struct { - stream *fsevents.EventStream - events chan FileEvent - errors chan error - stop chan struct{} - - pathsWereWatching map[string]any - closeOnce sync.Once -} - -func (d *fseventNotify) loop() { - for { - select { - case <-d.stop: - return - case events, ok := <-d.stream.Events: - if !ok { - return - } - - for _, e := range events { - e.Path = filepath.Join(string(os.PathSeparator), e.Path) - - _, isPathWereWatching := d.pathsWereWatching[e.Path] - if e.Flags&fsevents.ItemIsDir == fsevents.ItemIsDir && e.Flags&fsevents.ItemCreated == fsevents.ItemCreated && isPathWereWatching { - // This is the first create for the path that we're watching. We always get exactly one of these - // even after we get the HistoryDone event. Skip it. - continue - } - - d.events <- NewFileEvent(e.Path) - } - } - } -} - -// Add a path to be watched. Should only be called during initialization. -func (d *fseventNotify) initAdd(name string) { - d.stream.Paths = append(d.stream.Paths, name) - - if d.pathsWereWatching == nil { - d.pathsWereWatching = make(map[string]any) - } - d.pathsWereWatching[name] = struct{}{} -} - -func (d *fseventNotify) Start() error { - if len(d.stream.Paths) == 0 { - return nil - } - - d.closeOnce = sync.Once{} - - numberOfWatches.Add(int64(len(d.stream.Paths))) - - err := d.stream.Start() - if err != nil { - return err - } - go d.loop() - return nil -} - -func (d *fseventNotify) Close() error { - d.closeOnce.Do(func() { - numberOfWatches.Add(int64(-len(d.stream.Paths))) - - d.stream.Stop() - close(d.errors) - close(d.stop) - }) - - return nil -} - -func (d *fseventNotify) Events() chan FileEvent { - return d.events -} - -func (d *fseventNotify) Errors() chan error { - return d.errors -} - -func newWatcher(paths []string) (Notify, error) { - dw := &fseventNotify{ - stream: &fsevents.EventStream{ - Latency: 50 * time.Millisecond, - Flags: fsevents.FileEvents | fsevents.IgnoreSelf, - // NOTE(dmiller): this corresponds to the `sinceWhen` parameter in FSEventStreamCreate - // https://developer.apple.com/documentation/coreservices/1443980-fseventstreamcreate - EventID: fsevents.LatestEventID(), - }, - events: make(chan FileEvent), - errors: make(chan error), - stop: make(chan struct{}), - } - - paths = pathutil.EncompassingPaths(paths) - for _, path := range paths { - path, err := filepath.Abs(path) - if err != nil { - return nil, fmt.Errorf("newWatcher: %w", err) - } - dw.initAdd(path) - } - - return dw, nil -} - -var _ Notify = &fseventNotify{} diff --git a/pkg/watch/watcher_darwin_test.go b/pkg/watch/watcher_darwin_test.go deleted file mode 100644 index 50eb442b06b..00000000000 --- a/pkg/watch/watcher_darwin_test.go +++ /dev/null @@ -1,48 +0,0 @@ -//go:build fsnotify - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "testing" - - "gotest.tools/v3/assert" -) - -func TestFseventNotifyCloseIdempotent(t *testing.T) { - // Create a watcher with a temporary directory - tmpDir := t.TempDir() - watcher, err := newWatcher([]string{tmpDir}) - assert.NilError(t, err) - - // Start the watcher - err = watcher.Start() - assert.NilError(t, err) - - // Close should work the first time - err = watcher.Close() - assert.NilError(t, err) - - // Close should be idempotent - calling it again should not panic - err = watcher.Close() - assert.NilError(t, err) - - // Even a third time should be safe - err = watcher.Close() - assert.NilError(t, err) -} diff --git a/pkg/watch/watcher_naive.go b/pkg/watch/watcher_naive.go deleted file mode 100644 index 7798025e8bc..00000000000 --- a/pkg/watch/watcher_naive.go +++ /dev/null @@ -1,325 +0,0 @@ -//go:build !fsnotify - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "fmt" - "io/fs" - "os" - "path/filepath" - "runtime" - "strings" - - "github.com/sirupsen/logrus" - "github.com/tilt-dev/fsnotify" - - pathutil "github.com/docker/compose/v5/internal/paths" -) - -// A naive file watcher that uses the plain fsnotify API. -// Used on all non-Darwin systems (including Windows & Linux). -// -// All OS-specific codepaths are handled by fsnotify. -type naiveNotify struct { - // Paths that we're watching that should be passed up to the caller. - // Note that we may have to watch ancestors of these paths - // in order to fulfill the API promise. - // - // We often need to check if paths are a child of a path in - // the notify list. It might be better to store this in a tree - // structure, so we can filter the list quickly. - notifyList map[string]bool - - isWatcherRecursive bool - watcher *fsnotify.Watcher - events chan fsnotify.Event - wrappedEvents chan FileEvent - errors chan error - numWatches int64 -} - -func (d *naiveNotify) Start() error { - if len(d.notifyList) == 0 { - return nil - } - - pathsToWatch := []string{} - for path := range d.notifyList { - pathsToWatch = append(pathsToWatch, path) - } - - pathsToWatch, err := greatestExistingAncestors(pathsToWatch) - if err != nil { - return err - } - if d.isWatcherRecursive { - pathsToWatch = pathutil.EncompassingPaths(pathsToWatch) - } - - for _, name := range pathsToWatch { - fi, err := os.Stat(name) - if err != nil && !os.IsNotExist(err) { - return fmt.Errorf("notify.Add(%q): %w", name, err) - } - - // if it's a file that doesn't exist, - // we should have caught that above, let's just skip it. - if os.IsNotExist(err) { - continue - } - - if fi.IsDir() { - err = d.watchRecursively(name) - if err != nil { - return fmt.Errorf("notify.Add(%q): %w", name, err) - } - } else { - err = d.add(filepath.Dir(name)) - if err != nil { - return fmt.Errorf("notify.Add(%q): %w", filepath.Dir(name), err) - } - } - } - - go d.loop() - - return nil -} - -func (d *naiveNotify) watchRecursively(dir string) error { - if d.isWatcherRecursive { - err := d.add(dir) - if err == nil || os.IsNotExist(err) { - return nil - } - return fmt.Errorf("watcher.Add(%q): %w", dir, err) - } - - return filepath.WalkDir(dir, func(path string, info fs.DirEntry, err error) error { - if err != nil { - return err - } - - if !info.IsDir() { - return nil - } - - if d.shouldSkipDir(path) { - logrus.Debugf("Ignoring directory and its contents (recursively): %s", path) - return filepath.SkipDir - } - - err = d.add(path) - if err != nil { - if os.IsNotExist(err) { - return nil - } - return fmt.Errorf("watcher.Add(%q): %w", path, err) - } - return nil - }) -} - -func (d *naiveNotify) Close() error { - numberOfWatches.Add(-d.numWatches) - d.numWatches = 0 - return d.watcher.Close() -} - -func (d *naiveNotify) Events() chan FileEvent { - return d.wrappedEvents -} - -func (d *naiveNotify) Errors() chan error { - return d.errors -} - -func (d *naiveNotify) loop() { //nolint:gocyclo - defer close(d.wrappedEvents) - for e := range d.events { - // The Windows fsnotify event stream sometimes gets events with empty names - // that are also sent to the error stream. Hmmmm... - if e.Name == "" { - continue - } - - if e.Op&fsnotify.Create != fsnotify.Create { - if d.shouldNotify(e.Name) { - d.wrappedEvents <- FileEvent(e.Name) - } - continue - } - - if d.isWatcherRecursive { - if d.shouldNotify(e.Name) { - d.wrappedEvents <- FileEvent(e.Name) - } - continue - } - - // If the watcher is not recursive, we have to walk the tree - // and add watches manually. We fire the event while we're walking the tree. - // because it's a bit more elegant that way. - // - // TODO(dbentley): if there's a delete should we call d.watcher.Remove to prevent leaking? - err := filepath.WalkDir(e.Name, func(path string, info fs.DirEntry, err error) error { - if err != nil { - return err - } - - if d.shouldNotify(path) { - d.wrappedEvents <- FileEvent(path) - } - - // TODO(dmiller): symlinks 😭 - - shouldWatch := false - if info.IsDir() { - // watch directories unless we can skip them entirely - if d.shouldSkipDir(path) { - return filepath.SkipDir - } - - shouldWatch = true - } else { - // watch files that are explicitly named, but don't watch others - _, ok := d.notifyList[path] - if ok { - shouldWatch = true - } - } - if shouldWatch { - err := d.add(path) - if err != nil && !os.IsNotExist(err) { - logrus.Infof("Error watching path %s: %s", e.Name, err) - } - } - return nil - }) - if err != nil && !os.IsNotExist(err) { - logrus.Infof("Error walking directory %s: %s", e.Name, err) - } - } -} - -func (d *naiveNotify) shouldNotify(path string) bool { - if _, ok := d.notifyList[path]; ok { - // We generally don't care when directories change at the root of an ADD - stat, err := os.Lstat(path) - isDir := err == nil && stat.IsDir() - return !isDir - } - - for root := range d.notifyList { - if pathutil.IsChild(root, path) { - return true - } - } - return false -} - -func (d *naiveNotify) shouldSkipDir(path string) bool { - // If path is directly in the notifyList, we should always watch it. - if d.notifyList[path] { - return false - } - - // Suppose we're watching - // /src/.tiltignore - // but the .tiltignore file doesn't exist. - // - // Our watcher will create an inotify watch on /src/. - // - // But then we want to make sure we don't recurse from /src/ down to /src/node_modules. - // - // To handle this case, we only want to traverse dirs that are: - // - A child of a directory that's in our notify list, or - // - A parent of a directory that's in our notify list - // (i.e., to cover the "path doesn't exist" case). - for root := range d.notifyList { - if pathutil.IsChild(root, path) || pathutil.IsChild(path, root) { - return false - } - } - return true -} - -func (d *naiveNotify) add(path string) error { - err := d.watcher.Add(path) - if err != nil { - return err - } - d.numWatches++ - numberOfWatches.Add(1) - return nil -} - -func newWatcher(paths []string) (Notify, error) { - fsw, err := fsnotify.NewWatcher() - if err != nil { - if strings.Contains(err.Error(), "too many open files") && runtime.GOOS == "linux" { - return nil, fmt.Errorf("hit OS limits creating a watcher.\n" + - "Run 'sysctl fs.inotify.max_user_instances' to check your inotify limits.\n" + - "To raise them, run 'sudo sysctl fs.inotify.max_user_instances=1024'") - } - return nil, fmt.Errorf("creating file watcher: %w", err) - } - MaybeIncreaseBufferSize(fsw) - - err = fsw.SetRecursive() - isWatcherRecursive := err == nil - - wrappedEvents := make(chan FileEvent) - notifyList := make(map[string]bool, len(paths)) - if isWatcherRecursive { - paths = pathutil.EncompassingPaths(paths) - } - for _, path := range paths { - path, err := filepath.Abs(path) - if err != nil { - return nil, fmt.Errorf("newWatcher: %w", err) - } - notifyList[path] = true - } - - wmw := &naiveNotify{ - notifyList: notifyList, - watcher: fsw, - events: fsw.Events, - wrappedEvents: wrappedEvents, - errors: fsw.Errors, - isWatcherRecursive: isWatcherRecursive, - } - - return wmw, nil -} - -var _ Notify = &naiveNotify{} - -func greatestExistingAncestors(paths []string) ([]string, error) { - result := []string{} - for _, p := range paths { - newP, err := greatestExistingAncestor(p) - if err != nil { - return nil, fmt.Errorf("finding ancestor of %s: %w", p, err) - } - result = append(result, newP) - } - return result, nil -} diff --git a/pkg/watch/watcher_naive_test.go b/pkg/watch/watcher_naive_test.go deleted file mode 100644 index 78acfdc5fee..00000000000 --- a/pkg/watch/watcher_naive_test.go +++ /dev/null @@ -1,159 +0,0 @@ -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "fmt" - "os" - "os/exec" - "path/filepath" - "runtime" - "strconv" - "strings" - "testing" - - "github.com/stretchr/testify/require" -) - -func TestDontWatchEachFile(t *testing.T) { - if runtime.GOOS != "linux" { - t.Skip("This test uses linux-specific inotify checks") - } - - // fsnotify is not recursive, so we need to watch each directory - // you can watch individual files with fsnotify, but that is more prone to exhaust resources - // this test uses a Linux way to get the number of watches to make sure we're watching - // per-directory, not per-file - f := newNotifyFixture(t) - - watched := f.TempDir("watched") - - // there are a few different cases we want to test for because the code paths are slightly - // different: - // 1) initial: data there before we ever call watch - // 2) inplace: data we create while the watch is happening - // 3) staged: data we create in another directory and then atomically move into place - - // initial - f.WriteFile(f.JoinPath(watched, "initial.txt"), "initial data") - - initialDir := f.JoinPath(watched, "initial_dir") - if err := os.Mkdir(initialDir, 0o777); err != nil { - t.Fatal(err) - } - - for i := 0; i < 100; i++ { - f.WriteFile(f.JoinPath(initialDir, fmt.Sprintf("%d", i)), "initial data") - } - - f.watch(watched) - f.fsync() - if len(f.events) != 0 { - t.Fatalf("expected 0 initial events; got %d events: %v", len(f.events), f.events) - } - f.events = nil - - // inplace - inplace := f.JoinPath(watched, "inplace") - if err := os.Mkdir(inplace, 0o777); err != nil { - t.Fatal(err) - } - f.WriteFile(f.JoinPath(inplace, "inplace.txt"), "inplace data") - - inplaceDir := f.JoinPath(inplace, "inplace_dir") - if err := os.Mkdir(inplaceDir, 0o777); err != nil { - t.Fatal(err) - } - - for i := 0; i < 100; i++ { - f.WriteFile(f.JoinPath(inplaceDir, fmt.Sprintf("%d", i)), "inplace data") - } - - f.fsync() - if len(f.events) < 100 { - t.Fatalf("expected >100 inplace events; got %d events: %v", len(f.events), f.events) - } - f.events = nil - - // staged - staged := f.TempDir("staged") - f.WriteFile(f.JoinPath(staged, "staged.txt"), "staged data") - - stagedDir := f.JoinPath(staged, "staged_dir") - if err := os.Mkdir(stagedDir, 0o777); err != nil { - t.Fatal(err) - } - - for i := 0; i < 100; i++ { - f.WriteFile(f.JoinPath(stagedDir, fmt.Sprintf("%d", i)), "staged data") - } - - if err := os.Rename(staged, f.JoinPath(watched, "staged")); err != nil { - t.Fatal(err) - } - - f.fsync() - if len(f.events) < 100 { - t.Fatalf("expected >100 staged events; got %d events: %v", len(f.events), f.events) - } - f.events = nil - - n, err := inotifyNodes() - require.NoError(t, err) - if n > 10 { - t.Fatalf("watching more than 10 files: %d", n) - } -} - -func inotifyNodes() (int, error) { - pid := os.Getpid() - - output, err := exec.Command("/bin/sh", "-c", fmt.Sprintf( - "find /proc/%d/fd -lname anon_inode:inotify -printf '%%hinfo/%%f\n' | xargs cat | grep -c '^inotify'", pid)).Output() - if err != nil { - return 0, fmt.Errorf("error running command to determine number of watched files: %w\n %s", err, output) - } - - n, err := strconv.Atoi(strings.TrimSpace(string(output))) - if err != nil { - return 0, fmt.Errorf("couldn't parse number of watched files: %w", err) - } - return n, nil -} - -func TestDontRecurseWhenWatchingParentsOfNonExistentFiles(t *testing.T) { - if runtime.GOOS != "linux" { - t.Skip("This test uses linux-specific inotify checks") - } - - f := newNotifyFixture(t) - - watched := f.TempDir("watched") - f.watch(filepath.Join(watched, ".tiltignore")) - - excludedDir := f.JoinPath(watched, "excluded") - for i := 0; i < 10; i++ { - f.WriteFile(f.JoinPath(excludedDir, fmt.Sprintf("%d", i), "data.txt"), "initial data") - } - f.fsync() - - n, err := inotifyNodes() - require.NoError(t, err) - if n > 5 { - t.Fatalf("watching more than 5 files: %d", n) - } -} diff --git a/pkg/watch/watcher_nonwin.go b/pkg/watch/watcher_nonwin.go deleted file mode 100644 index 37222663b07..00000000000 --- a/pkg/watch/watcher_nonwin.go +++ /dev/null @@ -1,25 +0,0 @@ -//go:build !windows - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import "github.com/tilt-dev/fsnotify" - -func MaybeIncreaseBufferSize(w *fsnotify.Watcher) { - // Not needed on non-windows -} diff --git a/pkg/watch/watcher_windows.go b/pkg/watch/watcher_windows.go deleted file mode 100644 index a632967d772..00000000000 --- a/pkg/watch/watcher_windows.go +++ /dev/null @@ -1,35 +0,0 @@ -//go:build windows - -/* - Copyright 2020 Docker Compose CLI authors - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. -*/ - -package watch - -import ( - "github.com/tilt-dev/fsnotify" -) - -// TODO(nick): I think the ideal API would be to automatically increase the -// size of the buffer when we exceed capacity. But this gets messy, -// because each time we get a short read error, we need to invalidate -// everything we know about the currently changed files. So for now, -// we just provide a way for people to increase the buffer ourselves. -// -// It might also pay to be clever about sizing the buffer -// relative the number of files in the directory we're watching. -func MaybeIncreaseBufferSize(w *fsnotify.Watcher) { - w.SetBufferSize(DesiredWindowsBufferSize()) -} diff --git a/project/ISSUE-TRIAGE.md b/project/ISSUE-TRIAGE.md new file mode 100644 index 00000000000..b89cdc240a9 --- /dev/null +++ b/project/ISSUE-TRIAGE.md @@ -0,0 +1,35 @@ +Triaging of issues +------------------ + +The docker-compose issue triage process follows +https://github.com/docker/docker/blob/master/project/ISSUE-TRIAGE.md +with the following additions or exceptions. + + +### Classify the Issue + +The following labels are provided in additional to the standard labels: + +| Kind | Description | +|--------------|-------------------------------------------------------------------| +| kind/cleanup | A refactor or improvement that is related to quality not function | +| kind/parity | A request for feature parity with docker cli | + + +### Functional areas + +Most issues should fit into one of the following functional areas: + +| Area | +|-----------------| +| area/build | +| area/cli | +| area/config | +| area/logs | +| area/networking | +| area/packaging | +| area/run | +| area/scale | +| area/tests | +| area/up | +| area/volumes | diff --git a/project/RELEASE-PROCESS.md b/project/RELEASE-PROCESS.md new file mode 120000 index 00000000000..c8457671ac7 --- /dev/null +++ b/project/RELEASE-PROCESS.md @@ -0,0 +1 @@ +../script/release/README.md \ No newline at end of file diff --git a/pyinstaller/ldd b/pyinstaller/ldd new file mode 100755 index 00000000000..3f10ad2757e --- /dev/null +++ b/pyinstaller/ldd @@ -0,0 +1,13 @@ +#!/bin/sh + +# From http://wiki.musl-libc.org/wiki/FAQ#Q:_where_is_ldd_.3F +# +# Musl's dynlinker comes with ldd functionality built in. just create a +# symlink from ld-musl-$ARCH.so to /bin/ldd. If the dynlinker was started +# as "ldd", it will detect that and print the appropriate DSO information. +# +# Instead, this string replaced "ldd" with the package so that pyinstaller +# can find the actual lib. +exec /usr/bin/ldd "$@" | \ + sed -r 's/([^[:space:]]+) => ldd/\1 => \/lib\/\1/g' | \ + sed -r 's/ldd $.*$//g' diff --git a/requirements-build.txt b/requirements-build.txt new file mode 100644 index 00000000000..9ca8d66679a --- /dev/null +++ b/requirements-build.txt @@ -0,0 +1 @@ +pyinstaller==4.1 diff --git a/requirements-dev.txt b/requirements-dev.txt new file mode 100644 index 00000000000..00f7a455b07 --- /dev/null +++ b/requirements-dev.txt @@ -0,0 +1,10 @@ +Click==7.1.2 +coverage==5.2.1 +ddt==1.4.1 +flake8==3.8.3 +gitpython==3.1.11 +mock==3.0.5 +pytest==6.0.1; python_version >= '3.5' +pytest==4.6.5; python_version < '3.5' +pytest-cov==2.10.1 +PyYAML==5.3.1 diff --git a/requirements-indirect.txt b/requirements-indirect.txt new file mode 100644 index 00000000000..218a4eddd34 --- /dev/null +++ b/requirements-indirect.txt @@ -0,0 +1,28 @@ +altgraph==0.17 +appdirs==1.4.4 +attrs==20.3.0 +bcrypt==3.2.0 +cffi==1.14.4 +cryptography==3.2.1 +distlib==0.3.1 +entrypoints==0.3 +filelock==3.0.12 +gitdb2==4.0.2 +mccabe==0.6.1 +more-itertools==8.6.0; python_version >= '3.5' +more-itertools==5.0.0; python_version < '3.5' +packaging==20.4 +pluggy==0.13.1 +py==1.9.0 +pycodestyle==2.6.0 +pycparser==2.20 +pyflakes==2.2.0 +PyNaCl==1.4.0 +pyparsing==2.4.7 +pyrsistent==0.16.0 +smmap==3.0.4 +smmap2==3.0.1 +toml==0.10.1 +tox==3.20.1 +virtualenv==20.2.2 +wcwidth==0.2.5 diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 00000000000..82816e80f02 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,22 @@ +backports.shutil_get_terminal_size==1.0.0 +cached-property==1.5.1 +certifi==2020.6.20 +chardet==3.0.4 +colorama==0.4.3; sys_platform == 'win32' +distro==1.5.0 +docker==4.4.1 +docker-pycreds==0.4.0 +dockerpty==0.4.1 +docopt==0.6.2 +idna==2.10 +ipaddress==1.0.23 +jsonschema==3.2.0 +paramiko==2.7.1 +PySocks==1.7.1 +python-dotenv==0.14.0 +pywin32==227; sys_platform == 'win32' +PyYAML==5.3.1 +requests==2.24.0 +texttable==1.6.2 +urllib3==1.25.10; python_version == '3.3' +websocket-client==0.57.0 diff --git a/script/build/image b/script/build/image new file mode 100755 index 00000000000..fb3f856ee24 --- /dev/null +++ b/script/build/image @@ -0,0 +1,20 @@ +#!/bin/bash + +set -e + +if [ -z "$1" ]; then + >&2 echo "First argument must be image tag." + exit 1 +fi + +TAG="$1" + +VERSION="$(python setup.py --version)" + +DOCKER_COMPOSE_GITSHA="$(script/build/write-git-sha)" +echo "${DOCKER_COMPOSE_GITSHA}" > compose/GITSHA +python setup.py sdist bdist_wheel + +docker build \ + --build-arg GIT_COMMIT="${DOCKER_COMPOSE_GITSHA}" \ + -t "${TAG}" . diff --git a/script/build/linux b/script/build/linux new file mode 100755 index 00000000000..2e56b625c0f --- /dev/null +++ b/script/build/linux @@ -0,0 +1,16 @@ +#!/bin/bash + +set -ex + +./script/clean + +DOCKER_COMPOSE_GITSHA="$(script/build/write-git-sha)" + +docker build . \ + --target bin \ + --build-arg DISTRO=debian \ + --build-arg GIT_COMMIT="${DOCKER_COMPOSE_GITSHA}" \ + --output dist/ +ARCH=$(uname -m) +# Ensure that we output the binary with the same name as we did before +mv dist/docker-compose-linux-amd64 "dist/docker-compose-Linux-${ARCH}" diff --git a/script/build/linux-entrypoint b/script/build/linux-entrypoint new file mode 100755 index 00000000000..a0e86ee261a --- /dev/null +++ b/script/build/linux-entrypoint @@ -0,0 +1,40 @@ +#!/bin/bash + +set -ex + +CODE_PATH=/code +VENV="${CODE_PATH}"/.tox/py39 + +cd "${CODE_PATH}" +mkdir -p dist +chmod 777 dist + +"${VENV}"/bin/pip3 install -q -r requirements-build.txt + +# TODO(ulyssessouza) To check if really needed +if [ -z "${DOCKER_COMPOSE_GITSHA}" ]; then + DOCKER_COMPOSE_GITSHA="$(script/build/write-git-sha)" +fi +echo "${DOCKER_COMPOSE_GITSHA}" > compose/GITSHA + +export PATH="${CODE_PATH}/pyinstaller:${PATH}" + +if [ ! -z "${BUILD_BOOTLOADER}" ]; then + # Build bootloader for alpine; develop is the main branch + git clone --single-branch --branch develop https://github.com/pyinstaller/pyinstaller.git /tmp/pyinstaller + cd /tmp/pyinstaller/bootloader + # Checkout commit corresponding to version in requirements-build + git checkout v4.1 + "${VENV}"/bin/python3 ./waf configure --no-lsb all + "${VENV}"/bin/pip3 install .. + cd "${CODE_PATH}" + rm -Rf /tmp/pyinstaller +else + echo "NOT compiling bootloader!!!" +fi + +"${VENV}"/bin/pyinstaller --exclude-module pycrypto --exclude-module PyInstaller docker-compose.spec +ls -la dist/ +ldd dist/docker-compose +mv dist/docker-compose /usr/local/bin +docker-compose version diff --git a/script/build/osx b/script/build/osx new file mode 100755 index 00000000000..e2d17527b3a --- /dev/null +++ b/script/build/osx @@ -0,0 +1,25 @@ +#!/bin/bash +set -ex + +TOOLCHAIN_PATH="$(realpath $(dirname $0)/../../build/toolchain)" + +rm -rf venv + +virtualenv -p "${TOOLCHAIN_PATH}"/bin/python3 venv +venv/bin/pip install -r requirements-indirect.txt +venv/bin/pip install -r requirements.txt +venv/bin/pip install -r requirements-build.txt +venv/bin/pip install --no-deps . +DOCKER_COMPOSE_GITSHA="$(script/build/write-git-sha)" +echo "${DOCKER_COMPOSE_GITSHA}" > compose/GITSHA + +# Build as a folder for macOS Catalina. +venv/bin/pyinstaller docker-compose_darwin.spec +dist/docker-compose-Darwin-x86_64/docker-compose version +(cd dist/docker-compose-Darwin-x86_64/ && tar zcvf ../docker-compose-Darwin-x86_64.tgz .) +rm -rf dist/docker-compose-Darwin-x86_64 + +# Build static binary for legacy. +venv/bin/pyinstaller docker-compose.spec +mv dist/docker-compose dist/docker-compose-Darwin-x86_64 +dist/docker-compose-Darwin-x86_64 version diff --git a/script/build/test-image b/script/build/test-image new file mode 100755 index 00000000000..ddb8057d05c --- /dev/null +++ b/script/build/test-image @@ -0,0 +1,18 @@ +#!/bin/bash + +set -e + +if [ -z "$1" ]; then + >&2 echo "First argument must be image tag." + exit 1 +fi + +TAG="$1" +IMAGE="docker/compose-tests" + +DOCKER_COMPOSE_GITSHA="$(script/build/write-git-sha)" +docker build -t "${IMAGE}:${TAG}" . \ + --target build \ + --build-arg DISTRO="debian" \ + --build-arg GIT_COMMIT="${DOCKER_COMPOSE_GITSHA}" +docker tag "${IMAGE}":"${TAG}" "${IMAGE}":latest diff --git a/script/build/windows.ps1 b/script/build/windows.ps1 new file mode 100644 index 00000000000..147d0f07d88 --- /dev/null +++ b/script/build/windows.ps1 @@ -0,0 +1,60 @@ +# Builds the Windows binary. +# +# From a fresh 64-bit Windows 10 install, prepare the system as follows: +# +# 1. Install Git: +# +# http://git-scm.com/download/win +# +# 2. Install Python 3.9.x: +# +# https://www.python.org/downloads/ +# +# 3. Append ";C:\Python39;C:\Python39\Scripts" to the "Path" environment variable: +# +# https://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/sysdm_advancd_environmnt_addchange_variable.mspx?mfr=true +# +# 4. In Powershell, run the following commands: +# +# $ pip install 'virtualenv==20.2.2' +# $ Set-ExecutionPolicy -Scope CurrentUser RemoteSigned +# +# 5. Clone the repository: +# +# $ git clone https://github.com/docker/compose.git +# $ cd compose +# +# 6. Build the binary: +# +# .\script\build\windows.ps1 + +$ErrorActionPreference = "Stop" + +# Remove virtualenv +if (Test-Path venv) { + Remove-Item -Recurse -Force .\venv +} + +# Remove .pyc files +Get-ChildItem -Recurse -Include *.pyc | foreach ($_) { Remove-Item $_.FullName } + +# Create virtualenv +virtualenv -p C:\Python39\python.exe .\venv + +# pip and pyinstaller generate lots of warnings, so we need to ignore them +$ErrorActionPreference = "Continue" + +.\venv\Scripts\pip install pypiwin32==223 +.\venv\Scripts\pip install -r requirements-indirect.txt +.\venv\Scripts\pip install -r requirements.txt +.\venv\Scripts\pip install --no-deps . +.\venv\Scripts\pip install -r requirements-build.txt + +git rev-parse --short HEAD | out-file -encoding ASCII compose\GITSHA + +# Build binary +.\venv\Scripts\pyinstaller .\docker-compose.spec +$ErrorActionPreference = "Stop" + +Move-Item -Force .\dist\docker-compose.exe .\dist\docker-compose-Windows-x86_64.exe +.\dist\docker-compose-Windows-x86_64.exe --version diff --git a/script/build/write-git-sha b/script/build/write-git-sha new file mode 100755 index 00000000000..cac4b6fd3b1 --- /dev/null +++ b/script/build/write-git-sha @@ -0,0 +1,12 @@ +#!/bin/bash +# +# Write the current commit sha to the file GITSHA. This file is included in +# packaging so that `docker-compose version` can include the git sha. +# sets to 'unknown' and echoes a message if the command is not successful + +DOCKER_COMPOSE_GITSHA="$(git rev-parse --short HEAD)" +if [[ "${?}" != "0" ]]; then + echo "Couldn't get revision of the git repository. Setting to 'unknown' instead" + DOCKER_COMPOSE_GITSHA="unknown" +fi +echo "${DOCKER_COMPOSE_GITSHA}" diff --git a/script/ci b/script/ci new file mode 100755 index 00000000000..34bf9a4be65 --- /dev/null +++ b/script/ci @@ -0,0 +1,8 @@ +#!/bin/bash +# +# Backwards compatibility for jenkins +# +# TODO: remove this script after all current PRs and jenkins are updated with +# the new script/test/ci change +set -e +exec script/test/ci diff --git a/script/circle/bintray-deploy.sh b/script/circle/bintray-deploy.sh new file mode 100755 index 00000000000..a7cce726e60 --- /dev/null +++ b/script/circle/bintray-deploy.sh @@ -0,0 +1,35 @@ +#!/bin/bash + +curl -f -u$BINTRAY_USERNAME:$BINTRAY_API_KEY -X GET \ + https://api.bintray.com/repos/docker-compose/${CIRCLE_BRANCH} + +if test $? -ne 0; then + echo "Bintray repository ${CIRCLE_BRANCH} does not exist ; abandoning upload attempt" + exit 0 +fi + +curl -u$BINTRAY_USERNAME:$BINTRAY_API_KEY -X POST \ + -d "{\ + \"name\": \"${PKG_NAME}\", \"desc\": \"auto\", \"licenses\": [\"Apache-2.0\"], \ + \"vcs_url\": \"${CIRCLE_REPOSITORY_URL}\" \ + }" -H "Content-Type: application/json" \ + https://api.bintray.com/packages/docker-compose/${CIRCLE_BRANCH} + +curl -u$BINTRAY_USERNAME:$BINTRAY_API_KEY -X POST -d "{\ + \"name\": \"$CIRCLE_BRANCH\", \ + \"desc\": \"Automated build of the ${CIRCLE_BRANCH} branch.\", \ + }" -H "Content-Type: application/json" \ + https://api.bintray.com/packages/docker-compose/${CIRCLE_BRANCH}/${PKG_NAME}/versions + +curl -f -T dist/docker-compose-${OS_NAME}-x86_64 -u$BINTRAY_USERNAME:$BINTRAY_API_KEY \ + -H "X-Bintray-Package: ${PKG_NAME}" -H "X-Bintray-Version: $CIRCLE_BRANCH" \ + -H "X-Bintray-Override: 1" -H "X-Bintray-Publish: 1" -X PUT \ + https://api.bintray.com/content/docker-compose/${CIRCLE_BRANCH}/docker-compose-${OS_NAME}-x86_64 || exit 1 + +# Upload folder format of docker-compose for macOS in addition to binary. +if [ "${OS_NAME}" == "Darwin" ]; then + curl -f -T dist/docker-compose-${OS_NAME}-x86_64.tgz -u$BINTRAY_USERNAME:$BINTRAY_API_KEY \ + -H "X-Bintray-Package: ${PKG_NAME}" -H "X-Bintray-Version: $CIRCLE_BRANCH" \ + -H "X-Bintray-Override: 1" -H "X-Bintray-Publish: 1" -X PUT \ + https://api.bintray.com/content/docker-compose/${CIRCLE_BRANCH}/docker-compose-${OS_NAME}-x86_64.tgz || exit 1 +fi diff --git a/script/clean b/script/clean new file mode 100755 index 00000000000..2e1994df391 --- /dev/null +++ b/script/clean @@ -0,0 +1,8 @@ +#!/bin/sh +set -e + +find . -type f -name '*.pyc' -delete +rm -rf .coverage-binfiles +find . -name .coverage.* -delete +find . -name __pycache__ -delete +rm -rf docs/_site build dist docker-compose.egg-info diff --git a/script/docs/check_help.py b/script/docs/check_help.py new file mode 100755 index 00000000000..0904f00c4f6 --- /dev/null +++ b/script/docs/check_help.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python3 +import glob +import os.path +import re +import subprocess + +USAGE_RE = re.compile(r"```.*?\nUsage:.*?```", re.MULTILINE | re.DOTALL) +USAGE_IN_CMD_RE = re.compile(r"^Usage:.*", re.MULTILINE | re.DOTALL) + +HELP_CMD = "docker run --rm docker/compose:latest %s --help" + +for file in glob.glob("compose/reference/*.md"): + with open(file) as f: + data = f.read() + if not USAGE_RE.search(data): + print("Not a command:", file) + continue + subcmd = os.path.basename(file).replace(".md", "") + if subcmd == "overview": + continue + print(f"Found {subcmd}: {file}") + help_cmd = HELP_CMD % subcmd + help = subprocess.check_output(help_cmd.split()) + help = help.decode("utf-8") + help = USAGE_IN_CMD_RE.findall(help)[0] + help = help.strip() + data = USAGE_RE.sub(f"```none\n{help}\n```", data) + with open(file, "w") as f: + f.write(data) diff --git a/script/release/README.md b/script/release/README.md new file mode 100644 index 00000000000..b42e4fa1dfa --- /dev/null +++ b/script/release/README.md @@ -0,0 +1,23 @@ +# Release HOWTO + +The release process is fully automated by `Release.Jenkinsfile`. + +## Usage + +1. In the appropriate branch, run `./script/release/release.py tag ` + +By appropriate, we mean for a version `1.26.0` or `1.26.0-rc1` you should run the script in the `1.26.x` branch. + +The script should check the above then ask for changelog modifications. + +After the executions, you should have a commit with the proper bumps for `docker-compose version` and `run.sh` + +2. Run `git push --tags upstream ` +This should trigger a new CI build on the new tag. When the CI finishes with the tests and builds a new draft release would be available on github's releases page. + +3. Check and confirm the release on github's release page. + +4. In case of a GA version, please update `docker-compose`s release notes and version on [github documentation repository](https://github.com/docker/docker.github.io): + - [Release Notes](https://github.com/docker/docker.github.io/blob/master/compose/release-notes.md) + - [Config version](https://github.com/docker/docker.github.io/blob/master/_config.yml) + - [Config authoring version](https://github.com/docker/docker.github.io/blob/master/_config_authoring.yml) diff --git a/script/release/cherry-pick-pr b/script/release/cherry-pick-pr new file mode 100755 index 00000000000..f4a5a7406b3 --- /dev/null +++ b/script/release/cherry-pick-pr @@ -0,0 +1,34 @@ +#!/bin/bash +# +# Cherry-pick a PR into the release branch +# + +set -e +set -o pipefail + + +function usage() { + >&2 cat << EOM +Cherry-pick commits from a github pull request. + +Usage: + + $0 +EOM + exit 1 +} + +[ -n "$1" ] || usage + +if [ -z "$(command -v hub 2> /dev/null)" ]; then + >&2 echo "$0 requires https://hub.github.com/." + >&2 echo "Please install it and make sure it is available on your \$PATH." + exit 2 +fi + + +REPO=docker/compose +GITHUB=https://github.com/$REPO/pull +PR=$1 +url="$GITHUB/$PR" +hub am -3 $url diff --git a/script/release/const.py b/script/release/const.py new file mode 100644 index 00000000000..8c90eebca6e --- /dev/null +++ b/script/release/const.py @@ -0,0 +1,4 @@ +import os + + +REPO_ROOT = os.path.join(os.path.dirname(__file__), '..', '..') diff --git a/script/release/generate_changelog.sh b/script/release/generate_changelog.sh new file mode 100755 index 00000000000..018387de03d --- /dev/null +++ b/script/release/generate_changelog.sh @@ -0,0 +1,42 @@ +#!/bin/bash + +set -e +set -x + +## Usage : +## changelog PREVIOUS_TAG..HEAD + +# configure refs so we get pull-requests metadata +git config --add remote.origin.fetch +refs/pull/*/head:refs/remotes/origin/pull/* +git fetch origin + +RANGE=${1:-"$(git describe --tags --abbrev=0 HEAD^)..HEAD"} +echo "Generate changelog for range ${RANGE}" +echo + +pullrequests() { + for commit in $(git log ${RANGE} --format='format:%H'); do + # Get the oldest remotes/origin/pull/* branch to include this commit, i.e. the one to introduce it + git branch -a --sort=committerdate --contains $commit --list 'origin/pull/*' | head -1 | cut -d'/' -f4 + done +} + +changes=$(pullrequests | uniq) + +echo "pull requests merged within range:" +echo $changes + +echo '#Features' > FEATURES.md +echo '#Bugs' > BUGS.md +for pr in $changes; do + curl -fs -H "Authorization: token ${GITHUB_TOKEN}" https://api.github.com/repos/docker/compose/pulls/${pr} -o PR.json + + cat PR.json | jq -r ' select( .labels[].name | contains("kind/feature") ) | "- "+.title' >> FEATURES.md + cat PR.json | jq -r ' select( .labels[].name | contains("kind/bug") ) | "- "+.title' >> BUGS.md +done + +echo ${TAG_NAME} > CHANGELOG.md +echo >> CHANGELOG.md +cat FEATURES.md >> CHANGELOG.md +echo >> CHANGELOG.md +cat BUGS.md >> CHANGELOG.md diff --git a/script/release/release.md.tmpl b/script/release/release.md.tmpl new file mode 100644 index 00000000000..4d0ebe926eb --- /dev/null +++ b/script/release/release.md.tmpl @@ -0,0 +1,34 @@ +If you're a Mac or Windows user, the best way to install Compose and keep it up-to-date is **[Docker Desktop for Mac and Windows](https://www.docker.com/products/docker-desktop)**. + +Docker Desktop will automatically install the latest version of Docker Engine for you. + +Alternatively, you can use the usual commands to install or upgrade Compose: + +``` +curl -L https://github.com/docker/compose/releases/download/{{version}}/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose +chmod +x /usr/local/bin/docker-compose +``` + +See the [install docs](https://docs.docker.com/compose/install/) for more install options and instructions. + +## Compose file format compatibility matrix + +| Compose file format | Docker Engine | +| --- | --- | +{% for engine, formats in compat_matrix.items() -%} +| {% for format in formats %}{{format}}{% if not loop.last %}, {% endif %}{% endfor %} | {{engine}}+ | +{% endfor -%} + +## Changes + +{{changelog}} + +Thanks to {% for name in contributors %}@{{name}}{% if not loop.last %}, {% endif %}{% endfor %} for contributing to this release! + +## Integrity check + +Binary name | SHA-256 sum +| --- | --- | +{% for filename, sha in integrity.items() -%} +| `{{filename}}` | `{{sha[1]}}` | +{% endfor -%} diff --git a/script/release/release.py b/script/release/release.py new file mode 100644 index 00000000000..c8e5e7f767d --- /dev/null +++ b/script/release/release.py @@ -0,0 +1,123 @@ +#!/usr/bin/env python3 +import re + +import click +from git import Repo +from utils import update_init_py_version +from utils import update_run_sh_version +from utils import yesno + +VALID_VERSION_PATTERN = re.compile(r"^\d+\.\d+\.\d+(-rc\d+)?$") + + +class Version(str): + def matching_groups(self): + match = VALID_VERSION_PATTERN.match(self) + if not match: + return False + + return match.groups() + + def is_ga_version(self): + groups = self.matching_groups() + if not groups: + return False + + rc_suffix = groups[1] + return not rc_suffix + + def validate(self): + return len(self.matching_groups()) > 0 + + def branch_name(self): + if not self.validate(): + return None + + rc_part = self.matching_groups()[0] + ver = self + if rc_part: + ver = ver[:-len(rc_part)] + + tokens = ver.split(".") + tokens[-1] = 'x' + + return ".".join(tokens) + + +def create_bump_commit(repository, version): + print('Creating bump commit...') + repository.commit('-a', '-s', '-m "Bump {}"'.format(version), '--no-verify') + + +def validate_environment(version, repository): + if not version.validate(): + print('Version "{}" has an invalid format. This should follow D+.D+.D+(-rcD+). ' + 'Like: 1.26.0 or 1.26.0-rc1'.format(version)) + return False + + expected_branch = version.branch_name() + if str(repository.active_branch) != expected_branch: + print('Cannot tag in this branch with version "{}". ' + 'Please checkout "{}" to tag'.format(version, version.branch_name())) + return False + return True + + +@click.group() +def cli(): + pass + + +@cli.command() +@click.argument('version') +def tag(version): + """ + Updates the version related files and tag + """ + repo = Repo(".") + version = Version(version) + if not validate_environment(version, repo): + return + + update_init_py_version(version) + update_run_sh_version(version) + + input('Please add the release notes to the CHANGELOG.md file, then press Enter to continue.') + proceed = False + while not proceed: + print(repo.git.diff()) + proceed = yesno('Are these changes ok? y/N ', default=False) + + if repo.git.diff(): + create_bump_commit(repo.git, version) + else: + print('No changes to commit. Exiting...') + return + + repo.create_tag(version) + + print('Please, check the changes. If everything is OK, you just need to push with:\n' + '$ git push --tags upstream {}'.format(version.branch_name())) + + +@cli.command() +@click.argument('version') +def push_latest(version): + """ + TODO Pushes the latest tag pointing to a certain GA version + """ + raise NotImplementedError + + +@cli.command() +@click.argument('version') +def ghtemplate(version): + """ + TODO Generates the github release page content + """ + version = Version(version) + raise NotImplementedError + + +if __name__ == '__main__': + cli() diff --git a/script/release/utils.py b/script/release/utils.py new file mode 100644 index 00000000000..5ed53ec85b3 --- /dev/null +++ b/script/release/utils.py @@ -0,0 +1,44 @@ +import os +import re + +from const import REPO_ROOT + + +def update_init_py_version(version): + path = os.path.join(REPO_ROOT, 'compose', '__init__.py') + with open(path) as f: + contents = f.read() + contents = re.sub(r"__version__ = '[0-9a-z.-]+'", "__version__ = '{}'".format(version), contents) + with open(path, 'w') as f: + f.write(contents) + + +def update_run_sh_version(version): + path = os.path.join(REPO_ROOT, 'script', 'run', 'run.sh') + with open(path) as f: + contents = f.read() + contents = re.sub(r'VERSION="[0-9a-z.-]+"', 'VERSION="{}"'.format(version), contents) + with open(path, 'w') as f: + f.write(contents) + + +def yesno(prompt, default=None): + """ + Prompt the user for a yes or no. + + Can optionally specify a default value, which will only be + used if they enter a blank line. + + Unrecognised input (anything other than "y", "n", "yes", + "no" or "") will return None. + """ + answer = input(prompt).strip().lower() + + if answer == "y" or answer == "yes": + return True + elif answer == "n" or answer == "no": + return False + elif answer == "": + return default + else: + return None diff --git a/script/release/utils.sh b/script/release/utils.sh new file mode 100644 index 00000000000..321c1fb7b8c --- /dev/null +++ b/script/release/utils.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# +# Util functions for release scripts +# + +set -e +set -o pipefail + + +function browser() { + local url=$1 + xdg-open $url || open $url +} + + +function find_remote() { + local url=$1 + for remote in $(git remote); do + git config --get remote.${remote}.url | grep $url > /dev/null && echo -n $remote + done + # Always return true, extra remotes cause it to return false + true +} diff --git a/script/run/run.ps1 b/script/run/run.ps1 new file mode 100644 index 00000000000..47ec546925c --- /dev/null +++ b/script/run/run.ps1 @@ -0,0 +1,22 @@ +# Run docker-compose in a container via boot2docker. +# +# The current directory will be mirrored as a volume and additional +# volumes (or any other options) can be mounted by using +# $Env:DOCKER_COMPOSE_OPTIONS. + +if ($Env:DOCKER_COMPOSE_VERSION -eq $null -or $Env:DOCKER_COMPOSE_VERSION.Length -eq 0) { + $Env:DOCKER_COMPOSE_VERSION = "latest" +} + +if ($Env:DOCKER_COMPOSE_OPTIONS -eq $null) { + $Env:DOCKER_COMPOSE_OPTIONS = "" +} + +if (-not $Env:DOCKER_HOST) { + docker-machine env --shell=powershell default | Invoke-Expression + if (-not $?) { exit $LastExitCode } +} + +$local="/$($PWD -replace '^(.):(.*)$', '"$1".ToLower()+"$2".Replace("\","/")' | Invoke-Expression)" +docker run --rm -ti -v /var/run/docker.sock:/var/run/docker.sock -v "${local}:$local" -w "$local" $Env:DOCKER_COMPOSE_OPTIONS "docker/compose:$Env:DOCKER_COMPOSE_VERSION" $args +exit $LastExitCode diff --git a/script/run/run.sh b/script/run/run.sh new file mode 100755 index 00000000000..658cf47ac5a --- /dev/null +++ b/script/run/run.sh @@ -0,0 +1,91 @@ +#!/bin/sh +# +# Run docker-compose in a container +# +# This script will attempt to mirror the host paths by using volumes for the +# following paths: +# * $(pwd) +# * $(dirname $COMPOSE_FILE) if it's set +# * $HOME if it's set +# +# You can add additional volumes (or any docker run options) using +# the $COMPOSE_OPTIONS environment variable. +# + + +set -e + +VERSION="1.26.1" +IMAGE="docker/compose:$VERSION" + + +# Setup options for connecting to docker host +if [ -z "$DOCKER_HOST" ]; then + DOCKER_HOST='unix:///var/run/docker.sock' +fi +if [ -S "${DOCKER_HOST#unix://}" ]; then + DOCKER_ADDR="-v ${DOCKER_HOST#unix://}:${DOCKER_HOST#unix://} -e DOCKER_HOST" +else + DOCKER_ADDR="-e DOCKER_HOST -e DOCKER_TLS_VERIFY -e DOCKER_CERT_PATH" +fi + + +# Setup volume mounts for compose config and context +if [ "$(pwd)" != '/' ]; then + VOLUMES="-v $(pwd):$(pwd)" +fi +if [ -n "$COMPOSE_FILE" ]; then + COMPOSE_OPTIONS="$COMPOSE_OPTIONS -e COMPOSE_FILE=$COMPOSE_FILE" + compose_dir="$(dirname "$COMPOSE_FILE")" + # canonicalize dir, do not use realpath or readlink -f + # since they are not available in some systems (e.g. macOS). + compose_dir="$(cd "$compose_dir" && pwd)" +fi +if [ -n "$COMPOSE_PROJECT_NAME" ]; then + COMPOSE_OPTIONS="-e COMPOSE_PROJECT_NAME $COMPOSE_OPTIONS" +fi +if [ -n "$compose_dir" ]; then + VOLUMES="$VOLUMES -v $compose_dir:$compose_dir" +fi +if [ -n "$HOME" ]; then + VOLUMES="$VOLUMES -v $HOME:$HOME -e HOME" # Pass in HOME to share docker.config and allow ~/-relative paths to work. +fi +i=$# +while [ $i -gt 0 ]; do + arg=$1 + i=$((i - 1)) + shift + + case "$arg" in + -f|--file) + value=$1 + i=$((i - 1)) + shift + set -- "$@" "$arg" "$value" + + file_dir=$(realpath "$(dirname "$value")") + VOLUMES="$VOLUMES -v $file_dir:$file_dir" + ;; + *) set -- "$@" "$arg" ;; + esac +done + +# Setup environment variables for compose config and context +ENV_OPTIONS=$(printenv | sed -E "/^PATH=.*/d; s/^/-e /g; s/=.*//g; s/\n/ /g") + +# Only allocate tty if we detect one +if [ -t 0 ] && [ -t 1 ]; then + DOCKER_RUN_OPTIONS="$DOCKER_RUN_OPTIONS -t" +fi + +# Always set -i to support piped and terminal input in run/exec +DOCKER_RUN_OPTIONS="$DOCKER_RUN_OPTIONS -i" + + +# Handle userns security +if docker info --format '{{json .SecurityOptions}}' 2>/dev/null | grep -q 'name=userns'; then + DOCKER_RUN_OPTIONS="$DOCKER_RUN_OPTIONS --userns=host" +fi + +# shellcheck disable=SC2086 +exec docker run --rm $DOCKER_RUN_OPTIONS $DOCKER_ADDR $COMPOSE_OPTIONS $ENV_OPTIONS $VOLUMES -w "$(pwd)" $IMAGE "$@" diff --git a/script/setup/osx b/script/setup/osx new file mode 100755 index 00000000000..289155bafcd --- /dev/null +++ b/script/setup/osx @@ -0,0 +1,110 @@ +#!/usr/bin/env bash + +set -ex + +. $(dirname $0)/osx_helpers.sh + +DEPLOYMENT_TARGET=${DEPLOYMENT_TARGET:-"$(macos_version)"} +SDK_FETCH= +if ! [ ${DEPLOYMENT_TARGET} == "$(macos_version)" ]; then + SDK_FETCH=1 + # SDK URL from https://github.com/docker/golang-cross/blob/master/osx-cross.sh + SDK_URL=https://s3.dockerproject.org/darwin/v2/MacOSX${DEPLOYMENT_TARGET}.sdk.tar.xz + SDK_SHA1=dd228a335194e3392f1904ce49aff1b1da26ca62 +fi + +OPENSSL_VERSION=1.1.1h +OPENSSL_URL=https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz +OPENSSL_SHA1=8d0d099e8973ec851368c8c775e05e1eadca1794 + +PYTHON_VERSION=3.9.0 +PYTHON_URL=https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz +PYTHON_SHA1=5744a10ba989d2badacbab3c00cdcb83c83106c7 + +# +# Install prerequisites. +# +if ! [ -x "$(command -v brew)" ]; then + ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" +fi +if ! [ -x "$(command -v grealpath)" ]; then + brew update > /dev/null + brew install coreutils +fi +if ! [ -x "$(command -v python3)" ]; then + brew update > /dev/null + brew install python3 +fi +if ! [ -x "$(command -v virtualenv)" ]; then + pip3 install virtualenv==20.2.2 +fi + +# +# Create toolchain directory. +# +BUILD_PATH="$(grealpath $(dirname $0)/../../build)" +mkdir -p ${BUILD_PATH} +TOOLCHAIN_PATH="${BUILD_PATH}/toolchain" +mkdir -p ${TOOLCHAIN_PATH} + +# +# Set macOS SDK. +# +if [[ ${SDK_FETCH} && ! -f ${TOOLCHAIN_PATH}/MacOSX${DEPLOYMENT_TARGET}.sdk/SDKSettings.plist ]]; then + SDK_PATH=${TOOLCHAIN_PATH}/MacOSX${DEPLOYMENT_TARGET}.sdk + fetch_tarball ${SDK_URL} ${SDK_PATH} ${SDK_SHA1} +else + SDK_PATH="$(xcode-select --print-path)/Platforms/MacOSX.platform/Developer/SDKs/MacOSX${DEPLOYMENT_TARGET}.sdk" +fi + +# +# Build OpenSSL. +# +OPENSSL_SRC_PATH=${TOOLCHAIN_PATH}/openssl-${OPENSSL_VERSION} +if ! [[ $(${TOOLCHAIN_PATH}/bin/openssl version) == *"${OPENSSL_VERSION}"* ]]; then + rm -rf ${OPENSSL_SRC_PATH} + fetch_tarball ${OPENSSL_URL} ${OPENSSL_SRC_PATH} ${OPENSSL_SHA1} + ( + cd ${OPENSSL_SRC_PATH} + export MACOSX_DEPLOYMENT_TARGET=${DEPLOYMENT_TARGET} + export SDKROOT=${SDK_PATH} + ./Configure darwin64-x86_64-cc --prefix=${TOOLCHAIN_PATH} + make install_sw install_dev + ) +fi + +# +# Build Python. +# +PYTHON_SRC_PATH=${TOOLCHAIN_PATH}/Python-${PYTHON_VERSION} +if ! [[ $(${TOOLCHAIN_PATH}/bin/python3 --version) == *"${PYTHON_VERSION}"* ]]; then + rm -rf ${PYTHON_SRC_PATH} + fetch_tarball ${PYTHON_URL} ${PYTHON_SRC_PATH} ${PYTHON_SHA1} + ( + cd ${PYTHON_SRC_PATH} + ./configure --prefix=${TOOLCHAIN_PATH} \ + --enable-ipv6 --without-ensurepip --with-dtrace --without-gcc \ + --datarootdir=${TOOLCHAIN_PATH}/share \ + --datadir=${TOOLCHAIN_PATH}/share \ + --enable-framework=${TOOLCHAIN_PATH}/Frameworks \ + --with-openssl=${TOOLCHAIN_PATH} \ + MACOSX_DEPLOYMENT_TARGET=${DEPLOYMENT_TARGET} \ + CFLAGS="-isysroot ${SDK_PATH} -I${TOOLCHAIN_PATH}/include" \ + CPPFLAGS="-I${SDK_PATH}/usr/include -I${TOOLCHAIN_PATH}/include" \ + LDFLAGS="-isysroot ${SDK_PATH} -L ${TOOLCHAIN_PATH}/lib" + make -j 4 + make install PYTHONAPPSDIR=${TOOLCHAIN_PATH} + make frameworkinstallextras PYTHONAPPSDIR=${TOOLCHAIN_PATH}/share + ) +fi + +# +# Smoke test built Python. +# +openssl_version ${TOOLCHAIN_PATH} + +echo "" +echo "*** Targeting macOS: ${DEPLOYMENT_TARGET}" +echo "*** Using SDK ${SDK_PATH}" +echo "*** Using $(python3_version ${TOOLCHAIN_PATH})" +echo "*** Using $(openssl_version ${TOOLCHAIN_PATH})" diff --git a/script/setup/osx_helpers.sh b/script/setup/osx_helpers.sh new file mode 100644 index 00000000000..d60a30b6dd9 --- /dev/null +++ b/script/setup/osx_helpers.sh @@ -0,0 +1,41 @@ +#!/usr/bin/env bash + +# Check file's ($1) SHA1 ($2). +check_sha1() { + echo -n "$2 *$1" | shasum -c - +} + +# Download URL ($1) to path ($2). +download() { + curl -L $1 -o $2 +} + +# Extract tarball ($1) in folder ($2). +extract() { + tar xf $1 -C $2 +} + +# Download URL ($1), check SHA1 ($3), and extract utility ($2). +fetch_tarball() { + url=$1 + tarball=$2.tarball + sha1=$3 + download $url $tarball + check_sha1 $tarball $sha1 + extract $tarball $(dirname $tarball) +} + +# Version of Python at toolchain path ($1). +python3_version() { + $1/bin/python3 -V 2>&1 +} + +# Version of OpenSSL used by toolchain ($1) Python. +openssl_version() { + $1/bin/python3 -c "import ssl; print(ssl.OPENSSL_VERSION)" +} + +# System macOS version. +macos_version() { + sw_vers -productVersion | cut -f1,2 -d'.' +} diff --git a/script/test/acceptance b/script/test/acceptance new file mode 100755 index 00000000000..92710a76a8d --- /dev/null +++ b/script/test/acceptance @@ -0,0 +1,3 @@ +#!/usr/bin/env bash + +pytest --conformity --binary ${1:-docker-compose} tests/acceptance/ diff --git a/script/test/all b/script/test/all new file mode 100755 index 00000000000..1626fed37f9 --- /dev/null +++ b/script/test/all @@ -0,0 +1,64 @@ +#!/bin/bash +# This should be run inside a container built from the Dockerfile +# at the root of the repo - script/test will do it automatically. + +set -e + +>&2 echo "Running lint checks" +docker run --rm \ + --tty \ + ${GIT_VOLUME} \ + "$TAG" tox -e pre-commit + +get_versions="docker run --rm + --entrypoint=/code/.tox/py39/bin/python + $TAG + /code/script/test/versions.py docker/docker-ce,moby/moby" + +if [ "$DOCKER_VERSIONS" == "" ]; then + DOCKER_VERSIONS="$($get_versions default)" +elif [ "$DOCKER_VERSIONS" == "all" ]; then + DOCKER_VERSIONS=$($get_versions -n 2 recent) +fi + +DOCKER_VERSIONS=19.03.14 + +BUILD_NUMBER=${BUILD_NUMBER-$USER} +PY_TEST_VERSIONS=${PY_TEST_VERSIONS:-py39} + +for version in $DOCKER_VERSIONS; do + >&2 echo "Running tests against Docker $version" + + daemon_container="compose-dind-$version-$BUILD_NUMBER" + + function on_exit() { + if [[ "$?" != "0" ]]; then + docker logs "$daemon_container" 2>&1 | tail -n 100 + fi + docker rm -vf "$daemon_container" + } + + trap "on_exit" EXIT + + repo="dockerswarm/dind" + + docker run \ + -d \ + --name "$daemon_container" \ + --privileged \ + --volume="/var/lib/docker" \ + "$repo:$version" \ + dockerd -H tcp://0.0.0.0:2375 $DOCKER_DAEMON_ARGS \ + 2>&1 | tail -n 10 + + docker run \ + --rm \ + --tty \ + --link="$daemon_container:docker" \ + --env="DOCKER_HOST=tcp://docker:2375" \ + --env="DOCKER_VERSION=$version" \ + --entrypoint="tox" \ + "$TAG" \ + -e "$PY_TEST_VERSIONS" -- "$@" + +done diff --git a/script/test/ci b/script/test/ci new file mode 100755 index 00000000000..bbcedac47c5 --- /dev/null +++ b/script/test/ci @@ -0,0 +1,22 @@ +#!/bin/bash +# This should be run inside a container built from the Dockerfile +# at the root of the repo: +# +# $ TAG="docker-compose:$(git rev-parse --short HEAD)" +# $ docker build -t "$TAG" . +# $ docker run --rm \ +# --volume="/var/run/docker.sock:/var/run/docker.sock" \ +# --volume="$(pwd)/.git:/code/.git" \ +# -e "TAG=$TAG" \ +# --entrypoint="script/test/ci" "$TAG" + +set -ex + +docker version + +export DOCKER_VERSIONS=${DOCKER_VERSIONS:-all} +STORAGE_DRIVER=${STORAGE_DRIVER:-overlay} +export DOCKER_DAEMON_ARGS="--storage-driver=$STORAGE_DRIVER" + +GIT_VOLUME="--volumes-from=$(hostname)" +. script/test/all diff --git a/script/test/default b/script/test/default new file mode 100755 index 00000000000..4f307f2e927 --- /dev/null +++ b/script/test/default @@ -0,0 +1,20 @@ +#!/bin/bash +# See CONTRIBUTING.md for usage. + +set -ex + +TAG="docker-compose:alpine-$(git rev-parse --short HEAD)" + +# By default use the Dockerfile, but can be overridden to use an alternative file +# e.g DOCKERFILE=Dockerfile.s390x script/test/default +DOCKERFILE="${DOCKERFILE:-Dockerfile}" +DOCKER_BUILD_TARGET="${DOCKER_BUILD_TARGET:-build}" + +rm -rf coverage-html +# Create the host directory so it's owned by $USER +mkdir -p coverage-html + +docker build -f "${DOCKERFILE}" -t "${TAG}" --target "${DOCKER_BUILD_TARGET}" . + +GIT_VOLUME="--volume=$(pwd)/.git:/code/.git" +. script/test/all diff --git a/script/test/versions.py b/script/test/versions.py new file mode 100755 index 00000000000..1a28dc19ad3 --- /dev/null +++ b/script/test/versions.py @@ -0,0 +1,175 @@ +#!/usr/bin/env python +""" +Query the github API for the git tags of a project, and return a list of +version tags for recent releases, or the default release. + +The default release is the most recent non-RC version. + +Recent is a list of unique major.minor versions, where each is the most +recent version in the series. + +For example, if the list of versions is: + + 1.8.0-rc2 + 1.8.0-rc1 + 1.7.1 + 1.7.0 + 1.7.0-rc1 + 1.6.2 + 1.6.1 + +`default` would return `1.7.1` and +`recent -n 3` would return `1.8.0-rc2 1.7.1 1.6.2` +""" +import argparse +import itertools +import operator +import sys +from collections import namedtuple + +import requests + + +GITHUB_API = 'https://api.github.com/repos' + +STAGES = ['tp', 'beta', 'rc'] + + +class Version(namedtuple('_Version', 'major minor patch stage edition')): + + @classmethod + def parse(cls, version): + edition = None + version = version.lstrip('v') + version, _, stage = version.partition('-') + if stage: + if not any(marker in stage for marker in STAGES): + edition = stage + stage = None + elif '-' in stage: + edition, stage = stage.split('-') + major, minor, patch = version.split('.', 3) + return cls(major, minor, patch, stage, edition) + + @property + def major_minor(self): + return self.major, self.minor + + @property + def order(self): + """Return a representation that allows this object to be sorted + correctly with the default comparator. + """ + # non-GA releases should appear before GA releases + # Order: tp -> beta -> rc -> GA + if self.stage: + for st in STAGES: + if st in self.stage: + stage = (STAGES.index(st), self.stage) + break + else: + stage = (len(STAGES),) + + return (int(self.major), int(self.minor), int(self.patch)) + stage + + def __str__(self): + stage = '-{}'.format(self.stage) if self.stage else '' + edition = '-{}'.format(self.edition) if self.edition else '' + return '.'.join(map(str, self[:3])) + edition + stage + + +BLACKLIST = [ # List of versions known to be broken and should not be used + Version.parse('18.03.0-ce-rc2'), +] + + +def group_versions(versions): + """Group versions by `major.minor` releases. + + Example: + + >>> group_versions([ + Version(1, 0, 0), + Version(2, 0, 0, 'rc1'), + Version(2, 0, 0), + Version(2, 1, 0), + ]) + + [ + [Version(1, 0, 0)], + [Version(2, 0, 0), Version(2, 0, 0, 'rc1')], + [Version(2, 1, 0)], + ] + """ + return list( + list(releases) + for _, releases + in itertools.groupby(versions, operator.attrgetter('major_minor')) + ) + + +def get_latest_versions(versions, num=1): + """Return a list of the most recent versions for each major.minor version + group. + """ + versions = group_versions(versions) + num = min(len(versions), num) + return [versions[index][0] for index in range(num)] + + +def get_default(versions): + """Return a :class:`Version` for the latest GA version.""" + for version in versions: + if not version.stage: + return version + + +def get_versions(tags): + for tag in tags: + try: + v = Version.parse(tag['name']) + if v in BLACKLIST: + continue + yield v + except ValueError: + print("Skipping invalid tag: {name}".format(**tag), file=sys.stderr) + + +def get_github_releases(projects): + """Query the Github API for a list of version tags and return them in + sorted order. + + See https://developer.github.com/v3/repos/#list-tags + """ + versions = [] + for project in projects: + url = '{}/{}/tags'.format(GITHUB_API, project) + response = requests.get(url) + response.raise_for_status() + versions.extend(get_versions(response.json())) + return sorted(versions, reverse=True, key=operator.attrgetter('order')) + + +def parse_args(argv): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('project', help="Github project name (ex: docker/docker)") + parser.add_argument('command', choices=['recent', 'default']) + parser.add_argument('-n', '--num', type=int, default=2, + help="Number of versions to return from `recent`") + return parser.parse_args(argv) + + +def main(argv=None): + args = parse_args(argv) + versions = get_github_releases(args.project.split(',')) + + if args.command == 'recent': + print(' '.join(map(str, get_latest_versions(versions, args.num)))) + elif args.command == 'default': + print(get_default(versions)) + else: + raise ValueError("Unknown command {}".format(args.command)) + + +if __name__ == "__main__": + main() diff --git a/setup.cfg b/setup.cfg new file mode 100644 index 00000000000..3c6e79cf31d --- /dev/null +++ b/setup.cfg @@ -0,0 +1,2 @@ +[bdist_wheel] +universal=1 diff --git a/setup.py b/setup.py new file mode 100644 index 00000000000..57e1313355d --- /dev/null +++ b/setup.py @@ -0,0 +1,108 @@ +#!/usr/bin/env python +import codecs +import os +import re +import sys + +import pkg_resources +from setuptools import find_packages +from setuptools import setup + + +def read(*parts): + path = os.path.join(os.path.dirname(__file__), *parts) + with codecs.open(path, encoding='utf-8') as fobj: + return fobj.read() + + +def find_version(*file_paths): + version_file = read(*file_paths) + version_match = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]", + version_file, re.M) + if version_match: + return version_match.group(1) + raise RuntimeError("Unable to find version string.") + + +install_requires = [ + 'cached-property >= 1.2.0, < 2', + 'docopt >= 0.6.1, < 1', + 'PyYAML >= 3.10, < 6', + 'requests >= 2.20.0, < 3', + 'texttable >= 0.9.0, < 2', + 'websocket-client >= 0.32.0, < 1', + 'distro >= 1.5.0, < 2', + 'docker[ssh] >= 4.4.0, < 5', + 'dockerpty >= 0.4.1, < 1', + 'jsonschema >= 2.5.1, < 4', + 'python-dotenv >= 0.13.0, < 1', +] + + +tests_require = [ + 'ddt >= 1.2.2, < 2', + 'pytest < 6', +] + + +if sys.version_info[:2] < (3, 4): + tests_require.append('mock >= 1.0.1, < 4') + +extras_require = { + ':python_version < "3.5"': ['backports.ssl_match_hostname >= 3.5, < 4'], + ':sys_platform == "win32"': ['colorama >= 0.4, < 1'], + 'socks': ['PySocks >= 1.5.6, != 1.5.7, < 2'], + 'tests': tests_require, +} + + +try: + if 'bdist_wheel' not in sys.argv: + for key, value in extras_require.items(): + if key.startswith(':') and pkg_resources.evaluate_marker(key[1:]): + install_requires.extend(value) +except Exception as e: + print("Failed to compute platform dependencies: {}. ".format(e) + + "All dependencies will be installed as a result.", file=sys.stderr) + for key, value in extras_require.items(): + if key.startswith(':'): + install_requires.extend(value) + + +setup( + name='docker-compose', + version=find_version("compose", "__init__.py"), + description='Multi-container orchestration for Docker', + long_description=read('README.md'), + long_description_content_type='text/markdown', + url='https://www.docker.com/', + project_urls={ + 'Documentation': 'https://docs.docker.com/compose/overview', + 'Changelog': 'https://github.com/docker/compose/blob/release/CHANGELOG.md', + 'Source': 'https://github.com/docker/compose', + 'Tracker': 'https://github.com/docker/compose/issues', + }, + author='Docker, Inc.', + license='Apache License 2.0', + packages=find_packages(exclude=['tests.*', 'tests']), + include_package_data=True, + install_requires=install_requires, + extras_require=extras_require, + tests_require=tests_require, + python_requires='>=3.4', + entry_points={ + 'console_scripts': ['docker-compose=compose.cli.main:main'], + }, + classifiers=[ + 'Development Status :: 5 - Production/Stable', + 'Environment :: Console', + 'Intended Audience :: Developers', + 'License :: OSI Approved :: Apache Software License', + 'Programming Language :: Python :: 3', + 'Programming Language :: Python :: 3.4', + 'Programming Language :: Python :: 3.6', + 'Programming Language :: Python :: 3.7', + 'Programming Language :: Python :: 3.8', + 'Programming Language :: Python :: 3.9', + ], +) diff --git a/tests/__init__.py b/tests/__init__.py new file mode 100644 index 00000000000..9d732490098 --- /dev/null +++ b/tests/__init__.py @@ -0,0 +1,2 @@ +import unittest # NOQA +from unittest import mock # NOQA diff --git a/tests/acceptance/__init__.py b/tests/acceptance/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py new file mode 100644 index 00000000000..211579bd14a --- /dev/null +++ b/tests/acceptance/cli_test.py @@ -0,0 +1,3149 @@ +import datetime +import json +import os.path +import re +import signal +import subprocess +import time +from collections import Counter +from collections import namedtuple +from functools import reduce +from operator import attrgetter + +import pytest +import yaml +from docker import errors + +from .. import mock +from ..helpers import BUSYBOX_IMAGE_WITH_TAG +from ..helpers import create_host_file +from compose.cli.command import get_project +from compose.config.errors import DuplicateOverrideFileFound +from compose.const import COMPOSE_SPEC as VERSION +from compose.const import COMPOSEFILE_V1 as V1 +from compose.container import Container +from compose.project import OneOffFilter +from compose.utils import nanoseconds_from_time_seconds +from tests.integration.testcases import DockerClientTestCase +from tests.integration.testcases import get_links +from tests.integration.testcases import is_cluster +from tests.integration.testcases import no_cluster +from tests.integration.testcases import pull_busybox +from tests.integration.testcases import SWARM_SKIP_RM_VOLUMES + +DOCKER_COMPOSE_EXECUTABLE = 'docker-compose' + +ProcessResult = namedtuple('ProcessResult', 'stdout stderr') + + +BUILD_CACHE_TEXT = 'Using cache' +BUILD_PULL_TEXT = 'Status: Image is up to date for busybox:1.27.2' +COMPOSE_COMPATIBILITY_DICT = { + 'version': str(VERSION), + 'volumes': {'foo': {'driver': 'default'}}, + 'networks': {'bar': {}}, + 'services': { + 'foo': { + 'command': '/bin/true', + 'image': 'alpine:3.10.1', + 'scale': 3, + 'restart': 'always:7', + 'mem_limit': '300M', + 'mem_reservation': '100M', + 'cpus': 0.7, + 'volumes': ['foo:/bar:rw'], + 'networks': {'bar': None}, + } + }, +} + + +def start_process(base_dir, options, executable=None, env=None): + executable = executable or DOCKER_COMPOSE_EXECUTABLE + proc = subprocess.Popen( + [executable] + options, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + cwd=base_dir, + env=env, + ) + print("Running process: %s" % proc.pid) + return proc + + +def wait_on_process(proc, returncode=0, stdin=None): + stdout, stderr = proc.communicate(input=stdin) + if proc.returncode != returncode: + print("Stderr: {}".format(stderr)) + print("Stdout: {}".format(stdout)) + assert proc.returncode == returncode + return ProcessResult(stdout.decode('utf-8'), stderr.decode('utf-8')) + + +def dispatch(base_dir, options, + project_options=None, returncode=0, stdin=None, executable=None, env=None): + project_options = project_options or [] + proc = start_process(base_dir, project_options + options, executable=executable, env=env) + return wait_on_process(proc, returncode=returncode, stdin=stdin) + + +def wait_on_condition(condition, delay=0.1, timeout=40): + start_time = time.time() + while not condition(): + if time.time() - start_time > timeout: + raise AssertionError("Timeout: %s" % condition) + time.sleep(delay) + + +def kill_service(service): + for container in service.containers(): + if container.is_running: + container.kill() + + +class ContainerCountCondition: + + def __init__(self, project, expected): + self.project = project + self.expected = expected + + def __call__(self): + return len([c for c in self.project.containers() if c.is_running]) == self.expected + + def __str__(self): + return "waiting for counter count == %s" % self.expected + + +class ContainerStateCondition: + + def __init__(self, client, name, status): + self.client = client + self.name = name + self.status = status + + def __call__(self): + try: + if self.name.endswith('*'): + ctnrs = self.client.containers(all=True, filters={'name': self.name[:-1]}) + if len(ctnrs) > 0: + container = self.client.inspect_container(ctnrs[0]['Id']) + else: + return False + else: + container = self.client.inspect_container(self.name) + return container['State']['Status'] == self.status + except errors.APIError: + return False + + def __str__(self): + return "waiting for container to be %s" % self.status + + +class CLITestCase(DockerClientTestCase): + + def setUp(self): + super().setUp() + self.base_dir = 'tests/fixtures/simple-composefile' + self.override_dir = None + + def tearDown(self): + if self.base_dir: + self.project.kill() + self.project.down(None, True) + + for container in self.project.containers(stopped=True, one_off=OneOffFilter.only): + container.remove(force=True) + networks = self.client.networks() + for n in networks: + if n['Name'].split('/')[-1].startswith('{}_'.format(self.project.name)): + self.client.remove_network(n['Name']) + volumes = self.client.volumes().get('Volumes') or [] + for v in volumes: + if v['Name'].split('/')[-1].startswith('{}_'.format(self.project.name)): + self.client.remove_volume(v['Name']) + if hasattr(self, '_project'): + del self._project + + super().tearDown() + + @property + def project(self): + # Hack: allow project to be overridden + if not hasattr(self, '_project'): + self._project = get_project(self.base_dir, override_dir=self.override_dir) + return self._project + + def dispatch(self, options, project_options=None, returncode=0, stdin=None): + return dispatch(self.base_dir, options, project_options, returncode, stdin) + + def execute(self, container, cmd): + # Remove once Hijack and CloseNotifier sign a peace treaty + self.client.close() + exc = self.client.exec_create(container.id, cmd) + self.client.exec_start(exc) + return self.client.exec_inspect(exc)['ExitCode'] + + def lookup(self, container, hostname): + return self.execute(container, ["nslookup", hostname]) == 0 + + def test_help(self): + self.base_dir = 'tests/fixtures/no-composefile' + result = self.dispatch(['help', 'up'], returncode=0) + assert 'Usage: up [options] [--scale SERVICE=NUM...] [--] [SERVICE...]' in result.stdout + # Prevent tearDown from trying to create a project + self.base_dir = None + + def test_quiet_build(self): + self.base_dir = 'tests/fixtures/build-args' + result = self.dispatch(['build'], None) + quietResult = self.dispatch(['build', '-q'], None) + assert result.stdout != "" + assert quietResult.stdout == "" + + def test_help_nonexistent(self): + self.base_dir = 'tests/fixtures/no-composefile' + result = self.dispatch(['help', 'foobar'], returncode=1) + assert 'No such command' in result.stderr + self.base_dir = None + + def test_shorthand_host_opt(self): + self.dispatch( + ['-H={}'.format(os.environ.get('DOCKER_HOST', 'unix://')), + 'up', '-d'], + returncode=0 + ) + + def test_shorthand_host_opt_interactive(self): + self.dispatch( + ['-H={}'.format(os.environ.get('DOCKER_HOST', 'unix://')), + 'run', 'another', 'ls'], + returncode=0 + ) + + def test_host_not_reachable(self): + result = self.dispatch(['-H=tcp://doesnotexist:8000', 'ps'], returncode=1) + assert "Couldn't connect to Docker daemon" in result.stderr + + def test_host_not_reachable_volumes_from_container(self): + self.base_dir = 'tests/fixtures/volumes-from-container' + + container = self.client.create_container( + 'busybox', 'true', name='composetest_data_container', + host_config={} + ) + self.addCleanup(self.client.remove_container, container) + + result = self.dispatch(['-H=tcp://doesnotexist:8000', 'ps'], returncode=1) + assert "Couldn't connect to Docker daemon" in result.stderr + + def test_config_list_services(self): + self.base_dir = 'tests/fixtures/v2-full' + result = self.dispatch(['config', '--services']) + assert set(result.stdout.rstrip().split('\n')) == {'web', 'other'} + + def test_config_list_volumes(self): + self.base_dir = 'tests/fixtures/v2-full' + result = self.dispatch(['config', '--volumes']) + assert set(result.stdout.rstrip().split('\n')) == {'data'} + + def test_config_quiet_with_error(self): + self.base_dir = None + result = self.dispatch([ + '-f', 'tests/fixtures/invalid-composefile/invalid.yml', + 'config', '--quiet' + ], returncode=1) + assert "'notaservice' must be a mapping" in result.stderr + + def test_config_quiet(self): + self.base_dir = 'tests/fixtures/v2-full' + assert self.dispatch(['config', '--quiet']).stdout == '' + + def test_config_stdin(self): + config = b"""version: "3.7" +services: + web: + image: nginx + other: + image: alpine +""" + result = self.dispatch(['-f', '-', 'config', '--services'], stdin=config) + assert set(result.stdout.rstrip().split('\n')) == {'web', 'other'} + + def test_config_with_hash_option(self): + self.base_dir = 'tests/fixtures/v2-full' + result = self.dispatch(['config', '--hash=*']) + for service in self.project.get_services(): + assert '{} {}\n'.format(service.name, service.config_hash) in result.stdout + + svc = self.project.get_service('other') + result = self.dispatch(['config', '--hash=other']) + assert result.stdout == '{} {}\n'.format(svc.name, svc.config_hash) + + def test_config_default(self): + self.base_dir = 'tests/fixtures/v2-full' + result = self.dispatch(['config']) + # assert there are no python objects encoded in the output + assert '!!' not in result.stdout + + output = yaml.safe_load(result.stdout) + expected = { + 'version': '2', + 'volumes': {'data': {'driver': 'local'}}, + 'networks': {'front': {}}, + 'services': { + 'web': { + 'build': { + 'context': os.path.abspath(self.base_dir), + }, + 'networks': {'front': None, 'default': None}, + 'volumes_from': ['service:other:rw'], + }, + 'other': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'volumes': ['/data'], + }, + }, + } + assert output == expected + + def test_config_restart(self): + self.base_dir = 'tests/fixtures/restart' + result = self.dispatch(['config']) + assert yaml.safe_load(result.stdout) == { + 'version': '2', + 'services': { + 'never': { + 'image': 'busybox', + 'restart': 'no', + }, + 'always': { + 'image': 'busybox', + 'restart': 'always', + }, + 'on-failure': { + 'image': 'busybox', + 'restart': 'on-failure', + }, + 'on-failure-5': { + 'image': 'busybox', + 'restart': 'on-failure:5', + }, + 'restart-null': { + 'image': 'busybox', + 'restart': '' + }, + }, + } + + def test_config_external_network(self): + self.base_dir = 'tests/fixtures/networks' + result = self.dispatch(['-f', 'external-networks.yml', 'config']) + json_result = yaml.safe_load(result.stdout) + assert 'networks' in json_result + assert json_result['networks'] == { + 'networks_foo': { + 'external': True, + 'name': 'networks_foo' + }, + 'bar': { + 'external': True, + 'name': 'networks_bar' + } + } + + def test_config_with_dot_env(self): + self.base_dir = 'tests/fixtures/default-env-file' + result = self.dispatch(['config']) + json_result = yaml.safe_load(result.stdout) + assert json_result == { + 'version': '2.4', + 'services': { + 'web': { + 'command': 'true', + 'image': 'alpine:latest', + 'ports': [{'target': 5643}, {'target': 9999}] + } + } + } + + def test_config_with_env_file(self): + self.base_dir = 'tests/fixtures/default-env-file' + result = self.dispatch(['--env-file', '.env2', 'config']) + json_result = yaml.safe_load(result.stdout) + assert json_result == { + 'version': '2.4', + 'services': { + 'web': { + 'command': 'false', + 'image': 'alpine:latest', + 'ports': [{'target': 5644}, {'target': 9998}] + } + } + } + + def test_config_with_dot_env_and_override_dir(self): + self.base_dir = 'tests/fixtures/default-env-file' + result = self.dispatch(['--project-directory', 'alt/', 'config']) + json_result = yaml.safe_load(result.stdout) + assert json_result == { + 'version': '2.4', + 'services': { + 'web': { + 'command': 'echo uwu', + 'image': 'alpine:3.10.1', + 'ports': [{'target': 3341}, {'target': 4449}] + } + } + } + + def test_config_external_volume_v2(self): + self.base_dir = 'tests/fixtures/volumes' + result = self.dispatch(['-f', 'external-volumes-v2.yml', 'config']) + json_result = yaml.safe_load(result.stdout) + assert 'volumes' in json_result + assert json_result['volumes'] == { + 'foo': { + 'external': True, + 'name': 'foo', + }, + 'bar': { + 'external': True, + 'name': 'some_bar', + } + } + + def test_config_external_volume_v2_x(self): + self.base_dir = 'tests/fixtures/volumes' + result = self.dispatch(['-f', 'external-volumes-v2-x.yml', 'config']) + json_result = yaml.safe_load(result.stdout) + assert 'volumes' in json_result + assert json_result['volumes'] == { + 'foo': { + 'external': True, + 'name': 'some_foo', + }, + 'bar': { + 'external': True, + 'name': 'some_bar', + } + } + + def test_config_external_volume_v3_x(self): + self.base_dir = 'tests/fixtures/volumes' + result = self.dispatch(['-f', 'external-volumes-v3-x.yml', 'config']) + json_result = yaml.safe_load(result.stdout) + assert 'volumes' in json_result + assert json_result['volumes'] == { + 'foo': { + 'external': True, + 'name': 'foo', + }, + 'bar': { + 'external': True, + 'name': 'some_bar', + } + } + + def test_config_external_volume_v3_4(self): + self.base_dir = 'tests/fixtures/volumes' + result = self.dispatch(['-f', 'external-volumes-v3-4.yml', 'config']) + json_result = yaml.safe_load(result.stdout) + assert 'volumes' in json_result + assert json_result['volumes'] == { + 'foo': { + 'external': True, + 'name': 'some_foo', + }, + 'bar': { + 'external': True, + 'name': 'some_bar', + } + } + + def test_config_external_network_v3_5(self): + self.base_dir = 'tests/fixtures/networks' + result = self.dispatch(['-f', 'external-networks-v3-5.yml', 'config']) + json_result = yaml.safe_load(result.stdout) + assert 'networks' in json_result + assert json_result['networks'] == { + 'foo': { + 'external': True, + 'name': 'some_foo', + }, + 'bar': { + 'external': True, + 'name': 'some_bar', + }, + } + + def test_config_v1(self): + self.base_dir = 'tests/fixtures/v1-config' + result = self.dispatch(['config']) + assert yaml.safe_load(result.stdout) == { + 'version': str(V1), + 'services': { + 'net': { + 'image': 'busybox', + 'network_mode': 'bridge', + }, + 'volume': { + 'image': 'busybox', + 'volumes': ['/data'], + 'network_mode': 'bridge', + }, + 'app': { + 'image': 'busybox', + 'volumes_from': ['service:volume:rw'], + 'network_mode': 'service:net', + }, + }, + } + + def test_config_v3(self): + self.base_dir = 'tests/fixtures/v3-full' + result = self.dispatch(['config']) + assert yaml.safe_load(result.stdout) == { + 'version': '3.5', + 'volumes': { + 'foobar': { + 'labels': { + 'com.docker.compose.test': 'true', + }, + }, + }, + 'services': { + 'web': { + 'image': 'busybox', + 'deploy': { + 'mode': 'replicated', + 'replicas': 6, + 'labels': ['FOO=BAR'], + 'update_config': { + 'parallelism': 3, + 'delay': '10s', + 'failure_action': 'continue', + 'monitor': '60s', + 'max_failure_ratio': 0.3, + }, + 'resources': { + 'limits': { + 'cpus': 0.05, + 'memory': '50M', + }, + 'reservations': { + 'cpus': 0.01, + 'memory': '20M', + }, + }, + 'restart_policy': { + 'condition': 'on-failure', + 'delay': '5s', + 'max_attempts': 3, + 'window': '120s', + }, + 'placement': { + 'constraints': [ + 'node.hostname==foo', 'node.role != manager' + ], + 'preferences': [{'spread': 'node.labels.datacenter'}] + }, + }, + + 'healthcheck': { + 'test': 'cat /etc/passwd', + 'interval': '10s', + 'timeout': '1s', + 'retries': 5, + }, + 'volumes': [{ + 'read_only': True, + 'source': '/host/path', + 'target': '/container/path', + 'type': 'bind' + }, { + 'source': 'foobar', 'target': '/container/volumepath', 'type': 'volume' + }, { + 'target': '/anonymous', 'type': 'volume' + }, { + 'source': 'foobar', + 'target': '/container/volumepath2', + 'type': 'volume', + 'volume': {'nocopy': True} + }], + 'stop_grace_period': '20s', + }, + }, + } + + @pytest.mark.skip(reason='deprecated option') + def test_config_compatibility_mode(self): + self.base_dir = 'tests/fixtures/compatibility-mode' + result = self.dispatch(['--compatibility', 'config']) + + assert yaml.load(result.stdout) == COMPOSE_COMPATIBILITY_DICT + + @pytest.mark.skip(reason='deprecated option') + @mock.patch.dict(os.environ) + def test_config_compatibility_mode_from_env(self): + self.base_dir = 'tests/fixtures/compatibility-mode' + os.environ['COMPOSE_COMPATIBILITY'] = 'true' + result = self.dispatch(['config']) + + assert yaml.load(result.stdout) == COMPOSE_COMPATIBILITY_DICT + + @pytest.mark.skip(reason='deprecated option') + @mock.patch.dict(os.environ) + def test_config_compatibility_mode_from_env_and_option_precedence(self): + self.base_dir = 'tests/fixtures/compatibility-mode' + os.environ['COMPOSE_COMPATIBILITY'] = 'false' + result = self.dispatch(['--compatibility', 'config']) + + assert yaml.load(result.stdout) == COMPOSE_COMPATIBILITY_DICT + + def test_ps(self): + self.project.get_service('simple').create_container() + result = self.dispatch(['ps']) + assert 'simple-composefile_simple_1' in result.stdout + + def test_ps_default_composefile(self): + self.base_dir = 'tests/fixtures/multiple-composefiles' + self.dispatch(['up', '-d']) + result = self.dispatch(['ps']) + + assert 'multiple-composefiles_simple_1' in result.stdout + assert 'multiple-composefiles_another_1' in result.stdout + assert 'multiple-composefiles_yetanother_1' not in result.stdout + + def test_ps_alternate_composefile(self): + config_path = os.path.abspath( + 'tests/fixtures/multiple-composefiles/compose2.yml') + self._project = get_project(self.base_dir, [config_path]) + + self.base_dir = 'tests/fixtures/multiple-composefiles' + self.dispatch(['-f', 'compose2.yml', 'up', '-d']) + result = self.dispatch(['-f', 'compose2.yml', 'ps']) + + assert 'multiple-composefiles_simple_1' not in result.stdout + assert 'multiple-composefiles_another_1' not in result.stdout + assert 'multiple-composefiles_yetanother_1' in result.stdout + + def test_ps_services_filter_option(self): + self.base_dir = 'tests/fixtures/ps-services-filter' + image = self.dispatch(['ps', '--services', '--filter', 'source=image']) + build = self.dispatch(['ps', '--services', '--filter', 'source=build']) + all_services = self.dispatch(['ps', '--services']) + + assert 'with_build' in all_services.stdout + assert 'with_image' in all_services.stdout + assert 'with_build' in build.stdout + assert 'with_build' not in image.stdout + assert 'with_image' in image.stdout + assert 'with_image' not in build.stdout + + def test_ps_services_filter_status(self): + self.base_dir = 'tests/fixtures/ps-services-filter' + self.dispatch(['up', '-d']) + self.dispatch(['pause', 'with_image']) + paused = self.dispatch(['ps', '--services', '--filter', 'status=paused']) + stopped = self.dispatch(['ps', '--services', '--filter', 'status=stopped']) + running = self.dispatch(['ps', '--services', '--filter', 'status=running']) + + assert 'with_build' not in stopped.stdout + assert 'with_image' not in stopped.stdout + assert 'with_build' not in paused.stdout + assert 'with_image' in paused.stdout + assert 'with_build' in running.stdout + assert 'with_image' in running.stdout + + def test_ps_all(self): + self.project.get_service('simple').create_container(one_off='blahblah') + result = self.dispatch(['ps']) + assert 'simple-composefile_simple_run_' not in result.stdout + + result2 = self.dispatch(['ps', '--all']) + assert 'simple-composefile_simple_run_' in result2.stdout + + def test_pull(self): + result = self.dispatch(['pull']) + assert 'Pulling simple' in result.stderr + assert 'Pulling another' in result.stderr + assert 'done' in result.stderr + assert 'failed' not in result.stderr + + def test_pull_with_digest(self): + result = self.dispatch(['-f', 'digest.yml', 'pull', '--no-parallel']) + + assert 'Pulling simple ({})...'.format(BUSYBOX_IMAGE_WITH_TAG) in result.stderr + assert ('Pulling digest (busybox@' + 'sha256:38a203e1986cf79639cfb9b2e1d6e773de84002feea2d4eb006b520' + '04ee8502d)...') in result.stderr + + def test_pull_with_ignore_pull_failures(self): + result = self.dispatch([ + '-f', 'ignore-pull-failures.yml', + 'pull', '--ignore-pull-failures', '--no-parallel'] + ) + + assert 'Pulling simple ({})...'.format(BUSYBOX_IMAGE_WITH_TAG) in result.stderr + assert 'Pulling another (nonexisting-image:latest)...' in result.stderr + assert ('repository nonexisting-image not found' in result.stderr or + 'image library/nonexisting-image:latest not found' in result.stderr or + 'pull access denied for nonexisting-image' in result.stderr) + + def test_pull_with_quiet(self): + assert self.dispatch(['pull', '--quiet']).stderr == '' + assert self.dispatch(['pull', '--quiet']).stdout == '' + + def test_pull_with_parallel_failure(self): + result = self.dispatch([ + '-f', 'ignore-pull-failures.yml', 'pull'], + returncode=1 + ) + + assert re.search(re.compile('^Pulling simple', re.MULTILINE), result.stderr) + assert re.search(re.compile('^Pulling another', re.MULTILINE), result.stderr) + assert re.search( + re.compile('^ERROR: for another .*does not exist.*', re.MULTILINE), + result.stderr + ) + assert re.search( + re.compile('''^(ERROR: )?(b')?.* nonexisting-image''', re.MULTILINE), + result.stderr + ) + + def test_pull_can_build(self): + result = self.dispatch([ + '-f', 'can-build-pull-failures.yml', 'pull'], + returncode=0 + ) + assert 'Some service image(s) must be built from source' in result.stderr + assert 'docker-compose build can_build' in result.stderr + + def test_pull_with_no_deps(self): + self.base_dir = 'tests/fixtures/links-composefile' + result = self.dispatch(['pull', '--no-parallel', 'web']) + assert sorted(result.stderr.split('\n'))[1:] == [ + 'Pulling web (busybox:1.27.2)...', + ] + + def test_pull_with_include_deps(self): + self.base_dir = 'tests/fixtures/links-composefile' + result = self.dispatch(['pull', '--no-parallel', '--include-deps', 'web']) + assert sorted(result.stderr.split('\n'))[1:] == [ + 'Pulling db (busybox:1.27.2)...', + 'Pulling web (busybox:1.27.2)...', + ] + + def test_build_plain(self): + self.base_dir = 'tests/fixtures/simple-dockerfile' + self.dispatch(['build', 'simple']) + + result = self.dispatch(['build', 'simple']) + assert BUILD_PULL_TEXT not in result.stdout + + def test_build_no_cache(self): + self.base_dir = 'tests/fixtures/simple-dockerfile' + self.dispatch(['build', 'simple']) + + result = self.dispatch(['build', '--no-cache', 'simple']) + assert BUILD_CACHE_TEXT not in result.stdout + assert BUILD_PULL_TEXT not in result.stdout + + def test_up_ignore_missing_build_directory(self): + self.base_dir = 'tests/fixtures/no-build' + result = self.dispatch(['up', '--no-build']) + + assert 'alpine exited with code 0' in result.stdout + self.base_dir = None + + def test_pull_ignore_missing_build_directory(self): + self.base_dir = 'tests/fixtures/no-build' + result = self.dispatch(['pull']) + + assert 'Pulling my-alpine' in result.stderr + self.base_dir = None + + def test_build_pull(self): + # Make sure we have the latest busybox already + pull_busybox(self.client) + self.base_dir = 'tests/fixtures/simple-dockerfile' + self.dispatch(['build', 'simple'], None) + + result = self.dispatch(['build', '--pull', 'simple']) + if not is_cluster(self.client): + # If previous build happened on another node, cache won't be available + assert BUILD_CACHE_TEXT in result.stdout + assert BUILD_PULL_TEXT in result.stdout + + def test_build_no_cache_pull(self): + # Make sure we have the latest busybox already + pull_busybox(self.client) + self.base_dir = 'tests/fixtures/simple-dockerfile' + self.dispatch(['build', 'simple']) + + result = self.dispatch(['build', '--no-cache', '--pull', 'simple']) + assert BUILD_CACHE_TEXT not in result.stdout + assert BUILD_PULL_TEXT in result.stdout + + @mock.patch.dict(os.environ) + def test_build_log_level(self): + os.environ['COMPOSE_DOCKER_CLI_BUILD'] = '0' + os.environ['DOCKER_BUILDKIT'] = '0' + self.test_env_file_relative_to_compose_file() + self.base_dir = 'tests/fixtures/simple-dockerfile' + result = self.dispatch(['--log-level', 'warning', 'build', 'simple']) + assert result.stderr == '' + result = self.dispatch(['--log-level', 'debug', 'build', 'simple']) + assert 'Building simple' in result.stderr + assert 'Using configuration file' in result.stderr + self.base_dir = 'tests/fixtures/simple-failing-dockerfile' + result = self.dispatch(['--log-level', 'critical', 'build', 'simple'], returncode=1) + assert result.stderr == '' + result = self.dispatch(['--log-level', 'debug', 'build', 'simple'], returncode=1) + assert 'Building simple' in result.stderr + assert 'non-zero code' in result.stderr + + def test_build_failed(self): + self.base_dir = 'tests/fixtures/simple-failing-dockerfile' + self.dispatch(['build', 'simple'], returncode=1) + + labels = ["com.docker.compose.test_failing_image=true"] + containers = [ + Container.from_ps(self.project.client, c) + for c in self.project.client.containers( + all=True, + filters={"label": labels}) + ] + assert len(containers) == 1 + + def test_build_failed_forcerm(self): + self.base_dir = 'tests/fixtures/simple-failing-dockerfile' + self.dispatch(['build', '--force-rm', 'simple'], returncode=1) + + labels = ["com.docker.compose.test_failing_image=true"] + + containers = [ + Container.from_ps(self.project.client, c) + for c in self.project.client.containers( + all=True, + filters={"label": labels}) + ] + assert not containers + + @pytest.mark.xfail(True, reason='Flaky on local') + def test_build_rm(self): + containers = [ + Container.from_ps(self.project.client, c) + for c in self.project.client.containers(all=True) + ] + + assert not containers + + self.base_dir = 'tests/fixtures/simple-dockerfile' + self.dispatch(['build', '--no-rm', 'simple'], returncode=0) + + containers = [ + Container.from_ps(self.project.client, c) + for c in self.project.client.containers(all=True) + ] + assert containers + + for c in self.project.client.containers(all=True): + self.addCleanup(self.project.client.remove_container, c, force=True) + + @mock.patch.dict(os.environ) + def test_build_shm_size_build_option(self): + os.environ['COMPOSE_DOCKER_CLI_BUILD'] = '0' + pull_busybox(self.client) + self.base_dir = 'tests/fixtures/build-shm-size' + result = self.dispatch(['build', '--no-cache'], None) + assert 'shm_size: 96' in result.stdout + + @mock.patch.dict(os.environ) + def test_build_memory_build_option(self): + os.environ['COMPOSE_DOCKER_CLI_BUILD'] = '0' + pull_busybox(self.client) + self.base_dir = 'tests/fixtures/build-memory' + result = self.dispatch(['build', '--no-cache', '--memory', '96m', 'service'], None) + assert 'memory: 100663296' in result.stdout # 96 * 1024 * 1024 + + def test_build_with_buildarg_from_compose_file(self): + pull_busybox(self.client) + self.base_dir = 'tests/fixtures/build-args' + result = self.dispatch(['build'], None) + assert 'Favorite Touhou Character: mariya.kirisame' in result.stdout + + def test_build_with_buildarg_cli_override(self): + pull_busybox(self.client) + self.base_dir = 'tests/fixtures/build-args' + result = self.dispatch(['build', '--build-arg', 'favorite_th_character=sakuya.izayoi'], None) + assert 'Favorite Touhou Character: sakuya.izayoi' in result.stdout + + @mock.patch.dict(os.environ) + def test_build_with_buildarg_old_api_version(self): + pull_busybox(self.client) + self.base_dir = 'tests/fixtures/build-args' + os.environ['COMPOSE_API_VERSION'] = '1.24' + result = self.dispatch( + ['build', '--build-arg', 'favorite_th_character=reimu.hakurei'], None, returncode=1 + ) + assert '--build-arg is only supported when services are specified' in result.stderr + + result = self.dispatch( + ['build', '--build-arg', 'favorite_th_character=hong.meiling', 'web'], None + ) + assert 'Favorite Touhou Character: hong.meiling' in result.stdout + + def test_build_override_dir(self): + self.base_dir = 'tests/fixtures/build-path-override-dir' + self.override_dir = os.path.abspath('tests/fixtures') + result = self.dispatch([ + '--project-directory', self.override_dir, + 'build']) + + assert 'Successfully built' in result.stdout + + def test_build_override_dir_invalid_path(self): + config_path = os.path.abspath('tests/fixtures/build-path-override-dir/docker-compose.yml') + result = self.dispatch([ + '-f', config_path, + 'build'], returncode=1) + + assert 'does not exist, is not accessible, or is not a valid URL' in result.stderr + + def test_build_parallel(self): + self.base_dir = 'tests/fixtures/build-multiple-composefile' + result = self.dispatch(['build', '--parallel']) + assert 'Successfully tagged build-multiple-composefile_a:latest' in result.stdout + assert 'Successfully tagged build-multiple-composefile_b:latest' in result.stdout + assert 'Successfully built' in result.stdout + + def test_create(self): + self.dispatch(['create']) + service = self.project.get_service('simple') + another = self.project.get_service('another') + service_containers = service.containers(stopped=True) + another_containers = another.containers(stopped=True) + assert len(service_containers) == 1 + assert len(another_containers) == 1 + assert not service_containers[0].is_running + assert not another_containers[0].is_running + + def test_create_with_force_recreate(self): + self.dispatch(['create'], None) + service = self.project.get_service('simple') + service_containers = service.containers(stopped=True) + assert len(service_containers) == 1 + assert not service_containers[0].is_running + + old_ids = [c.id for c in service.containers(stopped=True)] + + self.dispatch(['create', '--force-recreate'], None) + service_containers = service.containers(stopped=True) + assert len(service_containers) == 1 + assert not service_containers[0].is_running + + new_ids = [c.id for c in service_containers] + + assert old_ids != new_ids + + def test_create_with_no_recreate(self): + self.dispatch(['create'], None) + service = self.project.get_service('simple') + service_containers = service.containers(stopped=True) + assert len(service_containers) == 1 + assert not service_containers[0].is_running + + old_ids = [c.id for c in service.containers(stopped=True)] + + self.dispatch(['create', '--no-recreate'], None) + service_containers = service.containers(stopped=True) + assert len(service_containers) == 1 + assert not service_containers[0].is_running + + new_ids = [c.id for c in service_containers] + + assert old_ids == new_ids + + def test_run_one_off_with_volume(self): + self.base_dir = 'tests/fixtures/simple-composefile-volume-ready' + volume_path = os.path.abspath(os.path.join(os.getcwd(), self.base_dir, 'files')) + node = create_host_file(self.client, os.path.join(volume_path, 'example.txt')) + + self.dispatch([ + 'run', + '-v', '{}:/data'.format(volume_path), + '-e', 'constraint:node=={}'.format(node if node is not None else '*'), + 'simple', + 'test', '-f', '/data/example.txt' + ], returncode=0) + + service = self.project.get_service('simple') + container_data = service.containers(one_off=OneOffFilter.only, stopped=True)[0] + mount = container_data.get('Mounts')[0] + assert mount['Source'] == volume_path + assert mount['Destination'] == '/data' + assert mount['Type'] == 'bind' + + def test_run_one_off_with_multiple_volumes(self): + self.base_dir = 'tests/fixtures/simple-composefile-volume-ready' + volume_path = os.path.abspath(os.path.join(os.getcwd(), self.base_dir, 'files')) + node = create_host_file(self.client, os.path.join(volume_path, 'example.txt')) + + self.dispatch([ + 'run', + '-v', '{}:/data'.format(volume_path), + '-v', '{}:/data1'.format(volume_path), + '-e', 'constraint:node=={}'.format(node if node is not None else '*'), + 'simple', + 'test', '-f', '/data/example.txt' + ], returncode=0) + + self.dispatch([ + 'run', + '-v', '{}:/data'.format(volume_path), + '-v', '{}:/data1'.format(volume_path), + '-e', 'constraint:node=={}'.format(node if node is not None else '*'), + 'simple', + 'test', '-f' '/data1/example.txt' + ], returncode=0) + + def test_run_one_off_with_volume_merge(self): + self.base_dir = 'tests/fixtures/simple-composefile-volume-ready' + volume_path = os.path.abspath(os.path.join(os.getcwd(), self.base_dir, 'files')) + node = create_host_file(self.client, os.path.join(volume_path, 'example.txt')) + + self.dispatch([ + '-f', 'docker-compose.merge.yml', + 'run', + '-v', '{}:/data'.format(volume_path), + '-e', 'constraint:node=={}'.format(node if node is not None else '*'), + 'simple', + 'test', '-f', '/data/example.txt' + ], returncode=0) + + service = self.project.get_service('simple') + container_data = service.containers(one_off=OneOffFilter.only, stopped=True)[0] + mounts = container_data.get('Mounts') + assert len(mounts) == 2 + config_mount = [m for m in mounts if m['Destination'] == '/data1'][0] + override_mount = [m for m in mounts if m['Destination'] == '/data'][0] + + assert config_mount['Type'] == 'volume' + assert override_mount['Source'] == volume_path + assert override_mount['Type'] == 'bind' + + def test_create_with_force_recreate_and_no_recreate(self): + self.dispatch( + ['create', '--force-recreate', '--no-recreate'], + returncode=1) + + def test_down_invalid_rmi_flag(self): + result = self.dispatch(['down', '--rmi', 'bogus'], returncode=1) + assert '--rmi flag must be' in result.stderr + + def test_down(self): + self.base_dir = 'tests/fixtures/v2-full' + + self.dispatch(['up', '-d']) + wait_on_condition(ContainerCountCondition(self.project, 2)) + + self.dispatch(['run', 'web', 'true']) + self.dispatch(['run', '-d', 'web', 'tail', '-f', '/dev/null']) + assert len(self.project.containers(one_off=OneOffFilter.only, stopped=True)) == 2 + + result = self.dispatch(['down', '--rmi=local', '--volumes']) + assert 'Stopping v2-full_web_1' in result.stderr + assert 'Stopping v2-full_other_1' in result.stderr + assert 'Stopping v2-full_web_run_' in result.stderr + assert 'Removing v2-full_web_1' in result.stderr + assert 'Removing v2-full_other_1' in result.stderr + assert 'Removing v2-full_web_run_' in result.stderr + assert 'Removing v2-full_web_run_' in result.stderr + assert 'Removing volume v2-full_data' in result.stderr + assert 'Removing image v2-full_web' in result.stderr + assert 'Removing image busybox' not in result.stderr + assert 'Removing network v2-full_default' in result.stderr + assert 'Removing network v2-full_front' in result.stderr + + def test_down_timeout(self): + self.dispatch(['up', '-d'], None) + service = self.project.get_service('simple') + assert len(service.containers()) == 1 + assert service.containers()[0].is_running + "" + + self.dispatch(['down', '-t', '1'], None) + + assert len(service.containers(stopped=True)) == 0 + + def test_down_signal(self): + self.base_dir = 'tests/fixtures/stop-signal-composefile' + self.dispatch(['up', '-d'], None) + service = self.project.get_service('simple') + assert len(service.containers()) == 1 + assert service.containers()[0].is_running + + self.dispatch(['down', '-t', '1'], None) + assert len(service.containers(stopped=True)) == 0 + + def test_up_detached(self): + self.dispatch(['up', '-d']) + service = self.project.get_service('simple') + another = self.project.get_service('another') + assert len(service.containers()) == 1 + assert len(another.containers()) == 1 + + # Ensure containers don't have stdin and stdout connected in -d mode + container, = service.containers() + assert not container.get('Config.AttachStderr') + assert not container.get('Config.AttachStdout') + assert not container.get('Config.AttachStdin') + + def test_up_detached_long_form(self): + self.dispatch(['up', '--detach']) + service = self.project.get_service('simple') + another = self.project.get_service('another') + assert len(service.containers()) == 1 + assert len(another.containers()) == 1 + + # Ensure containers don't have stdin and stdout connected in -d mode + container, = service.containers() + assert not container.get('Config.AttachStderr') + assert not container.get('Config.AttachStdout') + assert not container.get('Config.AttachStdin') + + def test_up_attached(self): + self.base_dir = 'tests/fixtures/echo-services' + result = self.dispatch(['up', '--no-color']) + simple_name = self.project.get_service('simple').containers(stopped=True)[0].name_without_project + another_name = self.project.get_service('another').containers( + stopped=True + )[0].name_without_project + + assert '{} | simple'.format(simple_name) in result.stdout + assert '{} | another'.format(another_name) in result.stdout + assert '{} exited with code 0'.format(simple_name) in result.stdout + assert '{} exited with code 0'.format(another_name) in result.stdout + + def test_up(self): + self.base_dir = 'tests/fixtures/v2-simple' + self.dispatch(['up', '-d'], None) + + services = self.project.get_services() + + network_name = self.project.networks.networks['default'].full_name + networks = self.client.networks(names=[network_name]) + assert len(networks) == 1 + assert networks[0]['Driver'] == 'bridge' if not is_cluster(self.client) else 'overlay' + assert 'com.docker.network.bridge.enable_icc' not in networks[0]['Options'] + + network = self.client.inspect_network(networks[0]['Id']) + + for service in services: + containers = service.containers() + assert len(containers) == 1 + + container = containers[0] + assert container.id in network['Containers'] + + networks = container.get('NetworkSettings.Networks') + assert list(networks) == [network['Name']] + + assert sorted(networks[network['Name']]['Aliases']) == sorted( + [service.name, container.short_id] + ) + + for service in services: + assert self.lookup(container, service.name) + + def test_up_no_start(self): + self.base_dir = 'tests/fixtures/v2-full' + self.dispatch(['up', '--no-start'], None) + + services = self.project.get_services() + + default_network = self.project.networks.networks['default'].full_name + front_network = self.project.networks.networks['front'].full_name + networks = self.client.networks(names=[default_network, front_network]) + assert len(networks) == 2 + + for service in services: + containers = service.containers(stopped=True) + assert len(containers) == 1 + + container = containers[0] + assert not container.is_running + assert container.get('State.Status') == 'created' + + volumes = self.project.volumes.volumes + assert 'data' in volumes + volume = volumes['data'] + + # The code below is a Swarm-compatible equivalent to volume.exists() + remote_volumes = [ + v for v in self.client.volumes().get('Volumes', []) + if v['Name'].split('/')[-1] == volume.full_name + ] + assert len(remote_volumes) > 0 + + def test_up_no_start_remove_orphans(self): + self.base_dir = 'tests/fixtures/v2-simple' + self.dispatch(['up', '--no-start'], None) + + services = self.project.get_services() + + stopped = reduce((lambda prev, next: prev.containers( + stopped=True) + next.containers(stopped=True)), services) + assert len(stopped) == 2 + + self.dispatch(['-f', 'one-container.yml', 'up', '--no-start', '--remove-orphans'], None) + stopped2 = reduce((lambda prev, next: prev.containers( + stopped=True) + next.containers(stopped=True)), services) + assert len(stopped2) == 1 + + def test_up_no_ansi(self): + self.base_dir = 'tests/fixtures/v2-simple' + result = self.dispatch(['--no-ansi', 'up', '-d'], None) + assert "%c[2K\r" % 27 not in result.stderr + assert "%c[1A" % 27 not in result.stderr + assert "%c[1B" % 27 not in result.stderr + + def test_up_with_default_network_config(self): + filename = 'default-network-config.yml' + + self.base_dir = 'tests/fixtures/networks' + self._project = get_project(self.base_dir, [filename]) + + self.dispatch(['-f', filename, 'up', '-d'], None) + + network_name = self.project.networks.networks['default'].full_name + networks = self.client.networks(names=[network_name]) + + assert networks[0]['Options']['com.docker.network.bridge.enable_icc'] == 'false' + + def test_up_with_network_aliases(self): + filename = 'network-aliases.yml' + self.base_dir = 'tests/fixtures/networks' + self.dispatch(['-f', filename, 'up', '-d'], None) + back_name = '{}_back'.format(self.project.name) + front_name = '{}_front'.format(self.project.name) + + networks = [ + n for n in self.client.networks() + if n['Name'].split('/')[-1].startswith('{}_'.format(self.project.name)) + ] + + # Two networks were created: back and front + assert sorted(n['Name'].split('/')[-1] for n in networks) == [back_name, front_name] + web_container = self.project.get_service('web').containers()[0] + + back_aliases = web_container.get( + 'NetworkSettings.Networks.{}.Aliases'.format(back_name) + ) + assert 'web' in back_aliases + front_aliases = web_container.get( + 'NetworkSettings.Networks.{}.Aliases'.format(front_name) + ) + assert 'web' in front_aliases + assert 'forward_facing' in front_aliases + assert 'ahead' in front_aliases + + def test_up_with_network_internal(self): + self.require_api_version('1.23') + filename = 'network-internal.yml' + self.base_dir = 'tests/fixtures/networks' + self.dispatch(['-f', filename, 'up', '-d'], None) + internal_net = '{}_internal'.format(self.project.name) + + networks = [ + n for n in self.client.networks() + if n['Name'].split('/')[-1].startswith('{}_'.format(self.project.name)) + ] + + # One network was created: internal + assert sorted(n['Name'].split('/')[-1] for n in networks) == [internal_net] + + assert networks[0]['Internal'] is True + + def test_up_with_network_static_addresses(self): + filename = 'network-static-addresses.yml' + ipv4_address = '172.16.100.100' + ipv6_address = 'fe80::1001:100' + self.base_dir = 'tests/fixtures/networks' + self.dispatch(['-f', filename, 'up', '-d'], None) + static_net = '{}_static_test'.format(self.project.name) + + networks = [ + n for n in self.client.networks() + if n['Name'].split('/')[-1].startswith('{}_'.format(self.project.name)) + ] + + # One networks was created: front + assert sorted(n['Name'].split('/')[-1] for n in networks) == [static_net] + web_container = self.project.get_service('web').containers()[0] + + ipam_config = web_container.get( + 'NetworkSettings.Networks.{}.IPAMConfig'.format(static_net) + ) + assert ipv4_address in ipam_config.values() + assert ipv6_address in ipam_config.values() + + def test_up_with_networks(self): + self.base_dir = 'tests/fixtures/networks' + self.dispatch(['up', '-d'], None) + + back_name = '{}_back'.format(self.project.name) + front_name = '{}_front'.format(self.project.name) + + networks = [ + n for n in self.client.networks() + if n['Name'].split('/')[-1].startswith('{}_'.format(self.project.name)) + ] + + # Two networks were created: back and front + assert sorted(n['Name'].split('/')[-1] for n in networks) == [back_name, front_name] + + # lookup by ID instead of name in case of duplicates + back_network = self.client.inspect_network( + [n for n in networks if n['Name'] == back_name][0]['Id'] + ) + front_network = self.client.inspect_network( + [n for n in networks if n['Name'] == front_name][0]['Id'] + ) + + web_container = self.project.get_service('web').containers()[0] + app_container = self.project.get_service('app').containers()[0] + db_container = self.project.get_service('db').containers()[0] + + for net_name in [front_name, back_name]: + links = app_container.get('NetworkSettings.Networks.{}.Links'.format(net_name)) + assert '{}:database'.format(db_container.name) in links + + # db and app joined the back network + assert sorted(back_network['Containers']) == sorted([db_container.id, app_container.id]) + + # web and app joined the front network + assert sorted(front_network['Containers']) == sorted([web_container.id, app_container.id]) + + # web can see app but not db + assert self.lookup(web_container, "app") + assert not self.lookup(web_container, "db") + + # app can see db + assert self.lookup(app_container, "db") + + # app has aliased db to "database" + assert self.lookup(app_container, "database") + + def test_up_missing_network(self): + self.base_dir = 'tests/fixtures/networks' + + result = self.dispatch( + ['-f', 'missing-network.yml', 'up', '-d'], + returncode=1) + + assert 'Service "web" uses an undefined network "foo"' in result.stderr + + @no_cluster('container networks not supported in Swarm') + def test_up_with_network_mode(self): + c = self.client.create_container( + 'busybox', 'top', name='composetest_network_mode_container', + host_config={} + ) + self.addCleanup(self.client.remove_container, c, force=True) + self.client.start(c) + container_mode_source = 'container:{}'.format(c['Id']) + + filename = 'network-mode.yml' + + self.base_dir = 'tests/fixtures/networks' + self._project = get_project(self.base_dir, [filename]) + + self.dispatch(['-f', filename, 'up', '-d'], None) + + networks = [ + n for n in self.client.networks() + if n['Name'].split('/')[-1].startswith('{}_'.format(self.project.name)) + ] + assert not networks + + for name in ['bridge', 'host', 'none']: + container = self.project.get_service(name).containers()[0] + assert list(container.get('NetworkSettings.Networks')) == [name] + assert container.get('HostConfig.NetworkMode') == name + + service_mode_source = 'container:{}'.format( + self.project.get_service('bridge').containers()[0].id) + service_mode_container = self.project.get_service('service').containers()[0] + assert not service_mode_container.get('NetworkSettings.Networks') + assert service_mode_container.get('HostConfig.NetworkMode') == service_mode_source + + container_mode_container = self.project.get_service('container').containers()[0] + assert not container_mode_container.get('NetworkSettings.Networks') + assert container_mode_container.get('HostConfig.NetworkMode') == container_mode_source + + def test_up_external_networks(self): + filename = 'external-networks.yml' + + self.base_dir = 'tests/fixtures/networks' + self._project = get_project(self.base_dir, [filename]) + + result = self.dispatch(['-f', filename, 'up', '-d'], returncode=1) + assert 'declared as external, but could not be found' in result.stderr + + networks = [ + n['Name'] for n in self.client.networks() + if n['Name'].startswith('{}_'.format(self.project.name)) + ] + assert not networks + + network_names = ['{}_{}'.format(self.project.name, n) for n in ['foo', 'bar']] + for name in network_names: + self.client.create_network(name, attachable=True) + + self.dispatch(['-f', filename, 'up', '-d']) + container = self.project.containers()[0] + assert sorted(list(container.get('NetworkSettings.Networks'))) == sorted(network_names) + + def test_up_with_external_default_network(self): + filename = 'external-default.yml' + + self.base_dir = 'tests/fixtures/networks' + self._project = get_project(self.base_dir, [filename]) + + result = self.dispatch(['-f', filename, 'up', '-d'], returncode=1) + assert 'declared as external, but could not be found' in result.stderr + + networks = [ + n['Name'] for n in self.client.networks() + if n['Name'].split('/')[-1].startswith('{}_'.format(self.project.name)) + ] + assert not networks + + network_name = 'composetest_external_network' + self.client.create_network(network_name, attachable=True) + + self.dispatch(['-f', filename, 'up', '-d']) + container = self.project.containers()[0] + assert list(container.get('NetworkSettings.Networks')) == [network_name] + + def test_up_with_network_labels(self): + filename = 'network-label.yml' + + self.base_dir = 'tests/fixtures/networks' + self._project = get_project(self.base_dir, [filename]) + + self.dispatch(['-f', filename, 'up', '-d'], returncode=0) + + network_with_label = '{}_network_with_label'.format(self.project.name) + + networks = [ + n for n in self.client.networks() + if n['Name'].split('/')[-1].startswith('{}_'.format(self.project.name)) + ] + + assert [n['Name'].split('/')[-1] for n in networks] == [network_with_label] + assert 'label_key' in networks[0]['Labels'] + assert networks[0]['Labels']['label_key'] == 'label_val' + + def test_up_with_volume_labels(self): + filename = 'volume-label.yml' + + self.base_dir = 'tests/fixtures/volumes' + self._project = get_project(self.base_dir, [filename]) + + self.dispatch(['-f', filename, 'up', '-d'], returncode=0) + + volume_with_label = '{}_volume_with_label'.format(self.project.name) + + volumes = [ + v for v in self.client.volumes().get('Volumes', []) + if v['Name'].split('/')[-1].startswith('{}_'.format(self.project.name)) + ] + + assert {v['Name'].split('/')[-1] for v in volumes} == {volume_with_label} + assert 'label_key' in volumes[0]['Labels'] + assert volumes[0]['Labels']['label_key'] == 'label_val' + + def test_up_no_services(self): + self.base_dir = 'tests/fixtures/no-services' + self.dispatch(['up', '-d'], None) + + network_names = [ + n['Name'] for n in self.client.networks() + if n['Name'].split('/')[-1].startswith('{}_'.format(self.project.name)) + ] + assert network_names == [] + + def test_up_with_links_v1(self): + self.base_dir = 'tests/fixtures/links-composefile' + self.dispatch(['up', '-d', 'web'], None) + + # No network was created + network_name = self.project.networks.networks['default'].full_name + networks = self.client.networks(names=[network_name]) + assert networks == [] + + web = self.project.get_service('web') + db = self.project.get_service('db') + console = self.project.get_service('console') + + # console was not started + assert len(web.containers()) == 1 + assert len(db.containers()) == 1 + assert len(console.containers()) == 0 + + # web has links + web_container = web.containers()[0] + assert web_container.get('HostConfig.Links') + + def test_up_with_net_is_invalid(self): + self.base_dir = 'tests/fixtures/net-container' + + result = self.dispatch( + ['-f', 'v2-invalid.yml', 'up', '-d'], + returncode=1) + + assert "Unsupported config option for services.bar: 'net'" in result.stderr + + @no_cluster("Legacy networking not supported on Swarm") + def test_up_with_net_v1(self): + self.base_dir = 'tests/fixtures/net-container' + self.dispatch(['up', '-d'], None) + + bar = self.project.get_service('bar') + bar_container = bar.containers()[0] + + foo = self.project.get_service('foo') + foo_container = foo.containers()[0] + + assert foo_container.get('HostConfig.NetworkMode') == 'container:{}'.format( + bar_container.id + ) + + def test_up_with_healthcheck(self): + def wait_on_health_status(container, status): + def condition(): + container.inspect() + return container.get('State.Health.Status') == status + + return wait_on_condition(condition, delay=0.5) + + self.base_dir = 'tests/fixtures/healthcheck' + self.dispatch(['up', '-d'], None) + + passes = self.project.get_service('passes') + passes_container = passes.containers()[0] + + assert passes_container.get('Config.Healthcheck') == { + "Test": ["CMD-SHELL", "/bin/true"], + "Interval": nanoseconds_from_time_seconds(1), + "Timeout": nanoseconds_from_time_seconds(30 * 60), + "Retries": 1, + } + + wait_on_health_status(passes_container, 'healthy') + + fails = self.project.get_service('fails') + fails_container = fails.containers()[0] + + assert fails_container.get('Config.Healthcheck') == { + "Test": ["CMD", "/bin/false"], + "Interval": nanoseconds_from_time_seconds(2.5), + "Retries": 2, + } + + wait_on_health_status(fails_container, 'unhealthy') + + disabled = self.project.get_service('disabled') + disabled_container = disabled.containers()[0] + + assert disabled_container.get('Config.Healthcheck') == { + "Test": ["NONE"], + } + + assert 'Health' not in disabled_container.get('State') + + def test_up_with_no_deps(self): + self.base_dir = 'tests/fixtures/links-composefile' + self.dispatch(['up', '-d', '--no-deps', 'web'], None) + web = self.project.get_service('web') + db = self.project.get_service('db') + console = self.project.get_service('console') + assert len(web.containers()) == 1 + assert len(db.containers()) == 0 + assert len(console.containers()) == 0 + + def test_up_with_attach_dependencies(self): + self.base_dir = 'tests/fixtures/echo-services-dependencies' + result = self.dispatch(['up', '--attach-dependencies', '--no-color', 'simple'], None) + simple_name = self.project.get_service('simple').containers(stopped=True)[0].name_without_project + another_name = self.project.get_service('another').containers( + stopped=True + )[0].name_without_project + + assert '{} | simple'.format(simple_name) in result.stdout + assert '{} | another'.format(another_name) in result.stdout + + def test_up_handles_aborted_dependencies(self): + self.base_dir = 'tests/fixtures/abort-on-container-exit-dependencies' + proc = start_process( + self.base_dir, + ['up', 'simple', '--attach-dependencies', '--abort-on-container-exit']) + wait_on_condition(ContainerCountCondition(self.project, 0)) + proc.wait() + assert proc.returncode == 1 + + def test_up_with_force_recreate(self): + self.dispatch(['up', '-d'], None) + service = self.project.get_service('simple') + assert len(service.containers()) == 1 + + old_ids = [c.id for c in service.containers()] + + self.dispatch(['up', '-d', '--force-recreate'], None) + assert len(service.containers()) == 1 + + new_ids = [c.id for c in service.containers()] + + assert old_ids != new_ids + + def test_up_with_no_recreate(self): + self.dispatch(['up', '-d'], None) + service = self.project.get_service('simple') + assert len(service.containers()) == 1 + + old_ids = [c.id for c in service.containers()] + + self.dispatch(['up', '-d', '--no-recreate'], None) + assert len(service.containers()) == 1 + + new_ids = [c.id for c in service.containers()] + + assert old_ids == new_ids + + def test_up_with_force_recreate_and_no_recreate(self): + self.dispatch( + ['up', '-d', '--force-recreate', '--no-recreate'], + returncode=1) + + def test_up_with_timeout(self): + self.dispatch(['up', '-d', '-t', '1']) + service = self.project.get_service('simple') + another = self.project.get_service('another') + assert len(service.containers()) == 1 + assert len(another.containers()) == 1 + + @mock.patch.dict(os.environ) + def test_up_with_ignore_remove_orphans(self): + os.environ["COMPOSE_IGNORE_ORPHANS"] = "True" + result = self.dispatch(['up', '-d', '--remove-orphans'], returncode=1) + assert "COMPOSE_IGNORE_ORPHANS and --remove-orphans cannot be combined." in result.stderr + + def test_up_handles_sigint(self): + proc = start_process(self.base_dir, ['up', '-t', '2']) + wait_on_condition(ContainerCountCondition(self.project, 2)) + + os.kill(proc.pid, signal.SIGINT) + wait_on_condition(ContainerCountCondition(self.project, 0)) + + def test_up_handles_sigterm(self): + proc = start_process(self.base_dir, ['up', '-t', '2']) + wait_on_condition(ContainerCountCondition(self.project, 2)) + + os.kill(proc.pid, signal.SIGTERM) + wait_on_condition(ContainerCountCondition(self.project, 0)) + + def test_up_handles_force_shutdown(self): + self.base_dir = 'tests/fixtures/sleeps-composefile' + proc = start_process(self.base_dir, ['up', '-t', '200']) + wait_on_condition(ContainerCountCondition(self.project, 2)) + + os.kill(proc.pid, signal.SIGTERM) + time.sleep(0.1) + os.kill(proc.pid, signal.SIGTERM) + wait_on_condition(ContainerCountCondition(self.project, 0)) + + def test_up_handles_abort_on_container_exit(self): + self.base_dir = 'tests/fixtures/abort-on-container-exit-0' + proc = start_process(self.base_dir, ['up', '--abort-on-container-exit']) + wait_on_condition(ContainerCountCondition(self.project, 0)) + proc.wait() + assert proc.returncode == 0 + + def test_up_handles_abort_on_container_exit_code(self): + self.base_dir = 'tests/fixtures/abort-on-container-exit-1' + proc = start_process(self.base_dir, ['up', '--abort-on-container-exit']) + wait_on_condition(ContainerCountCondition(self.project, 0)) + proc.wait() + assert proc.returncode == 1 + + @no_cluster('Container PID mode does not work across clusters') + def test_up_with_pid_mode(self): + c = self.client.create_container( + 'busybox', 'top', name='composetest_pid_mode_container', + host_config={} + ) + self.addCleanup(self.client.remove_container, c, force=True) + self.client.start(c) + container_mode_source = 'container:{}'.format(c['Id']) + + self.base_dir = 'tests/fixtures/pid-mode' + + self.dispatch(['up', '-d'], None) + + service_mode_source = 'container:{}'.format( + self.project.get_service('container').containers()[0].id) + service_mode_container = self.project.get_service('service').containers()[0] + assert service_mode_container.get('HostConfig.PidMode') == service_mode_source + + container_mode_container = self.project.get_service('container').containers()[0] + assert container_mode_container.get('HostConfig.PidMode') == container_mode_source + + host_mode_container = self.project.get_service('host').containers()[0] + assert host_mode_container.get('HostConfig.PidMode') == 'host' + + @no_cluster('Container IPC mode does not work across clusters') + def test_up_with_ipc_mode(self): + c = self.client.create_container( + 'busybox', 'top', name='composetest_ipc_mode_container', + host_config={} + ) + self.addCleanup(self.client.remove_container, c, force=True) + self.client.start(c) + container_mode_source = 'container:{}'.format(c['Id']) + + self.base_dir = 'tests/fixtures/ipc-mode' + + self.dispatch(['up', '-d'], None) + + service_mode_source = 'container:{}'.format( + self.project.get_service('shareable').containers()[0].id) + service_mode_container = self.project.get_service('service').containers()[0] + assert service_mode_container.get('HostConfig.IpcMode') == service_mode_source + + container_mode_container = self.project.get_service('container').containers()[0] + assert container_mode_container.get('HostConfig.IpcMode') == container_mode_source + + shareable_mode_container = self.project.get_service('shareable').containers()[0] + assert shareable_mode_container.get('HostConfig.IpcMode') == 'shareable' + + def test_profiles_up_with_no_profile(self): + self.base_dir = 'tests/fixtures/profiles' + self.dispatch(['up']) + + containers = self.project.containers(stopped=True) + service_names = [c.service for c in containers] + + assert 'foo' in service_names + assert len(containers) == 1 + + def test_profiles_up_with_profile(self): + self.base_dir = 'tests/fixtures/profiles' + self.dispatch(['--profile', 'test', 'up']) + + containers = self.project.containers(stopped=True) + service_names = [c.service for c in containers] + + assert 'foo' in service_names + assert 'bar' in service_names + assert 'baz' in service_names + assert len(containers) == 3 + + def test_profiles_up_invalid_dependency(self): + self.base_dir = 'tests/fixtures/profiles' + result = self.dispatch(['--profile', 'debug', 'up'], returncode=1) + + assert ('Service "bar" was pulled in as a dependency of service "zot" ' + 'but is not enabled by the active profiles.') in result.stderr + + def test_profiles_up_with_multiple_profiles(self): + self.base_dir = 'tests/fixtures/profiles' + self.dispatch(['--profile', 'debug', '--profile', 'test', 'up']) + + containers = self.project.containers(stopped=True) + service_names = [c.service for c in containers] + + assert 'foo' in service_names + assert 'bar' in service_names + assert 'baz' in service_names + assert 'zot' in service_names + assert len(containers) == 4 + + def test_profiles_up_with_profile_enabled_by_service(self): + self.base_dir = 'tests/fixtures/profiles' + self.dispatch(['up', 'bar']) + + containers = self.project.containers(stopped=True) + service_names = [c.service for c in containers] + + assert 'bar' in service_names + assert len(containers) == 1 + + def test_profiles_up_with_dependency_and_profile_enabled_by_service(self): + self.base_dir = 'tests/fixtures/profiles' + self.dispatch(['up', 'baz']) + + containers = self.project.containers(stopped=True) + service_names = [c.service for c in containers] + + assert 'bar' in service_names + assert 'baz' in service_names + assert len(containers) == 2 + + def test_profiles_up_with_invalid_dependency_for_target_service(self): + self.base_dir = 'tests/fixtures/profiles' + result = self.dispatch(['up', 'zot'], returncode=1) + + assert ('Service "bar" was pulled in as a dependency of service "zot" ' + 'but is not enabled by the active profiles.') in result.stderr + + def test_profiles_up_with_profile_for_dependency(self): + self.base_dir = 'tests/fixtures/profiles' + self.dispatch(['--profile', 'test', 'up', 'zot']) + + containers = self.project.containers(stopped=True) + service_names = [c.service for c in containers] + + assert 'bar' in service_names + assert 'zot' in service_names + assert len(containers) == 2 + + def test_profiles_up_with_merged_profiles(self): + self.base_dir = 'tests/fixtures/profiles' + self.dispatch(['-f', 'docker-compose.yml', '-f', 'merge-profiles.yml', 'up', 'zot']) + + containers = self.project.containers(stopped=True) + service_names = [c.service for c in containers] + + assert 'bar' in service_names + assert 'zot' in service_names + assert len(containers) == 2 + + def test_exec_without_tty(self): + self.base_dir = 'tests/fixtures/links-composefile' + self.dispatch(['up', '-d', 'console']) + assert len(self.project.containers()) == 1 + + stdout, stderr = self.dispatch(['exec', '-T', 'console', 'ls', '-1d', '/']) + assert stderr == "" + assert stdout == "/\n" + + @mock.patch.dict(os.environ) + def test_exec_novalue_var_dotenv_file(self): + os.environ['MYVAR'] = 'SUCCESS' + self.base_dir = 'tests/fixtures/exec-novalue-var' + self.dispatch(['up', '-d']) + assert len(self.project.containers()) == 1 + + stdout, stderr = self.dispatch(['exec', '-T', 'nginx', 'env']) + assert 'CHECK_VAR=SUCCESS' in stdout + assert not stderr + + def test_exec_detach_long_form(self): + self.base_dir = 'tests/fixtures/links-composefile' + self.dispatch(['up', '--detach', 'console']) + assert len(self.project.containers()) == 1 + + stdout, stderr = self.dispatch(['exec', '-T', 'console', 'ls', '-1d', '/']) + assert stderr == "" + assert stdout == "/\n" + + def test_exec_custom_user(self): + self.base_dir = 'tests/fixtures/links-composefile' + self.dispatch(['up', '-d', 'console']) + assert len(self.project.containers()) == 1 + + stdout, stderr = self.dispatch(['exec', '-T', '--user=operator', 'console', 'whoami']) + assert stdout == "operator\n" + assert stderr == "" + + def test_exec_workdir(self): + self.base_dir = 'tests/fixtures/links-composefile' + os.environ['COMPOSE_API_VERSION'] = '1.35' + self.dispatch(['up', '-d', 'console']) + assert len(self.project.containers()) == 1 + + stdout, stderr = self.dispatch(['exec', '-T', '--workdir', '/etc', 'console', 'ls']) + assert 'passwd' in stdout + + def test_exec_service_with_environment_overridden(self): + name = 'service' + self.base_dir = 'tests/fixtures/environment-exec' + self.dispatch(['up', '-d']) + assert len(self.project.containers()) == 1 + + stdout, stderr = self.dispatch([ + 'exec', + '-T', + '-e', 'foo=notbar', + '--env', 'alpha=beta', + name, + 'env', + ]) + + # env overridden + assert 'foo=notbar' in stdout + # keep environment from yaml + assert 'hello=world' in stdout + # added option from command line + assert 'alpha=beta' in stdout + + assert stderr == '' + + def test_run_service_without_links(self): + self.base_dir = 'tests/fixtures/links-composefile' + self.dispatch(['run', 'console', '/bin/true']) + assert len(self.project.containers()) == 0 + + # Ensure stdin/out was open + container = self.project.containers(stopped=True, one_off=OneOffFilter.only)[0] + config = container.inspect()['Config'] + assert config['AttachStderr'] + assert config['AttachStdout'] + assert config['AttachStdin'] + + def test_run_service_with_links(self): + self.base_dir = 'tests/fixtures/links-composefile' + self.dispatch(['run', 'web', '/bin/true'], None) + db = self.project.get_service('db') + console = self.project.get_service('console') + assert len(db.containers()) == 1 + assert len(console.containers()) == 0 + + def test_run_service_with_dependencies(self): + self.base_dir = 'tests/fixtures/v2-dependencies' + self.dispatch(['run', 'web', '/bin/true'], None) + db = self.project.get_service('db') + console = self.project.get_service('console') + assert len(db.containers()) == 1 + assert len(console.containers()) == 0 + + def test_run_service_with_unhealthy_dependencies(self): + self.base_dir = 'tests/fixtures/v2-unhealthy-dependencies' + result = self.dispatch(['run', 'web', '/bin/true'], returncode=1) + assert re.search( + re.compile('for web .*is unhealthy.*', re.MULTILINE), + result.stderr + ) + + def test_run_service_with_scaled_dependencies(self): + self.base_dir = 'tests/fixtures/v2-dependencies' + self.dispatch(['up', '-d', '--scale', 'db=2', '--scale', 'console=0']) + db = self.project.get_service('db') + console = self.project.get_service('console') + assert len(db.containers()) == 2 + assert len(console.containers()) == 0 + self.dispatch(['run', 'web', '/bin/true'], None) + assert len(db.containers()) == 2 + assert len(console.containers()) == 0 + + def test_run_with_no_deps(self): + self.base_dir = 'tests/fixtures/links-composefile' + self.dispatch(['run', '--no-deps', 'web', '/bin/true']) + db = self.project.get_service('db') + assert len(db.containers()) == 0 + + def test_run_does_not_recreate_linked_containers(self): + self.base_dir = 'tests/fixtures/links-composefile' + self.dispatch(['up', '-d', 'db']) + db = self.project.get_service('db') + assert len(db.containers()) == 1 + + old_ids = [c.id for c in db.containers()] + + self.dispatch(['run', 'web', '/bin/true'], None) + assert len(db.containers()) == 1 + + new_ids = [c.id for c in db.containers()] + + assert old_ids == new_ids + + def test_run_without_command(self): + self.base_dir = 'tests/fixtures/commands-composefile' + self.check_build('tests/fixtures/simple-dockerfile', tag='composetest_test') + + self.dispatch(['run', 'implicit']) + service = self.project.get_service('implicit') + containers = service.containers(stopped=True, one_off=OneOffFilter.only) + assert [c.human_readable_command for c in containers] == ['/bin/sh -c echo "success"'] + + self.dispatch(['run', 'explicit']) + service = self.project.get_service('explicit') + containers = service.containers(stopped=True, one_off=OneOffFilter.only) + assert [c.human_readable_command for c in containers] == ['/bin/true'] + + @pytest.mark.skipif(SWARM_SKIP_RM_VOLUMES, reason='Swarm DELETE /containers/ bug') + def test_run_rm(self): + self.base_dir = 'tests/fixtures/volume' + proc = start_process(self.base_dir, ['run', '--rm', 'test']) + service = self.project.get_service('test') + wait_on_condition(ContainerStateCondition( + self.project.client, + 'volume_test_run_*', + 'running') + ) + containers = service.containers(one_off=OneOffFilter.only) + assert len(containers) == 1 + mounts = containers[0].get('Mounts') + for mount in mounts: + if mount['Destination'] == '/container-path': + anonymous_name = mount['Name'] + break + os.kill(proc.pid, signal.SIGINT) + wait_on_process(proc, 1) + + assert len(service.containers(stopped=True, one_off=OneOffFilter.only)) == 0 + + volumes = self.client.volumes()['Volumes'] + assert volumes is not None + for volume in service.options.get('volumes'): + if volume.internal == '/container-named-path': + name = volume.external + break + volume_names = [v['Name'].split('/')[-1] for v in volumes] + assert name in volume_names + assert anonymous_name not in volume_names + + def test_run_service_with_dockerfile_entrypoint(self): + self.base_dir = 'tests/fixtures/entrypoint-dockerfile' + self.dispatch(['run', 'test']) + container = self.project.containers(stopped=True, one_off=OneOffFilter.only)[0] + assert container.get('Config.Entrypoint') == ['printf'] + assert container.get('Config.Cmd') == ['default', 'args'] + + def test_run_service_with_unset_entrypoint(self): + self.base_dir = 'tests/fixtures/entrypoint-dockerfile' + self.dispatch(['run', '--entrypoint=""', 'test', 'true']) + container = self.project.containers(stopped=True, one_off=OneOffFilter.only)[0] + assert container.get('Config.Entrypoint') is None + assert container.get('Config.Cmd') == ['true'] + + self.dispatch(['run', '--entrypoint', '""', 'test', 'true']) + container = self.project.containers(stopped=True, one_off=OneOffFilter.only)[0] + assert container.get('Config.Entrypoint') is None + assert container.get('Config.Cmd') == ['true'] + + def test_run_service_with_dockerfile_entrypoint_overridden(self): + self.base_dir = 'tests/fixtures/entrypoint-dockerfile' + self.dispatch(['run', '--entrypoint', 'echo', 'test']) + container = self.project.containers(stopped=True, one_off=OneOffFilter.only)[0] + assert container.get('Config.Entrypoint') == ['echo'] + assert not container.get('Config.Cmd') + + def test_run_service_with_dockerfile_entrypoint_and_command_overridden(self): + self.base_dir = 'tests/fixtures/entrypoint-dockerfile' + self.dispatch(['run', '--entrypoint', 'echo', 'test', 'foo']) + container = self.project.containers(stopped=True, one_off=OneOffFilter.only)[0] + assert container.get('Config.Entrypoint') == ['echo'] + assert container.get('Config.Cmd') == ['foo'] + + def test_run_service_with_compose_file_entrypoint(self): + self.base_dir = 'tests/fixtures/entrypoint-composefile' + self.dispatch(['run', 'test']) + container = self.project.containers(stopped=True, one_off=OneOffFilter.only)[0] + assert container.get('Config.Entrypoint') == ['printf'] + assert container.get('Config.Cmd') == ['default', 'args'] + + def test_run_service_with_compose_file_entrypoint_overridden(self): + self.base_dir = 'tests/fixtures/entrypoint-composefile' + self.dispatch(['run', '--entrypoint', 'echo', 'test']) + container = self.project.containers(stopped=True, one_off=OneOffFilter.only)[0] + assert container.get('Config.Entrypoint') == ['echo'] + assert not container.get('Config.Cmd') + + def test_run_service_with_compose_file_entrypoint_and_command_overridden(self): + self.base_dir = 'tests/fixtures/entrypoint-composefile' + self.dispatch(['run', '--entrypoint', 'echo', 'test', 'foo']) + container = self.project.containers(stopped=True, one_off=OneOffFilter.only)[0] + assert container.get('Config.Entrypoint') == ['echo'] + assert container.get('Config.Cmd') == ['foo'] + + def test_run_service_with_compose_file_entrypoint_and_empty_string_command(self): + self.base_dir = 'tests/fixtures/entrypoint-composefile' + self.dispatch(['run', '--entrypoint', 'echo', 'test', '']) + container = self.project.containers(stopped=True, one_off=OneOffFilter.only)[0] + assert container.get('Config.Entrypoint') == ['echo'] + assert container.get('Config.Cmd') == [''] + + def test_run_service_with_user_overridden(self): + self.base_dir = 'tests/fixtures/user-composefile' + name = 'service' + user = 'sshd' + self.dispatch(['run', '--user={user}'.format(user=user), name], returncode=1) + service = self.project.get_service(name) + container = service.containers(stopped=True, one_off=OneOffFilter.only)[0] + assert user == container.get('Config.User') + + def test_run_service_with_user_overridden_short_form(self): + self.base_dir = 'tests/fixtures/user-composefile' + name = 'service' + user = 'sshd' + self.dispatch(['run', '-u', user, name], returncode=1) + service = self.project.get_service(name) + container = service.containers(stopped=True, one_off=OneOffFilter.only)[0] + assert user == container.get('Config.User') + + def test_run_service_with_environment_overridden(self): + name = 'service' + self.base_dir = 'tests/fixtures/environment-composefile' + self.dispatch([ + 'run', '-e', 'foo=notbar', + '-e', 'allo=moto=bobo', + '-e', 'alpha=beta', + name, + '/bin/true', + ]) + service = self.project.get_service(name) + container = service.containers(stopped=True, one_off=OneOffFilter.only)[0] + # env overridden + assert 'notbar' == container.environment['foo'] + # keep environment from yaml + assert 'world' == container.environment['hello'] + # added option from command line + assert 'beta' == container.environment['alpha'] + # make sure a value with a = don't crash out + assert 'moto=bobo' == container.environment['allo'] + + def test_run_service_without_map_ports(self): + # create one off container + self.base_dir = 'tests/fixtures/ports-composefile' + self.dispatch(['run', '-d', 'simple']) + container = self.project.get_service('simple').containers(one_off=OneOffFilter.only)[0] + + # get port information + port_random = container.get_local_port(3000) + port_assigned = container.get_local_port(3001) + + # close all one off containers we just created + container.stop() + + # check the ports + assert port_random is None + assert port_assigned is None + + def test_run_service_with_map_ports(self): + # create one off container + self.base_dir = 'tests/fixtures/ports-composefile' + self.dispatch(['run', '-d', '--service-ports', 'simple']) + container = self.project.get_service('simple').containers(one_off=OneOffFilter.only)[0] + + # get port information + port_random = container.get_local_port(3000) + port_assigned = container.get_local_port(3001) + port_range = container.get_local_port(3002), container.get_local_port(3003) + + # close all one off containers we just created + container.stop() + + # check the ports + assert port_random is not None + assert port_assigned.endswith(':49152') + assert port_range[0].endswith(':49153') + assert port_range[1].endswith(':49154') + + def test_run_service_with_explicitly_mapped_ports(self): + # create one off container + self.base_dir = 'tests/fixtures/ports-composefile' + self.dispatch(['run', '-d', '-p', '30000:3000', '--publish', '30001:3001', 'simple']) + container = self.project.get_service('simple').containers(one_off=OneOffFilter.only)[0] + + # get port information + port_short = container.get_local_port(3000) + port_full = container.get_local_port(3001) + + # close all one off containers we just created + container.stop() + + # check the ports + assert port_short.endswith(':30000') + assert port_full.endswith(':30001') + + def test_run_service_with_explicitly_mapped_ip_ports(self): + # create one off container + self.base_dir = 'tests/fixtures/ports-composefile' + self.dispatch([ + 'run', '-d', + '-p', '127.0.0.1:30000:3000', + '--publish', '127.0.0.1:30001:3001', + 'simple' + ]) + container = self.project.get_service('simple').containers(one_off=OneOffFilter.only)[0] + + # get port information + port_short = container.get_local_port(3000) + port_full = container.get_local_port(3001) + + # close all one off containers we just created + container.stop() + + # check the ports + assert port_short == "127.0.0.1:30000" + assert port_full == "127.0.0.1:30001" + + def test_run_with_expose_ports(self): + # create one off container + self.base_dir = 'tests/fixtures/expose-composefile' + self.dispatch(['run', '-d', '--service-ports', 'simple']) + container = self.project.get_service('simple').containers(one_off=OneOffFilter.only)[0] + + ports = container.ports + assert len(ports) == 9 + # exposed ports are not mapped to host ports + assert ports['3000/tcp'] is None + assert ports['3001/tcp'] is None + assert ports['3001/udp'] is None + assert ports['3002/tcp'] is None + assert ports['3003/tcp'] is None + assert ports['3004/tcp'] is None + assert ports['3005/tcp'] is None + assert ports['3006/udp'] is None + assert ports['3007/udp'] is None + + # close all one off containers we just created + container.stop() + + def test_run_with_custom_name(self): + self.base_dir = 'tests/fixtures/environment-composefile' + name = 'the-container-name' + self.dispatch(['run', '--name', name, 'service', '/bin/true']) + + service = self.project.get_service('service') + container, = service.containers(stopped=True, one_off=OneOffFilter.only) + assert container.name == name + + def test_run_service_with_workdir_overridden(self): + self.base_dir = 'tests/fixtures/run-workdir' + name = 'service' + workdir = '/var' + self.dispatch(['run', '--workdir={workdir}'.format(workdir=workdir), name]) + service = self.project.get_service(name) + container = service.containers(stopped=True, one_off=True)[0] + assert workdir == container.get('Config.WorkingDir') + + def test_run_service_with_workdir_overridden_short_form(self): + self.base_dir = 'tests/fixtures/run-workdir' + name = 'service' + workdir = '/var' + self.dispatch(['run', '-w', workdir, name]) + service = self.project.get_service(name) + container = service.containers(stopped=True, one_off=True)[0] + assert workdir == container.get('Config.WorkingDir') + + def test_run_service_with_use_aliases(self): + filename = 'network-aliases.yml' + self.base_dir = 'tests/fixtures/networks' + self.dispatch(['-f', filename, 'run', '-d', '--use-aliases', 'web', 'top']) + + back_name = '{}_back'.format(self.project.name) + front_name = '{}_front'.format(self.project.name) + + web_container = self.project.get_service('web').containers(one_off=OneOffFilter.only)[0] + + back_aliases = web_container.get( + 'NetworkSettings.Networks.{}.Aliases'.format(back_name) + ) + assert 'web' in back_aliases + front_aliases = web_container.get( + 'NetworkSettings.Networks.{}.Aliases'.format(front_name) + ) + assert 'web' in front_aliases + assert 'forward_facing' in front_aliases + assert 'ahead' in front_aliases + + def test_run_interactive_connects_to_network(self): + self.base_dir = 'tests/fixtures/networks' + + self.dispatch(['up', '-d']) + self.dispatch(['run', 'app', 'nslookup', 'app']) + self.dispatch(['run', 'app', 'nslookup', 'db']) + + containers = self.project.get_service('app').containers( + stopped=True, one_off=OneOffFilter.only) + assert len(containers) == 2 + + for container in containers: + networks = container.get('NetworkSettings.Networks') + + assert sorted(list(networks)) == [ + '{}_{}'.format(self.project.name, name) + for name in ['back', 'front'] + ] + + for _, config in networks.items(): + # TODO: once we drop support for API <1.24, this can be changed to: + # assert config['Aliases'] == [container.short_id] + aliases = set(config['Aliases'] or []) - {container.short_id} + assert not aliases + + def test_run_detached_connects_to_network(self): + self.base_dir = 'tests/fixtures/networks' + self.dispatch(['up', '-d']) + self.dispatch(['run', '-d', 'app', 'top']) + + container = self.project.get_service('app').containers(one_off=OneOffFilter.only)[0] + networks = container.get('NetworkSettings.Networks') + + assert sorted(list(networks)) == [ + '{}_{}'.format(self.project.name, name) + for name in ['back', 'front'] + ] + + for _, config in networks.items(): + # TODO: once we drop support for API <1.24, this can be changed to: + # assert config['Aliases'] == [container.short_id] + aliases = set(config['Aliases'] or []) - {container.short_id} + assert not aliases + + assert self.lookup(container, 'app') + assert self.lookup(container, 'db') + + def test_run_handles_sigint(self): + proc = start_process(self.base_dir, ['run', '-T', 'simple', 'top']) + wait_on_condition(ContainerStateCondition( + self.project.client, + 'simple-composefile_simple_run_*', + 'running')) + + os.kill(proc.pid, signal.SIGINT) + wait_on_condition(ContainerStateCondition( + self.project.client, + 'simple-composefile_simple_run_*', + 'exited')) + + def test_run_handles_sigterm(self): + proc = start_process(self.base_dir, ['run', '-T', 'simple', 'top']) + wait_on_condition(ContainerStateCondition( + self.project.client, + 'simple-composefile_simple_run_*', + 'running')) + + os.kill(proc.pid, signal.SIGTERM) + wait_on_condition(ContainerStateCondition( + self.project.client, + 'simple-composefile_simple_run_*', + 'exited')) + + def test_run_handles_sighup(self): + proc = start_process(self.base_dir, ['run', '-T', 'simple', 'top']) + wait_on_condition(ContainerStateCondition( + self.project.client, + 'simple-composefile_simple_run_*', + 'running')) + + os.kill(proc.pid, signal.SIGHUP) + wait_on_condition(ContainerStateCondition( + self.project.client, + 'simple-composefile_simple_run_*', + 'exited')) + + @mock.patch.dict(os.environ) + def test_run_unicode_env_values_from_system(self): + value = 'ą, ć, ę, ł, ń, ó, ś, ź, ż' + os.environ['BAR'] = value + self.base_dir = 'tests/fixtures/unicode-environment' + self.dispatch(['run', 'simple']) + + container = self.project.containers(one_off=OneOffFilter.only, stopped=True)[0] + environment = container.get('Config.Env') + assert 'FOO={}'.format(value) in environment + + @mock.patch.dict(os.environ) + def test_run_env_values_from_system(self): + os.environ['FOO'] = 'bar' + os.environ['BAR'] = 'baz' + + self.dispatch(['run', '-e', 'FOO', 'simple', 'true'], None) + + container = self.project.containers(one_off=OneOffFilter.only, stopped=True)[0] + environment = container.get('Config.Env') + assert 'FOO=bar' in environment + assert 'BAR=baz' not in environment + + def test_run_label_flag(self): + self.base_dir = 'tests/fixtures/run-labels' + name = 'service' + self.dispatch(['run', '-l', 'default', '--label', 'foo=baz', name, '/bin/true']) + service = self.project.get_service(name) + container, = service.containers(stopped=True, one_off=OneOffFilter.only) + labels = container.labels + assert labels['default'] == '' + assert labels['foo'] == 'baz' + assert labels['hello'] == 'world' + + def test_rm(self): + service = self.project.get_service('simple') + service.create_container() + kill_service(service) + assert len(service.containers(stopped=True)) == 1 + self.dispatch(['rm', '--force'], None) + assert len(service.containers(stopped=True)) == 0 + service = self.project.get_service('simple') + service.create_container() + kill_service(service) + assert len(service.containers(stopped=True)) == 1 + self.dispatch(['rm', '-f'], None) + assert len(service.containers(stopped=True)) == 0 + service = self.project.get_service('simple') + service.create_container() + self.dispatch(['rm', '-fs'], None) + assert len(service.containers(stopped=True)) == 0 + + def test_rm_stop(self): + self.dispatch(['up', '-d'], None) + simple = self.project.get_service('simple') + another = self.project.get_service('another') + assert len(simple.containers()) == 1 + assert len(another.containers()) == 1 + self.dispatch(['rm', '-fs'], None) + assert len(simple.containers(stopped=True)) == 0 + assert len(another.containers(stopped=True)) == 0 + + self.dispatch(['up', '-d'], None) + assert len(simple.containers()) == 1 + assert len(another.containers()) == 1 + self.dispatch(['rm', '-fs', 'another'], None) + assert len(simple.containers()) == 1 + assert len(another.containers(stopped=True)) == 0 + + def test_rm_all(self): + service = self.project.get_service('simple') + service.create_container(one_off=False) + service.create_container(one_off=True) + kill_service(service) + assert len(service.containers(stopped=True)) == 1 + assert len(service.containers(stopped=True, one_off=OneOffFilter.only)) == 1 + self.dispatch(['rm', '-f'], None) + assert len(service.containers(stopped=True)) == 0 + assert len(service.containers(stopped=True, one_off=OneOffFilter.only)) == 0 + + service.create_container(one_off=False) + service.create_container(one_off=True) + kill_service(service) + assert len(service.containers(stopped=True)) == 1 + assert len(service.containers(stopped=True, one_off=OneOffFilter.only)) == 1 + self.dispatch(['rm', '-f', '--all'], None) + assert len(service.containers(stopped=True)) == 0 + assert len(service.containers(stopped=True, one_off=OneOffFilter.only)) == 0 + + def test_stop(self): + self.dispatch(['up', '-d'], None) + service = self.project.get_service('simple') + assert len(service.containers()) == 1 + assert service.containers()[0].is_running + + self.dispatch(['stop', '-t', '1'], None) + + assert len(service.containers(stopped=True)) == 1 + assert not service.containers(stopped=True)[0].is_running + + def test_stop_signal(self): + self.base_dir = 'tests/fixtures/stop-signal-composefile' + self.dispatch(['up', '-d'], None) + service = self.project.get_service('simple') + assert len(service.containers()) == 1 + assert service.containers()[0].is_running + + self.dispatch(['stop', '-t', '1'], None) + assert len(service.containers(stopped=True)) == 1 + assert not service.containers(stopped=True)[0].is_running + assert service.containers(stopped=True)[0].exit_code == 0 + + def test_start_no_containers(self): + result = self.dispatch(['start'], returncode=1) + assert 'failed' in result.stderr + assert 'No containers to start' in result.stderr + + def test_up_logging(self): + self.base_dir = 'tests/fixtures/logging-composefile' + self.dispatch(['up', '-d']) + simple = self.project.get_service('simple').containers()[0] + log_config = simple.get('HostConfig.LogConfig') + assert log_config + assert log_config.get('Type') == 'none' + + another = self.project.get_service('another').containers()[0] + log_config = another.get('HostConfig.LogConfig') + assert log_config + assert log_config.get('Type') == 'json-file' + assert log_config.get('Config')['max-size'] == '10m' + + def test_up_logging_legacy(self): + self.base_dir = 'tests/fixtures/logging-composefile-legacy' + self.dispatch(['up', '-d']) + simple = self.project.get_service('simple').containers()[0] + log_config = simple.get('HostConfig.LogConfig') + assert log_config + assert log_config.get('Type') == 'none' + + another = self.project.get_service('another').containers()[0] + log_config = another.get('HostConfig.LogConfig') + assert log_config + assert log_config.get('Type') == 'json-file' + assert log_config.get('Config')['max-size'] == '10m' + + def test_pause_unpause(self): + self.dispatch(['up', '-d'], None) + service = self.project.get_service('simple') + assert not service.containers()[0].is_paused + + self.dispatch(['pause'], None) + assert service.containers()[0].is_paused + + self.dispatch(['unpause'], None) + assert not service.containers()[0].is_paused + + def test_pause_no_containers(self): + result = self.dispatch(['pause'], returncode=1) + assert 'No containers to pause' in result.stderr + + def test_unpause_no_containers(self): + result = self.dispatch(['unpause'], returncode=1) + assert 'No containers to unpause' in result.stderr + + def test_logs_invalid_service_name(self): + self.dispatch(['logs', 'madeupname'], returncode=1) + + def test_logs_follow(self): + self.base_dir = 'tests/fixtures/echo-services' + self.dispatch(['up', '-d']) + + result = self.dispatch(['logs', '-f']) + + if not is_cluster(self.client): + assert result.stdout.count('\n') == 5 + else: + # Sometimes logs are picked up from old containers that haven't yet + # been removed (removal in Swarm is async) + assert result.stdout.count('\n') >= 5 + + assert 'simple' in result.stdout + assert 'another' in result.stdout + assert 'exited with code 0' in result.stdout + + @pytest.mark.skip(reason="race condition between up and logs") + def test_logs_follow_logs_from_new_containers(self): + self.base_dir = 'tests/fixtures/logs-composefile' + self.dispatch(['up', '-d', 'simple']) + + proc = start_process(self.base_dir, ['logs', '-f']) + + self.dispatch(['up', '-d', 'another']) + another_name = self.project.get_service('another').get_container().name_without_project + wait_on_condition( + ContainerStateCondition( + self.project.client, + 'logs-composefile_another_*', + 'exited' + ) + ) + + simple_name = self.project.get_service('simple').get_container().name_without_project + self.dispatch(['kill', 'simple']) + + result = wait_on_process(proc) + + assert 'hello' in result.stdout + assert 'test' in result.stdout + assert '{} exited with code 0'.format(another_name) in result.stdout + assert '{} exited with code 137'.format(simple_name) in result.stdout + + @pytest.mark.skip(reason="race condition between up and logs") + def test_logs_follow_logs_from_restarted_containers(self): + self.base_dir = 'tests/fixtures/logs-restart-composefile' + proc = start_process(self.base_dir, ['up']) + + wait_on_condition( + ContainerStateCondition( + self.project.client, + 'logs-restart-composefile_another_*', + 'exited' + ) + ) + self.dispatch(['kill', 'simple']) + + result = wait_on_process(proc) + + assert result.stdout.count( + r'logs-restart-composefile_another_1 exited with code 1' + ) == 3 + assert result.stdout.count('world') == 3 + + @pytest.mark.skip(reason="race condition between up and logs") + def test_logs_default(self): + self.base_dir = 'tests/fixtures/logs-composefile' + self.dispatch(['up', '-d']) + + result = self.dispatch(['logs']) + assert 'hello' in result.stdout + assert 'test' in result.stdout + assert 'exited with' not in result.stdout + + def test_logs_on_stopped_containers_exits(self): + self.base_dir = 'tests/fixtures/echo-services' + self.dispatch(['up']) + + result = self.dispatch(['logs']) + assert 'simple' in result.stdout + assert 'another' in result.stdout + assert 'exited with' not in result.stdout + + def test_logs_timestamps(self): + self.base_dir = 'tests/fixtures/echo-services' + self.dispatch(['up', '-d']) + + result = self.dispatch(['logs', '-f', '-t']) + assert re.search(r'(\d{4})-(\d{2})-(\d{2})T(\d{2})\:(\d{2})\:(\d{2})', result.stdout) + + def test_logs_tail(self): + self.base_dir = 'tests/fixtures/logs-tail-composefile' + self.dispatch(['up']) + + result = self.dispatch(['logs', '--tail', '2']) + assert 'y\n' in result.stdout + assert 'z\n' in result.stdout + assert 'w\n' not in result.stdout + assert 'x\n' not in result.stdout + + def test_kill(self): + self.dispatch(['up', '-d'], None) + service = self.project.get_service('simple') + assert len(service.containers()) == 1 + assert service.containers()[0].is_running + + self.dispatch(['kill'], None) + + assert len(service.containers(stopped=True)) == 1 + assert not service.containers(stopped=True)[0].is_running + + def test_kill_signal_sigstop(self): + self.dispatch(['up', '-d'], None) + service = self.project.get_service('simple') + assert len(service.containers()) == 1 + assert service.containers()[0].is_running + + self.dispatch(['kill', '-s', 'SIGSTOP'], None) + + assert len(service.containers()) == 1 + # The container is still running. It has only been paused + assert service.containers()[0].is_running + + def test_kill_stopped_service(self): + self.dispatch(['up', '-d'], None) + service = self.project.get_service('simple') + self.dispatch(['kill', '-s', 'SIGSTOP'], None) + assert service.containers()[0].is_running + + self.dispatch(['kill', '-s', 'SIGKILL'], None) + + assert len(service.containers(stopped=True)) == 1 + assert not service.containers(stopped=True)[0].is_running + + def test_restart(self): + service = self.project.get_service('simple') + container = service.create_container() + service.start_container(container) + started_at = container.dictionary['State']['StartedAt'] + self.dispatch(['restart', '-t', '1'], None) + container.inspect() + assert container.dictionary['State']['FinishedAt'] != '0001-01-01T00:00:00Z' + assert container.dictionary['State']['StartedAt'] != started_at + + def test_restart_stopped_container(self): + service = self.project.get_service('simple') + container = service.create_container() + container.start() + container.kill() + assert len(service.containers(stopped=True)) == 1 + self.dispatch(['restart', '-t', '1'], None) + assert len(service.containers(stopped=False)) == 1 + + def test_restart_no_containers(self): + result = self.dispatch(['restart'], returncode=1) + assert 'No containers to restart' in result.stderr + + def test_scale(self): + project = self.project + + self.dispatch(['scale', 'simple=1']) + assert len(project.get_service('simple').containers()) == 1 + + self.dispatch(['scale', 'simple=3', 'another=2']) + assert len(project.get_service('simple').containers()) == 3 + assert len(project.get_service('another').containers()) == 2 + + self.dispatch(['scale', 'simple=1', 'another=1']) + assert len(project.get_service('simple').containers()) == 1 + assert len(project.get_service('another').containers()) == 1 + + self.dispatch(['scale', 'simple=1', 'another=1']) + assert len(project.get_service('simple').containers()) == 1 + assert len(project.get_service('another').containers()) == 1 + + self.dispatch(['scale', 'simple=0', 'another=0']) + assert len(project.get_service('simple').containers()) == 0 + assert len(project.get_service('another').containers()) == 0 + + def test_up_scale_scale_up(self): + self.base_dir = 'tests/fixtures/scale' + project = self.project + + self.dispatch(['up', '-d']) + assert len(project.get_service('web').containers()) == 2 + assert len(project.get_service('db').containers()) == 1 + assert len(project.get_service('worker').containers()) == 0 + + self.dispatch(['up', '-d', '--scale', 'web=3', '--scale', 'worker=1']) + assert len(project.get_service('web').containers()) == 3 + assert len(project.get_service('db').containers()) == 1 + assert len(project.get_service('worker').containers()) == 1 + + def test_up_scale_scale_down(self): + self.base_dir = 'tests/fixtures/scale' + project = self.project + + self.dispatch(['up', '-d']) + assert len(project.get_service('web').containers()) == 2 + assert len(project.get_service('db').containers()) == 1 + assert len(project.get_service('worker').containers()) == 0 + + self.dispatch(['up', '-d', '--scale', 'web=1']) + assert len(project.get_service('web').containers()) == 1 + assert len(project.get_service('db').containers()) == 1 + assert len(project.get_service('worker').containers()) == 0 + + def test_up_scale_reset(self): + self.base_dir = 'tests/fixtures/scale' + project = self.project + + self.dispatch(['up', '-d', '--scale', 'web=3', '--scale', 'db=3', '--scale', 'worker=3']) + assert len(project.get_service('web').containers()) == 3 + assert len(project.get_service('db').containers()) == 3 + assert len(project.get_service('worker').containers()) == 3 + + self.dispatch(['up', '-d']) + assert len(project.get_service('web').containers()) == 2 + assert len(project.get_service('db').containers()) == 1 + assert len(project.get_service('worker').containers()) == 0 + + def test_up_scale_to_zero(self): + self.base_dir = 'tests/fixtures/scale' + project = self.project + + self.dispatch(['up', '-d']) + assert len(project.get_service('web').containers()) == 2 + assert len(project.get_service('db').containers()) == 1 + assert len(project.get_service('worker').containers()) == 0 + + self.dispatch(['up', '-d', '--scale', 'web=0', '--scale', 'db=0', '--scale', 'worker=0']) + assert len(project.get_service('web').containers()) == 0 + assert len(project.get_service('db').containers()) == 0 + assert len(project.get_service('worker').containers()) == 0 + + def test_port(self): + self.base_dir = 'tests/fixtures/ports-composefile' + self.dispatch(['up', '-d'], None) + container = self.project.get_service('simple').get_container() + + def get_port(number): + result = self.dispatch(['port', 'simple', str(number)]) + return result.stdout.rstrip() + + assert get_port(3000) == container.get_local_port(3000) + assert ':49152' in get_port(3001) + assert ':49153' in get_port(3002) + + def test_expanded_port(self): + self.base_dir = 'tests/fixtures/ports-composefile' + self.dispatch(['-f', 'expanded-notation.yml', 'up', '-d']) + container = self.project.get_service('simple').get_container() + + def get_port(number): + result = self.dispatch(['port', 'simple', str(number)]) + return result.stdout.rstrip() + + assert get_port(3000) == container.get_local_port(3000) + assert ':53222' in get_port(3001) + assert ':53223' in get_port(3002) + + def test_port_with_scale(self): + self.base_dir = 'tests/fixtures/ports-composefile-scale' + self.dispatch(['scale', 'simple=2'], None) + containers = sorted( + self.project.containers(service_names=['simple']), + key=attrgetter('name')) + + def get_port(number, index=None): + if index is None: + result = self.dispatch(['port', 'simple', str(number)]) + else: + result = self.dispatch(['port', '--index=' + str(index), 'simple', str(number)]) + return result.stdout.rstrip() + + assert get_port(3000) in (containers[0].get_local_port(3000), containers[1].get_local_port(3000)) + assert get_port(3000, index=containers[0].number) == containers[0].get_local_port(3000) + assert get_port(3000, index=containers[1].number) == containers[1].get_local_port(3000) + assert get_port(3002) == "" + + def test_events_json(self): + events_proc = start_process(self.base_dir, ['events', '--json']) + self.dispatch(['up', '-d']) + wait_on_condition(ContainerCountCondition(self.project, 2)) + + os.kill(events_proc.pid, signal.SIGINT) + result = wait_on_process(events_proc, returncode=1) + lines = [json.loads(line) for line in result.stdout.rstrip().split('\n')] + assert Counter(e['action'] for e in lines) == {'create': 2, 'start': 2} + + def test_events_human_readable(self): + + def has_timestamp(string): + str_iso_date, str_iso_time, container_info = string.split(' ', 2) + try: + return isinstance(datetime.datetime.strptime( + '{} {}'.format(str_iso_date, str_iso_time), + '%Y-%m-%d %H:%M:%S.%f'), + datetime.datetime) + except ValueError: + return False + + events_proc = start_process(self.base_dir, ['events']) + self.dispatch(['up', '-d', 'simple']) + wait_on_condition(ContainerCountCondition(self.project, 1)) + + os.kill(events_proc.pid, signal.SIGINT) + result = wait_on_process(events_proc, returncode=1) + lines = result.stdout.rstrip().split('\n') + assert len(lines) == 2 + + container, = self.project.containers() + expected_template = ' container {} {}' + expected_meta_info = ['image=busybox:1.27.2', 'name=simple-composefile_simple_'] + + assert expected_template.format('create', container.id) in lines[0] + assert expected_template.format('start', container.id) in lines[1] + for line in lines: + for info in expected_meta_info: + assert info in line + + assert has_timestamp(lines[0]) + + def test_env_file_relative_to_compose_file(self): + config_path = os.path.abspath('tests/fixtures/env-file/docker-compose.yml') + self.dispatch(['-f', config_path, 'up', '-d'], None) + self._project = get_project(self.base_dir, [config_path]) + + containers = self.project.containers(stopped=True) + assert len(containers) == 1 + assert "FOO=1" in containers[0].get('Config.Env') + + @mock.patch.dict(os.environ) + def test_home_and_env_var_in_volume_path(self): + os.environ['VOLUME_NAME'] = 'my-volume' + os.environ['HOME'] = '/tmp/home-dir' + + self.base_dir = 'tests/fixtures/volume-path-interpolation' + self.dispatch(['up', '-d'], None) + + container = self.project.containers(stopped=True)[0] + actual_host_path = container.get_mount('/container-path')['Source'] + components = actual_host_path.split('/') + assert components[-2:] == ['home-dir', 'my-volume'] + + def test_up_with_default_override_file(self): + self.base_dir = 'tests/fixtures/override-files' + self.dispatch(['up', '-d'], None) + + containers = self.project.containers() + assert len(containers) == 2 + + web, db = containers + assert web.human_readable_command == 'top' + assert db.human_readable_command == 'top' + + def test_up_with_multiple_files(self): + self.base_dir = 'tests/fixtures/override-files' + config_paths = [ + 'docker-compose.yml', + 'docker-compose.override.yml', + 'extra.yml', + ] + self._project = get_project(self.base_dir, config_paths) + self.dispatch( + [ + '-f', config_paths[0], + '-f', config_paths[1], + '-f', config_paths[2], + 'up', '-d', + ], + None) + + containers = self.project.containers() + assert len(containers) == 3 + + web, other, db = containers + assert web.human_readable_command == 'top' + assert db.human_readable_command == 'top' + assert other.human_readable_command == 'top' + + def test_up_with_extends(self): + self.base_dir = 'tests/fixtures/extends' + self.dispatch(['up', '-d'], None) + + assert {s.name for s in self.project.services} == {'mydb', 'myweb'} + + # Sort by name so we get [db, web] + containers = sorted( + self.project.containers(stopped=True), + key=lambda c: c.name, + ) + + assert len(containers) == 2 + web = containers[1] + db_name = containers[0].name_without_project + + assert set(get_links(web)) == {'db', db_name, 'extends_{}'.format(db_name)} + + expected_env = {"FOO=1", "BAR=2", "BAZ=2"} + assert expected_env <= set(web.get('Config.Env')) + + def test_top_services_not_running(self): + self.base_dir = 'tests/fixtures/top' + result = self.dispatch(['top']) + assert len(result.stdout) == 0 + + def test_top_services_running(self): + self.base_dir = 'tests/fixtures/top' + self.dispatch(['up', '-d']) + result = self.dispatch(['top']) + + assert 'top_service_a' in result.stdout + assert 'top_service_b' in result.stdout + assert 'top_not_a_service' not in result.stdout + + def test_top_processes_running(self): + self.base_dir = 'tests/fixtures/top' + self.dispatch(['up', '-d']) + result = self.dispatch(['top']) + assert result.stdout.count("top") == 4 + + def test_forward_exitval(self): + self.base_dir = 'tests/fixtures/exit-code-from' + proc = start_process( + self.base_dir, + ['up', '--abort-on-container-exit', '--exit-code-from', 'another'] + ) + + result = wait_on_process(proc, returncode=1) + assert 'exit-code-from_another_1 exited with code 1' in result.stdout + + def test_exit_code_from_signal_stop(self): + self.base_dir = 'tests/fixtures/exit-code-from' + proc = start_process( + self.base_dir, + ['up', '--abort-on-container-exit', '--exit-code-from', 'simple'] + ) + result = wait_on_process(proc, returncode=137) # SIGKILL + name = self.project.get_service('another').containers(stopped=True)[0].name_without_project + assert '{} exited with code 1'.format(name) in result.stdout + + def test_images(self): + self.project.get_service('simple').create_container() + result = self.dispatch(['images']) + assert 'busybox' in result.stdout + assert 'simple-composefile_simple_' in result.stdout + + def test_images_default_composefile(self): + self.base_dir = 'tests/fixtures/multiple-composefiles' + self.dispatch(['up', '-d']) + result = self.dispatch(['images']) + + assert 'busybox' in result.stdout + assert '_another_1' in result.stdout + assert '_simple_1' in result.stdout + + @mock.patch.dict(os.environ) + def test_images_tagless_image(self): + self.base_dir = 'tests/fixtures/tagless-image' + stream = self.client.build(self.base_dir, decode=True) + img_id = None + for data in stream: + if 'aux' in data: + img_id = data['aux']['ID'] + break + if 'stream' in data and 'Successfully built' in data['stream']: + img_id = self.client.inspect_image(data['stream'].split(' ')[2].strip())['Id'] + + assert img_id + + os.environ['IMAGE_ID'] = img_id + self.project.get_service('foo').create_container() + result = self.dispatch(['images']) + assert '' in result.stdout + assert 'tagless-image_foo_1' in result.stdout + + def test_up_with_override_yaml(self): + self.base_dir = 'tests/fixtures/override-yaml-files' + self._project = get_project(self.base_dir, []) + self.dispatch(['up', '-d'], None) + + containers = self.project.containers() + assert len(containers) == 2 + + web, db = containers + assert web.human_readable_command == 'sleep 100' + assert db.human_readable_command == 'top' + + def test_up_with_duplicate_override_yaml_files(self): + self.base_dir = 'tests/fixtures/duplicate-override-yaml-files' + with pytest.raises(DuplicateOverrideFileFound): + get_project(self.base_dir, []) + self.base_dir = None + + def test_images_use_service_tag(self): + pull_busybox(self.client) + self.base_dir = 'tests/fixtures/images-service-tag' + self.dispatch(['up', '-d', '--build']) + result = self.dispatch(['images']) + + assert re.search(r'foo1.+test[ \t]+dev', result.stdout) is not None + assert re.search(r'foo2.+test[ \t]+prod', result.stdout) is not None + assert re.search(r'foo3.+test[ \t]+latest', result.stdout) is not None + + def test_build_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + result = self.dispatch(['build', '--pull', '--', '--test-service']) + + assert BUILD_PULL_TEXT in result.stdout + + def test_events_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + events_proc = start_process(self.base_dir, ['events', '--json', '--', '--test-service']) + self.dispatch(['up', '-d', '--', '--test-service']) + wait_on_condition(ContainerCountCondition(self.project, 1)) + + os.kill(events_proc.pid, signal.SIGINT) + result = wait_on_process(events_proc, returncode=1) + lines = [json.loads(line) for line in result.stdout.rstrip().split('\n')] + assert Counter(e['action'] for e in lines) == {'create': 1, 'start': 1} + + def test_exec_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + self.dispatch(['up', '-d', '--', '--test-service']) + assert len(self.project.containers()) == 1 + + stdout, stderr = self.dispatch(['exec', '-T', '--', '--test-service', 'ls', '-1d', '/']) + + assert stderr == "" + assert stdout == "/\n" + + def test_images_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + self.dispatch(['up', '-d', '--', '--test-service']) + result = self.dispatch(['images', '--', '--test-service']) + + assert "busybox" in result.stdout + + def test_kill_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + self.dispatch(['up', '-d', '--', '--test-service']) + service = self.project.get_service('--test-service') + + assert len(service.containers()) == 1 + assert service.containers()[0].is_running + + self.dispatch(['kill', '--', '--test-service']) + + assert len(service.containers(stopped=True)) == 1 + assert not service.containers(stopped=True)[0].is_running + + def test_logs_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + self.dispatch(['up', '-d', '--', '--log-service']) + result = self.dispatch(['logs', '--', '--log-service']) + + assert 'hello' in result.stdout + assert 'exited with' not in result.stdout + + def test_port_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + self.dispatch(['up', '-d', '--', '--test-service']) + result = self.dispatch(['port', '--', '--test-service', '80']) + + assert result.stdout.strip() == "0.0.0.0:8080" + + def test_ps_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + self.dispatch(['up', '-d', '--', '--test-service']) + + result = self.dispatch(['ps', '--', '--test-service']) + + assert 'flag-as-service-name_--test-service_1' in result.stdout + + def test_pull_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + result = self.dispatch(['pull', '--', '--test-service']) + + assert 'Pulling --test-service' in result.stderr + assert 'failed' not in result.stderr + + def test_rm_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + self.dispatch(['up', '--no-start', '--', '--test-service']) + service = self.project.get_service('--test-service') + assert len(service.containers(stopped=True)) == 1 + + self.dispatch(['rm', '--force', '--', '--test-service']) + assert len(service.containers(stopped=True)) == 0 + + def test_run_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + result = self.dispatch(['run', '--no-deps', '--', '--test-service', 'echo', '-hello']) + + assert 'hello' in result.stdout + assert len(self.project.containers()) == 0 + + def test_stop_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + self.dispatch(['up', '-d', '--', '--test-service']) + service = self.project.get_service('--test-service') + assert len(service.containers()) == 1 + assert service.containers()[0].is_running + + self.dispatch(['stop', '-t', '1', '--', '--test-service']) + + assert len(service.containers(stopped=True)) == 1 + assert not service.containers(stopped=True)[0].is_running + + def test_restart_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + self.dispatch(['up', '-d', '--', '--test-service']) + service = self.project.get_service('--test-service') + assert len(service.containers()) == 1 + assert service.containers()[0].is_running + + self.dispatch(['restart', '-t', '1', '--', '--test-service']) + + assert len(service.containers()) == 1 + assert service.containers()[0].is_running + + def test_up_with_stop_process_flag(self): + self.base_dir = 'tests/fixtures/flag-as-service-name' + self.dispatch(['up', '-d', '--', '--test-service', '--log-service']) + + service = self.project.get_service('--test-service') + another = self.project.get_service('--log-service') + assert len(service.containers()) == 1 + assert len(another.containers()) == 1 + + def test_up_no_log_prefix(self): + self.base_dir = 'tests/fixtures/echo-services' + result = self.dispatch(['up', '--no-log-prefix']) + + assert 'simple' in result.stdout + assert 'another' in result.stdout + assert 'exited with code 0' in result.stdout + assert 'exited with code 0' in result.stdout diff --git a/tests/acceptance/context_test.py b/tests/acceptance/context_test.py new file mode 100644 index 00000000000..a5d0c14730f --- /dev/null +++ b/tests/acceptance/context_test.py @@ -0,0 +1,44 @@ +import os +import shutil +import unittest + +from docker import ContextAPI + +from tests.acceptance.cli_test import dispatch + + +class ContextTestCase(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.docker_dir = os.path.join(os.environ.get("HOME", "/tmp"), '.docker') + if not os.path.exists(cls.docker_dir): + os.makedirs(cls.docker_dir) + f = open(os.path.join(cls.docker_dir, "config.json"), "w") + f.write("{}") + f.close() + cls.docker_config = os.path.join(cls.docker_dir, "config.json") + os.environ['DOCKER_CONFIG'] = cls.docker_config + ContextAPI.create_context("testcontext", host="tcp://doesnotexist:8000") + + @classmethod + def tearDownClass(cls): + shutil.rmtree(cls.docker_dir, ignore_errors=True) + + def setUp(self): + self.base_dir = 'tests/fixtures/simple-composefile' + self.override_dir = None + + def dispatch(self, options, project_options=None, returncode=0, stdin=None): + return dispatch(self.base_dir, options, project_options, returncode, stdin) + + def test_help(self): + result = self.dispatch(['help'], returncode=0) + assert '-c, --context NAME' in result.stdout + + def test_fail_on_both_host_and_context_opt(self): + result = self.dispatch(['-H', 'unix://', '-c', 'default', 'up'], returncode=1) + assert '-H, --host and -c, --context are mutually exclusive' in result.stderr + + def test_fail_run_on_inexistent_context(self): + result = self.dispatch(['-c', 'testcontext', 'up', '-d'], returncode=1) + assert "Couldn't connect to Docker daemon" in result.stderr diff --git a/tests/conftest.py b/tests/conftest.py new file mode 100644 index 00000000000..fd31a974840 --- /dev/null +++ b/tests/conftest.py @@ -0,0 +1,240 @@ +import pytest + +import tests.acceptance.cli_test + +# FIXME Skipping all the acceptance tests when in `--conformity` +non_conformity_tests = [ + "test_build_failed", + "test_build_failed_forcerm", + "test_build_log_level", + "test_build_memory_build_option", + "test_build_no_cache", + "test_build_no_cache_pull", + "test_build_override_dir", + "test_build_override_dir_invalid_path", + "test_build_parallel", + "test_build_plain", + "test_build_pull", + "test_build_rm", + "test_build_shm_size_build_option", + "test_build_with_buildarg_cli_override", + "test_build_with_buildarg_from_compose_file", + "test_build_with_buildarg_old_api_version", + "test_config_compatibility_mode", + "test_config_compatibility_mode_from_env", + "test_config_compatibility_mode_from_env_and_option_precedence", + "test_config_default", + "test_config_external_network", + "test_config_external_network_v3_5", + "test_config_external_volume_v2", + "test_config_external_volume_v2_x", + "test_config_external_volume_v3_4", + "test_config_external_volume_v3_x", + "test_config_list_services", + "test_config_list_volumes", + "test_config_quiet", + "test_config_quiet_with_error", + "test_config_restart", + "test_config_stdin", + "test_config_v1", + "test_config_v3", + "test_config_with_dot_env", + "test_config_with_dot_env_and_override_dir", + "test_config_with_env_file", + "test_config_with_hash_option", + "test_create", + "test_create_with_force_recreate", + "test_create_with_force_recreate_and_no_recreate", + "test_create_with_no_recreate", + "test_down", + "test_down_invalid_rmi_flag", + "test_down_signal", + "test_down_timeout", + "test_env_file_relative_to_compose_file", + "test_events_human_readable", + "test_events_json", + "test_exec_custom_user", + "test_exec_detach_long_form", + "test_exec_novalue_var_dotenv_file", + "test_exec_service_with_environment_overridden", + "test_exec_without_tty", + "test_exec_workdir", + "test_exit_code_from_signal_stop", + "test_expanded_port", + "test_forward_exitval", + "test_help", + "test_help_nonexistent", + "test_home_and_env_var_in_volume_path", + "test_host_not_reachable", + "test_host_not_reachable_volumes_from_container", + "test_host_not_reachable_volumes_from_container", + "test_images", + "test_images_default_composefile", + "test_images_tagless_image", + "test_images_use_service_tag", + "test_kill", + "test_kill_signal_sigstop", + "test_kill_stopped_service", + "test_logs_default", + "test_logs_follow", + "test_logs_follow_logs_from_new_containers", + "test_logs_follow_logs_from_restarted_containers", + "test_logs_invalid_service_name", + "test_logs_on_stopped_containers_exits", + "test_logs_tail", + "test_logs_timestamps", + "test_pause_no_containers", + "test_pause_unpause", + "test_port", + "test_port_with_scale", + "test_ps", + "test_ps_all", + "test_ps_alternate_composefile", + "test_ps_default_composefile", + "test_ps_services_filter_option", + "test_ps_services_filter_status", + "test_pull", + "test_pull_can_build", + "test_pull_with_digest", + "test_pull_with_ignore_pull_failures", + "test_pull_with_include_deps", + "test_pull_with_no_deps", + "test_pull_with_parallel_failure", + "test_pull_with_quiet", + "test_quiet_build", + "test_restart", + "test_restart_no_containers", + "test_restart_stopped_container", + "test_rm", + "test_rm_all", + "test_rm_stop", + "test_run_detached_connects_to_network", + "test_run_does_not_recreate_linked_containers", + "test_run_env_values_from_system", + "test_run_handles_sighup", + "test_run_handles_sigint", + "test_run_handles_sigterm", + "test_run_interactive_connects_to_network", + "test_run_label_flag", + "test_run_one_off_with_multiple_volumes", + "test_run_one_off_with_volume", + "test_run_one_off_with_volume_merge", + "test_run_rm", + "test_run_service_with_compose_file_entrypoint", + "test_run_service_with_compose_file_entrypoint_and_command_overridden", + "test_run_service_with_compose_file_entrypoint_and_empty_string_command", + "test_run_service_with_compose_file_entrypoint_overridden", + "test_run_service_with_dependencies", + "test_run_service_with_dockerfile_entrypoint", + "test_run_service_with_dockerfile_entrypoint_and_command_overridden", + "test_run_service_with_dockerfile_entrypoint_overridden", + "test_run_service_with_environment_overridden", + "test_run_service_with_explicitly_mapped_ip_ports", + "test_run_service_with_explicitly_mapped_ports", + "test_run_service_with_links", + "test_run_service_with_map_ports", + "test_run_service_with_scaled_dependencies", + "test_run_service_with_unset_entrypoint", + "test_run_service_with_use_aliases", + "test_run_service_with_user_overridden", + "test_run_service_with_user_overridden_short_form", + "test_run_service_with_workdir_overridden", + "test_run_service_with_workdir_overridden_short_form", + "test_run_service_without_links", + "test_run_service_without_map_ports", + "test_run_unicode_env_values_from_system", + "test_run_with_custom_name", + "test_run_with_expose_ports", + "test_run_with_no_deps", + "test_run_without_command", + "test_scale", + "test_scale_v2_2", + "test_shorthand_host_opt", + "test_shorthand_host_opt_interactive", + "test_start_no_containers", + "test_stop", + "test_stop_signal", + "test_top_processes_running", + "test_top_services_not_running", + "test_top_services_running", + "test_unpause_no_containers", + "test_up", + "test_up_attached", + "test_up_detached", + "test_up_detached_long_form", + "test_up_external_networks", + "test_up_handles_abort_on_container_exit", + "test_up_handles_abort_on_container_exit_code", + "test_up_handles_aborted_dependencies", + "test_up_handles_force_shutdown", + "test_up_handles_sigint", + "test_up_handles_sigterm", + "test_up_logging", + "test_up_logging_legacy", + "test_up_missing_network", + "test_up_no_ansi", + "test_up_no_services", + "test_up_no_start", + "test_up_no_start_remove_orphans", + "test_up_scale_reset", + "test_up_scale_scale_down", + "test_up_scale_scale_up", + "test_up_scale_to_zero", + "test_up_with_attach_dependencies", + "test_up_with_default_network_config", + "test_up_with_default_override_file", + "test_up_with_duplicate_override_yaml_files", + "test_up_with_extends", + "test_up_with_external_default_network", + "test_up_with_force_recreate", + "test_up_with_force_recreate_and_no_recreate", + "test_up_with_healthcheck", + "test_up_with_ignore_remove_orphans", + "test_up_with_links_v1", + "test_up_with_multiple_files", + "test_up_with_net_is_invalid", + "test_up_with_net_v1", + "test_up_with_network_aliases", + "test_up_with_network_internal", + "test_up_with_network_labels", + "test_up_with_network_mode", + "test_up_with_network_static_addresses", + "test_up_with_networks", + "test_up_with_no_deps", + "test_up_with_no_recreate", + "test_up_with_override_yaml", + "test_up_with_pid_mode", + "test_up_with_timeout", + "test_up_with_volume_labels", + "test_fail_on_both_host_and_context_opt", + "test_fail_run_on_inexistent_context", +] + + +def pytest_addoption(parser): + parser.addoption( + "--conformity", + action="store_true", + default=False, + help="Only runs tests that are not black listed as non conformity test. " + "The conformity tests check for compatibility with the Compose spec." + ) + parser.addoption( + "--binary", + default=tests.acceptance.cli_test.DOCKER_COMPOSE_EXECUTABLE, + help="Forces the execution of a binary in the PATH. Default is `docker-compose`." + ) + + +def pytest_collection_modifyitems(config, items): + if not config.getoption("--conformity"): + return + if config.getoption("--binary"): + tests.acceptance.cli_test.DOCKER_COMPOSE_EXECUTABLE = config.getoption("--binary") + + print("Binary -> {}".format(tests.acceptance.cli_test.DOCKER_COMPOSE_EXECUTABLE)) + skip_non_conformity = pytest.mark.skip(reason="skipping because that's not a conformity test") + for item in items: + if item.name in non_conformity_tests: + print("Skipping '{}' when running in compatibility mode".format(item.name)) + item.add_marker(skip_non_conformity) diff --git a/tests/fixtures/UpperCaseDir/docker-compose.yml b/tests/fixtures/UpperCaseDir/docker-compose.yml new file mode 100644 index 00000000000..09cc9519fed --- /dev/null +++ b/tests/fixtures/UpperCaseDir/docker-compose.yml @@ -0,0 +1,6 @@ +simple: + image: busybox:1.31.0-uclibc + command: top +another: + image: busybox:1.31.0-uclibc + command: top diff --git a/tests/fixtures/abort-on-container-exit-0/docker-compose.yml b/tests/fixtures/abort-on-container-exit-0/docker-compose.yml new file mode 100644 index 00000000000..77307ef29cd --- /dev/null +++ b/tests/fixtures/abort-on-container-exit-0/docker-compose.yml @@ -0,0 +1,6 @@ +simple: + image: busybox:1.31.0-uclibc + command: top +another: + image: busybox:1.31.0-uclibc + command: ls . diff --git a/tests/fixtures/abort-on-container-exit-1/docker-compose.yml b/tests/fixtures/abort-on-container-exit-1/docker-compose.yml new file mode 100644 index 00000000000..23290964e51 --- /dev/null +++ b/tests/fixtures/abort-on-container-exit-1/docker-compose.yml @@ -0,0 +1,6 @@ +simple: + image: busybox:1.31.0-uclibc + command: top +another: + image: busybox:1.31.0-uclibc + command: ls /thecakeisalie diff --git a/tests/fixtures/abort-on-container-exit-dependencies/docker-compose.yml b/tests/fixtures/abort-on-container-exit-dependencies/docker-compose.yml new file mode 100644 index 00000000000..cd10c851c1b --- /dev/null +++ b/tests/fixtures/abort-on-container-exit-dependencies/docker-compose.yml @@ -0,0 +1,10 @@ +version: "2.0" +services: + simple: + image: busybox:1.31.0-uclibc + command: top + depends_on: + - another + another: + image: busybox:1.31.0-uclibc + command: ls /thecakeisalie diff --git a/tests/fixtures/build-args/Dockerfile b/tests/fixtures/build-args/Dockerfile new file mode 100644 index 00000000000..d1534068ae6 --- /dev/null +++ b/tests/fixtures/build-args/Dockerfile @@ -0,0 +1,4 @@ +FROM busybox:1.31.0-uclibc +LABEL com.docker.compose.test_image=true +ARG favorite_th_character +RUN echo "Favorite Touhou Character: ${favorite_th_character}" diff --git a/tests/fixtures/build-args/docker-compose.yml b/tests/fixtures/build-args/docker-compose.yml new file mode 100644 index 00000000000..ed60a337b86 --- /dev/null +++ b/tests/fixtures/build-args/docker-compose.yml @@ -0,0 +1,7 @@ +version: '2.2' +services: + web: + build: + context: . + args: + - favorite_th_character=mariya.kirisame diff --git a/tests/fixtures/build-ctx/Dockerfile b/tests/fixtures/build-ctx/Dockerfile new file mode 100644 index 00000000000..4acac9c7ac4 --- /dev/null +++ b/tests/fixtures/build-ctx/Dockerfile @@ -0,0 +1,3 @@ +FROM busybox:1.31.0-uclibc +LABEL com.docker.compose.test_image=true +CMD echo "success" diff --git a/tests/fixtures/build-memory/Dockerfile b/tests/fixtures/build-memory/Dockerfile new file mode 100644 index 00000000000..076b84d771e --- /dev/null +++ b/tests/fixtures/build-memory/Dockerfile @@ -0,0 +1,4 @@ +FROM busybox:1.31.0-uclibc + +# Report the memory (through the size of the group memory) +RUN echo "memory:" $(cat /sys/fs/cgroup/memory/memory.limit_in_bytes) diff --git a/tests/fixtures/build-memory/docker-compose.yml b/tests/fixtures/build-memory/docker-compose.yml new file mode 100644 index 00000000000..f98355851b1 --- /dev/null +++ b/tests/fixtures/build-memory/docker-compose.yml @@ -0,0 +1,6 @@ +version: '3.5' + +services: + service: + build: + context: . diff --git a/tests/fixtures/build-multiple-composefile/a/Dockerfile b/tests/fixtures/build-multiple-composefile/a/Dockerfile new file mode 100644 index 00000000000..52ed15ec671 --- /dev/null +++ b/tests/fixtures/build-multiple-composefile/a/Dockerfile @@ -0,0 +1,4 @@ + +FROM busybox:1.31.0-uclibc +RUN echo a +CMD top diff --git a/tests/fixtures/build-multiple-composefile/b/Dockerfile b/tests/fixtures/build-multiple-composefile/b/Dockerfile new file mode 100644 index 00000000000..932d851d98a --- /dev/null +++ b/tests/fixtures/build-multiple-composefile/b/Dockerfile @@ -0,0 +1,4 @@ + +FROM busybox:1.31.0-uclibc +RUN echo b +CMD top diff --git a/tests/fixtures/build-multiple-composefile/docker-compose.yml b/tests/fixtures/build-multiple-composefile/docker-compose.yml new file mode 100644 index 00000000000..efa70d7e060 --- /dev/null +++ b/tests/fixtures/build-multiple-composefile/docker-compose.yml @@ -0,0 +1,8 @@ + +version: "2" + +services: + a: + build: ./a + b: + build: ./b diff --git a/tests/fixtures/build-path-override-dir/docker-compose.yml b/tests/fixtures/build-path-override-dir/docker-compose.yml new file mode 100644 index 00000000000..15dbb3e68ea --- /dev/null +++ b/tests/fixtures/build-path-override-dir/docker-compose.yml @@ -0,0 +1,2 @@ +foo: + build: ./build-ctx/ diff --git a/tests/fixtures/build-path/docker-compose.yml b/tests/fixtures/build-path/docker-compose.yml new file mode 100644 index 00000000000..66e8916e9d4 --- /dev/null +++ b/tests/fixtures/build-path/docker-compose.yml @@ -0,0 +1,2 @@ +foo: + build: ../build-ctx/ diff --git a/tests/fixtures/build-shm-size/Dockerfile b/tests/fixtures/build-shm-size/Dockerfile new file mode 100644 index 00000000000..f91733d6301 --- /dev/null +++ b/tests/fixtures/build-shm-size/Dockerfile @@ -0,0 +1,4 @@ +FROM busybox + +# Report the shm_size (through the size of /dev/shm) +RUN echo "shm_size:" $(df -h /dev/shm | tail -n 1 | awk '{print $2}') diff --git a/tests/fixtures/build-shm-size/docker-compose.yml b/tests/fixtures/build-shm-size/docker-compose.yml new file mode 100644 index 00000000000..238a513223b --- /dev/null +++ b/tests/fixtures/build-shm-size/docker-compose.yml @@ -0,0 +1,7 @@ +version: '3.5' + +services: + custom_shm_size: + build: + context: . + shm_size: 100663296 # =96M diff --git a/tests/fixtures/commands-composefile/docker-compose.yml b/tests/fixtures/commands-composefile/docker-compose.yml new file mode 100644 index 00000000000..87602bd6ef1 --- /dev/null +++ b/tests/fixtures/commands-composefile/docker-compose.yml @@ -0,0 +1,5 @@ +implicit: + image: composetest_test +explicit: + image: composetest_test + command: [ "/bin/true" ] diff --git a/tests/fixtures/compatibility-mode/docker-compose.yml b/tests/fixtures/compatibility-mode/docker-compose.yml new file mode 100644 index 00000000000..4b63fadfb9e --- /dev/null +++ b/tests/fixtures/compatibility-mode/docker-compose.yml @@ -0,0 +1,28 @@ +version: '3.5' +services: + foo: + image: alpine:3.10.1 + command: /bin/true + deploy: + replicas: 3 + restart_policy: + condition: any + max_attempts: 7 + resources: + limits: + memory: 300M + cpus: '0.7' + reservations: + memory: 100M + volumes: + - foo:/bar + networks: + - bar + +volumes: + foo: + driver: default + +networks: + bar: + attachable: true diff --git a/tests/fixtures/default-env-file/.env b/tests/fixtures/default-env-file/.env new file mode 100644 index 00000000000..9056de724cd --- /dev/null +++ b/tests/fixtures/default-env-file/.env @@ -0,0 +1,4 @@ +IMAGE=alpine:latest +COMMAND=true +PORT1=5643 +PORT2=9999 diff --git a/tests/fixtures/default-env-file/.env2 b/tests/fixtures/default-env-file/.env2 new file mode 100644 index 00000000000..d754523fc50 --- /dev/null +++ b/tests/fixtures/default-env-file/.env2 @@ -0,0 +1,4 @@ +IMAGE=alpine:latest +COMMAND=false +PORT1=5644 +PORT2=9998 diff --git a/tests/fixtures/default-env-file/alt/.env b/tests/fixtures/default-env-file/alt/.env new file mode 100644 index 00000000000..981c7207b19 --- /dev/null +++ b/tests/fixtures/default-env-file/alt/.env @@ -0,0 +1,4 @@ +IMAGE=alpine:3.10.1 +COMMAND=echo uwu +PORT1=3341 +PORT2=4449 diff --git a/tests/fixtures/default-env-file/docker-compose.yml b/tests/fixtures/default-env-file/docker-compose.yml new file mode 100644 index 00000000000..79363586185 --- /dev/null +++ b/tests/fixtures/default-env-file/docker-compose.yml @@ -0,0 +1,8 @@ +version: '2.4' +services: + web: + image: ${IMAGE} + command: ${COMMAND} + ports: + - $PORT1 + - $PORT2 diff --git a/tests/fixtures/dockerfile-with-volume/Dockerfile b/tests/fixtures/dockerfile-with-volume/Dockerfile new file mode 100644 index 00000000000..f38e1d579ec --- /dev/null +++ b/tests/fixtures/dockerfile-with-volume/Dockerfile @@ -0,0 +1,4 @@ +FROM busybox:1.31.0-uclibc +LABEL com.docker.compose.test_image=true +VOLUME /data +CMD top diff --git a/tests/fixtures/duplicate-override-yaml-files/docker-compose.override.yaml b/tests/fixtures/duplicate-override-yaml-files/docker-compose.override.yaml new file mode 100644 index 00000000000..58c67348295 --- /dev/null +++ b/tests/fixtures/duplicate-override-yaml-files/docker-compose.override.yaml @@ -0,0 +1,3 @@ + +db: + command: "top" diff --git a/tests/fixtures/duplicate-override-yaml-files/docker-compose.override.yml b/tests/fixtures/duplicate-override-yaml-files/docker-compose.override.yml new file mode 100644 index 00000000000..f1b8ef181f7 --- /dev/null +++ b/tests/fixtures/duplicate-override-yaml-files/docker-compose.override.yml @@ -0,0 +1,3 @@ + +db: + command: "sleep 300" diff --git a/tests/fixtures/duplicate-override-yaml-files/docker-compose.yml b/tests/fixtures/duplicate-override-yaml-files/docker-compose.yml new file mode 100644 index 00000000000..6880435b384 --- /dev/null +++ b/tests/fixtures/duplicate-override-yaml-files/docker-compose.yml @@ -0,0 +1,10 @@ + +web: + image: busybox:1.31.0-uclibc + command: "sleep 100" + links: + - db + +db: + image: busybox:1.31.0-uclibc + command: "sleep 200" diff --git a/tests/fixtures/echo-services-dependencies/docker-compose.yml b/tests/fixtures/echo-services-dependencies/docker-compose.yml new file mode 100644 index 00000000000..5329e0033df --- /dev/null +++ b/tests/fixtures/echo-services-dependencies/docker-compose.yml @@ -0,0 +1,10 @@ +version: "2.0" +services: + simple: + image: busybox:1.31.0-uclibc + command: echo simple + depends_on: + - another + another: + image: busybox:1.31.0-uclibc + command: echo another diff --git a/tests/fixtures/echo-services/docker-compose.yml b/tests/fixtures/echo-services/docker-compose.yml new file mode 100644 index 00000000000..75fc45d95bb --- /dev/null +++ b/tests/fixtures/echo-services/docker-compose.yml @@ -0,0 +1,6 @@ +simple: + image: busybox:1.31.0-uclibc + command: echo simple +another: + image: busybox:1.31.0-uclibc + command: echo another diff --git a/tests/fixtures/entrypoint-composefile/docker-compose.yml b/tests/fixtures/entrypoint-composefile/docker-compose.yml new file mode 100644 index 00000000000..e9880973fb2 --- /dev/null +++ b/tests/fixtures/entrypoint-composefile/docker-compose.yml @@ -0,0 +1,6 @@ +version: "2" +services: + test: + image: busybox + entrypoint: printf + command: default args diff --git a/tests/fixtures/entrypoint-dockerfile/Dockerfile b/tests/fixtures/entrypoint-dockerfile/Dockerfile new file mode 100644 index 00000000000..30ec50bac09 --- /dev/null +++ b/tests/fixtures/entrypoint-dockerfile/Dockerfile @@ -0,0 +1,4 @@ +FROM busybox:1.31.0-uclibc +LABEL com.docker.compose.test_image=true +ENTRYPOINT ["printf"] +CMD ["default", "args"] diff --git a/tests/fixtures/entrypoint-dockerfile/docker-compose.yml b/tests/fixtures/entrypoint-dockerfile/docker-compose.yml new file mode 100644 index 00000000000..8318e61f31b --- /dev/null +++ b/tests/fixtures/entrypoint-dockerfile/docker-compose.yml @@ -0,0 +1,4 @@ +version: "2" +services: + test: + build: . diff --git a/tests/fixtures/env-file-override/.env.conf b/tests/fixtures/env-file-override/.env.conf new file mode 100644 index 00000000000..90b8b495a4b --- /dev/null +++ b/tests/fixtures/env-file-override/.env.conf @@ -0,0 +1,2 @@ +WHEREAMI +DEFAULT_CONF_LOADED=true diff --git a/pkg/e2e/fixtures/environment/env-priority/.env.override b/tests/fixtures/env-file-override/.env.override similarity index 100% rename from pkg/e2e/fixtures/environment/env-priority/.env.override rename to tests/fixtures/env-file-override/.env.override diff --git a/tests/fixtures/env-file-override/docker-compose.yml b/tests/fixtures/env-file-override/docker-compose.yml new file mode 100644 index 00000000000..fdae6d826c7 --- /dev/null +++ b/tests/fixtures/env-file-override/docker-compose.yml @@ -0,0 +1,6 @@ +version: '3.7' +services: + test: + image: busybox + env_file: .env.conf + entrypoint: env diff --git a/tests/fixtures/env-file/docker-compose.yml b/tests/fixtures/env-file/docker-compose.yml new file mode 100644 index 00000000000..d9366ace233 --- /dev/null +++ b/tests/fixtures/env-file/docker-compose.yml @@ -0,0 +1,4 @@ +web: + image: busybox + command: /bin/true + env_file: ./test.env diff --git a/tests/fixtures/env-file/test.env b/tests/fixtures/env-file/test.env new file mode 100644 index 00000000000..d99cd41a4b7 --- /dev/null +++ b/tests/fixtures/env-file/test.env @@ -0,0 +1 @@ +FOO=1 diff --git a/tests/fixtures/env/one.env b/tests/fixtures/env/one.env new file mode 100644 index 00000000000..45b59fe6337 --- /dev/null +++ b/tests/fixtures/env/one.env @@ -0,0 +1,11 @@ +# Keep the blank lines and comments in this file, please + +ONE=2 +TWO=1 + + # (thanks) + +THREE=3 + +FOO=bar +# FOO=somethingelse diff --git a/tests/fixtures/env/resolve.env b/tests/fixtures/env/resolve.env new file mode 100644 index 00000000000..b4f76b29edc --- /dev/null +++ b/tests/fixtures/env/resolve.env @@ -0,0 +1,4 @@ +FILE_DEF=bär +FILE_DEF_EMPTY= +ENV_DEF +NO_DEF diff --git a/tests/fixtures/env/three.env b/tests/fixtures/env/three.env new file mode 100644 index 00000000000..c2da74f19ee --- /dev/null +++ b/tests/fixtures/env/three.env @@ -0,0 +1,2 @@ +FOO=NO $ENV VAR +DOO=NO ${ENV} VAR diff --git a/tests/fixtures/env/two.env b/tests/fixtures/env/two.env new file mode 100644 index 00000000000..3b21871a046 --- /dev/null +++ b/tests/fixtures/env/two.env @@ -0,0 +1,2 @@ +FOO=baz +DOO=dah diff --git a/tests/fixtures/environment-composefile/docker-compose.yml b/tests/fixtures/environment-composefile/docker-compose.yml new file mode 100644 index 00000000000..5650c7c8e7d --- /dev/null +++ b/tests/fixtures/environment-composefile/docker-compose.yml @@ -0,0 +1,7 @@ +service: + image: busybox:1.31.0-uclibc + command: top + + environment: + foo: bar + hello: world diff --git a/tests/fixtures/environment-exec/docker-compose.yml b/tests/fixtures/environment-exec/docker-compose.yml new file mode 100644 index 00000000000..e284ba8cb36 --- /dev/null +++ b/tests/fixtures/environment-exec/docker-compose.yml @@ -0,0 +1,10 @@ +version: "2.2" + +services: + service: + image: busybox:1.27.2 + command: top + + environment: + foo: bar + hello: world diff --git a/tests/fixtures/environment-interpolation-with-defaults/docker-compose.yml b/tests/fixtures/environment-interpolation-with-defaults/docker-compose.yml new file mode 100644 index 00000000000..42e7cbb6a5b --- /dev/null +++ b/tests/fixtures/environment-interpolation-with-defaults/docker-compose.yml @@ -0,0 +1,13 @@ +version: "2.1" + +services: + web: + # set value with default, default must be ignored + image: ${IMAGE:-alpine} + + # unset value with default value + ports: + - "${HOST_PORT:-80}:8000" + + # unset value with empty default + hostname: "host-${UNSET_VALUE:-}" diff --git a/tests/fixtures/environment-interpolation/docker-compose.yml b/tests/fixtures/environment-interpolation/docker-compose.yml new file mode 100644 index 00000000000..7ed43a812cb --- /dev/null +++ b/tests/fixtures/environment-interpolation/docker-compose.yml @@ -0,0 +1,17 @@ +web: + # unbracketed name + image: $IMAGE + + # array element + ports: + - "${HOST_PORT}:8000" + + # dictionary item value + labels: + mylabel: "${LABEL_VALUE}" + + # unset value + hostname: "host-${UNSET_VALUE}" + + # escaped interpolation + command: "$${ESCAPED}" diff --git a/tests/fixtures/exec-novalue-var/docker-compose.yml b/tests/fixtures/exec-novalue-var/docker-compose.yml new file mode 100644 index 00000000000..1f8502f957a --- /dev/null +++ b/tests/fixtures/exec-novalue-var/docker-compose.yml @@ -0,0 +1,6 @@ +version: '3' +services: + nginx: + image: nginx + environment: + - CHECK_VAR=${MYVAR} diff --git a/tests/fixtures/exit-code-from/docker-compose.yml b/tests/fixtures/exit-code-from/docker-compose.yml new file mode 100644 index 00000000000..c38bd549b5b --- /dev/null +++ b/tests/fixtures/exit-code-from/docker-compose.yml @@ -0,0 +1,6 @@ +simple: + image: busybox:1.31.0-uclibc + command: sh -c "echo hello && tail -f /dev/null" +another: + image: busybox:1.31.0-uclibc + command: /bin/false diff --git a/tests/fixtures/expose-composefile/docker-compose.yml b/tests/fixtures/expose-composefile/docker-compose.yml new file mode 100644 index 00000000000..c2a3dc42490 --- /dev/null +++ b/tests/fixtures/expose-composefile/docker-compose.yml @@ -0,0 +1,11 @@ + +simple: + image: busybox:1.31.0-uclibc + command: top + expose: + - '3000' + - '3001/tcp' + - '3001/udp' + - '3002-3003' + - '3004-3005/tcp' + - '3006-3007/udp' diff --git a/tests/fixtures/extends/circle-1.yml b/tests/fixtures/extends/circle-1.yml new file mode 100644 index 00000000000..d88ea61d0ea --- /dev/null +++ b/tests/fixtures/extends/circle-1.yml @@ -0,0 +1,12 @@ +foo: + image: busybox +bar: + image: busybox +web: + extends: + file: circle-2.yml + service: other +baz: + image: busybox +quux: + image: busybox diff --git a/tests/fixtures/extends/circle-2.yml b/tests/fixtures/extends/circle-2.yml new file mode 100644 index 00000000000..de05bc8da0a --- /dev/null +++ b/tests/fixtures/extends/circle-2.yml @@ -0,0 +1,12 @@ +foo: + image: busybox +bar: + image: busybox +other: + extends: + file: circle-1.yml + service: web +baz: + image: busybox +quux: + image: busybox diff --git a/tests/fixtures/extends/common-env-labels-ulimits.yml b/tests/fixtures/extends/common-env-labels-ulimits.yml new file mode 100644 index 00000000000..09efb4e75d2 --- /dev/null +++ b/tests/fixtures/extends/common-env-labels-ulimits.yml @@ -0,0 +1,13 @@ +web: + extends: + file: common.yml + service: web + environment: + - FOO=2 + - BAZ=3 + labels: ['label=one'] + ulimits: + nproc: 65535 + memlock: + soft: 1024 + hard: 2048 diff --git a/tests/fixtures/extends/common.yml b/tests/fixtures/extends/common.yml new file mode 100644 index 00000000000..b2d86aa4caf --- /dev/null +++ b/tests/fixtures/extends/common.yml @@ -0,0 +1,7 @@ +web: + image: busybox + command: /bin/true + net: host + environment: + - FOO=1 + - BAR=1 diff --git a/tests/fixtures/extends/docker-compose.yml b/tests/fixtures/extends/docker-compose.yml new file mode 100644 index 00000000000..8e37d404a0f --- /dev/null +++ b/tests/fixtures/extends/docker-compose.yml @@ -0,0 +1,17 @@ +myweb: + extends: + file: common.yml + service: web + command: top + links: + - "mydb:db" + environment: + # leave FOO alone + # override BAR + BAR: "2" + # add BAZ + BAZ: "2" + net: bridge +mydb: + image: busybox + command: top diff --git a/tests/fixtures/extends/healthcheck-1.yml b/tests/fixtures/extends/healthcheck-1.yml new file mode 100644 index 00000000000..4c311e62caa --- /dev/null +++ b/tests/fixtures/extends/healthcheck-1.yml @@ -0,0 +1,9 @@ +version: '2.1' +services: + demo: + image: foobar:latest + healthcheck: + test: ["CMD", "/health.sh"] + interval: 10s + timeout: 5s + retries: 36 diff --git a/tests/fixtures/extends/healthcheck-2.yml b/tests/fixtures/extends/healthcheck-2.yml new file mode 100644 index 00000000000..11bc9f09da8 --- /dev/null +++ b/tests/fixtures/extends/healthcheck-2.yml @@ -0,0 +1,6 @@ +version: '2.1' +services: + demo: + extends: + file: healthcheck-1.yml + service: demo diff --git a/tests/fixtures/extends/invalid-links.yml b/tests/fixtures/extends/invalid-links.yml new file mode 100644 index 00000000000..cea740cb7b1 --- /dev/null +++ b/tests/fixtures/extends/invalid-links.yml @@ -0,0 +1,11 @@ +mydb: + build: '.' +myweb: + build: '.' + extends: + service: web + command: top +web: + build: '.' + links: + - "mydb:db" diff --git a/tests/fixtures/extends/invalid-net-v2.yml b/tests/fixtures/extends/invalid-net-v2.yml new file mode 100644 index 00000000000..7ba714e89db --- /dev/null +++ b/tests/fixtures/extends/invalid-net-v2.yml @@ -0,0 +1,12 @@ +version: "2" +services: + myweb: + build: '.' + extends: + service: web + command: top + web: + build: '.' + network_mode: "service:net" + net: + build: '.' diff --git a/tests/fixtures/extends/invalid-net.yml b/tests/fixtures/extends/invalid-net.yml new file mode 100644 index 00000000000..fbcd020bcf4 --- /dev/null +++ b/tests/fixtures/extends/invalid-net.yml @@ -0,0 +1,8 @@ +myweb: + build: '.' + extends: + service: web + command: top +web: + build: '.' + net: "container:db" diff --git a/tests/fixtures/extends/invalid-volumes.yml b/tests/fixtures/extends/invalid-volumes.yml new file mode 100644 index 00000000000..3db0118e0ef --- /dev/null +++ b/tests/fixtures/extends/invalid-volumes.yml @@ -0,0 +1,9 @@ +myweb: + build: '.' + extends: + service: web + command: top +web: + build: '.' + volumes_from: + - "db" diff --git a/tests/fixtures/extends/nested-intermediate.yml b/tests/fixtures/extends/nested-intermediate.yml new file mode 100644 index 00000000000..c2dd8c94329 --- /dev/null +++ b/tests/fixtures/extends/nested-intermediate.yml @@ -0,0 +1,6 @@ +webintermediate: + extends: + file: common.yml + service: web + environment: + - "FOO=2" diff --git a/tests/fixtures/extends/nested.yml b/tests/fixtures/extends/nested.yml new file mode 100644 index 00000000000..6025e6d530d --- /dev/null +++ b/tests/fixtures/extends/nested.yml @@ -0,0 +1,6 @@ +myweb: + extends: + file: nested-intermediate.yml + service: webintermediate + environment: + - "BAR=2" diff --git a/tests/fixtures/extends/no-file-specified.yml b/tests/fixtures/extends/no-file-specified.yml new file mode 100644 index 00000000000..40e43c4bf4c --- /dev/null +++ b/tests/fixtures/extends/no-file-specified.yml @@ -0,0 +1,9 @@ +myweb: + extends: + service: web + environment: + - "BAR=1" +web: + image: busybox + environment: + - "BAZ=3" diff --git a/tests/fixtures/extends/nonexistent-path-base.yml b/tests/fixtures/extends/nonexistent-path-base.yml new file mode 100644 index 00000000000..4e6c82b0d72 --- /dev/null +++ b/tests/fixtures/extends/nonexistent-path-base.yml @@ -0,0 +1,6 @@ +dnebase: + build: nonexistent.path + command: /bin/true + environment: + - FOO=1 + - BAR=1 diff --git a/tests/fixtures/extends/nonexistent-path-child.yml b/tests/fixtures/extends/nonexistent-path-child.yml new file mode 100644 index 00000000000..d3b732f2a3c --- /dev/null +++ b/tests/fixtures/extends/nonexistent-path-child.yml @@ -0,0 +1,8 @@ +dnechild: + extends: + file: nonexistent-path-base.yml + service: dnebase + image: busybox + command: /bin/true + environment: + - BAR=2 diff --git a/tests/fixtures/extends/nonexistent-service.yml b/tests/fixtures/extends/nonexistent-service.yml new file mode 100644 index 00000000000..e9e17f1bdc1 --- /dev/null +++ b/tests/fixtures/extends/nonexistent-service.yml @@ -0,0 +1,4 @@ +web: + image: busybox + extends: + service: foo diff --git a/tests/fixtures/extends/service-with-invalid-schema.yml b/tests/fixtures/extends/service-with-invalid-schema.yml new file mode 100644 index 00000000000..00c36647ef5 --- /dev/null +++ b/tests/fixtures/extends/service-with-invalid-schema.yml @@ -0,0 +1,4 @@ +myweb: + extends: + file: valid-composite-extends.yml + service: web diff --git a/tests/fixtures/extends/service-with-valid-composite-extends.yml b/tests/fixtures/extends/service-with-valid-composite-extends.yml new file mode 100644 index 00000000000..6c419ed0702 --- /dev/null +++ b/tests/fixtures/extends/service-with-valid-composite-extends.yml @@ -0,0 +1,5 @@ +myweb: + build: '.' + extends: + file: 'valid-composite-extends.yml' + service: web diff --git a/tests/fixtures/extends/specify-file-as-self.yml b/tests/fixtures/extends/specify-file-as-self.yml new file mode 100644 index 00000000000..c24f10bc92b --- /dev/null +++ b/tests/fixtures/extends/specify-file-as-self.yml @@ -0,0 +1,17 @@ +myweb: + extends: + file: specify-file-as-self.yml + service: web + environment: + - "BAR=1" +web: + extends: + file: specify-file-as-self.yml + service: otherweb + image: busybox + environment: + - "BAZ=3" +otherweb: + image: busybox + environment: + - "YEP=1" diff --git a/tests/fixtures/extends/valid-common-config.yml b/tests/fixtures/extends/valid-common-config.yml new file mode 100644 index 00000000000..d8f13e7a863 --- /dev/null +++ b/tests/fixtures/extends/valid-common-config.yml @@ -0,0 +1,6 @@ +myweb: + build: '.' + extends: + file: valid-common.yml + service: common-config + command: top diff --git a/tests/fixtures/extends/valid-common.yml b/tests/fixtures/extends/valid-common.yml new file mode 100644 index 00000000000..07ad68e3e7a --- /dev/null +++ b/tests/fixtures/extends/valid-common.yml @@ -0,0 +1,3 @@ +common-config: + environment: + - FOO=1 diff --git a/tests/fixtures/extends/valid-composite-extends.yml b/tests/fixtures/extends/valid-composite-extends.yml new file mode 100644 index 00000000000..8816c3f3b2f --- /dev/null +++ b/tests/fixtures/extends/valid-composite-extends.yml @@ -0,0 +1,2 @@ +web: + command: top diff --git a/tests/fixtures/extends/valid-interpolation-2.yml b/tests/fixtures/extends/valid-interpolation-2.yml new file mode 100644 index 00000000000..cb7bd93fc2a --- /dev/null +++ b/tests/fixtures/extends/valid-interpolation-2.yml @@ -0,0 +1,3 @@ +web: + build: '.' + hostname: "host-${HOSTNAME_VALUE}" diff --git a/tests/fixtures/extends/valid-interpolation.yml b/tests/fixtures/extends/valid-interpolation.yml new file mode 100644 index 00000000000..68e8740fb49 --- /dev/null +++ b/tests/fixtures/extends/valid-interpolation.yml @@ -0,0 +1,5 @@ +myweb: + extends: + service: web + file: valid-interpolation-2.yml + command: top diff --git a/tests/fixtures/extends/verbose-and-shorthand.yml b/tests/fixtures/extends/verbose-and-shorthand.yml new file mode 100644 index 00000000000..d381630275e --- /dev/null +++ b/tests/fixtures/extends/verbose-and-shorthand.yml @@ -0,0 +1,15 @@ +base: + image: busybox + environment: + - "BAR=1" + +verbose: + extends: + service: base + environment: + - "FOO=1" + +shorthand: + extends: base + environment: + - "FOO=2" diff --git a/tests/fixtures/flag-as-service-name/Dockerfile b/tests/fixtures/flag-as-service-name/Dockerfile new file mode 100644 index 00000000000..098ff3eb195 --- /dev/null +++ b/tests/fixtures/flag-as-service-name/Dockerfile @@ -0,0 +1,3 @@ +FROM busybox:1.27.2 +LABEL com.docker.compose.test_image=true +CMD echo "success" diff --git a/tests/fixtures/flag-as-service-name/docker-compose.yml b/tests/fixtures/flag-as-service-name/docker-compose.yml new file mode 100644 index 00000000000..5b519a63efe --- /dev/null +++ b/tests/fixtures/flag-as-service-name/docker-compose.yml @@ -0,0 +1,12 @@ +version: "2" +services: + --test-service: + image: busybox:1.27.0.2 + build: . + command: top + ports: + - "8080:80" + + --log-service: + image: busybox:1.31.0-uclibc + command: sh -c "echo hello && tail -f /dev/null" diff --git a/tests/fixtures/healthcheck/docker-compose.yml b/tests/fixtures/healthcheck/docker-compose.yml new file mode 100644 index 00000000000..2c45b8d8cd4 --- /dev/null +++ b/tests/fixtures/healthcheck/docker-compose.yml @@ -0,0 +1,24 @@ +version: "3" +services: + passes: + image: busybox + command: top + healthcheck: + test: "/bin/true" + interval: 1s + timeout: 30m + retries: 1 + + fails: + image: busybox + command: top + healthcheck: + test: ["CMD", "/bin/false"] + interval: 2.5s + retries: 2 + + disabled: + image: busybox + command: top + healthcheck: + disable: true diff --git a/tests/fixtures/images-service-tag/Dockerfile b/tests/fixtures/images-service-tag/Dockerfile new file mode 100644 index 00000000000..1e1a1b2e0e9 --- /dev/null +++ b/tests/fixtures/images-service-tag/Dockerfile @@ -0,0 +1,2 @@ +FROM busybox:1.31.0-uclibc +RUN touch /foo diff --git a/tests/fixtures/images-service-tag/docker-compose.yml b/tests/fixtures/images-service-tag/docker-compose.yml new file mode 100644 index 00000000000..a46b32bf578 --- /dev/null +++ b/tests/fixtures/images-service-tag/docker-compose.yml @@ -0,0 +1,11 @@ +version: "2.4" +services: + foo1: + build: . + image: test:dev + foo2: + build: . + image: test:prod + foo3: + build: . + image: test:latest diff --git a/tests/fixtures/invalid-composefile/invalid.yml b/tests/fixtures/invalid-composefile/invalid.yml new file mode 100644 index 00000000000..0e74be440a8 --- /dev/null +++ b/tests/fixtures/invalid-composefile/invalid.yml @@ -0,0 +1,5 @@ + +notaservice: oops + +web: + image: 'alpine:edge' diff --git a/tests/fixtures/ipc-mode/docker-compose.yml b/tests/fixtures/ipc-mode/docker-compose.yml new file mode 100644 index 00000000000..c58ce24484d --- /dev/null +++ b/tests/fixtures/ipc-mode/docker-compose.yml @@ -0,0 +1,17 @@ +version: "2.4" + +services: + service: + image: busybox + command: top + ipc: "service:shareable" + + container: + image: busybox + command: top + ipc: "container:composetest_ipc_mode_container" + + shareable: + image: busybox + command: top + ipc: shareable diff --git a/tests/fixtures/links-composefile/docker-compose.yml b/tests/fixtures/links-composefile/docker-compose.yml new file mode 100644 index 00000000000..0a2f3d9ef77 --- /dev/null +++ b/tests/fixtures/links-composefile/docker-compose.yml @@ -0,0 +1,11 @@ +db: + image: busybox:1.27.2 + command: top +web: + image: busybox:1.27.2 + command: top + links: + - db:db +console: + image: busybox:1.27.2 + command: top diff --git a/tests/fixtures/logging-composefile-legacy/docker-compose.yml b/tests/fixtures/logging-composefile-legacy/docker-compose.yml new file mode 100644 index 00000000000..efac1d6a353 --- /dev/null +++ b/tests/fixtures/logging-composefile-legacy/docker-compose.yml @@ -0,0 +1,10 @@ +simple: + image: busybox:1.31.0-uclibc + command: top + log_driver: "none" +another: + image: busybox:1.31.0-uclibc + command: top + log_driver: "json-file" + log_opt: + max-size: "10m" diff --git a/tests/fixtures/logging-composefile/docker-compose.yml b/tests/fixtures/logging-composefile/docker-compose.yml new file mode 100644 index 00000000000..ac231b89862 --- /dev/null +++ b/tests/fixtures/logging-composefile/docker-compose.yml @@ -0,0 +1,14 @@ +version: "2" +services: + simple: + image: busybox:1.31.0-uclibc + command: top + logging: + driver: "none" + another: + image: busybox:1.31.0-uclibc + command: top + logging: + driver: "json-file" + options: + max-size: "10m" diff --git a/tests/fixtures/logs-composefile/docker-compose.yml b/tests/fixtures/logs-composefile/docker-compose.yml new file mode 100644 index 00000000000..3ffaa98491f --- /dev/null +++ b/tests/fixtures/logs-composefile/docker-compose.yml @@ -0,0 +1,6 @@ +simple: + image: busybox:1.31.0-uclibc + command: sh -c "sleep 1 && echo hello && tail -f /dev/null" +another: + image: busybox:1.31.0-uclibc + command: sh -c "sleep 1 && echo test" diff --git a/tests/fixtures/logs-restart-composefile/docker-compose.yml b/tests/fixtures/logs-restart-composefile/docker-compose.yml new file mode 100644 index 00000000000..2179d54de4d --- /dev/null +++ b/tests/fixtures/logs-restart-composefile/docker-compose.yml @@ -0,0 +1,7 @@ +simple: + image: busybox:1.31.0-uclibc + command: sh -c "echo hello && tail -f /dev/null" +another: + image: busybox:1.31.0-uclibc + command: sh -c "sleep 2 && echo world && /bin/false" + restart: "on-failure:2" diff --git a/tests/fixtures/logs-tail-composefile/docker-compose.yml b/tests/fixtures/logs-tail-composefile/docker-compose.yml new file mode 100644 index 00000000000..18dad986ec7 --- /dev/null +++ b/tests/fixtures/logs-tail-composefile/docker-compose.yml @@ -0,0 +1,3 @@ +simple: + image: busybox:1.31.0-uclibc + command: sh -c "echo w && echo x && echo y && echo z" diff --git a/tests/fixtures/longer-filename-composefile/docker-compose.yaml b/tests/fixtures/longer-filename-composefile/docker-compose.yaml new file mode 100644 index 00000000000..5dadce44a63 --- /dev/null +++ b/tests/fixtures/longer-filename-composefile/docker-compose.yaml @@ -0,0 +1,3 @@ +definedinyamlnotyml: + image: busybox:1.31.0-uclibc + command: top diff --git a/tests/fixtures/multiple-composefiles/compose2.yml b/tests/fixtures/multiple-composefiles/compose2.yml new file mode 100644 index 00000000000..530d92df685 --- /dev/null +++ b/tests/fixtures/multiple-composefiles/compose2.yml @@ -0,0 +1,3 @@ +yetanother: + image: busybox:1.31.0-uclibc + command: top diff --git a/tests/fixtures/multiple-composefiles/docker-compose.yml b/tests/fixtures/multiple-composefiles/docker-compose.yml new file mode 100644 index 00000000000..09cc9519fed --- /dev/null +++ b/tests/fixtures/multiple-composefiles/docker-compose.yml @@ -0,0 +1,6 @@ +simple: + image: busybox:1.31.0-uclibc + command: top +another: + image: busybox:1.31.0-uclibc + command: top diff --git a/tests/fixtures/net-container/docker-compose.yml b/tests/fixtures/net-container/docker-compose.yml new file mode 100644 index 00000000000..b5506e0e17b --- /dev/null +++ b/tests/fixtures/net-container/docker-compose.yml @@ -0,0 +1,7 @@ +foo: + image: busybox + command: top + net: "container:bar" +bar: + image: busybox + command: top diff --git a/tests/fixtures/net-container/v2-invalid.yml b/tests/fixtures/net-container/v2-invalid.yml new file mode 100644 index 00000000000..9b846295828 --- /dev/null +++ b/tests/fixtures/net-container/v2-invalid.yml @@ -0,0 +1,10 @@ +version: "2" + +services: + foo: + image: busybox + command: top + bar: + image: busybox + command: top + net: "container:foo" diff --git a/tests/fixtures/networks/bridge.yml b/tests/fixtures/networks/bridge.yml new file mode 100644 index 00000000000..9fa7db820d0 --- /dev/null +++ b/tests/fixtures/networks/bridge.yml @@ -0,0 +1,9 @@ +version: "2" + +services: + web: + image: busybox + command: top + networks: + - bridge + - default diff --git a/tests/fixtures/networks/default-network-config.yml b/tests/fixtures/networks/default-network-config.yml new file mode 100644 index 00000000000..556ca9805d6 --- /dev/null +++ b/tests/fixtures/networks/default-network-config.yml @@ -0,0 +1,13 @@ +version: "2" +services: + simple: + image: busybox:1.31.0-uclibc + command: top + another: + image: busybox:1.31.0-uclibc + command: top +networks: + default: + driver: bridge + driver_opts: + "com.docker.network.bridge.enable_icc": "false" diff --git a/tests/fixtures/networks/docker-compose.yml b/tests/fixtures/networks/docker-compose.yml new file mode 100644 index 00000000000..b911c752bd6 --- /dev/null +++ b/tests/fixtures/networks/docker-compose.yml @@ -0,0 +1,21 @@ +version: "2" + +services: + web: + image: alpine:3.10.1 + command: top + networks: ["front"] + app: + image: alpine:3.10.1 + command: top + networks: ["front", "back"] + links: + - "db:database" + db: + image: alpine:3.10.1 + command: top + networks: ["back"] + +networks: + front: {} + back: {} diff --git a/tests/fixtures/networks/external-default.yml b/tests/fixtures/networks/external-default.yml new file mode 100644 index 00000000000..42a39565675 --- /dev/null +++ b/tests/fixtures/networks/external-default.yml @@ -0,0 +1,12 @@ +version: "2" +services: + simple: + image: busybox:1.31.0-uclibc + command: top + another: + image: busybox:1.31.0-uclibc + command: top +networks: + default: + external: + name: composetest_external_network diff --git a/tests/fixtures/networks/external-networks-v3-5.yml b/tests/fixtures/networks/external-networks-v3-5.yml new file mode 100644 index 00000000000..9ac7b14b5f7 --- /dev/null +++ b/tests/fixtures/networks/external-networks-v3-5.yml @@ -0,0 +1,17 @@ +version: "3.5" + +services: + web: + image: busybox + command: top + networks: + - foo + - bar + +networks: + foo: + external: true + name: some_foo + bar: + external: + name: some_bar diff --git a/tests/fixtures/networks/external-networks.yml b/tests/fixtures/networks/external-networks.yml new file mode 100644 index 00000000000..db75b780660 --- /dev/null +++ b/tests/fixtures/networks/external-networks.yml @@ -0,0 +1,16 @@ +version: "2" + +services: + web: + image: busybox + command: top + networks: + - networks_foo + - bar + +networks: + networks_foo: + external: true + bar: + external: + name: networks_bar diff --git a/tests/fixtures/networks/missing-network.yml b/tests/fixtures/networks/missing-network.yml new file mode 100644 index 00000000000..41012535d14 --- /dev/null +++ b/tests/fixtures/networks/missing-network.yml @@ -0,0 +1,10 @@ +version: "2" + +services: + web: + image: busybox + command: top + networks: ["foo"] + +networks: + bar: {} diff --git a/tests/fixtures/networks/network-aliases.yml b/tests/fixtures/networks/network-aliases.yml new file mode 100644 index 00000000000..8cf7d5af941 --- /dev/null +++ b/tests/fixtures/networks/network-aliases.yml @@ -0,0 +1,16 @@ +version: "2" + +services: + web: + image: busybox + command: top + networks: + front: + aliases: + - forward_facing + - ahead + back: + +networks: + front: {} + back: {} diff --git a/tests/fixtures/networks/network-internal.yml b/tests/fixtures/networks/network-internal.yml new file mode 100755 index 00000000000..1fa339b1f85 --- /dev/null +++ b/tests/fixtures/networks/network-internal.yml @@ -0,0 +1,13 @@ +version: "2" + +services: + web: + image: busybox + command: top + networks: + - internal + +networks: + internal: + driver: bridge + internal: True diff --git a/tests/fixtures/networks/network-label.yml b/tests/fixtures/networks/network-label.yml new file mode 100644 index 00000000000..fdb24f652d7 --- /dev/null +++ b/tests/fixtures/networks/network-label.yml @@ -0,0 +1,13 @@ +version: "2.1" + +services: + web: + image: busybox + command: top + networks: + - network_with_label + +networks: + network_with_label: + labels: + - "label_key=label_val" diff --git a/tests/fixtures/networks/network-mode.yml b/tests/fixtures/networks/network-mode.yml new file mode 100644 index 00000000000..e4d070b4444 --- /dev/null +++ b/tests/fixtures/networks/network-mode.yml @@ -0,0 +1,27 @@ +version: "2" + +services: + bridge: + image: busybox + command: top + network_mode: bridge + + service: + image: busybox + command: top + network_mode: "service:bridge" + + container: + image: busybox + command: top + network_mode: "container:composetest_network_mode_container" + + host: + image: busybox + command: top + network_mode: host + + none: + image: busybox + command: top + network_mode: none diff --git a/tests/fixtures/networks/network-static-addresses.yml b/tests/fixtures/networks/network-static-addresses.yml new file mode 100755 index 00000000000..f820ff6a4a4 --- /dev/null +++ b/tests/fixtures/networks/network-static-addresses.yml @@ -0,0 +1,23 @@ +version: "2" + +services: + web: + image: busybox + command: top + networks: + static_test: + ipv4_address: 172.16.100.100 + ipv6_address: fe80::1001:100 + +networks: + static_test: + driver: bridge + driver_opts: + com.docker.network.enable_ipv6: "true" + ipam: + driver: default + config: + - subnet: 172.16.100.0/24 + gateway: 172.16.100.1 + - subnet: fe80::/64 + gateway: fe80::1001:1 diff --git a/tests/fixtures/no-build/docker-compose.yml b/tests/fixtures/no-build/docker-compose.yml new file mode 100644 index 00000000000..f320d17c394 --- /dev/null +++ b/tests/fixtures/no-build/docker-compose.yml @@ -0,0 +1,8 @@ +version: "3" +services: + my-alpine: + image: alpine:3.12 + container_name: alpine + entrypoint: 'echo It works!' + build: + context: /this/path/doesnt/exist # and we don't really care. We just want to run containers already pulled. diff --git a/tests/fixtures/no-composefile/.gitignore b/tests/fixtures/no-composefile/.gitignore new file mode 100644 index 00000000000..e69de29bb2d diff --git a/tests/fixtures/no-links-composefile/docker-compose.yml b/tests/fixtures/no-links-composefile/docker-compose.yml new file mode 100644 index 00000000000..54936f30617 --- /dev/null +++ b/tests/fixtures/no-links-composefile/docker-compose.yml @@ -0,0 +1,9 @@ +db: + image: busybox:1.31.0-uclibc + command: top +web: + image: busybox:1.31.0-uclibc + command: top +console: + image: busybox:1.31.0-uclibc + command: top diff --git a/tests/fixtures/no-services/docker-compose.yml b/tests/fixtures/no-services/docker-compose.yml new file mode 100644 index 00000000000..6e76ec0c5a9 --- /dev/null +++ b/tests/fixtures/no-services/docker-compose.yml @@ -0,0 +1,5 @@ +version: "2" + +networks: + foo: {} + bar: {} diff --git a/tests/fixtures/override-files/docker-compose.override.yml b/tests/fixtures/override-files/docker-compose.override.yml new file mode 100644 index 00000000000..b2c54060124 --- /dev/null +++ b/tests/fixtures/override-files/docker-compose.override.yml @@ -0,0 +1,7 @@ +version: '2.2' +services: + web: + command: "top" + + db: + command: "top" diff --git a/tests/fixtures/override-files/docker-compose.yml b/tests/fixtures/override-files/docker-compose.yml new file mode 100644 index 00000000000..0119ec73817 --- /dev/null +++ b/tests/fixtures/override-files/docker-compose.yml @@ -0,0 +1,10 @@ +version: '2.2' +services: + web: + image: busybox:1.31.0-uclibc + command: "sleep 200" + depends_on: + - db + db: + image: busybox:1.31.0-uclibc + command: "sleep 200" diff --git a/tests/fixtures/override-files/extra.yml b/tests/fixtures/override-files/extra.yml new file mode 100644 index 00000000000..d03c5096d55 --- /dev/null +++ b/tests/fixtures/override-files/extra.yml @@ -0,0 +1,10 @@ +version: '2.2' +services: + web: + depends_on: + - db + - other + + other: + image: busybox:1.31.0-uclibc + command: "top" diff --git a/tests/fixtures/override-yaml-files/docker-compose.override.yaml b/tests/fixtures/override-yaml-files/docker-compose.override.yaml new file mode 100644 index 00000000000..58c67348295 --- /dev/null +++ b/tests/fixtures/override-yaml-files/docker-compose.override.yaml @@ -0,0 +1,3 @@ + +db: + command: "top" diff --git a/tests/fixtures/override-yaml-files/docker-compose.yml b/tests/fixtures/override-yaml-files/docker-compose.yml new file mode 100644 index 00000000000..6880435b384 --- /dev/null +++ b/tests/fixtures/override-yaml-files/docker-compose.yml @@ -0,0 +1,10 @@ + +web: + image: busybox:1.31.0-uclibc + command: "sleep 100" + links: + - db + +db: + image: busybox:1.31.0-uclibc + command: "sleep 200" diff --git a/tests/fixtures/pid-mode/docker-compose.yml b/tests/fixtures/pid-mode/docker-compose.yml new file mode 100644 index 00000000000..fece5a9f08b --- /dev/null +++ b/tests/fixtures/pid-mode/docker-compose.yml @@ -0,0 +1,17 @@ +version: "2.2" + +services: + service: + image: busybox + command: top + pid: "service:container" + + container: + image: busybox + command: top + pid: "container:composetest_pid_mode_container" + + host: + image: busybox + command: top + pid: host diff --git a/tests/fixtures/ports-composefile-scale/docker-compose.yml b/tests/fixtures/ports-composefile-scale/docker-compose.yml new file mode 100644 index 00000000000..bdd39cef3e5 --- /dev/null +++ b/tests/fixtures/ports-composefile-scale/docker-compose.yml @@ -0,0 +1,6 @@ + +simple: + image: busybox:1.31.0-uclibc + command: /bin/sleep 300 + ports: + - '3000' diff --git a/tests/fixtures/ports-composefile/docker-compose.yml b/tests/fixtures/ports-composefile/docker-compose.yml new file mode 100644 index 00000000000..f4987027846 --- /dev/null +++ b/tests/fixtures/ports-composefile/docker-compose.yml @@ -0,0 +1,8 @@ + +simple: + image: busybox:1.31.0-uclibc + command: top + ports: + - '3000' + - '49152:3001' + - '49153-49154:3002-3003' diff --git a/tests/fixtures/ports-composefile/expanded-notation.yml b/tests/fixtures/ports-composefile/expanded-notation.yml new file mode 100644 index 00000000000..6510e4281f0 --- /dev/null +++ b/tests/fixtures/ports-composefile/expanded-notation.yml @@ -0,0 +1,15 @@ +version: '3.2' +services: + simple: + image: busybox:1.31.0-uclibc + command: top + ports: + - target: 3000 + - target: 3001 + published: 53222 + - target: 3002 + published: 53223 + protocol: tcp + - target: 3003 + published: 53224 + protocol: udp diff --git a/tests/fixtures/profiles/docker-compose.yml b/tests/fixtures/profiles/docker-compose.yml new file mode 100644 index 00000000000..ba77f03b446 --- /dev/null +++ b/tests/fixtures/profiles/docker-compose.yml @@ -0,0 +1,20 @@ +version: "3" +services: + foo: + image: busybox:1.31.0-uclibc + bar: + image: busybox:1.31.0-uclibc + profiles: + - test + baz: + image: busybox:1.31.0-uclibc + depends_on: + - bar + profiles: + - test + zot: + image: busybox:1.31.0-uclibc + depends_on: + - bar + profiles: + - debug diff --git a/tests/fixtures/profiles/merge-profiles.yml b/tests/fixtures/profiles/merge-profiles.yml new file mode 100644 index 00000000000..42b0cfa4308 --- /dev/null +++ b/tests/fixtures/profiles/merge-profiles.yml @@ -0,0 +1,5 @@ +version: "3" +services: + bar: + profiles: + - debug diff --git a/tests/fixtures/ps-services-filter/docker-compose.yml b/tests/fixtures/ps-services-filter/docker-compose.yml new file mode 100644 index 00000000000..180f515aaef --- /dev/null +++ b/tests/fixtures/ps-services-filter/docker-compose.yml @@ -0,0 +1,6 @@ +with_image: + image: busybox:1.31.0-uclibc + command: top +with_build: + build: ../build-ctx/ + command: top diff --git a/tests/fixtures/restart/docker-compose.yml b/tests/fixtures/restart/docker-compose.yml new file mode 100644 index 00000000000..ecfdfbf5377 --- /dev/null +++ b/tests/fixtures/restart/docker-compose.yml @@ -0,0 +1,17 @@ +version: "2" +services: + never: + image: busybox + restart: "no" + always: + image: busybox + restart: always + on-failure: + image: busybox + restart: on-failure + on-failure-5: + image: busybox + restart: "on-failure:5" + restart-null: + image: busybox + restart: "" diff --git a/tests/fixtures/run-labels/docker-compose.yml b/tests/fixtures/run-labels/docker-compose.yml new file mode 100644 index 00000000000..e3b237fd51a --- /dev/null +++ b/tests/fixtures/run-labels/docker-compose.yml @@ -0,0 +1,7 @@ +service: + image: busybox:1.31.0-uclibc + command: top + + labels: + foo: bar + hello: world diff --git a/tests/fixtures/run-workdir/docker-compose.yml b/tests/fixtures/run-workdir/docker-compose.yml new file mode 100644 index 00000000000..9d092a55fcb --- /dev/null +++ b/tests/fixtures/run-workdir/docker-compose.yml @@ -0,0 +1,4 @@ +service: + image: busybox:1.31.0-uclibc + working_dir: /etc + command: /bin/true diff --git a/tests/fixtures/scale/docker-compose.yml b/tests/fixtures/scale/docker-compose.yml new file mode 100644 index 00000000000..53ae1342d3c --- /dev/null +++ b/tests/fixtures/scale/docker-compose.yml @@ -0,0 +1,13 @@ +version: '2.2' +services: + web: + image: busybox + command: top + scale: 2 + db: + image: busybox + command: top + worker: + image: busybox + command: top + scale: 0 diff --git a/tests/fixtures/secrets/default b/tests/fixtures/secrets/default new file mode 100644 index 00000000000..f9dc20149ee --- /dev/null +++ b/tests/fixtures/secrets/default @@ -0,0 +1 @@ +This is the secret diff --git a/tests/fixtures/simple-composefile-volume-ready/docker-compose.merge.yml b/tests/fixtures/simple-composefile-volume-ready/docker-compose.merge.yml new file mode 100644 index 00000000000..45b626d02a1 --- /dev/null +++ b/tests/fixtures/simple-composefile-volume-ready/docker-compose.merge.yml @@ -0,0 +1,9 @@ +version: '2.2' +services: + simple: + image: busybox:1.31.0-uclibc + volumes: + - datastore:/data1 + +volumes: + datastore: diff --git a/tests/fixtures/simple-composefile-volume-ready/docker-compose.yml b/tests/fixtures/simple-composefile-volume-ready/docker-compose.yml new file mode 100644 index 00000000000..088d71c9990 --- /dev/null +++ b/tests/fixtures/simple-composefile-volume-ready/docker-compose.yml @@ -0,0 +1,2 @@ +simple: + image: busybox:1.31.0-uclibc diff --git a/tests/fixtures/simple-composefile-volume-ready/files/example.txt b/tests/fixtures/simple-composefile-volume-ready/files/example.txt new file mode 100644 index 00000000000..edb4d33905a --- /dev/null +++ b/tests/fixtures/simple-composefile-volume-ready/files/example.txt @@ -0,0 +1 @@ +FILE_CONTENT diff --git a/tests/fixtures/simple-composefile/can-build-pull-failures.yml b/tests/fixtures/simple-composefile/can-build-pull-failures.yml new file mode 100644 index 00000000000..1ffe8e0fbd8 --- /dev/null +++ b/tests/fixtures/simple-composefile/can-build-pull-failures.yml @@ -0,0 +1,6 @@ +version: '3' +services: + can_build: + image: nonexisting-image-but-can-build:latest + build: . + command: top diff --git a/tests/fixtures/simple-composefile/digest.yml b/tests/fixtures/simple-composefile/digest.yml new file mode 100644 index 00000000000..79f043baa02 --- /dev/null +++ b/tests/fixtures/simple-composefile/digest.yml @@ -0,0 +1,6 @@ +simple: + image: busybox:1.31.0-uclibc + command: top +digest: + image: busybox@sha256:38a203e1986cf79639cfb9b2e1d6e773de84002feea2d4eb006b52004ee8502d + command: top diff --git a/tests/fixtures/simple-composefile/docker-compose.yml b/tests/fixtures/simple-composefile/docker-compose.yml new file mode 100644 index 00000000000..b66a065275b --- /dev/null +++ b/tests/fixtures/simple-composefile/docker-compose.yml @@ -0,0 +1,6 @@ +simple: + image: busybox:1.27.2 + command: top +another: + image: busybox:1.31.0-uclibc + command: top diff --git a/tests/fixtures/simple-composefile/ignore-pull-failures.yml b/tests/fixtures/simple-composefile/ignore-pull-failures.yml new file mode 100644 index 00000000000..7e7d560dac4 --- /dev/null +++ b/tests/fixtures/simple-composefile/ignore-pull-failures.yml @@ -0,0 +1,6 @@ +simple: + image: busybox:1.31.0-uclibc + command: top +another: + image: nonexisting-image:latest + command: top diff --git a/tests/fixtures/simple-composefile/pull-with-build.yml b/tests/fixtures/simple-composefile/pull-with-build.yml new file mode 100644 index 00000000000..3bff35c515a --- /dev/null +++ b/tests/fixtures/simple-composefile/pull-with-build.yml @@ -0,0 +1,11 @@ +version: "3" +services: + build_simple: + image: simple + build: . + command: top + from_simple: + image: simple + another: + image: busybox:1.31.0-uclibc + command: top diff --git a/tests/fixtures/simple-dockerfile/Dockerfile b/tests/fixtures/simple-dockerfile/Dockerfile new file mode 100644 index 00000000000..098ff3eb195 --- /dev/null +++ b/tests/fixtures/simple-dockerfile/Dockerfile @@ -0,0 +1,3 @@ +FROM busybox:1.27.2 +LABEL com.docker.compose.test_image=true +CMD echo "success" diff --git a/tests/fixtures/simple-dockerfile/docker-compose.yml b/tests/fixtures/simple-dockerfile/docker-compose.yml new file mode 100644 index 00000000000..b0357541ee3 --- /dev/null +++ b/tests/fixtures/simple-dockerfile/docker-compose.yml @@ -0,0 +1,2 @@ +simple: + build: . diff --git a/tests/fixtures/simple-failing-dockerfile/Dockerfile b/tests/fixtures/simple-failing-dockerfile/Dockerfile new file mode 100644 index 00000000000..a3328b0d5ec --- /dev/null +++ b/tests/fixtures/simple-failing-dockerfile/Dockerfile @@ -0,0 +1,7 @@ +FROM busybox:1.31.0-uclibc +LABEL com.docker.compose.test_image=true +LABEL com.docker.compose.test_failing_image=true +# With the following label the container will be cleaned up automatically +# Must be kept in sync with LABEL_PROJECT from compose/const.py +LABEL com.docker.compose.project=composetest +RUN exit 1 diff --git a/tests/fixtures/simple-failing-dockerfile/docker-compose.yml b/tests/fixtures/simple-failing-dockerfile/docker-compose.yml new file mode 100644 index 00000000000..b0357541ee3 --- /dev/null +++ b/tests/fixtures/simple-failing-dockerfile/docker-compose.yml @@ -0,0 +1,2 @@ +simple: + build: . diff --git a/tests/fixtures/sleeps-composefile/docker-compose.yml b/tests/fixtures/sleeps-composefile/docker-compose.yml new file mode 100644 index 00000000000..26feb502a58 --- /dev/null +++ b/tests/fixtures/sleeps-composefile/docker-compose.yml @@ -0,0 +1,10 @@ + +version: "2" + +services: + simple: + image: busybox:1.31.0-uclibc + command: sleep 200 + another: + image: busybox:1.31.0-uclibc + command: sleep 200 diff --git a/tests/fixtures/stop-signal-composefile/docker-compose.yml b/tests/fixtures/stop-signal-composefile/docker-compose.yml new file mode 100644 index 00000000000..9f99b0c75df --- /dev/null +++ b/tests/fixtures/stop-signal-composefile/docker-compose.yml @@ -0,0 +1,10 @@ +simple: + image: busybox:1.31.0-uclibc + command: + - sh + - '-c' + - | + trap 'exit 0' SIGINT + trap 'exit 1' SIGTERM + while true; do :; done + stop_signal: SIGINT diff --git a/tests/fixtures/tagless-image/Dockerfile b/tests/fixtures/tagless-image/Dockerfile new file mode 100644 index 00000000000..92305555136 --- /dev/null +++ b/tests/fixtures/tagless-image/Dockerfile @@ -0,0 +1,2 @@ +FROM busybox:1.31.0-uclibc +RUN touch /blah diff --git a/tests/fixtures/tagless-image/docker-compose.yml b/tests/fixtures/tagless-image/docker-compose.yml new file mode 100644 index 00000000000..c4baf2ba153 --- /dev/null +++ b/tests/fixtures/tagless-image/docker-compose.yml @@ -0,0 +1,5 @@ +version: '2.3' +services: + foo: + image: ${IMAGE_ID} + command: top diff --git a/tests/fixtures/tls/ca.pem b/tests/fixtures/tls/ca.pem new file mode 100644 index 00000000000..e69de29bb2d diff --git a/tests/fixtures/tls/cert.pem b/tests/fixtures/tls/cert.pem new file mode 100644 index 00000000000..e69de29bb2d diff --git a/tests/fixtures/tls/key.pem b/tests/fixtures/tls/key.pem new file mode 100644 index 00000000000..e69de29bb2d diff --git a/tests/fixtures/top/docker-compose.yml b/tests/fixtures/top/docker-compose.yml new file mode 100644 index 00000000000..36a3917d756 --- /dev/null +++ b/tests/fixtures/top/docker-compose.yml @@ -0,0 +1,6 @@ +service_a: + image: busybox:1.31.0-uclibc + command: top +service_b: + image: busybox:1.31.0-uclibc + command: top diff --git a/tests/fixtures/unicode-environment/docker-compose.yml b/tests/fixtures/unicode-environment/docker-compose.yml new file mode 100644 index 00000000000..307678cd056 --- /dev/null +++ b/tests/fixtures/unicode-environment/docker-compose.yml @@ -0,0 +1,7 @@ +version: '2' +services: + simple: + image: busybox:1.31.0-uclibc + command: sh -c 'echo $$FOO' + environment: + FOO: ${BAR} diff --git a/tests/fixtures/user-composefile/docker-compose.yml b/tests/fixtures/user-composefile/docker-compose.yml new file mode 100644 index 00000000000..11283d9d991 --- /dev/null +++ b/tests/fixtures/user-composefile/docker-compose.yml @@ -0,0 +1,4 @@ +service: + image: busybox:1.31.0-uclibc + user: notauser + command: id diff --git a/tests/fixtures/v1-config/docker-compose.yml b/tests/fixtures/v1-config/docker-compose.yml new file mode 100644 index 00000000000..8646c4edb91 --- /dev/null +++ b/tests/fixtures/v1-config/docker-compose.yml @@ -0,0 +1,10 @@ +net: + image: busybox +volume: + image: busybox + volumes: + - /data +app: + image: busybox + net: "container:net" + volumes_from: ["volume"] diff --git a/tests/fixtures/v2-dependencies/docker-compose.yml b/tests/fixtures/v2-dependencies/docker-compose.yml new file mode 100644 index 00000000000..45ec8501ea4 --- /dev/null +++ b/tests/fixtures/v2-dependencies/docker-compose.yml @@ -0,0 +1,13 @@ +version: "2.0" +services: + db: + image: busybox:1.31.0-uclibc + command: top + web: + image: busybox:1.31.0-uclibc + command: top + depends_on: + - db + console: + image: busybox:1.31.0-uclibc + command: top diff --git a/tests/fixtures/v2-full/Dockerfile b/tests/fixtures/v2-full/Dockerfile new file mode 100644 index 00000000000..6fa7a726c23 --- /dev/null +++ b/tests/fixtures/v2-full/Dockerfile @@ -0,0 +1,4 @@ + +FROM busybox:1.31.0-uclibc +RUN echo something +CMD top diff --git a/tests/fixtures/v2-full/docker-compose.yml b/tests/fixtures/v2-full/docker-compose.yml new file mode 100644 index 00000000000..20c14f0f7b9 --- /dev/null +++ b/tests/fixtures/v2-full/docker-compose.yml @@ -0,0 +1,24 @@ + +version: "2" + +volumes: + data: + driver: local + +networks: + front: {} + +services: + web: + build: . + networks: + - front + - default + volumes_from: + - other + + other: + image: busybox:1.31.0-uclibc + command: top + volumes: + - /data diff --git a/tests/fixtures/v2-simple/docker-compose.yml b/tests/fixtures/v2-simple/docker-compose.yml new file mode 100644 index 00000000000..ac754eeea0b --- /dev/null +++ b/tests/fixtures/v2-simple/docker-compose.yml @@ -0,0 +1,8 @@ +version: "2" +services: + simple: + image: busybox:1.27.2 + command: top + another: + image: busybox:1.27.2 + command: top diff --git a/tests/fixtures/v2-simple/links-invalid.yml b/tests/fixtures/v2-simple/links-invalid.yml new file mode 100644 index 00000000000..a88eb1d5241 --- /dev/null +++ b/tests/fixtures/v2-simple/links-invalid.yml @@ -0,0 +1,10 @@ +version: "2" +services: + simple: + image: busybox:1.31.0-uclibc + command: top + links: + - another + another: + image: busybox:1.31.0-uclibc + command: top diff --git a/tests/fixtures/v2-simple/one-container.yml b/tests/fixtures/v2-simple/one-container.yml new file mode 100644 index 00000000000..2d5c2ca668f --- /dev/null +++ b/tests/fixtures/v2-simple/one-container.yml @@ -0,0 +1,5 @@ +version: "2" +services: + simple: + image: busybox:1.31.0-uclibc + command: top diff --git a/tests/fixtures/v2-unhealthy-dependencies/docker-compose.yml b/tests/fixtures/v2-unhealthy-dependencies/docker-compose.yml new file mode 100644 index 00000000000..d96473e5a9f --- /dev/null +++ b/tests/fixtures/v2-unhealthy-dependencies/docker-compose.yml @@ -0,0 +1,19 @@ +version: "2.1" +services: + db: + image: busybox:1.31.0-uclibc + command: top + healthcheck: + test: exit 1 + interval: 1s + timeout: 1s + retries: 1 + web: + image: busybox:1.31.0-uclibc + command: top + depends_on: + db: + condition: service_healthy + console: + image: busybox:1.31.0-uclibc + command: top diff --git a/tests/fixtures/v3-full/docker-compose.yml b/tests/fixtures/v3-full/docker-compose.yml new file mode 100644 index 00000000000..0a51565826e --- /dev/null +++ b/tests/fixtures/v3-full/docker-compose.yml @@ -0,0 +1,60 @@ +version: "3.5" +services: + web: + image: busybox + deploy: + mode: replicated + replicas: 6 + labels: [FOO=BAR] + update_config: + parallelism: 3 + delay: 10s + failure_action: continue + monitor: 60s + max_failure_ratio: 0.3 + resources: + limits: + cpus: 0.05 + memory: 50M + reservations: + cpus: 0.01 + memory: 20M + restart_policy: + condition: on-failure + delay: 5s + max_attempts: 3 + window: 120s + placement: + constraints: + - node.hostname==foo + - node.role != manager + preferences: + - spread: node.labels.datacenter + + healthcheck: + test: cat /etc/passwd + interval: 10s + timeout: 1s + retries: 5 + + volumes: + - source: /host/path + target: /container/path + type: bind + read_only: true + - source: foobar + type: volume + target: /container/volumepath + - type: volume + target: /anonymous + - type: volume + source: foobar + target: /container/volumepath2 + volume: + nocopy: true + + stop_grace_period: 20s +volumes: + foobar: + labels: + com.docker.compose.test: 'true' diff --git a/tests/fixtures/volume-path-interpolation/docker-compose.yml b/tests/fixtures/volume-path-interpolation/docker-compose.yml new file mode 100644 index 00000000000..6d4e236af93 --- /dev/null +++ b/tests/fixtures/volume-path-interpolation/docker-compose.yml @@ -0,0 +1,5 @@ +test: + image: busybox + command: top + volumes: + - "~/${VOLUME_NAME}:/container-path" diff --git a/tests/fixtures/volume-path/common/services.yml b/tests/fixtures/volume-path/common/services.yml new file mode 100644 index 00000000000..2dbf75961d0 --- /dev/null +++ b/tests/fixtures/volume-path/common/services.yml @@ -0,0 +1,5 @@ +db: + image: busybox + volumes: + - ./foo:/foo + - ./bar:/bar diff --git a/tests/fixtures/volume-path/docker-compose.yml b/tests/fixtures/volume-path/docker-compose.yml new file mode 100644 index 00000000000..af433c52f79 --- /dev/null +++ b/tests/fixtures/volume-path/docker-compose.yml @@ -0,0 +1,6 @@ +db: + extends: + file: common/services.yml + service: db + volumes: + - ./bar:/bar diff --git a/tests/fixtures/volume/docker-compose.yml b/tests/fixtures/volume/docker-compose.yml new file mode 100644 index 00000000000..4335b0a094f --- /dev/null +++ b/tests/fixtures/volume/docker-compose.yml @@ -0,0 +1,11 @@ +version: '2' +services: + test: + image: busybox + command: top + volumes: + - /container-path + - testvolume:/container-named-path + +volumes: + testvolume: {} diff --git a/tests/fixtures/volumes-from-container/docker-compose.yml b/tests/fixtures/volumes-from-container/docker-compose.yml new file mode 100644 index 00000000000..495fcaae5c0 --- /dev/null +++ b/tests/fixtures/volumes-from-container/docker-compose.yml @@ -0,0 +1,5 @@ +version: "2" +services: + test: + image: busybox + volumes_from: ["container:composetest_data_container"] diff --git a/tests/fixtures/volumes/docker-compose.yml b/tests/fixtures/volumes/docker-compose.yml new file mode 100644 index 00000000000..da711ac42bb --- /dev/null +++ b/tests/fixtures/volumes/docker-compose.yml @@ -0,0 +1,2 @@ +version: '2.1' +services: {} diff --git a/tests/fixtures/volumes/external-volumes-v2-x.yml b/tests/fixtures/volumes/external-volumes-v2-x.yml new file mode 100644 index 00000000000..3b736c5f481 --- /dev/null +++ b/tests/fixtures/volumes/external-volumes-v2-x.yml @@ -0,0 +1,17 @@ +version: "2.1" + +services: + web: + image: busybox + command: top + volumes: + - foo:/var/lib/ + - bar:/etc/ + +volumes: + foo: + external: true + name: some_foo + bar: + external: + name: some_bar diff --git a/tests/fixtures/volumes/external-volumes-v2.yml b/tests/fixtures/volumes/external-volumes-v2.yml new file mode 100644 index 00000000000..4025b53b19f --- /dev/null +++ b/tests/fixtures/volumes/external-volumes-v2.yml @@ -0,0 +1,16 @@ +version: "2" + +services: + web: + image: busybox + command: top + volumes: + - foo:/var/lib/ + - bar:/etc/ + +volumes: + foo: + external: true + bar: + external: + name: some_bar diff --git a/tests/fixtures/volumes/external-volumes-v3-4.yml b/tests/fixtures/volumes/external-volumes-v3-4.yml new file mode 100644 index 00000000000..76c8421dc54 --- /dev/null +++ b/tests/fixtures/volumes/external-volumes-v3-4.yml @@ -0,0 +1,17 @@ +version: "3.4" + +services: + web: + image: busybox + command: top + volumes: + - foo:/var/lib/ + - bar:/etc/ + +volumes: + foo: + external: true + name: some_foo + bar: + external: + name: some_bar diff --git a/tests/fixtures/volumes/external-volumes-v3-x.yml b/tests/fixtures/volumes/external-volumes-v3-x.yml new file mode 100644 index 00000000000..903fee64728 --- /dev/null +++ b/tests/fixtures/volumes/external-volumes-v3-x.yml @@ -0,0 +1,16 @@ +version: "3.0" + +services: + web: + image: busybox + command: top + volumes: + - foo:/var/lib/ + - bar:/etc/ + +volumes: + foo: + external: true + bar: + external: + name: some_bar diff --git a/tests/fixtures/volumes/volume-label.yml b/tests/fixtures/volumes/volume-label.yml new file mode 100644 index 00000000000..a5f33a5aaa4 --- /dev/null +++ b/tests/fixtures/volumes/volume-label.yml @@ -0,0 +1,13 @@ +version: "2.1" + +services: + web: + image: busybox + command: top + volumes: + - volume_with_label:/data + +volumes: + volume_with_label: + labels: + - "label_key=label_val" diff --git a/tests/helpers.py b/tests/helpers.py new file mode 100644 index 00000000000..3642e6ebc5f --- /dev/null +++ b/tests/helpers.py @@ -0,0 +1,69 @@ +import contextlib +import os + +from compose.config.config import ConfigDetails +from compose.config.config import ConfigFile +from compose.config.config import load + +BUSYBOX_IMAGE_NAME = 'busybox' +BUSYBOX_DEFAULT_TAG = '1.31.0-uclibc' +BUSYBOX_IMAGE_WITH_TAG = '{}:{}'.format(BUSYBOX_IMAGE_NAME, BUSYBOX_DEFAULT_TAG) + + +def build_config(contents, **kwargs): + return load(build_config_details(contents, **kwargs)) + + +def build_config_details(contents, working_dir='working_dir', filename='filename.yml'): + return ConfigDetails( + working_dir, + [ConfigFile(filename, contents)], + ) + + +def create_custom_host_file(client, filename, content): + dirname = os.path.dirname(filename) + container = client.create_container( + BUSYBOX_IMAGE_WITH_TAG, + ['sh', '-c', 'echo -n "{}" > {}'.format(content, filename)], + volumes={dirname: {}}, + host_config=client.create_host_config( + binds={dirname: {'bind': dirname, 'ro': False}}, + network_mode='none', + ), + ) + try: + client.start(container) + exitcode = client.wait(container)['StatusCode'] + + if exitcode != 0: + output = client.logs(container) + raise Exception( + "Container exited with code {}:\n{}".format(exitcode, output)) + + container_info = client.inspect_container(container) + if 'Node' in container_info: + return container_info['Node']['Name'] + finally: + client.remove_container(container, force=True) + + +def create_host_file(client, filename): + with open(filename) as fh: + content = fh.read() + + return create_custom_host_file(client, filename, content) + + +@contextlib.contextmanager +def cd(path): + """ + A context manager which changes the working directory to the given + path, and then changes it back to its previous value on exit. + """ + prev_cwd = os.getcwd() + os.chdir(path) + try: + yield + finally: + os.chdir(prev_cwd) diff --git a/tests/integration/__init__.py b/tests/integration/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/tests/integration/environment_test.py b/tests/integration/environment_test.py new file mode 100644 index 00000000000..12a969c9428 --- /dev/null +++ b/tests/integration/environment_test.py @@ -0,0 +1,67 @@ +import tempfile + +from ddt import data +from ddt import ddt + +from .. import mock +from ..acceptance.cli_test import dispatch +from compose.cli.command import get_project +from compose.cli.command import project_from_options +from compose.config.environment import Environment +from tests.integration.testcases import DockerClientTestCase + + +@ddt +class EnvironmentTest(DockerClientTestCase): + @classmethod + def setUpClass(cls): + super().setUpClass() + cls.compose_file = tempfile.NamedTemporaryFile(mode='w+b') + cls.compose_file.write(bytes("""version: '3.2' +services: + svc: + image: busybox:1.31.0-uclibc + environment: + TEST_VARIABLE: ${TEST_VARIABLE}""", encoding='utf-8')) + cls.compose_file.flush() + + @classmethod + def tearDownClass(cls): + super().tearDownClass() + cls.compose_file.close() + + @data('events', + 'exec', + 'kill', + 'logs', + 'pause', + 'ps', + 'restart', + 'rm', + 'start', + 'stop', + 'top', + 'unpause') + def _test_no_warning_on_missing_host_environment_var_on_silent_commands(self, cmd): + options = {'COMMAND': cmd, '--file': [EnvironmentTest.compose_file.name]} + with mock.patch('compose.config.environment.log') as fake_log: + # Note that the warning silencing and the env variables check is + # done in `project_from_options` + # So no need to have a proper options map, the `COMMAND` key is enough + project_from_options('.', options) + assert fake_log.warn.call_count == 0 + + +class EnvironmentOverrideFileTest(DockerClientTestCase): + def test_env_file_override(self): + base_dir = 'tests/fixtures/env-file-override' + dispatch(base_dir, ['--env-file', '.env.override', 'up']) + project = get_project(project_dir=base_dir, + config_path=['docker-compose.yml'], + environment=Environment.from_env_file(base_dir, '.env.override'), + override_dir=base_dir) + containers = project.containers(stopped=True) + assert len(containers) == 1 + assert "WHEREAMI=override" in containers[0].get('Config.Env') + assert "DEFAULT_CONF_LOADED=true" in containers[0].get('Config.Env') + dispatch(base_dir, ['--env-file', '.env.override', 'down'], None) diff --git a/tests/integration/metrics_test.py b/tests/integration/metrics_test.py new file mode 100644 index 00000000000..3d6e3fe220e --- /dev/null +++ b/tests/integration/metrics_test.py @@ -0,0 +1,125 @@ +import logging +import os +import socket +from http.server import BaseHTTPRequestHandler +from http.server import HTTPServer +from threading import Thread + +import requests +from docker.transport import UnixHTTPAdapter + +from tests.acceptance.cli_test import dispatch +from tests.integration.testcases import DockerClientTestCase + + +TEST_SOCKET_FILE = '/tmp/test-metrics-docker-cli.sock' + + +class MetricsTest(DockerClientTestCase): + test_session = requests.sessions.Session() + test_env = None + base_dir = 'tests/fixtures/v3-full' + + @classmethod + def setUpClass(cls): + super().setUpClass() + MetricsTest.test_session.mount("http+unix://", UnixHTTPAdapter(TEST_SOCKET_FILE)) + MetricsTest.test_env = os.environ.copy() + MetricsTest.test_env['METRICS_SOCKET_FILE'] = TEST_SOCKET_FILE + MetricsServer().start() + + @classmethod + def test_metrics_help(cls): + # root `docker-compose` command is considered as a `--help` + dispatch(cls.base_dir, [], env=MetricsTest.test_env) + assert cls.get_content() == \ + b'{"command": "compose --help", "context": "moby", ' \ + b'"source": "docker-compose", "status": "success"}' + dispatch(cls.base_dir, ['help', 'run'], env=MetricsTest.test_env) + assert cls.get_content() == \ + b'{"command": "compose help", "context": "moby", ' \ + b'"source": "docker-compose", "status": "success"}' + dispatch(cls.base_dir, ['--help'], env=MetricsTest.test_env) + assert cls.get_content() == \ + b'{"command": "compose --help", "context": "moby", ' \ + b'"source": "docker-compose", "status": "success"}' + dispatch(cls.base_dir, ['run', '--help'], env=MetricsTest.test_env) + assert cls.get_content() == \ + b'{"command": "compose --help run", "context": "moby", ' \ + b'"source": "docker-compose", "status": "success"}' + dispatch(cls.base_dir, ['up', '--help', 'extra_args'], env=MetricsTest.test_env) + assert cls.get_content() == \ + b'{"command": "compose --help up", "context": "moby", ' \ + b'"source": "docker-compose", "status": "success"}' + + @classmethod + def test_metrics_simple_commands(cls): + dispatch(cls.base_dir, ['ps'], env=MetricsTest.test_env) + assert cls.get_content() == \ + b'{"command": "compose ps", "context": "moby", ' \ + b'"source": "docker-compose", "status": "success"}' + dispatch(cls.base_dir, ['version'], env=MetricsTest.test_env) + assert cls.get_content() == \ + b'{"command": "compose version", "context": "moby", ' \ + b'"source": "docker-compose", "status": "success"}' + dispatch(cls.base_dir, ['version', '--yyy'], env=MetricsTest.test_env) + assert cls.get_content() == \ + b'{"command": "compose version", "context": "moby", ' \ + b'"source": "docker-compose", "status": "failure"}' + + @staticmethod + def get_content(): + resp = MetricsTest.test_session.get("http+unix://localhost") + print(resp.content) + return resp.content + + +def start_server(uri=TEST_SOCKET_FILE): + try: + os.remove(uri) + except OSError: + pass + httpd = HTTPServer(uri, MetricsHTTPRequestHandler, False) + sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + sock.bind(TEST_SOCKET_FILE) + sock.listen(0) + httpd.socket = sock + print('Serving on ', uri) + httpd.serve_forever() + sock.shutdown(socket.SHUT_RDWR) + sock.close() + os.remove(uri) + + +class MetricsServer: + @classmethod + def start(cls): + t = Thread(target=start_server, daemon=True) + t.start() + + +class MetricsHTTPRequestHandler(BaseHTTPRequestHandler): + usages = [] + + def do_GET(self): + self.client_address = ('',) # avoid exception in BaseHTTPServer.py log_message() + self.send_response(200) + self.end_headers() + for u in MetricsHTTPRequestHandler.usages: + self.wfile.write(u) + MetricsHTTPRequestHandler.usages = [] + + def do_POST(self): + self.client_address = ('',) # avoid exception in BaseHTTPServer.py log_message() + content_length = int(self.headers['Content-Length']) + body = self.rfile.read(content_length) + print(body) + MetricsHTTPRequestHandler.usages.append(body) + self.send_response(200) + self.end_headers() + + +if __name__ == '__main__': + logging.getLogger("urllib3").propagate = False + logging.getLogger("requests").propagate = False + start_server() diff --git a/tests/integration/network_test.py b/tests/integration/network_test.py new file mode 100644 index 00000000000..23c9e9a4bda --- /dev/null +++ b/tests/integration/network_test.py @@ -0,0 +1,34 @@ +import pytest + +from .testcases import DockerClientTestCase +from compose.config.errors import ConfigurationError +from compose.const import LABEL_NETWORK +from compose.const import LABEL_PROJECT +from compose.network import Network + + +class NetworkTest(DockerClientTestCase): + def test_network_default_labels(self): + net = Network(self.client, 'composetest', 'foonet') + net.ensure() + net_data = net.inspect() + labels = net_data['Labels'] + assert labels[LABEL_NETWORK] == net.name + assert labels[LABEL_PROJECT] == net.project + + def test_network_external_default_ensure(self): + net = Network( + self.client, 'composetest', 'foonet', + external=True + ) + + with pytest.raises(ConfigurationError): + net.ensure() + + def test_network_external_overlay_ensure(self): + net = Network( + self.client, 'composetest', 'foonet', + driver='overlay', external=True + ) + + assert net.ensure() is None diff --git a/tests/integration/project_test.py b/tests/integration/project_test.py new file mode 100644 index 00000000000..c4210291f14 --- /dev/null +++ b/tests/integration/project_test.py @@ -0,0 +1,1994 @@ +import copy +import json +import os +import random +import shutil +import tempfile + +import pytest +from docker.errors import APIError +from docker.errors import NotFound + +from .. import mock +from ..helpers import build_config as load_config +from ..helpers import BUSYBOX_IMAGE_WITH_TAG +from ..helpers import cd +from ..helpers import create_host_file +from .testcases import DockerClientTestCase +from .testcases import SWARM_SKIP_CONTAINERS_ALL +from compose.config import config +from compose.config import ConfigurationError +from compose.config import types +from compose.config.types import VolumeFromSpec +from compose.config.types import VolumeSpec +from compose.const import COMPOSE_SPEC as VERSION +from compose.const import LABEL_PROJECT +from compose.const import LABEL_SERVICE +from compose.container import Container +from compose.errors import HealthCheckFailed +from compose.errors import NoHealthCheckConfigured +from compose.project import Project +from compose.project import ProjectError +from compose.service import ConvergenceStrategy +from tests.integration.testcases import if_runtime_available +from tests.integration.testcases import is_cluster +from tests.integration.testcases import no_cluster + + +def build_config(**kwargs): + return config.Config( + config_version=kwargs.get('version', VERSION), + version=kwargs.get('version', VERSION), + services=kwargs.get('services'), + volumes=kwargs.get('volumes'), + networks=kwargs.get('networks'), + secrets=kwargs.get('secrets'), + configs=kwargs.get('configs'), + ) + + +class ProjectTest(DockerClientTestCase): + + def test_containers(self): + web = self.create_service('web') + db = self.create_service('db') + project = Project('composetest', [web, db], self.client) + + project.up() + + containers = project.containers() + assert len(containers) == 2 + + @pytest.mark.skipif(SWARM_SKIP_CONTAINERS_ALL, reason='Swarm /containers/json bug') + def test_containers_stopped(self): + web = self.create_service('web') + db = self.create_service('db') + project = Project('composetest', [web, db], self.client) + + project.up() + assert len(project.containers()) == 2 + assert len(project.containers(stopped=True)) == 2 + + project.stop() + assert len(project.containers()) == 0 + assert len(project.containers(stopped=True)) == 2 + + def test_containers_with_service_names(self): + web = self.create_service('web') + db = self.create_service('db') + project = Project('composetest', [web, db], self.client) + + project.up() + + containers = project.containers(['web']) + assert len(containers) == 1 + assert containers[0].name.startswith('composetest_web_') + + def test_containers_with_extra_service(self): + web = self.create_service('web') + web_1 = web.create_container() + + db = self.create_service('db') + db_1 = db.create_container() + + self.create_service('extra').create_container() + + project = Project('composetest', [web, db], self.client) + assert set(project.containers(stopped=True)) == {web_1, db_1} + + def test_parallel_pull_with_no_image(self): + config_data = build_config( + services=[{ + 'name': 'web', + 'build': {'context': '.'}, + }], + ) + + project = Project.from_config( + name='composetest', + config_data=config_data, + client=self.client + ) + + project.pull(parallel_pull=True) + + def test_volumes_from_service(self): + project = Project.from_config( + name='composetest', + config_data=load_config({ + 'data': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes': ['/var/data'], + }, + 'db': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes_from': ['data'], + }, + }), + client=self.client, + ) + db = project.get_service('db') + data = project.get_service('data') + assert db.volumes_from == [VolumeFromSpec(data, 'rw', 'service')] + + def test_volumes_from_container(self): + data_container = Container.create( + self.client, + image=BUSYBOX_IMAGE_WITH_TAG, + volumes=['/var/data'], + name='composetest_data_container', + labels={LABEL_PROJECT: 'composetest'}, + host_config={}, + ) + project = Project.from_config( + name='composetest', + config_data=load_config({ + 'db': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes_from': ['composetest_data_container'], + }, + }), + client=self.client, + ) + db = project.get_service('db') + assert db._get_volumes_from() == [data_container.id + ':rw'] + + @no_cluster('container networks not supported in Swarm') + def test_network_mode_from_service(self): + project = Project.from_config( + name='composetest', + client=self.client, + config_data=load_config({ + 'services': { + 'net': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': ["top"] + }, + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'network_mode': 'service:net', + 'command': ["top"] + }, + }, + }), + ) + + project.up() + + web = project.get_service('web') + net = project.get_service('net') + assert web.network_mode.mode == 'container:' + net.containers()[0].id + + @no_cluster('container networks not supported in Swarm') + def test_network_mode_from_container(self): + def get_project(): + return Project.from_config( + name='composetest', + config_data=load_config({ + 'services': { + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'network_mode': 'container:composetest_net_container' + }, + }, + }), + client=self.client, + ) + + with pytest.raises(ConfigurationError) as excinfo: + get_project() + + assert "container 'composetest_net_container' which does not exist" in excinfo.exconly() + + net_container = Container.create( + self.client, + image=BUSYBOX_IMAGE_WITH_TAG, + name='composetest_net_container', + command='top', + labels={LABEL_PROJECT: 'composetest'}, + host_config={}, + ) + net_container.start() + + project = get_project() + project.up() + + web = project.get_service('web') + assert web.network_mode.mode == 'container:' + net_container.id + + @no_cluster('container networks not supported in Swarm') + def test_net_from_service_v1(self): + project = Project.from_config( + name='composetest', + config_data=load_config({ + 'net': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': ["top"] + }, + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'net': 'container:net', + 'command': ["top"] + }, + }), + client=self.client, + ) + + project.up() + + web = project.get_service('web') + net = project.get_service('net') + assert web.network_mode.mode == 'container:' + net.containers()[0].id + + @no_cluster('container networks not supported in Swarm') + def test_net_from_container_v1(self): + def get_project(): + return Project.from_config( + name='composetest', + config_data=load_config({ + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'net': 'container:composetest_net_container' + }, + }), + client=self.client, + ) + + with pytest.raises(ConfigurationError) as excinfo: + get_project() + + assert "container 'composetest_net_container' which does not exist" in excinfo.exconly() + + net_container = Container.create( + self.client, + image=BUSYBOX_IMAGE_WITH_TAG, + name='composetest_net_container', + command='top', + labels={LABEL_PROJECT: 'composetest'}, + host_config={}, + ) + net_container.start() + + project = get_project() + project.up() + + web = project.get_service('web') + assert web.network_mode.mode == 'container:' + net_container.id + + def test_start_pause_unpause_stop_kill_remove(self): + web = self.create_service('web') + db = self.create_service('db') + project = Project('composetest', [web, db], self.client) + + project.start() + + assert len(web.containers()) == 0 + assert len(db.containers()) == 0 + + web_container_1 = web.create_container() + web_container_2 = web.create_container() + db_container = db.create_container() + + project.start(service_names=['web']) + assert {c.name for c in project.containers() if c.is_running} == { + web_container_1.name, web_container_2.name} + + project.start() + assert {c.name for c in project.containers() if c.is_running} == { + web_container_1.name, web_container_2.name, db_container.name} + + project.pause(service_names=['web']) + assert {c.name for c in project.containers() if c.is_paused} == { + web_container_1.name, web_container_2.name} + + project.pause() + assert {c.name for c in project.containers() if c.is_paused} == { + web_container_1.name, web_container_2.name, db_container.name} + + project.unpause(service_names=['db']) + assert len([c.name for c in project.containers() if c.is_paused]) == 2 + + project.unpause() + assert len([c.name for c in project.containers() if c.is_paused]) == 0 + + project.stop(service_names=['web'], timeout=1) + assert {c.name for c in project.containers() if c.is_running} == {db_container.name} + + project.kill(service_names=['db']) + assert len([c for c in project.containers() if c.is_running]) == 0 + assert len(project.containers(stopped=True)) == 3 + + project.remove_stopped(service_names=['web']) + assert len(project.containers(stopped=True)) == 1 + + project.remove_stopped() + assert len(project.containers(stopped=True)) == 0 + + def test_create(self): + web = self.create_service('web') + db = self.create_service('db', volumes=[VolumeSpec.parse('/var/db')]) + project = Project('composetest', [web, db], self.client) + + project.create(['db']) + containers = project.containers(stopped=True) + assert len(containers) == 1 + assert not containers[0].is_running + db_containers = db.containers(stopped=True) + assert len(db_containers) == 1 + assert not db_containers[0].is_running + assert len(web.containers(stopped=True)) == 0 + + def test_create_twice(self): + web = self.create_service('web') + db = self.create_service('db', volumes=[VolumeSpec.parse('/var/db')]) + project = Project('composetest', [web, db], self.client) + + project.create(['db', 'web']) + project.create(['db', 'web']) + containers = project.containers(stopped=True) + assert len(containers) == 2 + db_containers = db.containers(stopped=True) + assert len(db_containers) == 1 + assert not db_containers[0].is_running + web_containers = web.containers(stopped=True) + assert len(web_containers) == 1 + assert not web_containers[0].is_running + + def test_create_with_links(self): + db = self.create_service('db') + web = self.create_service('web', links=[(db, 'db')]) + project = Project('composetest', [db, web], self.client) + + project.create(['web']) + # self.assertEqual(len(project.containers()), 0) + assert len(project.containers(stopped=True)) == 2 + assert not [c for c in project.containers(stopped=True) if c.is_running] + assert len(db.containers(stopped=True)) == 1 + assert len(web.containers(stopped=True)) == 1 + + def test_create_strategy_always(self): + db = self.create_service('db') + project = Project('composetest', [db], self.client) + project.create(['db']) + old_id = project.containers(stopped=True)[0].id + + project.create(['db'], strategy=ConvergenceStrategy.always) + assert len(project.containers(stopped=True)) == 1 + + db_container = project.containers(stopped=True)[0] + assert not db_container.is_running + assert db_container.id != old_id + + def test_create_strategy_never(self): + db = self.create_service('db') + project = Project('composetest', [db], self.client) + project.create(['db']) + old_id = project.containers(stopped=True)[0].id + + project.create(['db'], strategy=ConvergenceStrategy.never) + assert len(project.containers(stopped=True)) == 1 + + db_container = project.containers(stopped=True)[0] + assert not db_container.is_running + assert db_container.id == old_id + + def test_project_up(self): + web = self.create_service('web') + db = self.create_service('db', volumes=[VolumeSpec.parse('/var/db')]) + project = Project('composetest', [web, db], self.client) + project.start() + assert len(project.containers()) == 0 + + project.up(['db']) + assert len(project.containers()) == 1 + assert len(db.containers()) == 1 + assert len(web.containers()) == 0 + + def test_project_up_starts_uncreated_services(self): + db = self.create_service('db') + web = self.create_service('web', links=[(db, 'db')]) + project = Project('composetest', [db, web], self.client) + project.up(['db']) + assert len(project.containers()) == 1 + + project.up() + assert len(project.containers()) == 2 + assert len(db.containers()) == 1 + assert len(web.containers()) == 1 + + def test_recreate_preserves_volumes(self): + web = self.create_service('web') + db = self.create_service('db', volumes=[VolumeSpec.parse('/etc')]) + project = Project('composetest', [web, db], self.client) + project.start() + assert len(project.containers()) == 0 + + project.up(['db']) + assert len(project.containers()) == 1 + old_db_id = project.containers()[0].id + db_volume_path = project.containers()[0].get('Volumes./etc') + + project.up(strategy=ConvergenceStrategy.always) + assert len(project.containers()) == 2 + + db_container = [c for c in project.containers() if c.service == 'db'][0] + assert db_container.id != old_db_id + assert db_container.get('Volumes./etc') == db_volume_path + + def test_recreate_preserves_mounts(self): + web = self.create_service('web') + db = self.create_service('db', volumes=[types.MountSpec(type='volume', target='/etc')]) + project = Project('composetest', [web, db], self.client) + project.start() + assert len(project.containers()) == 0 + + project.up(['db']) + assert len(project.containers()) == 1 + old_db_id = project.containers()[0].id + db_volume_path = project.containers()[0].get_mount('/etc')['Source'] + + project.up(strategy=ConvergenceStrategy.always) + assert len(project.containers()) == 2 + + db_container = [c for c in project.containers() if c.service == 'db'][0] + assert db_container.id != old_db_id + assert db_container.get_mount('/etc')['Source'] == db_volume_path + + def test_project_up_with_no_recreate_running(self): + web = self.create_service('web') + db = self.create_service('db', volumes=[VolumeSpec.parse('/var/db')]) + project = Project('composetest', [web, db], self.client) + project.start() + assert len(project.containers()) == 0 + + project.up(['db']) + assert len(project.containers()) == 1 + container, = project.containers() + old_db_id = container.id + db_volume_path = container.get_mount('/var/db')['Source'] + + project.up(strategy=ConvergenceStrategy.never) + assert len(project.containers()) == 2 + + db_container = [c for c in project.containers() if c.name == container.name][0] + assert db_container.id == old_db_id + assert db_container.get_mount('/var/db')['Source'] == db_volume_path + + def test_project_up_with_no_recreate_stopped(self): + web = self.create_service('web') + db = self.create_service('db', volumes=[VolumeSpec.parse('/var/db')]) + project = Project('composetest', [web, db], self.client) + project.start() + assert len(project.containers()) == 0 + + project.up(['db']) + project.kill() + + old_containers = project.containers(stopped=True) + + assert len(old_containers) == 1 + old_container, = old_containers + old_db_id = old_container.id + db_volume_path = old_container.get_mount('/var/db')['Source'] + + project.up(strategy=ConvergenceStrategy.never) + + new_containers = project.containers(stopped=True) + assert len(new_containers) == 2 + assert [c.is_running for c in new_containers] == [True, True] + + db_container = [c for c in new_containers if c.service == 'db'][0] + assert db_container.id == old_db_id + assert db_container.get_mount('/var/db')['Source'] == db_volume_path + + def test_project_up_without_all_services(self): + console = self.create_service('console') + db = self.create_service('db') + project = Project('composetest', [console, db], self.client) + project.start() + assert len(project.containers()) == 0 + + project.up() + assert len(project.containers()) == 2 + assert len(db.containers()) == 1 + assert len(console.containers()) == 1 + + def test_project_up_starts_links(self): + console = self.create_service('console') + db = self.create_service('db', volumes=[VolumeSpec.parse('/var/db')]) + web = self.create_service('web', links=[(db, 'db')]) + + project = Project('composetest', [web, db, console], self.client) + project.start() + assert len(project.containers()) == 0 + + project.up(['web']) + assert len(project.containers()) == 2 + assert len(web.containers()) == 1 + assert len(db.containers()) == 1 + assert len(console.containers()) == 0 + + def test_project_up_starts_depends(self): + project = Project.from_config( + name='composetest', + config_data=load_config({ + 'console': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': ["top"], + }, + 'data': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': ["top"] + }, + 'db': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': ["top"], + 'volumes_from': ['data'], + }, + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': ["top"], + 'links': ['db'], + }, + }), + client=self.client, + ) + project.start() + assert len(project.containers()) == 0 + + project.up(['web']) + assert len(project.containers()) == 3 + assert len(project.get_service('web').containers()) == 1 + assert len(project.get_service('db').containers()) == 1 + assert len(project.get_service('data').containers()) == 1 + assert len(project.get_service('console').containers()) == 0 + + def test_project_up_with_no_deps(self): + project = Project.from_config( + name='composetest', + config_data=load_config({ + 'console': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': ["top"], + }, + 'data': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': ["top"] + }, + 'db': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': ["top"], + 'volumes_from': ['data'], + }, + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': ["top"], + 'links': ['db'], + }, + }), + client=self.client, + ) + project.start() + assert len(project.containers()) == 0 + + project.up(['db'], start_deps=False) + assert len(project.containers(stopped=True)) == 2 + assert len(project.get_service('web').containers()) == 0 + assert len(project.get_service('db').containers()) == 1 + assert len(project.get_service('data').containers(stopped=True)) == 1 + assert not project.get_service('data').containers(stopped=True)[0].is_running + assert len(project.get_service('console').containers()) == 0 + + def test_project_up_recreate_with_tmpfs_volume(self): + # https://github.com/docker/compose/issues/4751 + project = Project.from_config( + name='composetest', + config_data=load_config({ + 'version': '2.1', + 'services': { + 'foo': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'tmpfs': ['/dev/shm'], + 'volumes': ['/dev/shm'] + } + } + }), client=self.client + ) + project.up() + project.up(strategy=ConvergenceStrategy.always) + + def test_unscale_after_restart(self): + web = self.create_service('web') + project = Project('composetest', [web], self.client) + + project.start() + + service = project.get_service('web') + service.scale(1) + assert len(service.containers()) == 1 + service.scale(3) + assert len(service.containers()) == 3 + project.up() + service = project.get_service('web') + assert len(service.containers()) == 1 + service.scale(1) + assert len(service.containers()) == 1 + project.up(scale_override={'web': 3}) + service = project.get_service('web') + assert len(service.containers()) == 3 + # does scale=0 ,makes any sense? after recreating at least 1 container is running + service.scale(0) + project.up() + service = project.get_service('web') + assert len(service.containers()) == 1 + + def test_project_up_networks(self): + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'networks': { + 'foo': None, + 'bar': None, + 'baz': {'aliases': ['extra']}, + }, + }], + networks={ + 'foo': {'driver': 'bridge'}, + 'bar': {'driver': None}, + 'baz': {}, + }, + ) + + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data, + ) + project.up() + + containers = project.containers() + assert len(containers) == 1 + container, = containers + + for net_name in ['foo', 'bar', 'baz']: + full_net_name = 'composetest_{}'.format(net_name) + network_data = self.client.inspect_network(full_net_name) + assert network_data['Name'] == full_net_name + + aliases_key = 'NetworkSettings.Networks.{net}.Aliases' + assert 'web' in container.get(aliases_key.format(net='composetest_foo')) + assert 'web' in container.get(aliases_key.format(net='composetest_baz')) + assert 'extra' in container.get(aliases_key.format(net='composetest_baz')) + + foo_data = self.client.inspect_network('composetest_foo') + assert foo_data['Driver'] == 'bridge' + + def test_up_with_ipam_config(self): + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'networks': {'front': None}, + }], + networks={ + 'front': { + 'driver': 'bridge', + 'driver_opts': { + "com.docker.network.bridge.enable_icc": "false", + }, + 'ipam': { + 'driver': 'default', + 'config': [{ + "subnet": "172.28.0.0/16", + "ip_range": "172.28.5.0/24", + "gateway": "172.28.5.254", + "aux_addresses": { + "a": "172.28.1.5", + "b": "172.28.1.6", + "c": "172.28.1.7", + }, + }], + }, + }, + }, + ) + + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data, + ) + project.up() + + network = self.client.networks(names=['composetest_front'])[0] + + assert network['Options'] == { + "com.docker.network.bridge.enable_icc": "false" + } + + assert network['IPAM'] == { + 'Driver': 'default', + 'Options': None, + 'Config': [{ + 'Subnet': "172.28.0.0/16", + 'IPRange': "172.28.5.0/24", + 'Gateway': "172.28.5.254", + 'AuxiliaryAddresses': { + 'a': '172.28.1.5', + 'b': '172.28.1.6', + 'c': '172.28.1.7', + }, + }], + } + + def test_up_with_ipam_options(self): + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'networks': {'front': None}, + }], + networks={ + 'front': { + 'driver': 'bridge', + 'ipam': { + 'driver': 'default', + 'options': { + "com.docker.compose.network.test": "9-29-045" + } + }, + }, + }, + ) + + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data, + ) + project.up() + + network = self.client.networks(names=['composetest_front'])[0] + + assert network['IPAM']['Options'] == { + "com.docker.compose.network.test": "9-29-045" + } + + def test_up_with_network_static_addresses(self): + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'networks': { + 'static_test': { + 'ipv4_address': '172.16.100.100', + 'ipv6_address': 'fe80::1001:102' + } + }, + }], + networks={ + 'static_test': { + 'driver': 'bridge', + 'driver_opts': { + "com.docker.network.enable_ipv6": "true", + }, + 'ipam': { + 'driver': 'default', + 'config': [ + {"subnet": "172.16.100.0/24", + "gateway": "172.16.100.1"}, + {"subnet": "fe80::/64", + "gateway": "fe80::1001:1"} + ] + }, + 'enable_ipv6': True, + } + } + ) + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data, + ) + project.up(detached=True) + + service_container = project.get_service('web').containers()[0] + + ipam_config = (service_container.inspect().get('NetworkSettings', {}). + get('Networks', {}).get('composetest_static_test', {}). + get('IPAMConfig', {})) + assert ipam_config.get('IPv4Address') == '172.16.100.100' + assert ipam_config.get('IPv6Address') == 'fe80::1001:102' + + def test_up_with_network_priorities(self): + mac_address = '74:6f:75:68:6f:75' + + def get_config_data(p1, p2, p3): + return build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'networks': { + 'n1': { + 'priority': p1, + }, + 'n2': { + 'priority': p2, + }, + 'n3': { + 'priority': p3, + } + }, + 'command': 'top', + 'mac_address': mac_address + }], + networks={ + 'n1': {}, + 'n2': {}, + 'n3': {} + } + ) + + config1 = get_config_data(1000, 1, 1) + config2 = get_config_data(2, 3, 1) + config3 = get_config_data(5, 40, 100) + + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config1 + ) + project.up(detached=True) + service_container = project.get_service('web').containers()[0] + net_config = service_container.inspect()['NetworkSettings']['Networks']['composetest_n1'] + assert net_config['MacAddress'] == mac_address + + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config2 + ) + project.up(detached=True) + service_container = project.get_service('web').containers()[0] + net_config = service_container.inspect()['NetworkSettings']['Networks']['composetest_n2'] + assert net_config['MacAddress'] == mac_address + + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config3 + ) + project.up(detached=True) + service_container = project.get_service('web').containers()[0] + net_config = service_container.inspect()['NetworkSettings']['Networks']['composetest_n3'] + assert net_config['MacAddress'] == mac_address + + def test_up_with_enable_ipv6(self): + self.require_api_version('1.23') + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'networks': { + 'static_test': { + 'ipv6_address': 'fe80::1001:102' + } + }, + }], + networks={ + 'static_test': { + 'driver': 'bridge', + 'enable_ipv6': True, + 'ipam': { + 'driver': 'default', + 'config': [ + {"subnet": "fe80::/64", + "gateway": "fe80::1001:1"} + ] + } + } + } + ) + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data, + ) + project.up(detached=True) + network = [n for n in self.client.networks() if 'static_test' in n['Name']][0] + service_container = project.get_service('web').containers()[0] + + assert network['EnableIPv6'] is True + ipam_config = (service_container.inspect().get('NetworkSettings', {}). + get('Networks', {}).get('composetest_static_test', {}). + get('IPAMConfig', {})) + assert ipam_config.get('IPv6Address') == 'fe80::1001:102' + + def test_up_with_network_static_addresses_missing_subnet(self): + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'networks': { + 'static_test': { + 'ipv4_address': '172.16.100.100', + 'ipv6_address': 'fe80::1001:101' + } + }, + }], + networks={ + 'static_test': { + 'driver': 'bridge', + 'driver_opts': { + "com.docker.network.enable_ipv6": "true", + }, + 'ipam': { + 'driver': 'default', + }, + }, + }, + ) + + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data, + ) + + with pytest.raises(ProjectError): + project.up() + + def test_up_with_network_link_local_ips(self): + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'networks': { + 'linklocaltest': { + 'link_local_ips': ['169.254.8.8'] + } + } + }], + networks={ + 'linklocaltest': {'driver': 'bridge'} + } + ) + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data + ) + project.up(detached=True) + + service_container = project.get_service('web').containers(stopped=True)[0] + ipam_config = service_container.inspect().get( + 'NetworkSettings', {} + ).get( + 'Networks', {} + ).get( + 'composetest_linklocaltest', {} + ).get('IPAMConfig', {}) + assert 'LinkLocalIPs' in ipam_config + assert ipam_config['LinkLocalIPs'] == ['169.254.8.8'] + + def test_up_with_custom_name_resources(self): + config_data = build_config( + services=[{ + 'name': 'web', + 'volumes': [VolumeSpec.parse('foo:/container-path')], + 'networks': {'foo': {}}, + 'image': BUSYBOX_IMAGE_WITH_TAG + }], + networks={ + 'foo': { + 'name': 'zztop', + 'labels': {'com.docker.compose.test_value': 'sharpdressedman'} + } + }, + volumes={ + 'foo': { + 'name': 'acdc', + 'labels': {'com.docker.compose.test_value': 'thefuror'} + } + } + ) + + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data + ) + + project.up(detached=True) + network = [n for n in self.client.networks() if n['Name'] == 'zztop'][0] + volume = [v for v in self.client.volumes()['Volumes'] if v['Name'] == 'acdc'][0] + + assert network['Labels']['com.docker.compose.test_value'] == 'sharpdressedman' + assert volume['Labels']['com.docker.compose.test_value'] == 'thefuror' + + def test_up_with_isolation(self): + self.require_api_version('1.24') + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'isolation': 'default' + }], + ) + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data + ) + project.up(detached=True) + service_container = project.get_service('web').containers(stopped=True)[0] + assert service_container.inspect()['HostConfig']['Isolation'] == 'default' + + def test_up_with_invalid_isolation(self): + self.require_api_version('1.24') + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'isolation': 'foobar' + }], + ) + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data + ) + with pytest.raises(ProjectError): + project.up() + + @if_runtime_available('runc') + def test_up_with_runtime(self): + self.require_api_version('1.30') + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'runtime': 'runc' + }], + ) + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data + ) + project.up(detached=True) + service_container = project.get_service('web').containers(stopped=True)[0] + assert service_container.inspect()['HostConfig']['Runtime'] == 'runc' + + def test_up_with_invalid_runtime(self): + self.require_api_version('1.30') + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'runtime': 'foobar' + }], + ) + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data + ) + with pytest.raises(ProjectError): + project.up() + + @if_runtime_available('nvidia') + def test_up_with_nvidia_runtime(self): + self.require_api_version('1.30') + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'runtime': 'nvidia' + }], + ) + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data + ) + project.up(detached=True) + service_container = project.get_service('web').containers(stopped=True)[0] + assert service_container.inspect()['HostConfig']['Runtime'] == 'nvidia' + + def test_project_up_with_network_internal(self): + self.require_api_version('1.23') + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'networks': {'internal': None}, + }], + networks={ + 'internal': {'driver': 'bridge', 'internal': True}, + }, + ) + + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data, + ) + project.up() + + network = self.client.networks(names=['composetest_internal'])[0] + + assert network['Internal'] is True + + def test_project_up_with_network_label(self): + self.require_api_version('1.23') + + network_name = 'network_with_label' + + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'networks': {network_name: None} + }], + networks={ + network_name: {'labels': {'label_key': 'label_val'}} + } + ) + + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data + ) + + project.up() + + networks = [ + n for n in self.client.networks() + if n['Name'].startswith('composetest_') + ] + + assert [n['Name'] for n in networks] == ['composetest_{}'.format(network_name)] + assert 'label_key' in networks[0]['Labels'] + assert networks[0]['Labels']['label_key'] == 'label_val' + + def test_project_up_volumes(self): + vol_name = '{:x}'.format(random.getrandbits(32)) + full_vol_name = 'composetest_{}'.format(vol_name) + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top' + }], + volumes={vol_name: {'driver': 'local'}}, + ) + + project = Project.from_config( + name='composetest', + config_data=config_data, client=self.client + ) + project.up() + assert len(project.containers()) == 1 + + volume_data = self.get_volume_data(full_vol_name) + assert volume_data['Name'].split('/')[-1] == full_vol_name + assert volume_data['Driver'] == 'local' + + def test_project_up_with_volume_labels(self): + self.require_api_version('1.23') + + volume_name = 'volume_with_label' + + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes': [VolumeSpec.parse('{}:/data'.format(volume_name))] + }], + volumes={ + volume_name: { + 'labels': { + 'label_key': 'label_val' + } + } + }, + ) + + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data, + ) + + project.up() + + volumes = [ + v for v in self.client.volumes().get('Volumes', []) + if v['Name'].split('/')[-1].startswith('composetest_') + ] + + assert {v['Name'].split('/')[-1] for v in volumes} == { + 'composetest_{}'.format(volume_name) + } + + assert 'label_key' in volumes[0]['Labels'] + assert volumes[0]['Labels']['label_key'] == 'label_val' + + def test_project_up_logging_with_multiple_files(self): + base_file = config.ConfigFile( + 'base.yml', + { + 'services': { + 'simple': {'image': BUSYBOX_IMAGE_WITH_TAG, 'command': 'top'}, + 'another': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'logging': { + 'driver': "json-file", + 'options': { + 'max-size': "10m" + } + } + } + } + + }) + override_file = config.ConfigFile( + 'override.yml', + { + 'services': { + 'another': { + 'logging': { + 'driver': "none" + } + } + } + + }) + details = config.ConfigDetails('.', [base_file, override_file]) + + tmpdir = tempfile.mkdtemp('logging_test') + self.addCleanup(shutil.rmtree, tmpdir) + with cd(tmpdir): + config_data = config.load(details) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + project.up() + containers = project.containers() + assert len(containers) == 2 + + another = project.get_service('another').containers()[0] + log_config = another.get('HostConfig.LogConfig') + assert log_config + assert log_config.get('Type') == 'none' + + def test_project_up_port_mappings_with_multiple_files(self): + base_file = config.ConfigFile( + 'base.yml', + { + 'services': { + 'simple': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'ports': ['1234:1234'] + }, + }, + + }) + override_file = config.ConfigFile( + 'override.yml', + { + 'services': { + 'simple': { + 'ports': ['1234:1234'] + } + } + + }) + details = config.ConfigDetails('.', [base_file, override_file]) + + config_data = config.load(details) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + project.up() + containers = project.containers() + assert len(containers) == 1 + + def test_project_up_config_scale(self): + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'scale': 3 + }] + ) + + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + project.up() + assert len(project.containers()) == 3 + + project.up(scale_override={'web': 2}) + assert len(project.containers()) == 2 + + project.up(scale_override={'web': 4}) + assert len(project.containers()) == 4 + + project.stop() + project.up() + assert len(project.containers()) == 3 + + def test_project_up_scale_with_stopped_containers(self): + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'scale': 2 + }] + ) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + + project.up() + containers = project.containers() + assert len(containers) == 2 + + self.client.stop(containers[0].id) + project.up(scale_override={'web': 2}) + containers = project.containers() + assert len(containers) == 2 + + self.client.stop(containers[0].id) + project.up(scale_override={'web': 3}) + assert len(project.containers()) == 3 + + self.client.stop(containers[0].id) + project.up(scale_override={'web': 1}) + assert len(project.containers()) == 1 + + def test_initialize_volumes(self): + vol_name = '{:x}'.format(random.getrandbits(32)) + full_vol_name = 'composetest_{}'.format(vol_name) + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top' + }], + volumes={vol_name: {}}, + ) + + project = Project.from_config( + name='composetest', + config_data=config_data, client=self.client + ) + project.volumes.initialize() + + volume_data = self.get_volume_data(full_vol_name) + assert volume_data['Name'].split('/')[-1] == full_vol_name + assert volume_data['Driver'] == 'local' + + def test_project_up_implicit_volume_driver(self): + vol_name = '{:x}'.format(random.getrandbits(32)) + full_vol_name = 'composetest_{}'.format(vol_name) + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top' + }], + volumes={vol_name: {}}, + ) + + project = Project.from_config( + name='composetest', + config_data=config_data, client=self.client + ) + project.up() + + volume_data = self.get_volume_data(full_vol_name) + assert volume_data['Name'].split('/')[-1] == full_vol_name + assert volume_data['Driver'] == 'local' + + def test_project_up_with_secrets(self): + node = create_host_file(self.client, os.path.abspath('tests/fixtures/secrets/default')) + + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'cat /run/secrets/special', + 'secrets': [ + types.ServiceSecret.parse({'source': 'super', 'target': 'special'}), + ], + 'environment': ['constraint:node=={}'.format(node if node is not None else '*')] + }], + secrets={ + 'super': { + 'file': os.path.abspath('tests/fixtures/secrets/default'), + }, + }, + ) + + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data, + ) + project.up() + project.stop() + + containers = project.containers(stopped=True) + assert len(containers) == 1 + container, = containers + + output = container.logs() + assert output == b"This is the secret\n" + + def test_project_up_with_added_secrets(self): + node = create_host_file(self.client, os.path.abspath('tests/fixtures/secrets/default')) + + config_input1 = { + 'services': [ + { + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'cat /run/secrets/special', + 'environment': ['constraint:node=={}'.format(node if node is not None else '')] + } + + ], + 'secrets': { + 'super': { + 'file': os.path.abspath('tests/fixtures/secrets/default') + } + } + } + config_input2 = copy.deepcopy(config_input1) + # Add the secret + config_input2['services'][0]['secrets'] = [ + types.ServiceSecret.parse({'source': 'super', 'target': 'special'}) + ] + + config_data1 = build_config(**config_input1) + config_data2 = build_config(**config_input2) + + # First up with non-secret + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data1, + ) + project.up() + + # Then up with secret + project = Project.from_config( + client=self.client, + name='composetest', + config_data=config_data2, + ) + project.up() + project.stop() + + containers = project.containers(stopped=True) + assert len(containers) == 1 + container, = containers + + output = container.logs() + assert output == b"This is the secret\n" + + def test_initialize_volumes_invalid_volume_driver(self): + vol_name = '{:x}'.format(random.getrandbits(32)) + + config_data = build_config( + version=VERSION, + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top' + }], + volumes={vol_name: {'driver': 'foobar'}}, + ) + + project = Project.from_config( + name='composetest', + config_data=config_data, client=self.client + ) + with pytest.raises(APIError if is_cluster(self.client) else config.ConfigurationError): + project.volumes.initialize() + + @no_cluster('inspect volume by name defect on Swarm Classic') + def test_initialize_volumes_updated_driver(self): + vol_name = '{:x}'.format(random.getrandbits(32)) + full_vol_name = 'composetest_{}'.format(vol_name) + + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top' + }], + volumes={vol_name: {'driver': 'local'}}, + ) + project = Project.from_config( + name='composetest', + config_data=config_data, client=self.client + ) + project.volumes.initialize() + + volume_data = self.get_volume_data(full_vol_name) + assert volume_data['Name'].split('/')[-1] == full_vol_name + assert volume_data['Driver'] == 'local' + + config_data = config_data._replace( + volumes={vol_name: {'driver': 'smb'}} + ) + project = Project.from_config( + name='composetest', + config_data=config_data, + client=self.client + ) + with pytest.raises(config.ConfigurationError) as e: + project.volumes.initialize() + assert 'Configuration for volume {} specifies driver smb'.format( + vol_name + ) in str(e.value) + + @no_cluster('inspect volume by name defect on Swarm Classic') + def test_initialize_volumes_updated_driver_opts(self): + vol_name = '{:x}'.format(random.getrandbits(32)) + full_vol_name = 'composetest_{}'.format(vol_name) + tmpdir = tempfile.mkdtemp(prefix='compose_test_') + self.addCleanup(shutil.rmtree, tmpdir) + driver_opts = {'o': 'bind', 'device': tmpdir, 'type': 'none'} + + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top' + }], + volumes={ + vol_name: { + 'driver': 'local', + 'driver_opts': driver_opts + } + }, + ) + project = Project.from_config( + name='composetest', + config_data=config_data, client=self.client + ) + project.volumes.initialize() + + volume_data = self.get_volume_data(full_vol_name) + assert volume_data['Name'].split('/')[-1] == full_vol_name + assert volume_data['Driver'] == 'local' + assert volume_data['Options'] == driver_opts + + driver_opts['device'] = '/opt/data/localdata' + project = Project.from_config( + name='composetest', + config_data=config_data, + client=self.client + ) + with pytest.raises(config.ConfigurationError) as e: + project.volumes.initialize() + assert 'Configuration for volume {} specifies "device" driver_opt {}'.format( + vol_name, driver_opts['device'] + ) in str(e.value) + + def test_initialize_volumes_updated_blank_driver(self): + vol_name = '{:x}'.format(random.getrandbits(32)) + full_vol_name = 'composetest_{}'.format(vol_name) + + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top' + }], + volumes={vol_name: {'driver': 'local'}}, + ) + project = Project.from_config( + name='composetest', + config_data=config_data, client=self.client + ) + project.volumes.initialize() + + volume_data = self.get_volume_data(full_vol_name) + assert volume_data['Name'].split('/')[-1] == full_vol_name + assert volume_data['Driver'] == 'local' + + config_data = config_data._replace( + volumes={vol_name: {}} + ) + project = Project.from_config( + name='composetest', + config_data=config_data, + client=self.client + ) + project.volumes.initialize() + volume_data = self.get_volume_data(full_vol_name) + assert volume_data['Name'].split('/')[-1] == full_vol_name + assert volume_data['Driver'] == 'local' + + @no_cluster('inspect volume by name defect on Swarm Classic') + def test_initialize_volumes_external_volumes(self): + # Use composetest_ prefix so it gets garbage-collected in tearDown() + vol_name = 'composetest_{:x}'.format(random.getrandbits(32)) + full_vol_name = 'composetest_{}'.format(vol_name) + self.client.create_volume(vol_name) + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top' + }], + volumes={ + vol_name: {'external': True, 'name': vol_name} + }, + ) + project = Project.from_config( + name='composetest', + config_data=config_data, client=self.client + ) + project.volumes.initialize() + + with pytest.raises(NotFound): + self.client.inspect_volume(full_vol_name) + + def test_initialize_volumes_inexistent_external_volume(self): + vol_name = '{:x}'.format(random.getrandbits(32)) + + config_data = build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top' + }], + volumes={ + vol_name: {'external': True, 'name': vol_name} + }, + ) + project = Project.from_config( + name='composetest', + config_data=config_data, client=self.client + ) + with pytest.raises(config.ConfigurationError) as e: + project.volumes.initialize() + assert 'Volume {} declared as external'.format( + vol_name + ) in str(e.value) + + def test_project_up_named_volumes_in_binds(self): + vol_name = '{:x}'.format(random.getrandbits(32)) + full_vol_name = 'composetest_{}'.format(vol_name) + + base_file = config.ConfigFile( + 'base.yml', + { + 'services': { + 'simple': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'volumes': ['{}:/data'.format(vol_name)] + }, + }, + 'volumes': { + vol_name: {'driver': 'local'} + } + + }) + config_details = config.ConfigDetails('.', [base_file]) + config_data = config.load(config_details) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + service = project.services[0] + assert service.name == 'simple' + volumes = service.options.get('volumes') + assert len(volumes) == 1 + assert volumes[0].external == full_vol_name + project.up() + engine_volumes = self.client.volumes()['Volumes'] + container = service.get_container() + assert [mount['Name'] for mount in container.get('Mounts')] == [full_vol_name] + assert next((v for v in engine_volumes if v['Name'] == vol_name), None) is None + + def test_project_up_orphans(self): + config_dict = { + 'service1': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + } + } + + config_data = load_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + project.up() + config_dict['service2'] = config_dict['service1'] + del config_dict['service1'] + + config_data = load_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + with mock.patch('compose.project.log') as mock_log: + project.up() + + mock_log.warning.assert_called_once_with(mock.ANY) + + assert len([ + ctnr for ctnr in project._labeled_containers() + if ctnr.labels.get(LABEL_SERVICE) == 'service1' + ]) == 1 + + project.up(remove_orphans=True) + + assert len([ + ctnr for ctnr in project._labeled_containers() + if ctnr.labels.get(LABEL_SERVICE) == 'service1' + ]) == 0 + + def test_project_up_ignore_orphans(self): + config_dict = { + 'service1': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + } + } + + config_data = load_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + project.up() + config_dict['service2'] = config_dict['service1'] + del config_dict['service1'] + + config_data = load_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + with mock.patch('compose.project.log') as mock_log: + project.up(ignore_orphans=True) + + mock_log.warning.assert_not_called() + + def test_project_up_healthy_dependency(self): + config_dict = { + 'version': '2.1', + 'services': { + 'svc1': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'healthcheck': { + 'test': 'exit 0', + 'retries': 1, + 'timeout': '10s', + 'interval': '1s' + }, + }, + 'svc2': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'depends_on': { + 'svc1': {'condition': 'service_healthy'}, + } + } + } + } + config_data = load_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + project.up() + containers = project.containers() + assert len(containers) == 2 + + svc1 = project.get_service('svc1') + svc2 = project.get_service('svc2') + assert 'svc1' in svc2.get_dependency_names() + assert svc1.is_healthy() + + def test_project_up_unhealthy_dependency(self): + config_dict = { + 'version': '2.1', + 'services': { + 'svc1': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'healthcheck': { + 'test': 'exit 1', + 'retries': 1, + 'timeout': '10s', + 'interval': '1s' + }, + }, + 'svc2': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'depends_on': { + 'svc1': {'condition': 'service_healthy'}, + } + } + } + } + config_data = load_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + with pytest.raises(ProjectError): + project.up() + containers = project.containers() + assert len(containers) == 1 + + svc1 = project.get_service('svc1') + svc2 = project.get_service('svc2') + assert 'svc1' in svc2.get_dependency_names() + with pytest.raises(HealthCheckFailed): + svc1.is_healthy() + + def test_project_up_no_healthcheck_dependency(self): + config_dict = { + 'version': '2.1', + 'services': { + 'svc1': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'healthcheck': { + 'disable': True + }, + }, + 'svc2': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'depends_on': { + 'svc1': {'condition': 'service_healthy'}, + } + } + } + } + config_data = load_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + with pytest.raises(ProjectError): + project.up() + containers = project.containers() + assert len(containers) == 1 + + svc1 = project.get_service('svc1') + svc2 = project.get_service('svc2') + assert 'svc1' in svc2.get_dependency_names() + with pytest.raises(NoHealthCheckConfigured): + svc1.is_healthy() + + def test_project_up_seccomp_profile(self): + seccomp_data = { + 'defaultAction': 'SCMP_ACT_ALLOW', + 'syscalls': [] + } + fd, profile_path = tempfile.mkstemp('_seccomp.json') + self.addCleanup(os.remove, profile_path) + with os.fdopen(fd, 'w') as f: + json.dump(seccomp_data, f) + + config_dict = { + 'version': '2.3', + 'services': { + 'svc1': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'security_opt': ['seccomp:"{}"'.format(profile_path)] + } + } + } + + config_data = load_config(config_dict) + project = Project.from_config(name='composetest', config_data=config_data, client=self.client) + project.up() + containers = project.containers() + assert len(containers) == 1 + + remote_secopts = containers[0].get('HostConfig.SecurityOpt') + assert len(remote_secopts) == 1 + assert remote_secopts[0].startswith('seccomp=') + assert json.loads(remote_secopts[0].lstrip('seccomp=')) == seccomp_data + + @no_cluster('inspect volume by name defect on Swarm Classic') + def test_project_up_name_starts_with_illegal_char(self): + config_dict = { + 'version': '2.3', + 'services': { + 'svc1': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'ls', + 'volumes': ['foo:/foo:rw'], + 'networks': ['bar'], + }, + }, + 'volumes': { + 'foo': {}, + }, + 'networks': { + 'bar': {}, + } + } + config_data = load_config(config_dict) + project = Project.from_config( + name='_underscoretest', config_data=config_data, client=self.client + ) + project.up() + self.addCleanup(project.down, None, True) + + containers = project.containers(stopped=True) + assert len(containers) == 1 + assert containers[0].name.startswith('underscoretest_svc1_') + assert containers[0].project == '_underscoretest' + + full_vol_name = 'underscoretest_foo' + vol_data = self.get_volume_data(full_vol_name) + assert vol_data + assert vol_data['Labels'][LABEL_PROJECT] == '_underscoretest' + + full_net_name = '_underscoretest_bar' + net_data = self.client.inspect_network(full_net_name) + assert net_data + assert net_data['Labels'][LABEL_PROJECT] == '_underscoretest' + + project2 = Project.from_config( + name='-dashtest', config_data=config_data, client=self.client + ) + project2.up() + self.addCleanup(project2.down, None, True) + + containers = project2.containers(stopped=True) + assert len(containers) == 1 + assert containers[0].name.startswith('dashtest_svc1_') + assert containers[0].project == '-dashtest' + + full_vol_name = 'dashtest_foo' + vol_data = self.get_volume_data(full_vol_name) + assert vol_data + assert vol_data['Labels'][LABEL_PROJECT] == '-dashtest' + + full_net_name = '-dashtest_bar' + net_data = self.client.inspect_network(full_net_name) + assert net_data + assert net_data['Labels'][LABEL_PROJECT] == '-dashtest' diff --git a/tests/integration/resilience_test.py b/tests/integration/resilience_test.py new file mode 100644 index 00000000000..2fbaafb2894 --- /dev/null +++ b/tests/integration/resilience_test.py @@ -0,0 +1,56 @@ +import pytest + +from .. import mock +from .testcases import DockerClientTestCase +from compose.config.types import VolumeSpec +from compose.project import Project +from compose.service import ConvergenceStrategy + + +class ResilienceTest(DockerClientTestCase): + def setUp(self): + self.db = self.create_service( + 'db', + volumes=[VolumeSpec.parse('/var/db')], + command='top') + self.project = Project('composetest', [self.db], self.client) + + container = self.db.create_container() + self.db.start_container(container) + self.host_path = container.get_mount('/var/db')['Source'] + + def tearDown(self): + del self.project + del self.db + super().tearDown() + + def test_successful_recreate(self): + self.project.up(strategy=ConvergenceStrategy.always) + container = self.db.containers()[0] + assert container.get_mount('/var/db')['Source'] == self.host_path + + def test_create_failure(self): + with mock.patch('compose.service.Service.create_container', crash): + with pytest.raises(Crash): + self.project.up(strategy=ConvergenceStrategy.always) + + self.project.up() + container = self.db.containers()[0] + assert container.get_mount('/var/db')['Source'] == self.host_path + + def test_start_failure(self): + with mock.patch('compose.service.Service.start_container', crash): + with pytest.raises(Crash): + self.project.up(strategy=ConvergenceStrategy.always) + + self.project.up() + container = self.db.containers()[0] + assert container.get_mount('/var/db')['Source'] == self.host_path + + +class Crash(Exception): + pass + + +def crash(*args, **kwargs): + raise Crash() diff --git a/tests/integration/service_test.py b/tests/integration/service_test.py new file mode 100644 index 00000000000..06a97508d1f --- /dev/null +++ b/tests/integration/service_test.py @@ -0,0 +1,1784 @@ +import os +import re +import shutil +import tempfile +from distutils.spawn import find_executable +from io import StringIO +from os import path + +import pytest +from docker.errors import APIError +from docker.errors import ImageNotFound + +from .. import mock +from ..helpers import BUSYBOX_IMAGE_WITH_TAG +from .testcases import docker_client +from .testcases import DockerClientTestCase +from .testcases import get_links +from .testcases import pull_busybox +from .testcases import SWARM_SKIP_CONTAINERS_ALL +from .testcases import SWARM_SKIP_CPU_SHARES +from compose import __version__ +from compose.config.types import MountSpec +from compose.config.types import SecurityOpt +from compose.config.types import VolumeFromSpec +from compose.config.types import VolumeSpec +from compose.const import IS_WINDOWS_PLATFORM +from compose.const import LABEL_CONFIG_HASH +from compose.const import LABEL_CONTAINER_NUMBER +from compose.const import LABEL_ONE_OFF +from compose.const import LABEL_PROJECT +from compose.const import LABEL_SERVICE +from compose.const import LABEL_VERSION +from compose.container import Container +from compose.errors import OperationFailedError +from compose.parallel import ParallelStreamWriter +from compose.project import OneOffFilter +from compose.project import Project +from compose.service import BuildAction +from compose.service import BuildError +from compose.service import ConvergencePlan +from compose.service import ConvergenceStrategy +from compose.service import IpcMode +from compose.service import NetworkMode +from compose.service import PidMode +from compose.service import Service +from compose.utils import parse_nanoseconds_int +from tests.helpers import create_custom_host_file +from tests.integration.testcases import is_cluster +from tests.integration.testcases import no_cluster + + +def create_and_start_container(service, **override_options): + container = service.create_container(**override_options) + return service.start_container(container) + + +class ServiceTest(DockerClientTestCase): + + def test_containers(self): + foo = self.create_service('foo') + bar = self.create_service('bar') + + create_and_start_container(foo) + + assert len(foo.containers()) == 1 + assert foo.containers()[0].name.startswith('composetest_foo_') + assert len(bar.containers()) == 0 + + create_and_start_container(bar) + create_and_start_container(bar) + + assert len(foo.containers()) == 1 + assert len(bar.containers()) == 2 + + names = [c.name for c in bar.containers()] + assert len(names) == 2 + assert all(name.startswith('composetest_bar_') for name in names) + + def test_containers_one_off(self): + db = self.create_service('db') + container = db.create_container(one_off=True) + assert db.containers(stopped=True) == [] + assert db.containers(one_off=OneOffFilter.only, stopped=True) == [container] + + def test_project_is_added_to_container_name(self): + service = self.create_service('web') + create_and_start_container(service) + assert service.containers()[0].name.startswith('composetest_web_') + + def test_create_container_with_one_off(self): + db = self.create_service('db') + container = db.create_container(one_off=True) + assert container.name.startswith('composetest_db_run_') + + def test_create_container_with_one_off_when_existing_container_is_running(self): + db = self.create_service('db') + db.start() + container = db.create_container(one_off=True) + assert container.name.startswith('composetest_db_run_') + + def test_create_container_with_unspecified_volume(self): + service = self.create_service('db', volumes=[VolumeSpec.parse('/var/db')]) + container = service.create_container() + service.start_container(container) + assert container.get_mount('/var/db') + + def test_create_container_with_volume_driver(self): + service = self.create_service('db', volume_driver='foodriver') + container = service.create_container() + service.start_container(container) + assert 'foodriver' == container.get('HostConfig.VolumeDriver') + + @pytest.mark.skipif(SWARM_SKIP_CPU_SHARES, reason='Swarm --cpu-shares bug') + def test_create_container_with_cpu_shares(self): + service = self.create_service('db', cpu_shares=73) + container = service.create_container() + service.start_container(container) + assert container.get('HostConfig.CpuShares') == 73 + + def test_create_container_with_cpu_quota(self): + service = self.create_service('db', cpu_quota=40000, cpu_period=150000) + container = service.create_container() + container.start() + assert container.get('HostConfig.CpuQuota') == 40000 + assert container.get('HostConfig.CpuPeriod') == 150000 + + @pytest.mark.xfail(raises=OperationFailedError, reason='not supported by kernel') + def test_create_container_with_cpu_rt(self): + service = self.create_service('db', cpu_rt_runtime=40000, cpu_rt_period=150000) + container = service.create_container() + container.start() + assert container.get('HostConfig.CpuRealtimeRuntime') == 40000 + assert container.get('HostConfig.CpuRealtimePeriod') == 150000 + + def test_create_container_with_cpu_count(self): + self.require_api_version('1.25') + service = self.create_service('db', cpu_count=2) + container = service.create_container() + service.start_container(container) + assert container.get('HostConfig.CpuCount') == 2 + + @pytest.mark.skipif(not IS_WINDOWS_PLATFORM, reason='cpu_percent is not supported for Linux') + def test_create_container_with_cpu_percent(self): + self.require_api_version('1.25') + service = self.create_service('db', cpu_percent=12) + container = service.create_container() + service.start_container(container) + assert container.get('HostConfig.CpuPercent') == 12 + + def test_create_container_with_cpus(self): + self.require_api_version('1.25') + service = self.create_service('db', cpus=1) + container = service.create_container() + service.start_container(container) + assert container.get('HostConfig.NanoCpus') == 1000000000 + + def test_create_container_with_shm_size(self): + self.require_api_version('1.22') + service = self.create_service('db', shm_size=67108864) + container = service.create_container() + service.start_container(container) + assert container.get('HostConfig.ShmSize') == 67108864 + + def test_create_container_with_init_bool(self): + self.require_api_version('1.25') + service = self.create_service('db', init=True) + container = service.create_container() + service.start_container(container) + assert container.get('HostConfig.Init') is True + + @pytest.mark.xfail(True, reason='Option has been removed in Engine 17.06.0') + def test_create_container_with_init_path(self): + self.require_api_version('1.25') + docker_init_path = find_executable('docker-init') + service = self.create_service('db', init=docker_init_path) + container = service.create_container() + service.start_container(container) + assert container.get('HostConfig.InitPath') == docker_init_path + + @pytest.mark.xfail(True, reason='Some kernels/configs do not support pids_limit') + def test_create_container_with_pids_limit(self): + self.require_api_version('1.23') + service = self.create_service('db', pids_limit=10) + container = service.create_container() + service.start_container(container) + assert container.get('HostConfig.PidsLimit') == 10 + + def test_create_container_with_extra_hosts_list(self): + extra_hosts = ['somehost:162.242.195.82', 'otherhost:50.31.209.229'] + service = self.create_service('db', extra_hosts=extra_hosts) + container = service.create_container() + service.start_container(container) + assert set(container.get('HostConfig.ExtraHosts')) == set(extra_hosts) + + def test_create_container_with_extra_hosts_dicts(self): + extra_hosts = {'somehost': '162.242.195.82', 'otherhost': '50.31.209.229'} + extra_hosts_list = ['somehost:162.242.195.82', 'otherhost:50.31.209.229'] + service = self.create_service('db', extra_hosts=extra_hosts) + container = service.create_container() + service.start_container(container) + assert set(container.get('HostConfig.ExtraHosts')) == set(extra_hosts_list) + + def test_create_container_with_cpu_set(self): + service = self.create_service('db', cpuset='0') + container = service.create_container() + service.start_container(container) + assert container.get('HostConfig.CpusetCpus') == '0' + + def test_create_container_with_read_only_root_fs(self): + read_only = True + service = self.create_service('db', read_only=read_only) + container = service.create_container() + service.start_container(container) + assert container.get('HostConfig.ReadonlyRootfs') == read_only + + @pytest.mark.xfail(True, reason='Getting "Your kernel does not support ' + 'cgroup blkio weight and weight_device" on daemon start ' + 'on Linux kernel 5.3.x') + def test_create_container_with_blkio_config(self): + blkio_config = { + 'weight': 300, + 'weight_device': [{'path': '/dev/sda', 'weight': 200}], + 'device_read_bps': [{'path': '/dev/sda', 'rate': 1024 * 1024 * 100}], + 'device_read_iops': [{'path': '/dev/sda', 'rate': 1000}], + 'device_write_bps': [{'path': '/dev/sda', 'rate': 1024 * 1024}], + 'device_write_iops': [{'path': '/dev/sda', 'rate': 800}] + } + service = self.create_service('web', blkio_config=blkio_config) + container = service.create_container() + assert container.get('HostConfig.BlkioWeight') == 300 + assert container.get('HostConfig.BlkioWeightDevice') == [{ + 'Path': '/dev/sda', 'Weight': 200 + }] + assert container.get('HostConfig.BlkioDeviceReadBps') == [{ + 'Path': '/dev/sda', 'Rate': 1024 * 1024 * 100 + }] + assert container.get('HostConfig.BlkioDeviceWriteBps') == [{ + 'Path': '/dev/sda', 'Rate': 1024 * 1024 + }] + assert container.get('HostConfig.BlkioDeviceReadIOps') == [{ + 'Path': '/dev/sda', 'Rate': 1000 + }] + assert container.get('HostConfig.BlkioDeviceWriteIOps') == [{ + 'Path': '/dev/sda', 'Rate': 800 + }] + + def test_create_container_with_security_opt(self): + security_opt = [SecurityOpt.parse('label:disable')] + service = self.create_service('db', security_opt=security_opt) + container = service.create_container() + service.start_container(container) + assert set(container.get('HostConfig.SecurityOpt')) == {o.repr() for o in security_opt} + + @pytest.mark.xfail(True, reason='Not supported on most drivers') + def test_create_container_with_storage_opt(self): + storage_opt = {'size': '1G'} + service = self.create_service('db', storage_opt=storage_opt) + container = service.create_container() + service.start_container(container) + assert container.get('HostConfig.StorageOpt') == storage_opt + + def test_create_container_with_oom_kill_disable(self): + self.require_api_version('1.20') + service = self.create_service('db', oom_kill_disable=True) + container = service.create_container() + assert container.get('HostConfig.OomKillDisable') is True + + def test_create_container_with_mac_address(self): + service = self.create_service('db', mac_address='02:42:ac:11:65:43') + container = service.create_container() + service.start_container(container) + assert container.inspect()['Config']['MacAddress'] == '02:42:ac:11:65:43' + + def test_create_container_with_device_cgroup_rules(self): + service = self.create_service('db', device_cgroup_rules=['c 7:128 rwm']) + container = service.create_container() + assert container.get('HostConfig.DeviceCgroupRules') == ['c 7:128 rwm'] + + def test_create_container_with_specified_volume(self): + host_path = '/tmp/host-path' + container_path = '/container-path' + + service = self.create_service( + 'db', + volumes=[VolumeSpec(host_path, container_path, 'rw')]) + container = service.create_container() + service.start_container(container) + assert container.get_mount(container_path) + + # Match the last component ("host-path"), because boot2docker symlinks /tmp + actual_host_path = container.get_mount(container_path)['Source'] + + assert path.basename(actual_host_path) == path.basename(host_path), ( + "Last component differs: {}, {}".format(actual_host_path, host_path) + ) + + def test_create_container_with_host_mount(self): + host_path = '/tmp/host-path' + container_path = '/container-path' + + create_custom_host_file(self.client, path.join(host_path, 'a.txt'), 'test') + + service = self.create_service( + 'db', + volumes=[ + MountSpec(type='bind', source=host_path, target=container_path, read_only=True) + ] + ) + container = service.create_container() + service.start_container(container) + mount = container.get_mount(container_path) + assert mount + assert path.basename(mount['Source']) == path.basename(host_path) + assert mount['RW'] is False + + def test_create_container_with_tmpfs_mount(self): + container_path = '/container-tmpfs' + service = self.create_service( + 'db', + volumes=[MountSpec(type='tmpfs', target=container_path)] + ) + container = service.create_container() + service.start_container(container) + mount = container.get_mount(container_path) + assert mount + assert mount['Type'] == 'tmpfs' + + def test_create_container_with_tmpfs_mount_tmpfs_size(self): + container_path = '/container-tmpfs' + service = self.create_service( + 'db', + volumes=[MountSpec(type='tmpfs', target=container_path, tmpfs={'size': 5368709})] + ) + container = service.create_container() + service.start_container(container) + mount = container.get_mount(container_path) + assert mount + print(container.dictionary) + assert mount['Type'] == 'tmpfs' + assert container.get('HostConfig.Mounts')[0]['TmpfsOptions'] == { + 'SizeBytes': 5368709 + } + + def test_create_container_with_volume_mount(self): + container_path = '/container-volume' + volume_name = 'composetest_abcde' + self.client.create_volume(volume_name) + service = self.create_service( + 'db', + volumes=[MountSpec(type='volume', source=volume_name, target=container_path)] + ) + container = service.create_container() + service.start_container(container) + mount = container.get_mount(container_path) + assert mount + assert mount['Name'] == volume_name + + def test_create_container_with_legacy_mount(self): + # Ensure mounts are converted to volumes if API version < 1.30 + # Needed to support long syntax in the 3.2 format + client = docker_client({}, version='1.25') + container_path = '/container-volume' + volume_name = 'composetest_abcde' + self.client.create_volume(volume_name) + service = Service('db', client=client, volumes=[ + MountSpec(type='volume', source=volume_name, target=container_path) + ], image=BUSYBOX_IMAGE_WITH_TAG, command=['top'], project='composetest') + container = service.create_container() + service.start_container(container) + mount = container.get_mount(container_path) + assert mount + assert mount['Name'] == volume_name + + def test_create_container_with_legacy_tmpfs_mount(self): + # Ensure tmpfs mounts are converted to tmpfs entries if API version < 1.30 + # Needed to support long syntax in the 3.2 format + client = docker_client({}, version='1.25') + container_path = '/container-tmpfs' + service = Service('db', client=client, volumes=[ + MountSpec(type='tmpfs', target=container_path) + ], image=BUSYBOX_IMAGE_WITH_TAG, command=['top'], project='composetest') + container = service.create_container() + service.start_container(container) + mount = container.get_mount(container_path) + assert mount is None + assert container_path in container.get('HostConfig.Tmpfs') + + def test_create_container_with_healthcheck_config(self): + one_second = parse_nanoseconds_int('1s') + healthcheck = { + 'test': ['true'], + 'interval': 2 * one_second, + 'timeout': 5 * one_second, + 'retries': 5, + 'start_period': 2 * one_second + } + service = self.create_service('db', healthcheck=healthcheck) + container = service.create_container() + remote_healthcheck = container.get('Config.Healthcheck') + assert remote_healthcheck['Test'] == healthcheck['test'] + assert remote_healthcheck['Interval'] == healthcheck['interval'] + assert remote_healthcheck['Timeout'] == healthcheck['timeout'] + assert remote_healthcheck['Retries'] == healthcheck['retries'] + assert remote_healthcheck['StartPeriod'] == healthcheck['start_period'] + + def test_recreate_preserves_volume_with_trailing_slash(self): + """When the Compose file specifies a trailing slash in the container path, make + sure we copy the volume over when recreating. + """ + service = self.create_service('data', volumes=[VolumeSpec.parse('/data/')]) + old_container = create_and_start_container(service) + volume_path = old_container.get_mount('/data')['Source'] + + new_container = service.recreate_container(old_container) + assert new_container.get_mount('/data')['Source'] == volume_path + + def test_recreate_volume_to_mount(self): + # https://github.com/docker/compose/issues/6280 + service = Service( + project='composetest', + name='db', + client=self.client, + build={'context': 'tests/fixtures/dockerfile-with-volume'}, + volumes=[MountSpec.parse({ + 'type': 'volume', + 'target': '/data', + })] + ) + old_container = create_and_start_container(service) + new_container = service.recreate_container(old_container) + assert new_container.get_mount('/data')['Source'] + + def test_duplicate_volume_trailing_slash(self): + """ + When an image specifies a volume, and the Compose file specifies a host path + but adds a trailing slash, make sure that we don't create duplicate binds. + """ + host_path = '/tmp/data' + container_path = '/data' + volumes = [VolumeSpec.parse('{}:{}/'.format(host_path, container_path))] + + tmp_container = self.client.create_container( + 'busybox', 'true', + volumes={container_path: {}}, + labels={'com.docker.compose.test_image': 'true'}, + host_config={} + ) + image = self.client.commit(tmp_container)['Id'] + + service = self.create_service('db', image=image, volumes=volumes) + old_container = create_and_start_container(service) + + assert old_container.get('Config.Volumes') == {container_path: {}} + + service = self.create_service('db', image=image, volumes=volumes) + new_container = service.recreate_container(old_container) + + assert new_container.get('Config.Volumes') == {container_path: {}} + + assert service.containers(stopped=False) == [new_container] + + def test_create_container_with_volumes_from(self): + volume_service = self.create_service('data') + volume_container_1 = volume_service.create_container() + volume_container_2 = Container.create( + self.client, + image=BUSYBOX_IMAGE_WITH_TAG, + command=["top"], + labels={LABEL_PROJECT: 'composetest'}, + host_config={}, + environment=['affinity:container=={}'.format(volume_container_1.id)], + ) + host_service = self.create_service( + 'host', + volumes_from=[ + VolumeFromSpec(volume_service, 'rw', 'service'), + VolumeFromSpec(volume_container_2, 'rw', 'container') + ], + environment=['affinity:container=={}'.format(volume_container_1.id)], + ) + host_container = host_service.create_container() + host_service.start_container(host_container) + assert volume_container_1.id + ':rw' in host_container.get('HostConfig.VolumesFrom') + assert volume_container_2.id + ':rw' in host_container.get('HostConfig.VolumesFrom') + + def test_execute_convergence_plan_recreate(self): + service = self.create_service( + 'db', + environment={'FOO': '1'}, + volumes=[VolumeSpec.parse('/etc')], + entrypoint=['top'], + command=['-d', '1'] + ) + old_container = service.create_container() + assert old_container.get('Config.Entrypoint') == ['top'] + assert old_container.get('Config.Cmd') == ['-d', '1'] + assert 'FOO=1' in old_container.get('Config.Env') + assert old_container.name.startswith('composetest_db_') + service.start_container(old_container) + old_container.inspect() # reload volume data + volume_path = old_container.get_mount('/etc')['Source'] + + num_containers_before = len(self.client.containers(all=True)) + + service.options['environment']['FOO'] = '2' + new_container, = service.execute_convergence_plan( + ConvergencePlan('recreate', [old_container])) + + assert new_container.get('Config.Entrypoint') == ['top'] + assert new_container.get('Config.Cmd') == ['-d', '1'] + assert 'FOO=2' in new_container.get('Config.Env') + assert new_container.name.startswith('composetest_db_') + assert new_container.get_mount('/etc')['Source'] == volume_path + if not is_cluster(self.client): + assert ( + 'affinity:container==%s' % old_container.id in + new_container.get('Config.Env') + ) + else: + # In Swarm, the env marker is consumed and the container should be deployed + # on the same node. + assert old_container.get('Node.Name') == new_container.get('Node.Name') + + assert len(self.client.containers(all=True)) == num_containers_before + assert old_container.id != new_container.id + with pytest.raises(APIError): + self.client.inspect_container(old_container.id) + + def test_execute_convergence_plan_recreate_change_mount_target(self): + service = self.create_service( + 'db', + volumes=[MountSpec(target='/app1', type='volume')], + entrypoint=['top'], command=['-d', '1'] + ) + old_container = create_and_start_container(service) + assert ( + [mount['Destination'] for mount in old_container.get('Mounts')] == + ['/app1'] + ) + service.options['volumes'] = [MountSpec(target='/app2', type='volume')] + + new_container, = service.execute_convergence_plan( + ConvergencePlan('recreate', [old_container]) + ) + + assert ( + [mount['Destination'] for mount in new_container.get('Mounts')] == + ['/app2'] + ) + + def test_execute_convergence_plan_recreate_twice(self): + service = self.create_service( + 'db', + volumes=[VolumeSpec.parse('/etc')], + entrypoint=['top'], + command=['-d', '1']) + + orig_container = service.create_container() + service.start_container(orig_container) + + orig_container.inspect() # reload volume data + volume_path = orig_container.get_mount('/etc')['Source'] + + # Do this twice to reproduce the bug + for _ in range(2): + new_container, = service.execute_convergence_plan( + ConvergencePlan('recreate', [orig_container])) + + assert new_container.get_mount('/etc')['Source'] == volume_path + if not is_cluster(self.client): + assert ('affinity:container==%s' % orig_container.id in + new_container.get('Config.Env')) + else: + # In Swarm, the env marker is consumed and the container should be deployed + # on the same node. + assert orig_container.get('Node.Name') == new_container.get('Node.Name') + + orig_container = new_container + + def test_execute_convergence_plan_recreate_twice_with_mount(self): + service = self.create_service( + 'db', + volumes=[MountSpec(target='/etc', type='volume')], + entrypoint=['top'], + command=['-d', '1'] + ) + + orig_container = service.create_container() + service.start_container(orig_container) + + orig_container.inspect() # reload volume data + volume_path = orig_container.get_mount('/etc')['Source'] + + # Do this twice to reproduce the bug + for _ in range(2): + new_container, = service.execute_convergence_plan( + ConvergencePlan('recreate', [orig_container]) + ) + + assert new_container.get_mount('/etc')['Source'] == volume_path + if not is_cluster(self.client): + assert ('affinity:container==%s' % orig_container.id in + new_container.get('Config.Env')) + else: + # In Swarm, the env marker is consumed and the container should be deployed + # on the same node. + assert orig_container.get('Node.Name') == new_container.get('Node.Name') + + orig_container = new_container + + def test_execute_convergence_plan_when_containers_are_stopped(self): + service = self.create_service( + 'db', + environment={'FOO': '1'}, + volumes=[VolumeSpec.parse('/var/db')], + entrypoint=['top'], + command=['-d', '1'] + ) + service.create_container() + + containers = service.containers(stopped=True) + assert len(containers) == 1 + container, = containers + assert not container.is_running + + service.execute_convergence_plan(ConvergencePlan('start', [container])) + + containers = service.containers() + assert len(containers) == 1 + container.inspect() + assert container == containers[0] + assert container.is_running + + def test_execute_convergence_plan_with_image_declared_volume(self): + service = Service( + project='composetest', + name='db', + client=self.client, + build={'context': 'tests/fixtures/dockerfile-with-volume'}, + ) + + old_container = create_and_start_container(service) + assert [mount['Destination'] for mount in old_container.get('Mounts')] == ['/data'] + volume_path = old_container.get_mount('/data')['Source'] + + new_container, = service.execute_convergence_plan( + ConvergencePlan('recreate', [old_container])) + + assert [mount['Destination'] for mount in new_container.get('Mounts')] == ['/data'] + assert new_container.get_mount('/data')['Source'] == volume_path + + def test_execute_convergence_plan_with_image_declared_volume_renew(self): + service = Service( + project='composetest', + name='db', + client=self.client, + build={'context': 'tests/fixtures/dockerfile-with-volume'}, + ) + + old_container = create_and_start_container(service) + assert [mount['Destination'] for mount in old_container.get('Mounts')] == ['/data'] + volume_path = old_container.get_mount('/data')['Source'] + + new_container, = service.execute_convergence_plan( + ConvergencePlan('recreate', [old_container]), renew_anonymous_volumes=True + ) + + assert [mount['Destination'] for mount in new_container.get('Mounts')] == ['/data'] + assert new_container.get_mount('/data')['Source'] != volume_path + + def test_execute_convergence_plan_when_image_volume_masks_config(self): + service = self.create_service( + 'db', + build={'context': 'tests/fixtures/dockerfile-with-volume'}, + ) + + old_container = create_and_start_container(service) + assert [mount['Destination'] for mount in old_container.get('Mounts')] == ['/data'] + volume_path = old_container.get_mount('/data')['Source'] + + service.options['volumes'] = [VolumeSpec.parse('/tmp:/data')] + + with mock.patch('compose.service.log') as mock_log: + new_container, = service.execute_convergence_plan( + ConvergencePlan('recreate', [old_container])) + + mock_log.warning.assert_called_once_with(mock.ANY) + _, args, kwargs = mock_log.warning.mock_calls[0] + assert "Service \"db\" is using volume \"/data\" from the previous container" in args[0] + + assert [mount['Destination'] for mount in new_container.get('Mounts')] == ['/data'] + assert new_container.get_mount('/data')['Source'] == volume_path + + def test_execute_convergence_plan_when_host_volume_is_removed(self): + host_path = '/tmp/host-path' + service = self.create_service( + 'db', + build={'context': 'tests/fixtures/dockerfile-with-volume'}, + volumes=[VolumeSpec(host_path, '/data', 'rw')]) + + old_container = create_and_start_container(service) + assert ( + [mount['Destination'] for mount in old_container.get('Mounts')] == + ['/data'] + ) + service.options['volumes'] = [] + + with mock.patch('compose.service.log', autospec=True) as mock_log: + new_container, = service.execute_convergence_plan( + ConvergencePlan('recreate', [old_container])) + + assert not mock_log.warn.called + assert ( + [mount['Destination'] for mount in new_container.get('Mounts')] == + ['/data'] + ) + assert new_container.get_mount('/data')['Source'] != host_path + + def test_execute_convergence_plan_anonymous_volume_renew(self): + service = self.create_service( + 'db', + image='busybox', + volumes=[VolumeSpec(None, '/data', 'rw')]) + + old_container = create_and_start_container(service) + assert ( + [mount['Destination'] for mount in old_container.get('Mounts')] == + ['/data'] + ) + volume_path = old_container.get_mount('/data')['Source'] + + new_container, = service.execute_convergence_plan( + ConvergencePlan('recreate', [old_container]), + renew_anonymous_volumes=True + ) + + assert ( + [mount['Destination'] for mount in new_container.get('Mounts')] == + ['/data'] + ) + assert new_container.get_mount('/data')['Source'] != volume_path + + def test_execute_convergence_plan_anonymous_volume_recreate_then_renew(self): + service = self.create_service( + 'db', + image='busybox', + volumes=[VolumeSpec(None, '/data', 'rw')]) + + old_container = create_and_start_container(service) + assert ( + [mount['Destination'] for mount in old_container.get('Mounts')] == + ['/data'] + ) + volume_path = old_container.get_mount('/data')['Source'] + + mid_container, = service.execute_convergence_plan( + ConvergencePlan('recreate', [old_container]), + ) + + assert ( + [mount['Destination'] for mount in mid_container.get('Mounts')] == + ['/data'] + ) + assert mid_container.get_mount('/data')['Source'] == volume_path + + new_container, = service.execute_convergence_plan( + ConvergencePlan('recreate', [mid_container]), + renew_anonymous_volumes=True + ) + + assert ( + [mount['Destination'] for mount in new_container.get('Mounts')] == + ['/data'] + ) + assert new_container.get_mount('/data')['Source'] != volume_path + + def test_execute_convergence_plan_without_start(self): + service = self.create_service( + 'db', + build={'context': 'tests/fixtures/dockerfile-with-volume'} + ) + + containers = service.execute_convergence_plan(ConvergencePlan('create', []), start=False) + service_containers = service.containers(stopped=True) + assert len(service_containers) == 1 + assert not service_containers[0].is_running + + containers = service.execute_convergence_plan( + ConvergencePlan('recreate', containers), + start=False) + service_containers = service.containers(stopped=True) + assert len(service_containers) == 1 + assert not service_containers[0].is_running + + service.execute_convergence_plan(ConvergencePlan('start', containers), start=False) + service_containers = service.containers(stopped=True) + assert len(service_containers) == 1 + assert not service_containers[0].is_running + + def test_execute_convergence_plan_image_with_volume_is_removed(self): + service = self.create_service( + 'db', build={'context': 'tests/fixtures/dockerfile-with-volume'} + ) + + old_container = create_and_start_container(service) + assert ( + [mount['Destination'] for mount in old_container.get('Mounts')] == + ['/data'] + ) + volume_path = old_container.get_mount('/data')['Source'] + + old_container.stop() + self.client.remove_image(service.image(), force=True) + + service.ensure_image_exists() + with pytest.raises(ImageNotFound): + service.execute_convergence_plan( + ConvergencePlan('recreate', [old_container]) + ) + old_container.inspect() # retrieve new name from server + + new_container, = service.execute_convergence_plan( + ConvergencePlan('recreate', [old_container]), + reset_container_image=True + ) + assert [mount['Destination'] for mount in new_container.get('Mounts')] == ['/data'] + assert new_container.get_mount('/data')['Source'] == volume_path + + def test_start_container_passes_through_options(self): + db = self.create_service('db') + create_and_start_container(db, environment={'FOO': 'BAR'}) + assert db.containers()[0].environment['FOO'] == 'BAR' + + def test_start_container_inherits_options_from_constructor(self): + db = self.create_service('db', environment={'FOO': 'BAR'}) + create_and_start_container(db) + assert db.containers()[0].environment['FOO'] == 'BAR' + + @no_cluster('No legacy links support in Swarm') + def test_start_container_creates_links(self): + db = self.create_service('db') + web = self.create_service('web', links=[(db, None)]) + + db1 = create_and_start_container(db) + db2 = create_and_start_container(db) + create_and_start_container(web) + + assert set(get_links(web.containers()[0])) == { + db1.name, db1.name_without_project, + db2.name, db2.name_without_project, + 'db' + } + + @no_cluster('No legacy links support in Swarm') + def test_start_container_creates_links_with_names(self): + db = self.create_service('db') + web = self.create_service('web', links=[(db, 'custom_link_name')]) + + db1 = create_and_start_container(db) + db2 = create_and_start_container(db) + create_and_start_container(web) + + assert set(get_links(web.containers()[0])) == { + db1.name, db1.name_without_project, + db2.name, db2.name_without_project, + 'custom_link_name' + } + + @no_cluster('No legacy links support in Swarm') + def test_start_container_with_external_links(self): + db = self.create_service('db') + db_ctnrs = [create_and_start_container(db) for _ in range(3)] + web = self.create_service( + 'web', external_links=[ + db_ctnrs[0].name, + db_ctnrs[1].name, + '{}:db_3'.format(db_ctnrs[2].name) + ] + ) + + create_and_start_container(web) + + assert set(get_links(web.containers()[0])) == { + db_ctnrs[0].name, + db_ctnrs[1].name, + 'db_3' + } + + @no_cluster('No legacy links support in Swarm') + def test_start_normal_container_does_not_create_links_to_its_own_service(self): + db = self.create_service('db') + + create_and_start_container(db) + create_and_start_container(db) + + c = create_and_start_container(db) + assert set(get_links(c)) == set() + + @no_cluster('No legacy links support in Swarm') + def test_start_one_off_container_creates_links_to_its_own_service(self): + db = self.create_service('db') + + db1 = create_and_start_container(db) + db2 = create_and_start_container(db) + + c = create_and_start_container(db, one_off=OneOffFilter.only) + + assert set(get_links(c)) == { + db1.name, db1.name_without_project, + db2.name, db2.name_without_project, + 'db' + } + + def test_start_container_builds_images(self): + service = Service( + name='test', + client=self.client, + build={'context': 'tests/fixtures/simple-dockerfile'}, + project='composetest', + ) + container = create_and_start_container(service) + container.wait() + assert b'success' in container.logs() + assert len(self.client.images(name='composetest_test')) >= 1 + + def test_start_container_uses_tagged_image_if_it_exists(self): + self.check_build('tests/fixtures/simple-dockerfile', tag='composetest_test') + service = Service( + name='test', + client=self.client, + build={'context': 'this/does/not/exist/and/will/throw/error'}, + project='composetest', + ) + container = create_and_start_container(service) + container.wait() + assert b'success' in container.logs() + + def test_start_container_creates_ports(self): + service = self.create_service('web', ports=[8000]) + container = create_and_start_container(service).inspect() + assert list(container['NetworkSettings']['Ports'].keys()) == ['8000/tcp'] + assert container['NetworkSettings']['Ports']['8000/tcp'][0]['HostPort'] != '8000' + + def test_build(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write("FROM busybox\n") + + service = self.create_service('web', + build={'context': base_dir}, + environment={ + 'COMPOSE_DOCKER_CLI_BUILD': '0', + 'DOCKER_BUILDKIT': '0', + }) + service.build() + self.addCleanup(self.client.remove_image, service.image_name) + + assert self.client.inspect_image('composetest_web') + + def test_build_cli(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write("FROM busybox\n") + + service = self.create_service('web', + build={'context': base_dir}, + environment={ + 'DOCKER_BUILDKIT': '1', + }) + service.build(cli=True) + self.addCleanup(self.client.remove_image, service.image_name) + assert self.client.inspect_image('composetest_web') + + def test_build_cli_with_build_labels(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write("FROM busybox\n") + + service = self.create_service('web', + build={ + 'context': base_dir, + 'labels': {'com.docker.compose.test': 'true'}}, + ) + service.build(cli=True) + self.addCleanup(self.client.remove_image, service.image_name) + image = self.client.inspect_image('composetest_web') + assert image['Config']['Labels']['com.docker.compose.test'] + + def test_build_cli_with_build_error(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write('\n'.join([ + "FROM busybox", + "RUN exit 2", + ])) + service = self.create_service('web', + build={ + 'context': base_dir, + 'labels': {'com.docker.compose.test': 'true'}}, + ) + with pytest.raises(BuildError): + service.build(cli=True) + + def test_up_build_cli(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write("FROM busybox\n") + + web = self.create_service('web', + build={'context': base_dir}, + environment={ + 'DOCKER_BUILDKIT': '1', + }) + project = Project('composetest', [web], self.client) + project.up(do_build=BuildAction.force) + + containers = project.containers(['web']) + assert len(containers) == 1 + assert containers[0].name.startswith('composetest_web_') + + def test_build_non_ascii_filename(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write("FROM busybox\n") + + with open(os.path.join(base_dir.encode('utf8'), b'foo\xE2bar'), 'w') as f: + f.write("hello world\n") + + service = self.create_service('web', build={'context': str(base_dir)}) + service.build() + self.addCleanup(self.client.remove_image, service.image_name) + assert self.client.inspect_image('composetest_web') + + def test_build_with_image_name(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write("FROM busybox\n") + + image_name = 'examples/composetest:latest' + self.addCleanup(self.client.remove_image, image_name) + self.create_service('web', build={'context': base_dir}, image=image_name).build() + assert self.client.inspect_image(image_name) + + def test_build_with_git_url(self): + build_url = "https://github.com/dnephin/docker-build-from-url.git" + service = self.create_service('buildwithurl', build={'context': build_url}) + self.addCleanup(self.client.remove_image, service.image_name) + service.build() + assert service.image() + + def test_build_with_build_args(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write("FROM busybox\n") + f.write("ARG build_version\n") + f.write("RUN echo ${build_version}\n") + + service = self.create_service('buildwithargs', + build={'context': str(base_dir), + 'args': {"build_version": "1"}}) + service.build() + self.addCleanup(self.client.remove_image, service.image_name) + assert service.image() + assert "build_version=1" in service.image()['ContainerConfig']['Cmd'] + + def test_build_with_build_args_override(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write("FROM busybox\n") + f.write("ARG build_version\n") + f.write("RUN echo ${build_version}\n") + + service = self.create_service('buildwithargs', + build={'context': str(base_dir), + 'args': {"build_version": "1"}}) + service.build(build_args_override={'build_version': '2'}) + self.addCleanup(self.client.remove_image, service.image_name) + + assert service.image() + assert "build_version=2" in service.image()['ContainerConfig']['Cmd'] + + def test_build_with_build_labels(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write('FROM busybox\n') + + service = self.create_service('buildlabels', build={ + 'context': str(base_dir), + 'labels': {'com.docker.compose.test': 'true'} + }) + service.build() + self.addCleanup(self.client.remove_image, service.image_name) + + assert service.image() + assert service.image()['Config']['Labels']['com.docker.compose.test'] == 'true' + + @no_cluster('Container networks not on Swarm') + def test_build_with_network(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write('FROM busybox\n') + f.write('RUN ping -c1 google.local\n') + + net_container = self.client.create_container( + 'busybox', 'top', host_config=self.client.create_host_config( + extra_hosts={'google.local': '127.0.0.1'} + ), name='composetest_build_network' + ) + + self.addCleanup(self.client.remove_container, net_container, force=True) + self.client.start(net_container) + + service = self.create_service('buildwithnet', build={ + 'context': str(base_dir), + 'network': 'container:{}'.format(net_container['Id']) + }) + + service.build() + self.addCleanup(self.client.remove_image, service.image_name) + + assert service.image() + + @no_cluster('Not supported on UCP 2.2.0-beta1') # FIXME: remove once support is added + def test_build_with_target(self): + self.require_api_version('1.30') + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write('FROM busybox as one\n') + f.write('LABEL com.docker.compose.test=true\n') + f.write('LABEL com.docker.compose.test.target=one\n') + f.write('FROM busybox as two\n') + f.write('LABEL com.docker.compose.test.target=two\n') + + service = self.create_service('buildtarget', build={ + 'context': str(base_dir), + 'target': 'one' + }) + + service.build() + assert service.image() + assert service.image()['Config']['Labels']['com.docker.compose.test.target'] == 'one' + + def test_build_with_extra_hosts(self): + self.require_api_version('1.27') + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write('\n'.join([ + 'FROM busybox', + 'RUN ping -c1 foobar', + 'RUN ping -c1 baz', + ])) + + service = self.create_service('build_extra_hosts', build={ + 'context': str(base_dir), + 'extra_hosts': { + 'foobar': '127.0.0.1', + 'baz': '127.0.0.1' + } + }) + service.build() + assert service.image() + + def test_build_with_gzip(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write('\n'.join([ + 'FROM busybox', + 'COPY . /src', + 'RUN cat /src/hello.txt' + ])) + with open(os.path.join(base_dir, 'hello.txt'), 'w') as f: + f.write('hello world\n') + + service = self.create_service('build_gzip', build={ + 'context': str(base_dir), + }) + service.build(gzip=True) + assert service.image() + + def test_build_with_isolation(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write('FROM busybox\n') + + service = self.create_service('build_isolation', build={ + 'context': str(base_dir), + 'isolation': 'default', + }) + service.build() + assert service.image() + + def test_build_with_illegal_leading_chars(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write('FROM busybox\nRUN echo "Embodiment of Scarlet Devil"\n') + service = Service( + 'build_leading_slug', client=self.client, + project='___-composetest', build={ + 'context': str(base_dir) + } + ) + assert service.image_name == 'composetest_build_leading_slug' + service.build() + assert service.image() + + def test_start_container_stays_unprivileged(self): + service = self.create_service('web') + container = create_and_start_container(service).inspect() + assert container['HostConfig']['Privileged'] is False + + def test_start_container_becomes_privileged(self): + service = self.create_service('web', privileged=True) + container = create_and_start_container(service).inspect() + assert container['HostConfig']['Privileged'] is True + + def test_expose_does_not_publish_ports(self): + service = self.create_service('web', expose=["8000"]) + container = create_and_start_container(service).inspect() + assert container['NetworkSettings']['Ports'] == {'8000/tcp': None} + + def test_start_container_creates_port_with_explicit_protocol(self): + service = self.create_service('web', ports=['8000/udp']) + container = create_and_start_container(service).inspect() + assert list(container['NetworkSettings']['Ports'].keys()) == ['8000/udp'] + + def test_start_container_creates_fixed_external_ports(self): + service = self.create_service('web', ports=['8000:8000']) + container = create_and_start_container(service).inspect() + assert '8000/tcp' in container['NetworkSettings']['Ports'] + assert container['NetworkSettings']['Ports']['8000/tcp'][0]['HostPort'] == '8000' + + def test_start_container_creates_fixed_external_ports_when_it_is_different_to_internal_port(self): + service = self.create_service('web', ports=['8001:8000']) + container = create_and_start_container(service).inspect() + assert '8000/tcp' in container['NetworkSettings']['Ports'] + assert container['NetworkSettings']['Ports']['8000/tcp'][0]['HostPort'] == '8001' + + def test_port_with_explicit_interface(self): + service = self.create_service('web', ports=[ + '127.0.0.1:8001:8000', + '0.0.0.0:9001:9000/udp', + ]) + container = create_and_start_container(service).inspect() + assert container['NetworkSettings']['Ports']['8000/tcp'] == [{ + 'HostIp': '127.0.0.1', + 'HostPort': '8001', + }] + assert container['NetworkSettings']['Ports']['9000/udp'][0]['HostPort'] == '9001' + if not is_cluster(self.client): + assert container['NetworkSettings']['Ports']['9000/udp'][0]['HostIp'] == '0.0.0.0' + # self.assertEqual(container['NetworkSettings']['Ports'], { + # '8000/tcp': [ + # { + # 'HostIp': '127.0.0.1', + # 'HostPort': '8001', + # }, + # ], + # '9000/udp': [ + # { + # 'HostIp': '0.0.0.0', + # 'HostPort': '9001', + # }, + # ], + # }) + + def test_create_with_image_id(self): + pull_busybox(self.client) + image_id = self.client.inspect_image(BUSYBOX_IMAGE_WITH_TAG)['Id'][:12] + service = self.create_service('foo', image=image_id) + service.create_container() + + def test_scale(self): + service = self.create_service('web') + service.scale(1) + assert len(service.containers()) == 1 + + # Ensure containers don't have stdout or stdin connected + container = service.containers()[0] + config = container.inspect()['Config'] + assert not config['AttachStderr'] + assert not config['AttachStdout'] + assert not config['AttachStdin'] + + service.scale(3) + assert len(service.containers()) == 3 + service.scale(1) + assert len(service.containers()) == 1 + service.scale(0) + assert len(service.containers()) == 0 + + @pytest.mark.skipif( + SWARM_SKIP_CONTAINERS_ALL, + reason='Swarm /containers/json bug' + ) + def test_scale_with_stopped_containers(self): + """ + Given there are some stopped containers and scale is called with a + desired number that is the same as the number of stopped containers, + test that those containers are restarted and not removed/recreated. + """ + service = self.create_service('web') + service.create_container(number=1) + service.create_container(number=2) + + ParallelStreamWriter.instance = None + with mock.patch('sys.stderr', new_callable=StringIO) as mock_stderr: + service.scale(2) + for container in service.containers(): + assert container.is_running + assert container.number in [1, 2] + + captured_output = mock_stderr.getvalue() + assert 'Creating' not in captured_output + assert 'Starting' in captured_output + + def test_scale_with_stopped_containers_and_needing_creation(self): + """ + Given there are some stopped containers and scale is called with a + desired number that is greater than the number of stopped containers, + test that those containers are restarted and required number are created. + """ + service = self.create_service('web') + next_number = service._next_container_number() + service.create_container(number=next_number, quiet=True) + + for container in service.containers(): + assert not container.is_running + + ParallelStreamWriter.instance = None + with mock.patch('sys.stderr', new_callable=StringIO) as mock_stderr: + service.scale(2) + + assert len(service.containers()) == 2 + for container in service.containers(): + assert container.is_running + + captured_output = mock_stderr.getvalue() + assert 'Creating' in captured_output + assert 'Starting' in captured_output + + def test_scale_with_api_error(self): + """Test that when scaling if the API returns an error, that error is handled + and the remaining threads continue. + """ + service = self.create_service('web') + next_number = service._next_container_number() + service.create_container(number=next_number, quiet=True) + + with mock.patch( + 'compose.container.Container.create', + side_effect=APIError( + message="testing", + response={}, + explanation="Boom")): + with mock.patch('sys.stderr', new_callable=StringIO) as mock_stderr: + with pytest.raises(OperationFailedError): + service.scale(3) + + assert len(service.containers()) == 1 + assert service.containers()[0].is_running + assert "ERROR: for composetest_web_" in mock_stderr.getvalue() + assert "Cannot create container for service web: Boom" in mock_stderr.getvalue() + + def test_scale_with_unexpected_exception(self): + """Test that when scaling if the API returns an error, that is not of type + APIError, that error is re-raised. + """ + service = self.create_service('web') + next_number = service._next_container_number() + service.create_container(number=next_number, quiet=True) + + with mock.patch( + 'compose.container.Container.create', + side_effect=ValueError("BOOM") + ): + with pytest.raises(ValueError): + service.scale(3) + + assert len(service.containers()) == 1 + assert service.containers()[0].is_running + + @mock.patch('compose.service.log') + def test_scale_with_desired_number_already_achieved(self, mock_log): + """ + Test that calling scale with a desired number that is equal to the + number of containers already running results in no change. + """ + service = self.create_service('web') + next_number = service._next_container_number() + container = service.create_container(number=next_number, quiet=True) + container.start() + + container.inspect() + assert container.is_running + assert len(service.containers()) == 1 + + service.scale(1) + assert len(service.containers()) == 1 + container.inspect() + assert container.is_running + + captured_output = mock_log.info.call_args[0] + assert 'Desired container number already achieved' in captured_output + + @mock.patch('compose.service.log') + def test_scale_with_custom_container_name_outputs_warning(self, mock_log): + """Test that calling scale on a service that has a custom container name + results in warning output. + """ + service = self.create_service('app', container_name='custom-container') + assert service.custom_container_name == 'custom-container' + + with pytest.raises(OperationFailedError): + service.scale(3) + + captured_output = mock_log.warning.call_args[0][0] + + assert len(service.containers()) == 1 + assert "Remove the custom name to scale the service." in captured_output + + def test_scale_sets_ports(self): + service = self.create_service('web', ports=['8000']) + service.scale(2) + containers = service.containers() + assert len(containers) == 2 + for container in containers: + assert list(container.get('HostConfig.PortBindings')) == ['8000/tcp'] + + def test_scale_with_immediate_exit(self): + service = self.create_service('web', image='busybox', command='true') + service.scale(2) + assert len(service.containers(stopped=True)) == 2 + + def test_network_mode_none(self): + service = self.create_service('web', network_mode=NetworkMode('none')) + container = create_and_start_container(service) + assert container.get('HostConfig.NetworkMode') == 'none' + + def test_network_mode_bridged(self): + service = self.create_service('web', network_mode=NetworkMode('bridge')) + container = create_and_start_container(service) + assert container.get('HostConfig.NetworkMode') == 'bridge' + + def test_network_mode_host(self): + service = self.create_service('web', network_mode=NetworkMode('host')) + container = create_and_start_container(service) + assert container.get('HostConfig.NetworkMode') == 'host' + + def test_pid_mode_none_defined(self): + service = self.create_service('web', pid_mode=None) + container = create_and_start_container(service) + assert container.get('HostConfig.PidMode') == '' + + def test_pid_mode_host(self): + service = self.create_service('web', pid_mode=PidMode('host')) + container = create_and_start_container(service) + assert container.get('HostConfig.PidMode') == 'host' + + def test_ipc_mode_none_defined(self): + service = self.create_service('web', ipc_mode=None) + container = create_and_start_container(service) + print(container.get('HostConfig.IpcMode')) + assert container.get('HostConfig.IpcMode') == 'shareable' + + def test_ipc_mode_host(self): + service = self.create_service('web', ipc_mode=IpcMode('host')) + container = create_and_start_container(service) + assert container.get('HostConfig.IpcMode') == 'host' + + def test_userns_mode_none_defined(self): + service = self.create_service('web', userns_mode=None) + container = create_and_start_container(service) + assert container.get('HostConfig.UsernsMode') == '' + + def test_userns_mode_host(self): + service = self.create_service('web', userns_mode='host') + container = create_and_start_container(service) + assert container.get('HostConfig.UsernsMode') == 'host' + + def test_dns_no_value(self): + service = self.create_service('web') + container = create_and_start_container(service) + assert container.get('HostConfig.Dns') is None + + def test_dns_list(self): + service = self.create_service('web', dns=['8.8.8.8', '9.9.9.9']) + container = create_and_start_container(service) + assert container.get('HostConfig.Dns') == ['8.8.8.8', '9.9.9.9'] + + def test_mem_swappiness(self): + service = self.create_service('web', mem_swappiness=11) + container = create_and_start_container(service) + assert container.get('HostConfig.MemorySwappiness') == 11 + + def test_mem_reservation(self): + service = self.create_service('web', mem_reservation='20m') + container = create_and_start_container(service) + assert container.get('HostConfig.MemoryReservation') == 20 * 1024 * 1024 + + def test_restart_always_value(self): + service = self.create_service('web', restart={'Name': 'always'}) + container = create_and_start_container(service) + assert container.get('HostConfig.RestartPolicy.Name') == 'always' + + def test_oom_score_adj_value(self): + service = self.create_service('web', oom_score_adj=500) + container = create_and_start_container(service) + assert container.get('HostConfig.OomScoreAdj') == 500 + + def test_group_add_value(self): + service = self.create_service('web', group_add=["root", "1"]) + container = create_and_start_container(service) + + host_container_groupadd = container.get('HostConfig.GroupAdd') + assert "root" in host_container_groupadd + assert "1" in host_container_groupadd + + def test_dns_opt_value(self): + service = self.create_service('web', dns_opt=["use-vc", "no-tld-query"]) + container = create_and_start_container(service) + + dns_opt = container.get('HostConfig.DnsOptions') + assert 'use-vc' in dns_opt + assert 'no-tld-query' in dns_opt + + def test_restart_on_failure_value(self): + service = self.create_service('web', restart={ + 'Name': 'on-failure', + 'MaximumRetryCount': 5 + }) + container = create_and_start_container(service) + assert container.get('HostConfig.RestartPolicy.Name') == 'on-failure' + assert container.get('HostConfig.RestartPolicy.MaximumRetryCount') == 5 + + def test_cap_add_list(self): + service = self.create_service('web', cap_add=['SYS_ADMIN', 'NET_ADMIN']) + container = create_and_start_container(service) + assert container.get('HostConfig.CapAdd') == ['SYS_ADMIN', 'NET_ADMIN'] + + def test_cap_drop_list(self): + service = self.create_service('web', cap_drop=['SYS_ADMIN', 'NET_ADMIN']) + container = create_and_start_container(service) + assert container.get('HostConfig.CapDrop') == ['SYS_ADMIN', 'NET_ADMIN'] + + def test_dns_search(self): + service = self.create_service('web', dns_search=['dc1.example.com', 'dc2.example.com']) + container = create_and_start_container(service) + assert container.get('HostConfig.DnsSearch') == ['dc1.example.com', 'dc2.example.com'] + + def test_tmpfs(self): + service = self.create_service('web', tmpfs=['/run']) + container = create_and_start_container(service) + assert container.get('HostConfig.Tmpfs') == {'/run': ''} + + def test_working_dir_param(self): + service = self.create_service('container', working_dir='/working/dir/sample') + container = service.create_container() + assert container.get('Config.WorkingDir') == '/working/dir/sample' + + def test_split_env(self): + service = self.create_service( + 'web', + environment=['NORMAL=F1', 'CONTAINS_EQUALS=F=2', 'TRAILING_EQUALS=']) + env = create_and_start_container(service).environment + for k, v in {'NORMAL': 'F1', 'CONTAINS_EQUALS': 'F=2', 'TRAILING_EQUALS': ''}.items(): + assert env[k] == v + + def test_env_from_file_combined_with_env(self): + service = self.create_service( + 'web', + environment=['ONE=1', 'TWO=2', 'THREE=3'], + env_file=['tests/fixtures/env/one.env', 'tests/fixtures/env/two.env']) + env = create_and_start_container(service).environment + for k, v in { + 'ONE': '1', + 'TWO': '2', + 'THREE': '3', + 'FOO': 'baz', + 'DOO': 'dah' + }.items(): + assert env[k] == v + + def test_build_with_cachefrom(self): + base_dir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, base_dir) + + with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f: + f.write("FROM busybox\n") + + service = self.create_service('cache_from', + build={'context': base_dir, + 'cache_from': ['build1']}) + service.build() + self.addCleanup(self.client.remove_image, service.image_name) + + assert service.image() + + @mock.patch.dict(os.environ) + def test_resolve_env(self): + os.environ['FILE_DEF'] = 'E1' + os.environ['FILE_DEF_EMPTY'] = 'E2' + os.environ['ENV_DEF'] = 'E3' + service = self.create_service( + 'web', + environment={ + 'FILE_DEF': 'F1', + 'FILE_DEF_EMPTY': '', + 'ENV_DEF': None, + 'NO_DEF': None + } + ) + env = create_and_start_container(service).environment + for k, v in { + 'FILE_DEF': 'F1', + 'FILE_DEF_EMPTY': '', + 'ENV_DEF': 'E3', + 'NO_DEF': None + }.items(): + assert env[k] == v + + def test_with_high_enough_api_version_we_get_default_network_mode(self): + # TODO: remove this test once minimum docker version is 1.8.x + with mock.patch.object(self.client, '_version', '1.20'): + service = self.create_service('web') + service_config = service._get_container_host_config({}) + assert service_config['NetworkMode'] == 'default' + + def test_labels(self): + labels_dict = { + 'com.example.description': "Accounting webapp", + 'com.example.department': "Finance", + 'com.example.label-with-empty-value': "", + } + + compose_labels = { + LABEL_ONE_OFF: 'False', + LABEL_PROJECT: 'composetest', + LABEL_SERVICE: 'web', + LABEL_VERSION: __version__, + LABEL_CONTAINER_NUMBER: '1' + } + expected = dict(labels_dict, **compose_labels) + + service = self.create_service('web', labels=labels_dict) + ctnr = create_and_start_container(service) + labels = ctnr.labels.items() + for pair in expected.items(): + assert pair in labels + + def test_empty_labels(self): + labels_dict = {'foo': '', 'bar': ''} + service = self.create_service('web', labels=labels_dict) + labels = create_and_start_container(service).labels.items() + for name in labels_dict: + assert (name, '') in labels + + def test_stop_signal(self): + stop_signal = 'SIGINT' + service = self.create_service('web', stop_signal=stop_signal) + container = create_and_start_container(service) + assert container.stop_signal == stop_signal + + def test_custom_container_name(self): + service = self.create_service('web', container_name='my-web-container') + assert service.custom_container_name == 'my-web-container' + + container = create_and_start_container(service) + assert container.name == 'my-web-container' + + one_off_container = service.create_container(one_off=True) + assert one_off_container.name != 'my-web-container' + + @pytest.mark.skipif(True, reason="Broken on 1.11.0 - 17.03.0") + def test_log_drive_invalid(self): + service = self.create_service('web', logging={'driver': 'xxx'}) + expected_error_msg = "logger: no log driver named 'xxx' is registered" + + with pytest.raises(APIError) as excinfo: + create_and_start_container(service) + assert re.search(expected_error_msg, excinfo.value) + + def test_log_drive_empty_default_jsonfile(self): + service = self.create_service('web') + log_config = create_and_start_container(service).log_config + + assert 'json-file' == log_config['Type'] + assert not log_config['Config'] + + def test_log_drive_none(self): + service = self.create_service('web', logging={'driver': 'none'}) + log_config = create_and_start_container(service).log_config + + assert 'none' == log_config['Type'] + assert not log_config['Config'] + + def test_devices(self): + service = self.create_service('web', devices=["/dev/random:/dev/mapped-random"]) + device_config = create_and_start_container(service).get('HostConfig.Devices') + + device_dict = { + 'PathOnHost': '/dev/random', + 'CgroupPermissions': 'rwm', + 'PathInContainer': '/dev/mapped-random' + } + + assert 1 == len(device_config) + assert device_dict == device_config[0] + + def test_duplicate_containers(self): + service = self.create_service('web') + + options = service._get_container_create_options({}, service._next_container_number()) + original = Container.create(service.client, **options) + + assert set(service.containers(stopped=True)) == {original} + assert set(service.duplicate_containers()) == set() + + options['name'] = 'temporary_container_name' + duplicate = Container.create(service.client, **options) + + assert set(service.containers(stopped=True)) == {original, duplicate} + assert set(service.duplicate_containers()) == {duplicate} + + +def converge(service, strategy=ConvergenceStrategy.changed): + """Create a converge plan from a strategy and execute the plan.""" + plan = service.convergence_plan(strategy) + return service.execute_convergence_plan(plan, timeout=1) + + +class ConfigHashTest(DockerClientTestCase): + + def test_no_config_hash_when_one_off(self): + web = self.create_service('web') + container = web.create_container(one_off=True) + assert LABEL_CONFIG_HASH not in container.labels + + def test_no_config_hash_when_overriding_options(self): + web = self.create_service('web') + container = web.create_container(environment={'FOO': '1'}) + assert LABEL_CONFIG_HASH not in container.labels + + def test_config_hash_with_custom_labels(self): + web = self.create_service('web', labels={'foo': '1'}) + container = converge(web)[0] + assert LABEL_CONFIG_HASH in container.labels + assert 'foo' in container.labels + + def test_config_hash_sticks_around(self): + web = self.create_service('web', command=["top"]) + container = converge(web)[0] + assert LABEL_CONFIG_HASH in container.labels + + web = self.create_service('web', command=["top", "-d", "1"]) + container = converge(web)[0] + assert LABEL_CONFIG_HASH in container.labels diff --git a/tests/integration/state_test.py b/tests/integration/state_test.py new file mode 100644 index 00000000000..8168cddf010 --- /dev/null +++ b/tests/integration/state_test.py @@ -0,0 +1,462 @@ +""" +Integration tests which cover state convergence (aka smart recreate) performed +by `docker-compose up`. +""" +import copy +import os +import shutil +import tempfile + +from docker.errors import ImageNotFound + +from ..helpers import BUSYBOX_IMAGE_WITH_TAG +from .testcases import DockerClientTestCase +from .testcases import get_links +from .testcases import no_cluster +from compose.config import config +from compose.project import Project +from compose.service import ConvergenceStrategy + + +class ProjectTestCase(DockerClientTestCase): + def run_up(self, cfg, **kwargs): + kwargs.setdefault('timeout', 1) + kwargs.setdefault('detached', True) + + project = self.make_project(cfg) + project.up(**kwargs) + return set(project.containers(stopped=True)) + + def make_project(self, cfg): + details = config.ConfigDetails( + 'working_dir', + [config.ConfigFile(None, cfg)]) + return Project.from_config( + name='composetest', + client=self.client, + config_data=config.load(details)) + + +class BasicProjectTest(ProjectTestCase): + def setUp(self): + super().setUp() + + self.cfg = { + 'db': {'image': BUSYBOX_IMAGE_WITH_TAG, 'command': 'top'}, + 'web': {'image': BUSYBOX_IMAGE_WITH_TAG, 'command': 'top'}, + } + + def test_no_change(self): + old_containers = self.run_up(self.cfg) + assert len(old_containers) == 2 + + new_containers = self.run_up(self.cfg) + assert len(new_containers) == 2 + + assert old_containers == new_containers + + def test_partial_change(self): + old_containers = self.run_up(self.cfg) + old_db = [c for c in old_containers if c.name_without_project.startswith('db_')][0] + old_web = [c for c in old_containers if c.name_without_project.startswith('web_')][0] + + self.cfg['web']['command'] = '/bin/true' + + new_containers = self.run_up(self.cfg) + assert len(new_containers) == 2 + + preserved = list(old_containers & new_containers) + assert preserved == [old_db] + + removed = list(old_containers - new_containers) + assert removed == [old_web] + + created = list(new_containers - old_containers) + assert len(created) == 1 + assert created[0].name_without_project == old_web.name_without_project + assert created[0].get('Config.Cmd') == ['/bin/true'] + + def test_all_change(self): + old_containers = self.run_up(self.cfg) + assert len(old_containers) == 2 + + self.cfg['web']['command'] = '/bin/true' + self.cfg['db']['command'] = '/bin/true' + + new_containers = self.run_up(self.cfg) + assert len(new_containers) == 2 + + unchanged = old_containers & new_containers + assert len(unchanged) == 0 + + new = new_containers - old_containers + assert len(new) == 2 + + +class ProjectWithDependenciesTest(ProjectTestCase): + def setUp(self): + super().setUp() + + self.cfg = { + 'db': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'tail -f /dev/null', + }, + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'tail -f /dev/null', + 'links': ['db'], + }, + 'nginx': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'tail -f /dev/null', + 'links': ['web'], + }, + } + + def test_up(self): + containers = self.run_up(self.cfg) + assert {c.service for c in containers} == {'db', 'web', 'nginx'} + + def test_change_leaf(self): + old_containers = self.run_up(self.cfg) + + self.cfg['nginx']['environment'] = {'NEW_VAR': '1'} + new_containers = self.run_up(self.cfg) + + assert {c.service for c in new_containers - old_containers} == {'nginx'} + + def test_change_middle(self): + old_containers = self.run_up(self.cfg) + + self.cfg['web']['environment'] = {'NEW_VAR': '1'} + new_containers = self.run_up(self.cfg) + + assert {c.service for c in new_containers - old_containers} == {'web'} + + def test_change_middle_always_recreate_deps(self): + old_containers = self.run_up(self.cfg, always_recreate_deps=True) + + self.cfg['web']['environment'] = {'NEW_VAR': '1'} + new_containers = self.run_up(self.cfg, always_recreate_deps=True) + + assert {c.service for c in new_containers - old_containers} == {'web', 'nginx'} + + def test_change_root(self): + old_containers = self.run_up(self.cfg) + + self.cfg['db']['environment'] = {'NEW_VAR': '1'} + new_containers = self.run_up(self.cfg) + + assert {c.service for c in new_containers - old_containers} == {'db'} + + def test_change_root_always_recreate_deps(self): + old_containers = self.run_up(self.cfg, always_recreate_deps=True) + + self.cfg['db']['environment'] = {'NEW_VAR': '1'} + new_containers = self.run_up(self.cfg, always_recreate_deps=True) + + assert {c.service for c in new_containers - old_containers} == { + 'db', 'web', 'nginx' + } + + def test_change_root_no_recreate(self): + old_containers = self.run_up(self.cfg) + + self.cfg['db']['environment'] = {'NEW_VAR': '1'} + new_containers = self.run_up( + self.cfg, + strategy=ConvergenceStrategy.never) + + assert new_containers - old_containers == set() + + def test_service_removed_while_down(self): + next_cfg = { + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'tail -f /dev/null', + }, + 'nginx': self.cfg['nginx'], + } + + containers = self.run_up(self.cfg) + assert len(containers) == 3 + + project = self.make_project(self.cfg) + project.stop(timeout=1) + + containers = self.run_up(next_cfg) + assert len(containers) == 2 + + def test_service_recreated_when_dependency_created(self): + containers = self.run_up(self.cfg, service_names=['web'], start_deps=False) + assert len(containers) == 1 + + containers = self.run_up(self.cfg) + assert len(containers) == 3 + + web, = [c for c in containers if c.service == 'web'] + nginx, = [c for c in containers if c.service == 'nginx'] + db, = [c for c in containers if c.service == 'db'] + + assert set(get_links(web)) == { + 'composetest_db_1', + 'db', + 'db_1', + } + assert set(get_links(nginx)) == { + 'composetest_web_1', + 'web', + 'web_1', + } + + +class ProjectWithDependsOnDependenciesTest(ProjectTestCase): + def setUp(self): + super().setUp() + + self.cfg = { + 'version': '2', + 'services': { + 'db': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'tail -f /dev/null', + }, + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'tail -f /dev/null', + 'depends_on': ['db'], + }, + 'nginx': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'tail -f /dev/null', + 'depends_on': ['web'], + }, + } + } + + def test_up(self): + local_cfg = copy.deepcopy(self.cfg) + containers = self.run_up(local_cfg) + assert {c.service for c in containers} == {'db', 'web', 'nginx'} + + def test_change_leaf(self): + local_cfg = copy.deepcopy(self.cfg) + old_containers = self.run_up(local_cfg) + + local_cfg['services']['nginx']['environment'] = {'NEW_VAR': '1'} + new_containers = self.run_up(local_cfg) + + assert {c.service for c in new_containers - old_containers} == {'nginx'} + + def test_change_middle(self): + local_cfg = copy.deepcopy(self.cfg) + old_containers = self.run_up(local_cfg) + + local_cfg['services']['web']['environment'] = {'NEW_VAR': '1'} + new_containers = self.run_up(local_cfg) + + assert {c.service for c in new_containers - old_containers} == {'web'} + + def test_change_middle_always_recreate_deps(self): + local_cfg = copy.deepcopy(self.cfg) + old_containers = self.run_up(local_cfg, always_recreate_deps=True) + + local_cfg['services']['web']['environment'] = {'NEW_VAR': '1'} + new_containers = self.run_up(local_cfg, always_recreate_deps=True) + + assert {c.service for c in new_containers - old_containers} == {'web', 'nginx'} + + def test_change_root(self): + local_cfg = copy.deepcopy(self.cfg) + old_containers = self.run_up(local_cfg) + + local_cfg['services']['db']['environment'] = {'NEW_VAR': '1'} + new_containers = self.run_up(local_cfg) + + assert {c.service for c in new_containers - old_containers} == {'db'} + + def test_change_root_always_recreate_deps(self): + local_cfg = copy.deepcopy(self.cfg) + old_containers = self.run_up(local_cfg, always_recreate_deps=True) + + local_cfg['services']['db']['environment'] = {'NEW_VAR': '1'} + new_containers = self.run_up(local_cfg, always_recreate_deps=True) + + assert {c.service for c in new_containers - old_containers} == {'db', 'web', 'nginx'} + + def test_change_root_no_recreate(self): + local_cfg = copy.deepcopy(self.cfg) + old_containers = self.run_up(local_cfg) + + local_cfg['services']['db']['environment'] = {'NEW_VAR': '1'} + new_containers = self.run_up( + local_cfg, + strategy=ConvergenceStrategy.never) + + assert new_containers - old_containers == set() + + def test_service_removed_while_down(self): + local_cfg = copy.deepcopy(self.cfg) + next_cfg = copy.deepcopy(self.cfg) + del next_cfg['services']['db'] + del next_cfg['services']['web']['depends_on'] + + containers = self.run_up(local_cfg) + assert {c.service for c in containers} == {'db', 'web', 'nginx'} + + project = self.make_project(local_cfg) + project.stop(timeout=1) + + next_containers = self.run_up(next_cfg) + assert {c.service for c in next_containers} == {'web', 'nginx'} + + def test_service_removed_while_up(self): + local_cfg = copy.deepcopy(self.cfg) + containers = self.run_up(local_cfg) + assert {c.service for c in containers} == {'db', 'web', 'nginx'} + + del local_cfg['services']['db'] + del local_cfg['services']['web']['depends_on'] + + containers = self.run_up(local_cfg) + assert {c.service for c in containers} == {'web', 'nginx'} + + def test_dependency_removed(self): + local_cfg = copy.deepcopy(self.cfg) + next_cfg = copy.deepcopy(self.cfg) + del next_cfg['services']['nginx']['depends_on'] + + containers = self.run_up(local_cfg, service_names=['nginx']) + assert {c.service for c in containers} == {'db', 'web', 'nginx'} + + project = self.make_project(local_cfg) + project.stop(timeout=1) + + next_containers = self.run_up(next_cfg, service_names=['nginx']) + assert {c.service for c in next_containers if c.is_running} == {'nginx'} + + def test_dependency_added(self): + local_cfg = copy.deepcopy(self.cfg) + + del local_cfg['services']['nginx']['depends_on'] + containers = self.run_up(local_cfg, service_names=['nginx']) + assert {c.service for c in containers} == {'nginx'} + + local_cfg['services']['nginx']['depends_on'] = ['db'] + containers = self.run_up(local_cfg, service_names=['nginx']) + assert {c.service for c in containers} == {'nginx', 'db'} + + +class ServiceStateTest(DockerClientTestCase): + """Test cases for Service.convergence_plan.""" + + def test_trigger_create(self): + web = self.create_service('web') + assert ('create', []) == web.convergence_plan() + + def test_trigger_noop(self): + web = self.create_service('web') + container = web.create_container() + web.start() + + web = self.create_service('web') + assert ('noop', [container]) == web.convergence_plan() + + def test_trigger_start(self): + options = dict(command=["top"]) + + web = self.create_service('web', **options) + web.scale(2) + + containers = web.containers(stopped=True) + containers[0].stop() + containers[0].inspect() + + assert [c.is_running for c in containers] == [False, True] + + assert ('start', containers) == web.convergence_plan() + + def test_trigger_recreate_with_config_change(self): + web = self.create_service('web', command=["top"]) + container = web.create_container() + + web = self.create_service('web', command=["top", "-d", "1"]) + assert ('recreate', [container]) == web.convergence_plan() + + def test_trigger_recreate_with_nonexistent_image_tag(self): + web = self.create_service('web', image=BUSYBOX_IMAGE_WITH_TAG) + container = web.create_container() + + web = self.create_service('web', image="nonexistent-image") + assert ('recreate', [container]) == web.convergence_plan() + + def test_trigger_recreate_with_image_change(self): + repo = 'composetest_myimage' + tag = 'latest' + image = '{}:{}'.format(repo, tag) + + def safe_remove_image(image): + try: + self.client.remove_image(image) + except ImageNotFound: + pass + + image_id = self.client.images(name='busybox')[0]['Id'] + self.client.tag(image_id, repository=repo, tag=tag) + self.addCleanup(safe_remove_image, image) + + web = self.create_service('web', image=image) + container = web.create_container() + + # update the image + c = self.client.create_container(image, ['touch', '/hello.txt'], host_config={}) + + # In the case of a cluster, there's a chance we pick up the old image when + # calculating the new hash. To circumvent that, untag the old image first + # See also: https://github.com/moby/moby/issues/26852 + self.client.remove_image(image, force=True) + + self.client.commit(c, repository=repo, tag=tag) + self.client.remove_container(c) + + web = self.create_service('web', image=image) + assert ('recreate', [container]) == web.convergence_plan() + + @no_cluster('Can not guarantee the build will be run on the same node the service is deployed') + def test_trigger_recreate_with_build(self): + context = tempfile.mkdtemp('test_trigger_recreate_with_build') + self.addCleanup(shutil.rmtree, context) + + base_image = "FROM busybox\nLABEL com.docker.compose.test_image=true\n" + dockerfile = os.path.join(context, 'Dockerfile') + with open(dockerfile, mode="w") as dockerfile_fh: + dockerfile_fh.write(base_image) + + web = self.create_service('web', build={'context': str(context)}) + container = web.create_container() + + with open(dockerfile, mode="w") as dockerfile_fh: + dockerfile_fh.write(base_image + 'CMD echo hello world\n') + web.build() + + web = self.create_service('web', build={'context': str(context)}) + assert ('recreate', [container]) == web.convergence_plan() + + def test_image_changed_to_build(self): + context = tempfile.mkdtemp('test_image_changed_to_build') + self.addCleanup(shutil.rmtree, context) + with open(os.path.join(context, 'Dockerfile'), mode="w") as dockerfile: + dockerfile.write(""" + FROM busybox + LABEL com.docker.compose.test_image=true + """) + + web = self.create_service('web', image='busybox') + container = web.create_container() + + web = self.create_service('web', build={'context': str(context)}) + plan = web.convergence_plan() + assert ('recreate', [container]) == plan + containers = web.execute_convergence_plan(plan) + assert len(containers) == 1 diff --git a/tests/integration/testcases.py b/tests/integration/testcases.py new file mode 100644 index 00000000000..d4fbc9f61a2 --- /dev/null +++ b/tests/integration/testcases.py @@ -0,0 +1,168 @@ +import functools +import os + +import pytest +from docker.errors import APIError +from docker.utils import version_lt + +from .. import unittest +from ..helpers import BUSYBOX_IMAGE_WITH_TAG +from compose.cli.docker_client import docker_client +from compose.config.config import resolve_environment +from compose.config.environment import Environment +from compose.const import API_VERSIONS +from compose.const import COMPOSE_SPEC as VERSION +from compose.const import COMPOSEFILE_V1 as V1 +from compose.const import LABEL_PROJECT +from compose.progress_stream import stream_output +from compose.service import Service + +SWARM_SKIP_CONTAINERS_ALL = os.environ.get('SWARM_SKIP_CONTAINERS_ALL', '0') != '0' +SWARM_SKIP_CPU_SHARES = os.environ.get('SWARM_SKIP_CPU_SHARES', '0') != '0' +SWARM_SKIP_RM_VOLUMES = os.environ.get('SWARM_SKIP_RM_VOLUMES', '0') != '0' +SWARM_ASSUME_MULTINODE = os.environ.get('SWARM_ASSUME_MULTINODE', '0') != '0' + + +def pull_busybox(client): + client.pull(BUSYBOX_IMAGE_WITH_TAG, stream=False) + + +def get_links(container): + links = container.get('HostConfig.Links') or [] + + def format_link(link): + _, alias = link.split(':') + return alias.split('/')[-1] + + return [format_link(link) for link in links] + + +def engine_max_version(): + if 'DOCKER_VERSION' not in os.environ: + return VERSION + version = os.environ['DOCKER_VERSION'].partition('-')[0] + if version_lt(version, '1.10'): + return V1 + return VERSION + + +def min_version_skip(version): + return pytest.mark.skipif( + engine_max_version() < version, + reason="Engine version %s is too low" % version + ) + + +class DockerClientTestCase(unittest.TestCase): + @classmethod + def setUpClass(cls): + version = API_VERSIONS[engine_max_version()] + cls.client = docker_client(Environment(), version) + + @classmethod + def tearDownClass(cls): + cls.client.close() + del cls.client + + def tearDown(self): + for c in self.client.containers( + all=True, + filters={'label': '%s=composetest' % LABEL_PROJECT}): + self.client.remove_container(c['Id'], force=True) + + for i in self.client.images( + filters={'label': 'com.docker.compose.test_image'}): + try: + self.client.remove_image(i, force=True) + except APIError as e: + if e.is_server_error(): + pass + + volumes = self.client.volumes().get('Volumes') or [] + for v in volumes: + if 'composetest_' in v['Name']: + self.client.remove_volume(v['Name']) + + networks = self.client.networks() + for n in networks: + if 'composetest_' in n['Name']: + self.client.remove_network(n['Name']) + + def create_service(self, name, **kwargs): + if 'image' not in kwargs and 'build' not in kwargs: + kwargs['image'] = BUSYBOX_IMAGE_WITH_TAG + + if 'command' not in kwargs: + kwargs['command'] = ["top"] + + kwargs['environment'] = resolve_environment( + kwargs, Environment.from_env_file(None) + ) + labels = dict(kwargs.setdefault('labels', {})) + labels['com.docker.compose.test-name'] = self.id() + + return Service(name, client=self.client, project='composetest', **kwargs) + + def check_build(self, *args, **kwargs): + kwargs.setdefault('rm', True) + build_output = self.client.build(*args, **kwargs) + with open(os.devnull, 'w') as devnull: + for event in stream_output(build_output, devnull): + pass + + def require_api_version(self, minimum): + api_version = self.client.version()['ApiVersion'] + if version_lt(api_version, minimum): + pytest.skip("API version is too low ({} < {})".format(api_version, minimum)) + + def get_volume_data(self, volume_name): + if not is_cluster(self.client): + return self.client.inspect_volume(volume_name) + + volumes = self.client.volumes(filters={'name': volume_name})['Volumes'] + assert len(volumes) > 0 + return self.client.inspect_volume(volumes[0]['Name']) + + +def if_runtime_available(runtime): + def decorator(f): + @functools.wraps(f) + def wrapper(self, *args, **kwargs): + if runtime not in self.client.info().get('Runtimes', {}): + return pytest.skip("This daemon does not support the '{}'' runtime".format(runtime)) + return f(self, *args, **kwargs) + return wrapper + + return decorator + + +def is_cluster(client): + if SWARM_ASSUME_MULTINODE: + return True + + def get_nodes_number(): + try: + return len(client.nodes()) + except APIError: + # If the Engine is not part of a Swarm, the SDK will raise + # an APIError + return 0 + + if not hasattr(is_cluster, 'nodes') or is_cluster.nodes is None: + # Only make the API call if the value hasn't been cached yet + is_cluster.nodes = get_nodes_number() + + return is_cluster.nodes > 1 + + +def no_cluster(reason): + def decorator(f): + @functools.wraps(f) + def wrapper(self, *args, **kwargs): + if is_cluster(self.client): + pytest.skip("Test will not be run in cluster mode: %s" % reason) + return + return f(self, *args, **kwargs) + return wrapper + + return decorator diff --git a/tests/integration/volume_test.py b/tests/integration/volume_test.py new file mode 100644 index 00000000000..0e7c78bc25a --- /dev/null +++ b/tests/integration/volume_test.py @@ -0,0 +1,122 @@ +from docker.errors import DockerException + +from .testcases import DockerClientTestCase +from .testcases import no_cluster +from compose.const import LABEL_PROJECT +from compose.const import LABEL_VOLUME +from compose.volume import Volume + + +class VolumeTest(DockerClientTestCase): + def setUp(self): + self.tmp_volumes = [] + + def tearDown(self): + for volume in self.tmp_volumes: + try: + self.client.remove_volume(volume.full_name) + except DockerException: + pass + del self.tmp_volumes + super().tearDown() + + def create_volume(self, name, driver=None, opts=None, external=None, custom_name=False): + if external: + custom_name = True + if isinstance(external, str): + name = external + + vol = Volume( + self.client, 'composetest', name, driver=driver, driver_opts=opts, + external=bool(external), custom_name=custom_name + ) + self.tmp_volumes.append(vol) + return vol + + def test_create_volume(self): + vol = self.create_volume('volume01') + vol.create() + info = self.get_volume_data(vol.full_name) + assert info['Name'].split('/')[-1] == vol.full_name + + def test_create_volume_custom_name(self): + vol = self.create_volume('volume01', custom_name=True) + assert vol.name == vol.full_name + vol.create() + info = self.get_volume_data(vol.full_name) + assert info['Name'].split('/')[-1] == vol.name + + def test_recreate_existing_volume(self): + vol = self.create_volume('volume01') + + vol.create() + info = self.get_volume_data(vol.full_name) + assert info['Name'].split('/')[-1] == vol.full_name + + vol.create() + info = self.get_volume_data(vol.full_name) + assert info['Name'].split('/')[-1] == vol.full_name + + @no_cluster('inspect volume by name defect on Swarm Classic') + def test_inspect_volume(self): + vol = self.create_volume('volume01') + vol.create() + info = vol.inspect() + assert info['Name'] == vol.full_name + + @no_cluster('remove volume by name defect on Swarm Classic') + def test_remove_volume(self): + vol = Volume(self.client, 'composetest', 'volume01') + vol.create() + vol.remove() + volumes = self.client.volumes()['Volumes'] + assert len([v for v in volumes if v['Name'] == vol.full_name]) == 0 + + @no_cluster('inspect volume by name defect on Swarm Classic') + def test_external_volume(self): + vol = self.create_volume('composetest_volume_ext', external=True) + assert vol.external is True + assert vol.full_name == vol.name + vol.create() + info = vol.inspect() + assert info['Name'] == vol.name + + @no_cluster('inspect volume by name defect on Swarm Classic') + def test_external_aliased_volume(self): + alias_name = 'composetest_alias01' + vol = self.create_volume('volume01', external=alias_name) + assert vol.external is True + assert vol.full_name == alias_name + vol.create() + info = vol.inspect() + assert info['Name'] == alias_name + + @no_cluster('inspect volume by name defect on Swarm Classic') + def test_exists(self): + vol = self.create_volume('volume01') + assert vol.exists() is False + vol.create() + assert vol.exists() is True + + @no_cluster('inspect volume by name defect on Swarm Classic') + def test_exists_external(self): + vol = self.create_volume('volume01', external=True) + assert vol.exists() is False + vol.create() + assert vol.exists() is True + + @no_cluster('inspect volume by name defect on Swarm Classic') + def test_exists_external_aliased(self): + vol = self.create_volume('volume01', external='composetest_alias01') + assert vol.exists() is False + vol.create() + assert vol.exists() is True + + @no_cluster('inspect volume by name defect on Swarm Classic') + def test_volume_default_labels(self): + vol = self.create_volume('volume01') + vol.create() + vol_data = vol.inspect() + labels = vol_data['Labels'] + assert labels[LABEL_VOLUME] == vol.name + assert labels[LABEL_PROJECT] == vol.project diff --git a/tests/unit/__init__.py b/tests/unit/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/tests/unit/cli/__init__.py b/tests/unit/cli/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/tests/unit/cli/command_test.py b/tests/unit/cli/command_test.py new file mode 100644 index 00000000000..60638864c37 --- /dev/null +++ b/tests/unit/cli/command_test.py @@ -0,0 +1,54 @@ +import os + +import pytest + +from compose.cli.command import get_config_path_from_options +from compose.config.environment import Environment +from compose.const import IS_WINDOWS_PLATFORM +from tests import mock + + +class TestGetConfigPathFromOptions: + + def test_path_from_options(self): + paths = ['one.yml', 'two.yml'] + opts = {'--file': paths} + environment = Environment.from_env_file('.') + assert get_config_path_from_options(opts, environment) == paths + + def test_single_path_from_env(self): + with mock.patch.dict(os.environ): + os.environ['COMPOSE_FILE'] = 'one.yml' + environment = Environment.from_env_file('.') + assert get_config_path_from_options({}, environment) == ['one.yml'] + + @pytest.mark.skipif(IS_WINDOWS_PLATFORM, reason='posix separator') + def test_multiple_path_from_env(self): + with mock.patch.dict(os.environ): + os.environ['COMPOSE_FILE'] = 'one.yml:two.yml' + environment = Environment.from_env_file('.') + assert get_config_path_from_options({}, environment) == ['one.yml', 'two.yml'] + + @pytest.mark.skipif(not IS_WINDOWS_PLATFORM, reason='windows separator') + def test_multiple_path_from_env_windows(self): + with mock.patch.dict(os.environ): + os.environ['COMPOSE_FILE'] = 'one.yml;two.yml' + environment = Environment.from_env_file('.') + assert get_config_path_from_options({}, environment) == ['one.yml', 'two.yml'] + + def test_multiple_path_from_env_custom_separator(self): + with mock.patch.dict(os.environ): + os.environ['COMPOSE_PATH_SEPARATOR'] = '^' + os.environ['COMPOSE_FILE'] = 'c:\\one.yml^.\\semi;colon.yml' + environment = Environment.from_env_file('.') + assert get_config_path_from_options({}, environment) == ['c:\\one.yml', '.\\semi;colon.yml'] + + def test_no_path(self): + environment = Environment.from_env_file('.') + assert not get_config_path_from_options({}, environment) + + def test_unicode_path_from_options(self): + paths = [b'\xe5\xb0\xb1\xe5\x90\x83\xe9\xa5\xad/docker-compose.yml'] + opts = {'--file': paths} + environment = Environment.from_env_file('.') + assert get_config_path_from_options(opts, environment) == ['就吃饭/docker-compose.yml'] diff --git a/tests/unit/cli/docker_client_test.py b/tests/unit/cli/docker_client_test.py new file mode 100644 index 00000000000..307e47f1bae --- /dev/null +++ b/tests/unit/cli/docker_client_test.py @@ -0,0 +1,249 @@ +import os +import platform +import ssl + +import docker +import pytest +from docker.constants import DEFAULT_DOCKER_API_VERSION + +import compose +from compose.cli import errors +from compose.cli.docker_client import docker_client +from compose.cli.docker_client import get_tls_version +from compose.cli.docker_client import tls_config_from_options +from compose.config.environment import Environment +from tests import mock +from tests import unittest + + +class DockerClientTestCase(unittest.TestCase): + + def test_docker_client_no_home(self): + with mock.patch.dict(os.environ): + try: + del os.environ['HOME'] + except KeyError: + pass + docker_client(os.environ, version=DEFAULT_DOCKER_API_VERSION) + + @mock.patch.dict(os.environ) + def test_docker_client_with_custom_timeout(self): + os.environ['COMPOSE_HTTP_TIMEOUT'] = '123' + client = docker_client(os.environ, version=DEFAULT_DOCKER_API_VERSION) + assert client.timeout == 123 + + @mock.patch.dict(os.environ) + def test_custom_timeout_error(self): + os.environ['COMPOSE_HTTP_TIMEOUT'] = '123' + client = docker_client(os.environ, version=DEFAULT_DOCKER_API_VERSION) + + with mock.patch('compose.cli.errors.log') as fake_log: + with pytest.raises(errors.ConnectionError): + with errors.handle_connection_errors(client): + raise errors.RequestsConnectionError( + errors.ReadTimeoutError(None, None, None)) + + assert fake_log.error.call_count == 1 + assert '123' in fake_log.error.call_args[0][0] + + with mock.patch('compose.cli.errors.log') as fake_log: + with pytest.raises(errors.ConnectionError): + with errors.handle_connection_errors(client): + raise errors.ReadTimeout() + + assert fake_log.error.call_count == 1 + assert '123' in fake_log.error.call_args[0][0] + + def test_user_agent(self): + client = docker_client(os.environ, version=DEFAULT_DOCKER_API_VERSION) + expected = "docker-compose/{} docker-py/{} {}/{}".format( + compose.__version__, + docker.__version__, + platform.system(), + platform.release() + ) + assert client.headers['User-Agent'] == expected + + +class TLSConfigTestCase(unittest.TestCase): + cert_path = 'tests/fixtures/tls/' + ca_cert = os.path.join(cert_path, 'ca.pem') + client_cert = os.path.join(cert_path, 'cert.pem') + key = os.path.join(cert_path, 'key.pem') + + def test_simple_tls(self): + options = {'--tls': True} + result = tls_config_from_options(options) + assert result is True + + def test_tls_ca_cert(self): + options = { + '--tlscacert': self.ca_cert, '--tlsverify': True + } + result = tls_config_from_options(options) + assert isinstance(result, docker.tls.TLSConfig) + assert result.ca_cert == options['--tlscacert'] + assert result.verify is True + + def test_tls_ca_cert_explicit(self): + options = { + '--tlscacert': self.ca_cert, '--tls': True, + '--tlsverify': True + } + result = tls_config_from_options(options) + assert isinstance(result, docker.tls.TLSConfig) + assert result.ca_cert == options['--tlscacert'] + assert result.verify is True + + def test_tls_client_cert(self): + options = { + '--tlscert': self.client_cert, '--tlskey': self.key + } + result = tls_config_from_options(options) + assert isinstance(result, docker.tls.TLSConfig) + assert result.cert == (options['--tlscert'], options['--tlskey']) + + def test_tls_client_cert_explicit(self): + options = { + '--tlscert': self.client_cert, '--tlskey': self.key, + '--tls': True + } + result = tls_config_from_options(options) + assert isinstance(result, docker.tls.TLSConfig) + assert result.cert == (options['--tlscert'], options['--tlskey']) + + def test_tls_client_and_ca(self): + options = { + '--tlscert': self.client_cert, '--tlskey': self.key, + '--tlsverify': True, '--tlscacert': self.ca_cert + } + result = tls_config_from_options(options) + assert isinstance(result, docker.tls.TLSConfig) + assert result.cert == (options['--tlscert'], options['--tlskey']) + assert result.ca_cert == options['--tlscacert'] + assert result.verify is True + + def test_tls_client_and_ca_explicit(self): + options = { + '--tlscert': self.client_cert, '--tlskey': self.key, + '--tlsverify': True, '--tlscacert': self.ca_cert, + '--tls': True + } + result = tls_config_from_options(options) + assert isinstance(result, docker.tls.TLSConfig) + assert result.cert == (options['--tlscert'], options['--tlskey']) + assert result.ca_cert == options['--tlscacert'] + assert result.verify is True + + def test_tls_client_missing_key(self): + options = {'--tlscert': self.client_cert} + with pytest.raises(docker.errors.TLSParameterError): + tls_config_from_options(options) + + options = {'--tlskey': self.key} + with pytest.raises(docker.errors.TLSParameterError): + tls_config_from_options(options) + + def test_assert_hostname_explicit_skip(self): + options = {'--tlscacert': self.ca_cert, '--skip-hostname-check': True} + result = tls_config_from_options(options) + assert isinstance(result, docker.tls.TLSConfig) + assert result.assert_hostname is False + + def test_tls_client_and_ca_quoted_paths(self): + options = { + '--tlscacert': '"{}"'.format(self.ca_cert), + '--tlscert': '"{}"'.format(self.client_cert), + '--tlskey': '"{}"'.format(self.key), + '--tlsverify': True + } + result = tls_config_from_options(options) + assert isinstance(result, docker.tls.TLSConfig) + assert result.cert == (self.client_cert, self.key) + assert result.ca_cert == self.ca_cert + assert result.verify is True + + def test_tls_simple_with_tls_version(self): + tls_version = 'TLSv1' + options = {'--tls': True} + environment = Environment({'COMPOSE_TLS_VERSION': tls_version}) + result = tls_config_from_options(options, environment) + assert isinstance(result, docker.tls.TLSConfig) + assert result.ssl_version == ssl.PROTOCOL_TLSv1 + + def test_tls_mixed_environment_and_flags(self): + options = {'--tls': True, '--tlsverify': False} + environment = Environment({'DOCKER_CERT_PATH': 'tests/fixtures/tls/'}) + result = tls_config_from_options(options, environment) + assert isinstance(result, docker.tls.TLSConfig) + assert result.cert == (self.client_cert, self.key) + assert result.ca_cert == self.ca_cert + assert result.verify is False + + def test_tls_flags_override_environment(self): + environment = Environment({ + 'DOCKER_CERT_PATH': '/completely/wrong/path', + 'DOCKER_TLS_VERIFY': 'false' + }) + options = { + '--tlscacert': '"{}"'.format(self.ca_cert), + '--tlscert': '"{}"'.format(self.client_cert), + '--tlskey': '"{}"'.format(self.key), + '--tlsverify': True + } + + result = tls_config_from_options(options, environment) + assert isinstance(result, docker.tls.TLSConfig) + assert result.cert == (self.client_cert, self.key) + assert result.ca_cert == self.ca_cert + assert result.verify is True + + def test_tls_verify_flag_no_override(self): + environment = Environment({ + 'DOCKER_TLS_VERIFY': 'true', + 'COMPOSE_TLS_VERSION': 'TLSv1', + 'DOCKER_CERT_PATH': self.cert_path + }) + options = {'--tls': True, '--tlsverify': False} + + result = tls_config_from_options(options, environment) + assert isinstance(result, docker.tls.TLSConfig) + assert result.ssl_version == ssl.PROTOCOL_TLSv1 + # verify is a special case - since `--tlsverify` = False means it + # wasn't used, we set it if either the environment or the flag is True + # see https://github.com/docker/compose/issues/5632 + assert result.verify is True + + def test_tls_verify_env_falsy_value(self): + environment = Environment({'DOCKER_TLS_VERIFY': '0'}) + options = {'--tls': True} + assert tls_config_from_options(options, environment) is True + + def test_tls_verify_default_cert_path(self): + environment = Environment({'DOCKER_TLS_VERIFY': '1'}) + options = {'--tls': True} + with mock.patch('compose.cli.docker_client.default_cert_path') as dcp: + dcp.return_value = 'tests/fixtures/tls/' + result = tls_config_from_options(options, environment) + assert isinstance(result, docker.tls.TLSConfig) + assert result.verify is True + assert result.ca_cert == self.ca_cert + assert result.cert == (self.client_cert, self.key) + + +class TestGetTlsVersion: + def test_get_tls_version_default(self): + environment = {} + assert get_tls_version(environment) is None + + @pytest.mark.skipif(not hasattr(ssl, 'PROTOCOL_TLSv1_2'), reason='TLS v1.2 unsupported') + def test_get_tls_version_upgrade(self): + environment = {'COMPOSE_TLS_VERSION': 'TLSv1_2'} + assert get_tls_version(environment) == ssl.PROTOCOL_TLSv1_2 + + def test_get_tls_version_unavailable(self): + environment = {'COMPOSE_TLS_VERSION': 'TLSv5_5'} + with mock.patch('compose.cli.docker_client.log') as mock_log: + tls_version = get_tls_version(environment) + mock_log.warning.assert_called_once_with(mock.ANY) + assert tls_version is None diff --git a/tests/unit/cli/errors_test.py b/tests/unit/cli/errors_test.py new file mode 100644 index 00000000000..3b70ffe7b37 --- /dev/null +++ b/tests/unit/cli/errors_test.py @@ -0,0 +1,95 @@ +import pytest +from docker.errors import APIError +from requests.exceptions import ConnectionError + +from compose.cli import errors +from compose.cli.errors import handle_connection_errors +from compose.const import IS_WINDOWS_PLATFORM +from tests import mock + + +@pytest.yield_fixture +def mock_logging(): + with mock.patch('compose.cli.errors.log', autospec=True) as mock_log: + yield mock_log + + +def patch_find_executable(side_effect): + return mock.patch( + 'compose.cli.errors.find_executable', + autospec=True, + side_effect=side_effect) + + +class TestHandleConnectionErrors: + + def test_generic_connection_error(self, mock_logging): + with pytest.raises(errors.ConnectionError): + with patch_find_executable(['/bin/docker', None]): + with handle_connection_errors(mock.Mock()): + raise ConnectionError() + + _, args, _ = mock_logging.error.mock_calls[0] + assert "Couldn't connect to Docker daemon" in args[0] + + def test_api_error_version_mismatch(self, mock_logging): + with pytest.raises(errors.ConnectionError): + with handle_connection_errors(mock.Mock(api_version='1.38')): + raise APIError(None, None, b"client is newer than server") + + _, args, _ = mock_logging.error.mock_calls[0] + assert "Docker Engine of version 18.06.0 or greater" in args[0] + + def test_api_error_version_mismatch_unicode_explanation(self, mock_logging): + with pytest.raises(errors.ConnectionError): + with handle_connection_errors(mock.Mock(api_version='1.38')): + raise APIError(None, None, "client is newer than server") + + _, args, _ = mock_logging.error.mock_calls[0] + assert "Docker Engine of version 18.06.0 or greater" in args[0] + + def test_api_error_version_other(self, mock_logging): + msg = b"Something broke!" + with pytest.raises(errors.ConnectionError): + with handle_connection_errors(mock.Mock(api_version='1.22')): + raise APIError(None, None, msg) + + mock_logging.error.assert_called_once_with(msg.decode('utf-8')) + + def test_api_error_version_other_unicode_explanation(self, mock_logging): + msg = "Something broke!" + with pytest.raises(errors.ConnectionError): + with handle_connection_errors(mock.Mock(api_version='1.22')): + raise APIError(None, None, msg) + + mock_logging.error.assert_called_once_with(msg) + + @pytest.mark.skipif(not IS_WINDOWS_PLATFORM, reason='Needs pywin32') + def test_windows_pipe_error_no_data(self, mock_logging): + import pywintypes + with pytest.raises(errors.ConnectionError): + with handle_connection_errors(mock.Mock(api_version='1.22')): + raise pywintypes.error(232, 'WriteFile', 'The pipe is being closed.') + + _, args, _ = mock_logging.error.mock_calls[0] + assert "The current Compose file version is not compatible with your engine version." in args[0] + + @pytest.mark.skipif(not IS_WINDOWS_PLATFORM, reason='Needs pywin32') + def test_windows_pipe_error_misc(self, mock_logging): + import pywintypes + with pytest.raises(errors.ConnectionError): + with handle_connection_errors(mock.Mock(api_version='1.22')): + raise pywintypes.error(231, 'WriteFile', 'The pipe is busy.') + + _, args, _ = mock_logging.error.mock_calls[0] + assert "Windows named pipe error: The pipe is busy. (code: 231)" == args[0] + + @pytest.mark.skipif(not IS_WINDOWS_PLATFORM, reason='Needs pywin32') + def test_windows_pipe_error_encoding_issue(self, mock_logging): + import pywintypes + with pytest.raises(errors.ConnectionError): + with handle_connection_errors(mock.Mock(api_version='1.22')): + raise pywintypes.error(9999, 'WriteFile', 'I use weird characters \xe9') + + _, args, _ = mock_logging.error.mock_calls[0] + assert 'Windows named pipe error: I use weird characters \xe9 (code: 9999)' == args[0] diff --git a/tests/unit/cli/formatter_test.py b/tests/unit/cli/formatter_test.py new file mode 100644 index 00000000000..08752a6226b --- /dev/null +++ b/tests/unit/cli/formatter_test.py @@ -0,0 +1,49 @@ +import logging + +from compose.cli import colors +from compose.cli.formatter import ConsoleWarningFormatter +from tests import unittest + + +MESSAGE = 'this is the message' + + +def make_log_record(level, message=None): + return logging.LogRecord('name', level, 'pathame', 0, message or MESSAGE, (), None) + + +class ConsoleWarningFormatterTestCase(unittest.TestCase): + + def setUp(self): + self.formatter = ConsoleWarningFormatter() + + def test_format_warn(self): + output = self.formatter.format(make_log_record(logging.WARN)) + expected = colors.yellow('WARNING') + ': ' + assert output == expected + MESSAGE + + def test_format_error(self): + output = self.formatter.format(make_log_record(logging.ERROR)) + expected = colors.red('ERROR') + ': ' + assert output == expected + MESSAGE + + def test_format_info(self): + output = self.formatter.format(make_log_record(logging.INFO)) + assert output == MESSAGE + + def test_format_unicode_info(self): + message = b'\xec\xa0\x95\xec\x88\x98\xec\xa0\x95' + output = self.formatter.format(make_log_record(logging.INFO, message)) + assert output == message.decode('utf-8') + + def test_format_unicode_warn(self): + message = b'\xec\xa0\x95\xec\x88\x98\xec\xa0\x95' + output = self.formatter.format(make_log_record(logging.WARN, message)) + expected = colors.yellow('WARNING') + ': ' + assert output == '{}{}'.format(expected, message.decode('utf-8')) + + def test_format_unicode_error(self): + message = b'\xec\xa0\x95\xec\x88\x98\xec\xa0\x95' + output = self.formatter.format(make_log_record(logging.ERROR, message)) + expected = colors.red('ERROR') + ': ' + assert output == '{}{}'.format(expected, message.decode('utf-8')) diff --git a/tests/unit/cli/log_printer_test.py b/tests/unit/cli/log_printer_test.py new file mode 100644 index 00000000000..aeeed31f32f --- /dev/null +++ b/tests/unit/cli/log_printer_test.py @@ -0,0 +1,209 @@ +import itertools +from io import StringIO +from queue import Queue + +import pytest +import requests +from docker.errors import APIError + +from compose.cli.log_printer import build_log_generator +from compose.cli.log_printer import build_log_presenters +from compose.cli.log_printer import build_no_log_generator +from compose.cli.log_printer import consume_queue +from compose.cli.log_printer import QueueItem +from compose.cli.log_printer import wait_on_exit +from compose.cli.log_printer import watch_events +from compose.container import Container +from tests import mock + + +@pytest.fixture +def output_stream(): + output = StringIO() + output.flush = mock.Mock() + return output + + +@pytest.fixture +def mock_container(): + return mock.Mock(spec=Container, name_without_project='web_1') + + +class TestLogPresenter: + + def test_monochrome(self, mock_container): + presenters = build_log_presenters(['foo', 'bar'], True) + presenter = next(presenters) + actual = presenter.present(mock_container, "this line") + assert actual == "web_1 | this line" + + def test_polychrome(self, mock_container): + presenters = build_log_presenters(['foo', 'bar'], False) + presenter = next(presenters) + actual = presenter.present(mock_container, "this line") + assert '\033[' in actual + + +def test_wait_on_exit(): + exit_status = 3 + mock_container = mock.Mock( + spec=Container, + name='cname', + wait=mock.Mock(return_value=exit_status)) + + expected = '{} exited with code {}\n'.format(mock_container.name, exit_status) + assert expected == wait_on_exit(mock_container) + + +def test_wait_on_exit_raises(): + status_code = 500 + + def mock_wait(): + resp = requests.Response() + resp.status_code = status_code + raise APIError('Bad server', resp) + + mock_container = mock.Mock( + spec=Container, + name='cname', + wait=mock_wait + ) + + expected = 'Unexpected API error for {} (HTTP code {})\n'.format( + mock_container.name, status_code, + ) + assert expected in wait_on_exit(mock_container) + + +def test_build_no_log_generator(mock_container): + mock_container.has_api_logs = False + mock_container.log_driver = 'none' + output, = build_no_log_generator(mock_container, None) + assert "WARNING: no logs are available with the 'none' log driver\n" in output + assert "exited with code" not in output + + +class TestBuildLogGenerator: + + def test_no_log_stream(self, mock_container): + mock_container.log_stream = None + mock_container.logs.return_value = iter([b"hello\nworld"]) + log_args = {'follow': True} + + generator = build_log_generator(mock_container, log_args) + assert next(generator) == "hello\n" + assert next(generator) == "world" + mock_container.logs.assert_called_once_with( + stdout=True, + stderr=True, + stream=True, + **log_args) + + def test_with_log_stream(self, mock_container): + mock_container.log_stream = iter([b"hello\nworld"]) + log_args = {'follow': True} + + generator = build_log_generator(mock_container, log_args) + assert next(generator) == "hello\n" + assert next(generator) == "world" + + def test_unicode(self, output_stream): + glyph = '\u2022\n' + mock_container.log_stream = iter([glyph.encode('utf-8')]) + + generator = build_log_generator(mock_container, {}) + assert next(generator) == glyph + + +@pytest.fixture +def thread_map(): + return {'cid': mock.Mock()} + + +@pytest.fixture +def mock_presenters(): + return itertools.cycle([mock.Mock()]) + + +class TestWatchEvents: + + def test_stop_event(self, thread_map, mock_presenters): + event_stream = [{'action': 'stop', 'id': 'cid'}] + watch_events(thread_map, event_stream, mock_presenters, ()) + assert not thread_map + + def test_start_event(self, thread_map, mock_presenters): + container_id = 'abcd' + event = {'action': 'start', 'id': container_id, 'container': mock.Mock()} + event_stream = [event] + thread_args = 'foo', 'bar' + + with mock.patch( + 'compose.cli.log_printer.build_thread', + autospec=True + ) as mock_build_thread: + watch_events(thread_map, event_stream, mock_presenters, thread_args) + mock_build_thread.assert_called_once_with( + event['container'], + next(mock_presenters), + *thread_args) + assert container_id in thread_map + + def test_container_attach_event(self, thread_map, mock_presenters): + container_id = 'abcd' + mock_container = mock.Mock(is_restarting=False) + mock_container.attach_log_stream.side_effect = APIError("race condition") + event_die = {'action': 'die', 'id': container_id} + event_start = {'action': 'start', 'id': container_id, 'container': mock_container} + event_stream = [event_die, event_start] + thread_args = 'foo', 'bar' + watch_events(thread_map, event_stream, mock_presenters, thread_args) + assert mock_container.attach_log_stream.called + + def test_other_event(self, thread_map, mock_presenters): + container_id = 'abcd' + event_stream = [{'action': 'create', 'id': container_id}] + watch_events(thread_map, event_stream, mock_presenters, ()) + assert container_id not in thread_map + + +class TestConsumeQueue: + + def test_item_is_an_exception(self): + + class Problem(Exception): + pass + + queue = Queue() + error = Problem('oops') + for item in QueueItem.new('a'), QueueItem.new('b'), QueueItem.exception(error): + queue.put(item) + + generator = consume_queue(queue, False) + assert next(generator) == 'a' + assert next(generator) == 'b' + with pytest.raises(Problem): + next(generator) + + def test_item_is_stop_without_cascade_stop(self): + queue = Queue() + for item in QueueItem.stop(), QueueItem.new('a'), QueueItem.new('b'): + queue.put(item) + + generator = consume_queue(queue, False) + assert next(generator) == 'a' + assert next(generator) == 'b' + + def test_item_is_stop_with_cascade_stop(self): + """Return the name of the container that caused the cascade_stop""" + queue = Queue() + for item in QueueItem.stop('foobar-1'), QueueItem.new('a'), QueueItem.new('b'): + queue.put(item) + + generator = consume_queue(queue, True) + assert next(generator) == 'foobar-1' + + def test_item_is_none_when_timeout_is_hit(self): + queue = Queue() + generator = consume_queue(queue, False) + assert next(generator) is None diff --git a/tests/unit/cli/main_test.py b/tests/unit/cli/main_test.py new file mode 100644 index 00000000000..d75b6bd4c12 --- /dev/null +++ b/tests/unit/cli/main_test.py @@ -0,0 +1,250 @@ +import logging + +import docker +import pytest + +from compose import container +from compose.cli.errors import UserError +from compose.cli.formatter import ConsoleWarningFormatter +from compose.cli.main import build_one_off_container_options +from compose.cli.main import call_docker +from compose.cli.main import convergence_strategy_from_opts +from compose.cli.main import filter_attached_containers +from compose.cli.main import get_docker_start_call +from compose.cli.main import setup_console_handler +from compose.cli.main import warn_for_swarm_mode +from compose.service import ConvergenceStrategy +from tests import mock + + +def mock_container(service, number): + return mock.create_autospec( + container.Container, + service=service, + number=number, + name_without_project='{}_{}'.format(service, number)) + + +@pytest.fixture +def logging_handler(): + stream = mock.Mock() + stream.isatty.return_value = True + return logging.StreamHandler(stream=stream) + + +class TestCLIMainTestCase: + + def test_filter_attached_containers(self): + containers = [ + mock_container('web', 1), + mock_container('web', 2), + mock_container('db', 1), + mock_container('other', 1), + mock_container('another', 1), + ] + service_names = ['web', 'db'] + actual = filter_attached_containers(containers, service_names) + assert actual == containers[:3] + + def test_filter_attached_containers_with_dependencies(self): + containers = [ + mock_container('web', 1), + mock_container('web', 2), + mock_container('db', 1), + mock_container('other', 1), + mock_container('another', 1), + ] + service_names = ['web', 'db'] + actual = filter_attached_containers(containers, service_names, attach_dependencies=True) + assert actual == containers + + def test_filter_attached_containers_all(self): + containers = [ + mock_container('web', 1), + mock_container('db', 1), + mock_container('other', 1), + ] + service_names = [] + actual = filter_attached_containers(containers, service_names) + assert actual == containers + + def test_warning_in_swarm_mode(self): + mock_client = mock.create_autospec(docker.APIClient) + mock_client.info.return_value = {'Swarm': {'LocalNodeState': 'active'}} + + with mock.patch('compose.cli.main.log') as fake_log: + warn_for_swarm_mode(mock_client) + assert fake_log.warning.call_count == 1 + + def test_build_one_off_container_options(self): + command = 'build myservice' + detach = False + options = { + '-e': ['MYVAR=MYVALUE'], + '-T': True, + '--label': ['MYLABEL'], + '--entrypoint': 'bash', + '--user': 'MYUSER', + '--service-ports': [], + '--publish': '', + '--name': 'MYNAME', + '--workdir': '.', + '--volume': [], + 'stdin_open': False, + } + + expected_container_options = { + 'command': command, + 'tty': False, + 'stdin_open': False, + 'detach': detach, + 'entrypoint': 'bash', + 'environment': {'MYVAR': 'MYVALUE'}, + 'labels': {'MYLABEL': ''}, + 'name': 'MYNAME', + 'ports': [], + 'restart': None, + 'user': 'MYUSER', + 'working_dir': '.', + } + + container_options = build_one_off_container_options(options, detach, command) + assert container_options == expected_container_options + + def test_get_docker_start_call(self): + container_id = 'my_container_id' + + mock_container_options = {'detach': False, 'stdin_open': True} + expected_docker_start_call = ['start', '--attach', '--interactive', container_id] + docker_start_call = get_docker_start_call(mock_container_options, container_id) + assert expected_docker_start_call == docker_start_call + + mock_container_options = {'detach': False, 'stdin_open': False} + expected_docker_start_call = ['start', '--attach', container_id] + docker_start_call = get_docker_start_call(mock_container_options, container_id) + assert expected_docker_start_call == docker_start_call + + mock_container_options = {'detach': True, 'stdin_open': True} + expected_docker_start_call = ['start', '--interactive', container_id] + docker_start_call = get_docker_start_call(mock_container_options, container_id) + assert expected_docker_start_call == docker_start_call + + mock_container_options = {'detach': True, 'stdin_open': False} + expected_docker_start_call = ['start', container_id] + docker_start_call = get_docker_start_call(mock_container_options, container_id) + assert expected_docker_start_call == docker_start_call + + +class TestSetupConsoleHandlerTestCase: + + def test_with_tty_verbose(self, logging_handler): + setup_console_handler(logging_handler, True) + assert type(logging_handler.formatter) == ConsoleWarningFormatter + assert '%(name)s' in logging_handler.formatter._fmt + assert '%(funcName)s' in logging_handler.formatter._fmt + + def test_with_tty_not_verbose(self, logging_handler): + setup_console_handler(logging_handler, False) + assert type(logging_handler.formatter) == ConsoleWarningFormatter + assert '%(name)s' not in logging_handler.formatter._fmt + assert '%(funcName)s' not in logging_handler.formatter._fmt + + def test_with_not_a_tty(self, logging_handler): + logging_handler.stream.isatty.return_value = False + setup_console_handler(logging_handler, False) + assert type(logging_handler.formatter) == logging.Formatter + + +class TestConvergeStrategyFromOptsTestCase: + + def test_invalid_opts(self): + options = {'--force-recreate': True, '--no-recreate': True} + with pytest.raises(UserError): + convergence_strategy_from_opts(options) + + def test_always(self): + options = {'--force-recreate': True, '--no-recreate': False} + assert ( + convergence_strategy_from_opts(options) == + ConvergenceStrategy.always + ) + + def test_never(self): + options = {'--force-recreate': False, '--no-recreate': True} + assert ( + convergence_strategy_from_opts(options) == + ConvergenceStrategy.never + ) + + def test_changed(self): + options = {'--force-recreate': False, '--no-recreate': False} + assert ( + convergence_strategy_from_opts(options) == + ConvergenceStrategy.changed + ) + + +def mock_find_executable(exe): + return exe + + +@mock.patch('compose.cli.main.find_executable', mock_find_executable) +class TestCallDocker: + def test_simple_no_options(self): + with mock.patch('subprocess.call') as fake_call: + call_docker(['ps'], {}, {}) + + assert fake_call.call_args[0][0] == ['docker', 'ps'] + + def test_simple_tls_option(self): + with mock.patch('subprocess.call') as fake_call: + call_docker(['ps'], {'--tls': True}, {}) + + assert fake_call.call_args[0][0] == ['docker', '--tls', 'ps'] + + def test_advanced_tls_options(self): + with mock.patch('subprocess.call') as fake_call: + call_docker(['ps'], { + '--tls': True, + '--tlscacert': './ca.pem', + '--tlscert': './cert.pem', + '--tlskey': './key.pem', + }, {}) + + assert fake_call.call_args[0][0] == [ + 'docker', '--tls', '--tlscacert', './ca.pem', '--tlscert', + './cert.pem', '--tlskey', './key.pem', 'ps' + ] + + def test_with_host_option(self): + with mock.patch('subprocess.call') as fake_call: + call_docker(['ps'], {'--host': 'tcp://mydocker.net:2333'}, {}) + + assert fake_call.call_args[0][0] == [ + 'docker', '--host', 'tcp://mydocker.net:2333', 'ps' + ] + + def test_with_http_host(self): + with mock.patch('subprocess.call') as fake_call: + call_docker(['ps'], {'--host': 'http://mydocker.net:2333'}, {}) + + assert fake_call.call_args[0][0] == [ + 'docker', '--host', 'tcp://mydocker.net:2333', 'ps', + ] + + def test_with_host_option_shorthand_equal(self): + with mock.patch('subprocess.call') as fake_call: + call_docker(['ps'], {'--host': '=tcp://mydocker.net:2333'}, {}) + + assert fake_call.call_args[0][0] == [ + 'docker', '--host', 'tcp://mydocker.net:2333', 'ps' + ] + + def test_with_env(self): + with mock.patch('subprocess.call') as fake_call: + call_docker(['ps'], {}, {'DOCKER_HOST': 'tcp://mydocker.net:2333'}) + + assert fake_call.call_args[0][0] == [ + 'docker', 'ps' + ] + assert fake_call.call_args[1]['env'] == {'DOCKER_HOST': 'tcp://mydocker.net:2333'} diff --git a/tests/unit/cli/utils_test.py b/tests/unit/cli/utils_test.py new file mode 100644 index 00000000000..d67c8ba8aff --- /dev/null +++ b/tests/unit/cli/utils_test.py @@ -0,0 +1,45 @@ +import unittest + +from compose.cli.utils import human_readable_file_size +from compose.utils import unquote_path + + +class UnquotePathTest(unittest.TestCase): + def test_no_quotes(self): + assert unquote_path('hello') == 'hello' + + def test_simple_quotes(self): + assert unquote_path('"hello"') == 'hello' + + def test_uneven_quotes(self): + assert unquote_path('"hello') == '"hello' + assert unquote_path('hello"') == 'hello"' + + def test_nested_quotes(self): + assert unquote_path('""hello""') == '"hello"' + assert unquote_path('"hel"lo"') == 'hel"lo' + assert unquote_path('"hello""') == 'hello"' + + +class HumanReadableFileSizeTest(unittest.TestCase): + def test_100b(self): + assert human_readable_file_size(100) == '100 B' + + def test_1kb(self): + assert human_readable_file_size(1000) == '1 kB' + assert human_readable_file_size(1024) == '1.024 kB' + + def test_1023b(self): + assert human_readable_file_size(1023) == '1.023 kB' + + def test_999b(self): + assert human_readable_file_size(999) == '999 B' + + def test_units(self): + assert human_readable_file_size((10 ** 3) ** 0) == '1 B' + assert human_readable_file_size((10 ** 3) ** 1) == '1 kB' + assert human_readable_file_size((10 ** 3) ** 2) == '1 MB' + assert human_readable_file_size((10 ** 3) ** 3) == '1 GB' + assert human_readable_file_size((10 ** 3) ** 4) == '1 TB' + assert human_readable_file_size((10 ** 3) ** 5) == '1 PB' + assert human_readable_file_size((10 ** 3) ** 6) == '1 EB' diff --git a/tests/unit/cli/verbose_proxy_test.py b/tests/unit/cli/verbose_proxy_test.py new file mode 100644 index 00000000000..0da662fd0c0 --- /dev/null +++ b/tests/unit/cli/verbose_proxy_test.py @@ -0,0 +1,28 @@ +from compose.cli import verbose_proxy +from tests import unittest + + +class VerboseProxyTestCase(unittest.TestCase): + + def test_format_call(self): + prefix = '' + expected = "(%(p)s'arg1', True, key=%(p)s'value')" % dict(p=prefix) + actual = verbose_proxy.format_call( + ("arg1", True), + {'key': 'value'}) + + assert expected == actual + + def test_format_return_sequence(self): + expected = "(list with 10 items)" + actual = verbose_proxy.format_return(list(range(10)), 2) + assert expected == actual + + def test_format_return(self): + expected = repr({'Id': 'ok'}) + actual = verbose_proxy.format_return({'Id': 'ok'}, 2) + assert expected == actual + + def test_format_return_no_result(self): + actual = verbose_proxy.format_return(None, 2) + assert actual is None diff --git a/tests/unit/cli_test.py b/tests/unit/cli_test.py new file mode 100644 index 00000000000..fa6e76747f4 --- /dev/null +++ b/tests/unit/cli_test.py @@ -0,0 +1,301 @@ +import os +import shutil +import tempfile +from io import StringIO + +import docker +import py +import pytest +from docker.constants import DEFAULT_DOCKER_API_VERSION + +from .. import mock +from .. import unittest +from ..helpers import build_config +from compose.cli.command import get_project +from compose.cli.command import get_project_name +from compose.cli.docopt_command import NoSuchCommand +from compose.cli.errors import UserError +from compose.cli.main import TopLevelCommand +from compose.config.environment import Environment +from compose.const import IS_WINDOWS_PLATFORM +from compose.const import LABEL_SERVICE +from compose.container import Container +from compose.project import Project + + +class CLITestCase(unittest.TestCase): + + def test_default_project_name(self): + test_dir = py._path.local.LocalPath('tests/fixtures/simple-composefile') + with test_dir.as_cwd(): + project_name = get_project_name('.') + assert 'simple-composefile' == project_name + + def test_project_name_with_explicit_base_dir(self): + base_dir = 'tests/fixtures/simple-composefile' + project_name = get_project_name(base_dir) + assert 'simple-composefile' == project_name + + def test_project_name_with_explicit_uppercase_base_dir(self): + base_dir = 'tests/fixtures/UpperCaseDir' + project_name = get_project_name(base_dir) + assert 'uppercasedir' == project_name + + def test_project_name_with_explicit_project_name(self): + name = 'explicit-project-name' + project_name = get_project_name(None, project_name=name) + assert 'explicit-project-name' == project_name + + @mock.patch.dict(os.environ) + def test_project_name_from_environment_new_var(self): + name = 'namefromenv' + os.environ['COMPOSE_PROJECT_NAME'] = name + project_name = get_project_name(None) + assert project_name == name + + def test_project_name_with_empty_environment_var(self): + base_dir = 'tests/fixtures/simple-composefile' + with mock.patch.dict(os.environ): + os.environ['COMPOSE_PROJECT_NAME'] = '' + project_name = get_project_name(base_dir) + assert 'simple-composefile' == project_name + + @mock.patch.dict(os.environ) + def test_project_name_with_environment_file(self): + base_dir = tempfile.mkdtemp() + try: + name = 'namefromenvfile' + with open(os.path.join(base_dir, '.env'), 'w') as f: + f.write('COMPOSE_PROJECT_NAME={}'.format(name)) + project_name = get_project_name(base_dir) + assert project_name == name + + # Environment has priority over .env file + os.environ['COMPOSE_PROJECT_NAME'] = 'namefromenv' + assert get_project_name(base_dir) == os.environ['COMPOSE_PROJECT_NAME'] + finally: + shutil.rmtree(base_dir) + + def test_get_project(self): + base_dir = 'tests/fixtures/longer-filename-composefile' + env = Environment.from_env_file(base_dir) + env['COMPOSE_API_VERSION'] = DEFAULT_DOCKER_API_VERSION + project = get_project(base_dir, environment=env) + assert project.name == 'longer-filename-composefile' + assert project.client + assert project.services + + def test_command_help(self): + with mock.patch('sys.stdout', new=StringIO()) as fake_stdout: + TopLevelCommand.help({'COMMAND': 'up'}) + + assert "Usage: up" in fake_stdout.getvalue() + + def test_command_help_nonexistent(self): + with pytest.raises(NoSuchCommand): + TopLevelCommand.help({'COMMAND': 'nonexistent'}) + + @pytest.mark.xfail(IS_WINDOWS_PLATFORM, reason="requires dockerpty") + @mock.patch('compose.cli.main.RunOperation', autospec=True) + @mock.patch('compose.cli.main.PseudoTerminal', autospec=True) + @mock.patch('compose.service.Container.create') + @mock.patch.dict(os.environ) + def test_run_interactive_passes_logs_false( + self, + mock_container_create, + mock_pseudo_terminal, + mock_run_operation, + ): + os.environ['COMPOSE_INTERACTIVE_NO_CLI'] = 'true' + mock_client = mock.create_autospec(docker.APIClient) + mock_client.api_version = DEFAULT_DOCKER_API_VERSION + mock_client._general_configs = {} + mock_container_create.return_value = Container(mock_client, { + 'Id': '37b35e0ba80d91009d37e16f249b32b84f72bda269985578ed6c75a0a13fcaa8', + 'Config': { + 'Labels': { + LABEL_SERVICE: 'service', + } + }, + }, has_been_inspected=True) + project = Project.from_config( + name='composetest', + client=mock_client, + config_data=build_config({ + 'service': {'image': 'busybox'} + }), + ) + command = TopLevelCommand(project) + + with pytest.raises(SystemExit): + command.run({ + 'SERVICE': 'service', + 'COMMAND': None, + '-e': [], + '--label': [], + '--user': None, + '--no-deps': None, + '--detach': False, + '-T': None, + '--entrypoint': None, + '--service-ports': None, + '--use-aliases': None, + '--publish': [], + '--volume': [], + '--rm': None, + '--name': None, + '--workdir': None, + }) + + _, _, call_kwargs = mock_run_operation.mock_calls[0] + assert call_kwargs['logs'] is False + + @mock.patch('compose.service.Container.create') + def test_run_service_with_restart_always(self, mock_container_create): + mock_client = mock.create_autospec(docker.APIClient) + mock_client.api_version = DEFAULT_DOCKER_API_VERSION + mock_client._general_configs = {} + mock_container_create.return_value = Container(mock_client, { + 'Id': '37b35e0ba80d91009d37e16f249b32b84f72bda269985578ed6c75a0a13fcaa8', + 'Name': 'composetest_service_37b35', + 'Config': { + 'Labels': { + LABEL_SERVICE: 'service', + } + }, + }, has_been_inspected=True) + + project = Project.from_config( + name='composetest', + client=mock_client, + config_data=build_config({ + 'service': { + 'image': 'busybox', + 'restart': 'always', + } + }), + ) + + command = TopLevelCommand(project) + command.run({ + 'SERVICE': 'service', + 'COMMAND': None, + '-e': [], + '--label': [], + '--user': None, + '--no-deps': None, + '--detach': True, + '-T': None, + '--entrypoint': None, + '--service-ports': None, + '--use-aliases': None, + '--publish': [], + '--volume': [], + '--rm': None, + '--name': None, + '--workdir': None, + }) + + # NOTE: The "run" command is supposed to be a one-off tool; therefore restart policy "no" + # (the default) is enforced despite explicit wish for "always" in the project + # configuration file + assert not mock_client.create_host_config.call_args[1].get('restart_policy') + + command = TopLevelCommand(project) + command.run({ + 'SERVICE': 'service', + 'COMMAND': None, + '-e': [], + '--label': [], + '--user': None, + '--no-deps': None, + '--detach': True, + '-T': None, + '--entrypoint': None, + '--service-ports': None, + '--use-aliases': None, + '--publish': [], + '--volume': [], + '--rm': True, + '--name': None, + '--workdir': None, + }) + + assert not mock_client.create_host_config.call_args[1].get('restart_policy') + + @mock.patch('compose.project.Project.up') + @mock.patch.dict(os.environ) + def test_run_up_with_docker_cli_build(self, mock_project_up): + os.environ['COMPOSE_DOCKER_CLI_BUILD'] = '1' + mock_client = mock.create_autospec(docker.APIClient) + mock_client.api_version = DEFAULT_DOCKER_API_VERSION + mock_client._general_configs = {} + container = Container(mock_client, { + 'Id': '37b35e0ba80d91009d37e16f249b32b84f72bda269985578ed6c75a0a13fcaa8', + 'Name': 'composetest_service_37b35', + 'Config': { + 'Labels': { + LABEL_SERVICE: 'service', + } + }, + }, has_been_inspected=True) + mock_project_up.return_value = [container] + + project = Project.from_config( + name='composetest', + config_data=build_config({ + 'service': {'image': 'busybox'} + }), + client=mock_client, + ) + + command = TopLevelCommand(project) + command.run({ + 'SERVICE': 'service', + 'COMMAND': None, + '-e': [], + '--label': [], + '--user': None, + '--no-deps': None, + '--detach': True, + '-T': None, + '--entrypoint': None, + '--service-ports': None, + '--use-aliases': None, + '--publish': [], + '--volume': [], + '--rm': None, + '--name': None, + '--workdir': None, + }) + + _, _, call_kwargs = mock_project_up.mock_calls[0] + assert call_kwargs.get('cli') + + def test_command_manual_and_service_ports_together(self): + project = Project.from_config( + name='composetest', + client=None, + config_data=build_config({ + 'service': {'image': 'busybox'}, + }), + ) + command = TopLevelCommand(project) + + with pytest.raises(UserError): + command.run({ + 'SERVICE': 'service', + 'COMMAND': None, + '-e': [], + '--label': [], + '--user': None, + '--no-deps': None, + '--detach': True, + '-T': None, + '--entrypoint': None, + '--service-ports': True, + '--use-aliases': None, + '--publish': ['80:80'], + '--rm': None, + '--name': None, + }) diff --git a/tests/unit/config/__init__.py b/tests/unit/config/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py new file mode 100644 index 00000000000..ffc16e0853d --- /dev/null +++ b/tests/unit/config/config_test.py @@ -0,0 +1,5539 @@ +import codecs +import os +import shutil +import tempfile +from operator import itemgetter +from random import shuffle + +import pytest +import yaml +from ddt import data +from ddt import ddt + +from ...helpers import build_config_details +from ...helpers import BUSYBOX_IMAGE_WITH_TAG +from ...helpers import cd +from compose.config import config +from compose.config import types +from compose.config.config import ConfigFile +from compose.config.config import resolve_build_args +from compose.config.config import resolve_environment +from compose.config.environment import Environment +from compose.config.errors import ConfigurationError +from compose.config.errors import VERSION_EXPLANATION +from compose.config.serialize import denormalize_service_dict +from compose.config.serialize import serialize_config +from compose.config.serialize import serialize_ns_time_value +from compose.config.types import VolumeSpec +from compose.const import COMPOSE_SPEC as VERSION +from compose.const import COMPOSEFILE_V1 as V1 +from compose.const import IS_WINDOWS_PLATFORM +from tests import mock +from tests import unittest + +DEFAULT_VERSION = VERSION + + +def make_service_dict(name, service_dict, working_dir='.', filename=None): + """Test helper function to construct a ServiceExtendsResolver + """ + resolver = config.ServiceExtendsResolver( + config.ServiceConfig( + working_dir=working_dir, + filename=filename, + name=name, + config=service_dict), + config.ConfigFile(filename=filename, config={}), + environment=Environment.from_env_file(working_dir) + ) + return config.process_service(resolver.run()) + + +def service_sort(services): + return sorted(services, key=itemgetter('name')) + + +def secret_sort(secrets): + return sorted(secrets, key=itemgetter('source')) + + +@ddt +class ConfigTest(unittest.TestCase): + + def test_load(self): + service_dicts = config.load( + build_config_details( + { + 'services': { + 'foo': {'image': 'busybox'}, + 'bar': {'image': 'busybox', 'environment': ['FOO=1']}, + } + }, + 'tests/fixtures/extends', + 'common.yml' + ) + ).services + + assert service_sort(service_dicts) == service_sort([ + { + 'name': 'bar', + 'image': 'busybox', + 'environment': {'FOO': '1'}, + }, + { + 'name': 'foo', + 'image': 'busybox', + } + ]) + + def test_load_v2(self): + config_data = config.load( + build_config_details({ + 'version': '2', + 'services': { + 'foo': {'image': 'busybox'}, + 'bar': {'image': 'busybox', 'environment': ['FOO=1']}, + }, + 'volumes': { + 'hello': { + 'driver': 'default', + 'driver_opts': {'beep': 'boop'} + } + }, + 'networks': { + 'default': { + 'driver': 'bridge', + 'driver_opts': {'beep': 'boop'} + }, + 'with_ipam': { + 'ipam': { + 'driver': 'default', + 'config': [ + {'subnet': '172.28.0.0/16'} + ] + } + }, + 'internal': { + 'driver': 'bridge', + 'internal': True + } + } + }, 'working_dir', 'filename.yml') + ) + service_dicts = config_data.services + volume_dict = config_data.volumes + networks_dict = config_data.networks + assert service_sort(service_dicts) == service_sort([ + { + 'name': 'bar', + 'image': 'busybox', + 'environment': {'FOO': '1'}, + }, + { + 'name': 'foo', + 'image': 'busybox', + } + ]) + assert volume_dict == { + 'hello': { + 'driver': 'default', + 'driver_opts': {'beep': 'boop'} + } + } + assert networks_dict == { + 'default': { + 'driver': 'bridge', + 'driver_opts': {'beep': 'boop'} + }, + 'with_ipam': { + 'ipam': { + 'driver': 'default', + 'config': [ + {'subnet': '172.28.0.0/16'} + ] + } + }, + 'internal': { + 'driver': 'bridge', + 'internal': True + } + } + + def test_valid_versions(self): + cfg = config.load( + build_config_details({ + 'services': { + 'foo': {'image': 'busybox'}, + 'bar': {'image': 'busybox', 'environment': ['FOO=1']}, + } + }) + ) + assert cfg.config_version == VERSION + assert cfg.version == VERSION + + for version in ['2', '2.0', '2.1', '2.2', '2.3', + '3', '3.0', '3.1', '3.2', '3.3', '3.4', '3.5', '3.6', '3.7', '3.8']: + cfg = config.load(build_config_details({'version': version})) + assert cfg.config_version == version + assert cfg.version == VERSION + + def test_v1_file_version(self): + cfg = config.load(build_config_details({'web': {'image': 'busybox'}})) + assert cfg.version == V1 + assert list(s['name'] for s in cfg.services) == ['web'] + + cfg = config.load(build_config_details({'version': {'image': 'busybox'}})) + assert cfg.version == V1 + assert list(s['name'] for s in cfg.services) == ['version'] + + def test_wrong_version_type(self): + for version in [1, 2, 2.0]: + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + {'version': version}, + filename='filename.yml', + ) + ) + + assert 'Version in "filename.yml" is invalid - it should be a string.' \ + in excinfo.exconly() + + def test_unsupported_version(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + {'version': '1'}, + filename='filename.yml', + ) + ) + + assert 'Version in "filename.yml" is invalid' in excinfo.exconly() + assert VERSION_EXPLANATION in excinfo.exconly() + + def test_version_1_is_invalid(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': '1', + 'web': {'image': 'busybox'}, + }, + filename='filename.yml', + ) + ) + + assert 'Version in "filename.yml" is invalid' in excinfo.exconly() + assert VERSION_EXPLANATION in excinfo.exconly() + + def test_v1_file_with_version_is_invalid(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': '2', + 'web': {'image': 'busybox'}, + }, + filename='filename.yml', + ) + ) + + assert "compose.config.errors.ConfigurationError: " \ + "The Compose file 'filename.yml' is invalid because:\n" \ + "'web' does not match any of the regexes: '^x-'" in excinfo.exconly() + assert VERSION_EXPLANATION in excinfo.exconly() + + def test_named_volume_config_empty(self): + config_details = build_config_details({ + 'version': '2', + 'services': { + 'simple': {'image': 'busybox'} + }, + 'volumes': { + 'simple': None, + 'other': {}, + } + }) + config_result = config.load(config_details) + volumes = config_result.volumes + assert 'simple' in volumes + assert volumes['simple'] == {} + assert volumes['other'] == {} + + def test_named_volume_numeric_driver_opt(self): + config_details = build_config_details({ + 'version': '2', + 'services': { + 'simple': {'image': 'busybox'} + }, + 'volumes': { + 'simple': {'driver_opts': {'size': 42}}, + } + }) + cfg = config.load(config_details) + assert cfg.volumes['simple']['driver_opts']['size'] == '42' + + def test_volume_invalid_driver_opt(self): + config_details = build_config_details({ + 'version': '2', + 'services': { + 'simple': {'image': 'busybox'} + }, + 'volumes': { + 'simple': {'driver_opts': {'size': True}}, + } + }) + with pytest.raises(ConfigurationError) as exc: + config.load(config_details) + assert 'driver_opts.size contains an invalid type' in exc.exconly() + + def test_named_volume_invalid_type_list(self): + config_details = build_config_details({ + 'version': '2', + 'services': { + 'simple': {'image': 'busybox'} + }, + 'volumes': [] + }) + with pytest.raises(ConfigurationError) as exc: + config.load(config_details) + assert "volume must be a mapping, not an array" in exc.exconly() + + def test_networks_invalid_type_list(self): + config_details = build_config_details({ + 'version': '2', + 'services': { + 'simple': {'image': 'busybox'} + }, + 'networks': [] + }) + with pytest.raises(ConfigurationError) as exc: + config.load(config_details) + assert "network must be a mapping, not an array" in exc.exconly() + + def test_load_service_with_name_version(self): + with mock.patch('compose.config.config.log') as mock_logging: + config_data = config.load( + build_config_details({ + 'version': { + 'image': 'busybox' + } + }, 'working_dir', 'filename.yml') + ) + assert 'Unexpected type for "version" key in "filename.yml"' \ + in mock_logging.warning.call_args[0][0] + + service_dicts = config_data.services + assert service_sort(service_dicts) == service_sort([ + { + 'name': 'version', + 'image': 'busybox', + } + ]) + + def test_load_throws_error_when_not_dict(self): + with pytest.raises(ConfigurationError): + config.load( + build_config_details( + {'web': BUSYBOX_IMAGE_WITH_TAG}, + 'working_dir', + 'filename.yml' + ) + ) + + def test_load_throws_error_when_not_dict_v2(self): + with pytest.raises(ConfigurationError): + config.load( + build_config_details( + {'version': '2', 'services': {'web': BUSYBOX_IMAGE_WITH_TAG}}, + 'working_dir', + 'filename.yml' + ) + ) + + def test_load_throws_error_with_invalid_network_fields(self): + with pytest.raises(ConfigurationError): + config.load( + build_config_details({ + 'version': '2', + 'services': {'web': BUSYBOX_IMAGE_WITH_TAG}, + 'networks': { + 'invalid': {'foo', 'bar'} + } + }, 'working_dir', 'filename.yml') + ) + + def test_load_config_link_local_ips_network(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '2', + 'services': { + 'web': { + 'image': 'example/web', + 'networks': { + 'foobar': { + 'aliases': ['foo', 'bar'], + 'link_local_ips': ['169.254.8.8'] + } + } + } + }, + 'networks': {'foobar': {}} + } + ) + + details = config.ConfigDetails('.', [base_file]) + web_service = config.load(details).services[0] + assert web_service['networks'] == { + 'foobar': { + 'aliases': ['foo', 'bar'], + 'link_local_ips': ['169.254.8.8'] + } + } + + def test_load_config_service_labels(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '2.1', + 'services': { + 'web': { + 'image': 'example/web', + 'labels': ['label_key=label_val'] + }, + 'db': { + 'image': 'example/db', + 'labels': { + 'label_key': 'label_val' + } + } + }, + } + ) + details = config.ConfigDetails('.', [base_file]) + service_dicts = config.load(details).services + for service in service_dicts: + assert service['labels'] == { + 'label_key': 'label_val' + } + + def test_load_config_custom_resource_names(self): + base_file = config.ConfigFile( + 'base.yaml', { + 'version': '3.5', + 'volumes': { + 'abc': { + 'name': 'xyz' + } + }, + 'networks': { + 'abc': { + 'name': 'xyz' + } + }, + 'secrets': { + 'abc': { + 'name': 'xyz' + } + }, + 'configs': { + 'abc': { + 'name': 'xyz' + } + } + } + ) + details = config.ConfigDetails('.', [base_file]) + loaded_config = config.load(details) + + assert loaded_config.networks['abc'] == {'name': 'xyz'} + assert loaded_config.volumes['abc'] == {'name': 'xyz'} + assert loaded_config.secrets['abc']['name'] == 'xyz' + assert loaded_config.configs['abc']['name'] == 'xyz' + + def test_load_config_volume_and_network_labels(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '2.1', + 'services': { + 'web': { + 'image': 'example/web', + }, + }, + 'networks': { + 'with_label': { + 'labels': { + 'label_key': 'label_val' + } + } + }, + 'volumes': { + 'with_label': { + 'labels': { + 'label_key': 'label_val' + } + } + } + } + ) + + details = config.ConfigDetails('.', [base_file]) + loaded_config = config.load(details) + + assert loaded_config.networks == { + 'with_label': { + 'labels': { + 'label_key': 'label_val' + } + } + } + + assert loaded_config.volumes == { + 'with_label': { + 'labels': { + 'label_key': 'label_val' + } + } + } + + def test_load_config_invalid_service_names(self): + for invalid_name in ['?not?allowed', ' ', '', '!', '/', '\xe2']: + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details( + { + 'version': '2', + 'services': { + invalid_name: + { + 'image': 'busybox' + } + } + })) + assert 'Invalid service name \'%s\'' % invalid_name in exc.exconly() + + def test_load_config_invalid_service_names_v2(self): + for invalid_name in ['?not?allowed', ' ', '', '!', '/', '\xe2']: + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details( + { + 'version': '2', + 'services': {invalid_name: {'image': 'busybox'}}, + })) + assert 'Invalid service name \'%s\'' % invalid_name in exc.exconly() + + def test_load_with_invalid_field_name(self): + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details( + { + 'version': '2', + 'services': { + 'web': {'image': 'busybox', 'name': 'bogus'}, + } + }, + 'working_dir', + 'filename.yml', + )) + + assert "Unsupported config option for services.web: 'name'" in exc.exconly() + + def test_load_with_invalid_field_name_v1(self): + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details( + { + 'version': '2', + 'services': { + 'web': {'image': 'busybox', 'name': 'bogus'} + } + }, + 'working_dir', + 'filename.yml', + )) + assert "Unsupported config option for services.web: 'name'" in exc.exconly() + + def test_load_invalid_service_definition(self): + config_details = build_config_details( + { + 'version': '2', + 'services': { + 'web': 'wrong' + } + }, + 'working_dir', + 'filename.yml') + with pytest.raises(ConfigurationError) as exc: + config.load(config_details) + assert "service 'web' must be a mapping not a string." in exc.exconly() + + def test_load_with_empty_build_args(self): + config_details = build_config_details( + { + 'version': '2', + 'services': { + 'web': { + 'build': { + 'context': os.getcwd(), + 'args': None, + }, + }, + }, + } + ) + with pytest.raises(ConfigurationError) as exc: + config.load(config_details) + assert ( + "services.web.build.args contains an invalid type, it should be an " + "object, or an array" in exc.exconly() + ) + + def test_config_integer_service_name_raise_validation_error(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': '2', + 'services': {1: {'image': 'busybox'}} + }, + 'working_dir', + 'filename.yml' + ) + ) + + assert ( + "In file 'filename.yml', the service name 1 must be a quoted string, i.e. '1'" in + excinfo.exconly() + ) + + def test_config_integer_service_name_raise_validation_error_v2(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': '2', + 'services': {1: {'image': 'busybox'}} + }, + 'working_dir', + 'filename.yml' + ) + ) + + assert ( + "In file 'filename.yml', the service name 1 must be a quoted string, i.e. '1'." in + excinfo.exconly() + ) + + def test_config_integer_service_name_raise_validation_error_v2_when_no_interpolate(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': '2', + 'services': {1: {'image': 'busybox'}} + }, + 'working_dir', + 'filename.yml' + ), + interpolate=False + ) + + assert ( + "In file 'filename.yml', the service name 1 must be a quoted string, i.e. '1'." in + excinfo.exconly() + ) + + def test_config_integer_service_property_raise_validation_error(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details({ + 'version': '2.1', + 'services': {'foobar': {'image': 'busybox', 1234: 'hah'}} + }, 'working_dir', 'filename.yml') + ) + + assert ( + "Unsupported config option for services.foobar: '1234'" in excinfo.exconly() + ) + + def test_config_invalid_service_name_raise_validation_error(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details({ + 'version': '2', + 'services': { + 'test_app': {'build': '.'}, + 'mong\\o': {'image': 'mongo'}, + } + }) + ) + + assert 'Invalid service name \'mong\\o\'' in excinfo.exconly() + + def test_config_duplicate_cache_from_values_no_validation_error(self): + with pytest.raises(ConfigurationError) as exc: + config.load( + build_config_details({ + 'version': '2.3', + 'services': { + 'test': {'build': {'context': '.', 'cache_from': ['a', 'b', 'a']}} + } + + }) + ) + + assert 'build.cache_from contains non-unique items' not in exc.exconly() + + def test_load_with_multiple_files_v1(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'web': { + 'image': 'example/web', + 'links': ['db'], + }, + 'db': { + 'image': 'example/db', + }, + }) + override_file = config.ConfigFile( + 'override.yaml', + { + 'web': { + 'build': '/', + 'volumes': ['/home/user/project:/code'], + }, + }) + details = config.ConfigDetails('.', [base_file, override_file]) + + service_dicts = config.load(details).services + expected = [ + { + 'name': 'web', + 'build': {'context': os.path.abspath('/')}, + 'volumes': [VolumeSpec.parse('/home/user/project:/code')], + 'links': ['db'], + }, + { + 'name': 'db', + 'image': 'example/db', + }, + ] + assert service_sort(service_dicts) == service_sort(expected) + + def test_load_with_multiple_files_and_empty_override(self): + base_file = config.ConfigFile( + 'base.yml', + {'web': {'image': 'example/web'}}) + override_file = config.ConfigFile('override.yml', None) + details = config.ConfigDetails('.', [base_file, override_file]) + + with pytest.raises(ConfigurationError) as exc: + config.load(details) + error_msg = "Top level object in 'override.yml' needs to be an object" + assert error_msg in exc.exconly() + + def test_load_with_multiple_files_and_empty_override_v2(self): + base_file = config.ConfigFile( + 'base.yml', + {'version': '2', 'services': {'web': {'image': 'example/web'}}}) + override_file = config.ConfigFile('override.yml', None) + details = config.ConfigDetails('.', [base_file, override_file]) + + with pytest.raises(ConfigurationError) as exc: + config.load(details) + error_msg = "Top level object in 'override.yml' needs to be an object" + assert error_msg in exc.exconly() + + def test_load_with_multiple_files_and_empty_base(self): + base_file = config.ConfigFile('base.yml', None) + override_file = config.ConfigFile( + 'override.yml', + {'web': {'image': 'example/web'}}) + details = config.ConfigDetails('.', [base_file, override_file]) + + with pytest.raises(ConfigurationError) as exc: + config.load(details) + assert "Top level object in 'base.yml' needs to be an object" in exc.exconly() + + def test_load_with_multiple_files_and_empty_base_v2(self): + base_file = config.ConfigFile('base.yml', None) + override_file = config.ConfigFile( + 'override.tml', + {'version': '2', 'services': {'web': {'image': 'example/web'}}} + ) + details = config.ConfigDetails('.', [base_file, override_file]) + with pytest.raises(ConfigurationError) as exc: + config.load(details) + assert "Top level object in 'base.yml' needs to be an object" in exc.exconly() + + def test_load_with_multiple_files_and_extends_in_override_file(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'web': {'image': 'example/web'}, + }) + override_file = config.ConfigFile( + 'override.yaml', + { + 'web': { + 'extends': { + 'file': 'common.yml', + 'service': 'base', + }, + 'volumes': ['/home/user/project:/code'], + }, + }) + details = config.ConfigDetails('.', [base_file, override_file]) + + tmpdir = tempfile.mkdtemp('config_test') + self.addCleanup(shutil.rmtree, tmpdir) + with open(os.path.join(tmpdir, 'common.yml'), mode="w") as common_fh: + common_fh.write(""" + base: + labels: ['label=one'] + """) + with cd(tmpdir): + service_dicts = config.load(details).services + + expected = [ + { + 'name': 'web', + 'image': 'example/web', + 'volumes': [VolumeSpec.parse('/home/user/project:/code')], + 'labels': {'label': 'one'}, + }, + ] + assert service_sort(service_dicts) == service_sort(expected) + + def test_load_mixed_extends_resolution(self): + main_file = config.ConfigFile( + 'main.yml', { + 'version': '2.2', + 'services': { + 'prodweb': { + 'extends': { + 'service': 'web', + 'file': 'base.yml' + }, + 'environment': {'PROD': 'true'}, + }, + }, + } + ) + + tmpdir = tempfile.mkdtemp('config_test') + self.addCleanup(shutil.rmtree, tmpdir) + with open(os.path.join(tmpdir, 'base.yml'), mode="w") as base_fh: + base_fh.write(""" + version: '2.2' + services: + base: + image: base + web: + extends: base + """) + + details = config.ConfigDetails('.', [main_file]) + with cd(tmpdir): + service_dicts = config.load(details).services + assert service_dicts[0] == { + 'name': 'prodweb', + 'image': 'base', + 'environment': {'PROD': 'true'}, + } + + def test_load_with_multiple_files_and_invalid_override(self): + base_file = config.ConfigFile( + 'base.yaml', + {'version': '2', 'services': {'web': {'image': 'example/web'}}}) + override_file = config.ConfigFile( + 'override.yaml', + {'version': '2', 'services': {'bogus': 'thing'}}) + details = config.ConfigDetails('.', [base_file, override_file]) + + with pytest.raises(ConfigurationError) as exc: + config.load(details) + assert "service 'bogus' must be a mapping not a string." in exc.exconly() + assert "In file 'override.yaml'" in exc.exconly() + + def test_load_sorts_in_dependency_order(self): + config_details = build_config_details({ + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'links': ['db'], + }, + 'db': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes_from': ['volume:ro'] + }, + 'volume': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes': ['/tmp'], + } + }) + services = config.load(config_details).services + + assert services[0]['name'] == 'volume' + assert services[1]['name'] == 'db' + assert services[2]['name'] == 'web' + + def test_load_with_extensions(self): + config_details = build_config_details({ + 'version': '2.3', + 'x-data': { + 'lambda': 3, + 'excess': [True, {}] + } + }) + + config_data = config.load(config_details) + assert config_data.services == [] + + def test_config_build_configuration(self): + service = config.load( + build_config_details( + {'web': { + 'build': '.', + 'dockerfile': 'Dockerfile-alt' + }}, + 'tests/fixtures/extends', + 'filename.yml' + ) + ).services + assert 'context' in service[0]['build'] + assert service[0]['build']['dockerfile'] == 'Dockerfile-alt' + + def test_config_build_configuration_v2(self): + # service.dockerfile is invalid in v2 + with pytest.raises(ConfigurationError): + config.load( + build_config_details( + { + 'version': '2', + 'services': { + 'web': { + 'build': '.', + 'dockerfile': 'Dockerfile-alt' + } + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + service = config.load( + build_config_details({ + 'version': '2', + 'services': { + 'web': { + 'build': '.' + } + } + }, 'tests/fixtures/extends', 'filename.yml') + ).services[0] + assert 'context' in service['build'] + + service = config.load( + build_config_details( + { + 'version': '2', + 'services': { + 'web': { + 'build': { + 'context': '.', + 'dockerfile': 'Dockerfile-alt' + } + } + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ).services + assert 'context' in service[0]['build'] + assert service[0]['build']['dockerfile'] == 'Dockerfile-alt' + + def test_load_with_buildargs(self): + service = config.load( + build_config_details( + { + 'version': '2', + 'services': { + 'web': { + 'build': { + 'context': '.', + 'dockerfile': 'Dockerfile-alt', + 'args': { + 'opt1': 42, + 'opt2': 'foobar' + } + } + } + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ).services[0] + assert 'args' in service['build'] + assert 'opt1' in service['build']['args'] + assert isinstance(service['build']['args']['opt1'], str) + assert service['build']['args']['opt1'] == '42' + assert service['build']['args']['opt2'] == 'foobar' + + def test_load_build_labels_dict(self): + service = config.load( + build_config_details( + { + 'services': { + 'web': { + 'build': { + 'context': '.', + 'dockerfile': 'Dockerfile-alt', + 'labels': { + 'label1': 42, + 'label2': 'foobar' + } + } + } + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ).services[0] + assert 'labels' in service['build'] + assert 'label1' in service['build']['labels'] + assert service['build']['labels']['label1'] == '42' + assert service['build']['labels']['label2'] == 'foobar' + + def test_load_build_labels_list(self): + base_file = config.ConfigFile( + 'base.yml', + { + 'version': '2.3', + 'services': { + 'web': { + 'build': { + 'context': '.', + 'labels': ['foo=bar', 'baz=true', 'foobar=1'] + }, + }, + }, + } + ) + + details = config.ConfigDetails('.', [base_file]) + service = config.load(details).services[0] + assert service['build']['labels'] == { + 'foo': 'bar', 'baz': 'true', 'foobar': '1' + } + + def test_build_args_allow_empty_properties(self): + service = config.load( + build_config_details( + { + 'version': '2', + 'services': { + 'web': { + 'build': { + 'context': '.', + 'dockerfile': 'Dockerfile-alt', + 'args': { + 'foo': None + } + } + } + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ).services[0] + assert 'args' in service['build'] + assert 'foo' in service['build']['args'] + assert service['build']['args']['foo'] == '' + + # If build argument is None then it will be converted to the empty + # string. Make sure that int zero kept as it is, i.e. not converted to + # the empty string + def test_build_args_check_zero_preserved(self): + service = config.load( + build_config_details( + { + 'version': '2', + 'services': { + 'web': { + 'build': { + 'context': '.', + 'dockerfile': 'Dockerfile-alt', + 'args': { + 'foo': 0 + } + } + } + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ).services[0] + assert 'args' in service['build'] + assert 'foo' in service['build']['args'] + assert service['build']['args']['foo'] == '0' + + def test_load_with_multiple_files_mismatched_networks_format(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '2', + 'services': { + 'web': { + 'image': 'example/web', + 'networks': { + 'foobar': {'aliases': ['foo', 'bar']} + } + } + }, + 'networks': {'foobar': {}, 'baz': {}} + } + ) + + override_file = config.ConfigFile( + 'override.yaml', + { + 'version': '2', + 'services': { + 'web': { + 'networks': ['baz'] + } + } + } + ) + + details = config.ConfigDetails('.', [base_file, override_file]) + web_service = config.load(details).services[0] + assert web_service['networks'] == { + 'foobar': {'aliases': ['bar', 'foo']}, + 'baz': {} + } + + def test_load_with_multiple_files_mismatched_networks_format_inverse_order(self): + base_file = config.ConfigFile( + 'override.yaml', + { + 'version': '2', + 'services': { + 'web': { + 'networks': ['baz'] + } + } + } + ) + override_file = config.ConfigFile( + 'base.yaml', + { + 'version': '2', + 'services': { + 'web': { + 'image': 'example/web', + 'networks': { + 'foobar': {'aliases': ['foo', 'bar']} + } + } + }, + 'networks': {'foobar': {}, 'baz': {}} + } + ) + + details = config.ConfigDetails('.', [base_file, override_file]) + web_service = config.load(details).services[0] + assert web_service['networks'] == { + 'foobar': {'aliases': ['bar', 'foo']}, + 'baz': {} + } + + def test_load_with_multiple_files_v2(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '2', + 'services': { + 'web': { + 'image': 'example/web', + 'depends_on': ['db'], + }, + 'db': { + 'image': 'example/db', + } + }, + }) + override_file = config.ConfigFile( + 'override.yaml', + { + 'version': '2', + 'services': { + 'web': { + 'build': '/', + 'volumes': ['/home/user/project:/code'], + 'depends_on': ['other'], + }, + 'other': { + 'image': 'example/other', + } + } + }) + details = config.ConfigDetails('.', [base_file, override_file]) + + service_dicts = config.load(details).services + expected = [ + { + 'name': 'web', + 'build': {'context': os.path.abspath('/')}, + 'image': 'example/web', + 'volumes': [VolumeSpec.parse('/home/user/project:/code')], + 'depends_on': { + 'db': {'condition': 'service_started'}, + 'other': {'condition': 'service_started'}, + }, + }, + { + 'name': 'db', + 'image': 'example/db', + }, + { + 'name': 'other', + 'image': 'example/other', + }, + ] + assert service_sort(service_dicts) == service_sort(expected) + + @mock.patch.dict(os.environ) + def test_load_with_multiple_files_v3_2(self): + os.environ['COMPOSE_CONVERT_WINDOWS_PATHS'] = 'true' + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '3.2', + 'services': { + 'web': { + 'image': 'example/web', + 'volumes': [ + {'source': '/a', 'target': '/b', 'type': 'bind'}, + {'source': 'vol', 'target': '/x', 'type': 'volume', 'read_only': True} + ], + 'stop_grace_period': '30s', + } + }, + 'volumes': {'vol': {}} + } + ) + + override_file = config.ConfigFile( + 'override.yaml', + { + 'version': '3.2', + 'services': { + 'web': { + 'volumes': ['/c:/b', '/anonymous'] + } + } + } + ) + details = config.ConfigDetails('.', [base_file, override_file]) + service_dicts = config.load(details).services + svc_volumes = map(lambda v: v.repr(), service_dicts[0]['volumes']) + for vol in svc_volumes: + assert vol in [ + '/anonymous', + '/c:/b:rw', + {'source': 'vol', 'target': '/x', 'type': 'volume', 'read_only': True} + ] + assert service_dicts[0]['stop_grace_period'] == '30s' + + @mock.patch.dict(os.environ) + def test_volume_mode_override(self): + os.environ['COMPOSE_CONVERT_WINDOWS_PATHS'] = 'true' + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '2.3', + 'services': { + 'web': { + 'image': 'example/web', + 'volumes': ['/c:/b:rw'] + } + }, + } + ) + + override_file = config.ConfigFile( + 'override.yaml', + { + 'version': '2.3', + 'services': { + 'web': { + 'volumes': ['/c:/b:ro'] + } + } + } + ) + details = config.ConfigDetails('.', [base_file, override_file]) + service_dicts = config.load(details).services + svc_volumes = list(map(lambda v: v.repr(), service_dicts[0]['volumes'])) + assert svc_volumes == ['/c:/b:ro'] + + def test_undeclared_volume_v2(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '2', + 'services': { + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes': ['data0028:/data:ro'], + }, + }, + } + ) + details = config.ConfigDetails('.', [base_file]) + with pytest.raises(ConfigurationError): + config.load(details) + + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '2', + 'services': { + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes': ['./data0028:/data:ro'], + }, + }, + } + ) + details = config.ConfigDetails('.', [base_file]) + config_data = config.load(details) + volume = config_data.services[0].get('volumes')[0] + assert not volume.is_named_volume + + def test_undeclared_volume_v1(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes': ['data0028:/data:ro'], + }, + } + ) + details = config.ConfigDetails('.', [base_file]) + config_data = config.load(details) + volume = config_data.services[0].get('volumes')[0] + assert volume.external == 'data0028' + assert volume.is_named_volume + + def test_volumes_long_syntax(self): + base_file = config.ConfigFile( + 'base.yaml', { + 'version': '2.3', + 'services': { + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes': [ + { + 'target': '/anonymous', 'type': 'volume' + }, { + 'source': '/abc', 'target': '/xyz', 'type': 'bind' + }, { + 'source': '\\\\.\\pipe\\abcd', 'target': '/named_pipe', 'type': 'npipe' + }, { + 'type': 'tmpfs', 'target': '/tmpfs' + } + ] + }, + }, + }, + ) + details = config.ConfigDetails('.', [base_file]) + config_data = config.load(details) + volumes = config_data.services[0].get('volumes') + anon_volume = [v for v in volumes if v.target == '/anonymous'][0] + tmpfs_mount = [v for v in volumes if v.type == 'tmpfs'][0] + host_mount = [v for v in volumes if v.type == 'bind'][0] + npipe_mount = [v for v in volumes if v.type == 'npipe'][0] + + assert anon_volume.type == 'volume' + assert not anon_volume.is_named_volume + + assert tmpfs_mount.target == '/tmpfs' + assert not tmpfs_mount.is_named_volume + + assert host_mount.source == '/abc' + assert host_mount.target == '/xyz' + assert not host_mount.is_named_volume + + assert npipe_mount.source == '\\\\.\\pipe\\abcd' + assert npipe_mount.target == '/named_pipe' + assert not npipe_mount.is_named_volume + + def test_load_bind_mount_relative_path(self): + expected_source = 'C:\\tmp\\web' if IS_WINDOWS_PLATFORM else '/tmp/web' + base_file = config.ConfigFile( + 'base.yaml', { + 'version': '3.4', + 'services': { + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes': [ + {'type': 'bind', 'source': './web', 'target': '/web'}, + ], + }, + }, + }, + ) + + details = config.ConfigDetails('/tmp', [base_file]) + config_data = config.load(details) + mount = config_data.services[0].get('volumes')[0] + assert mount.target == '/web' + assert mount.type == 'bind' + assert mount.source == expected_source + + def test_load_bind_mount_relative_path_with_tilde(self): + base_file = config.ConfigFile( + 'base.yaml', { + 'version': '3.4', + 'services': { + 'web': { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes': [ + {'type': 'bind', 'source': '~/web', 'target': '/web'}, + ], + }, + }, + }, + ) + + details = config.ConfigDetails('.', [base_file]) + config_data = config.load(details) + mount = config_data.services[0].get('volumes')[0] + assert mount.target == '/web' + assert mount.type == 'bind' + assert ( + not mount.source.startswith('~') and mount.source.endswith( + '{}web'.format(os.path.sep) + ) + ) + + def test_config_invalid_ipam_config(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': str(VERSION), + 'networks': { + 'foo': { + 'driver': 'default', + 'ipam': { + 'driver': 'default', + 'config': ['172.18.0.0/16'], + } + } + } + }, + filename='filename.yml', + ) + ) + assert ('networks.foo.ipam.config contains an invalid type,' + ' it should be an object') in excinfo.exconly() + + def test_config_valid_ipam_config(self): + ipam_config = { + 'subnet': '172.28.0.0/16', + 'ip_range': '172.28.5.0/24', + 'gateway': '172.28.5.254', + 'aux_addresses': { + 'host1': '172.28.1.5', + 'host2': '172.28.1.6', + 'host3': '172.28.1.7', + }, + } + networks = config.load( + build_config_details( + { + 'networks': { + 'foo': { + 'driver': 'default', + 'ipam': { + 'driver': 'default', + 'config': [ipam_config], + } + } + } + }, + filename='filename.yml', + ) + ).networks + + assert 'foo' in networks + assert networks['foo']['ipam']['config'] == [ipam_config] + + def test_config_valid_service_names(self): + for valid_name in ['_', '-', '.__.', '_what-up.', 'what_.up----', 'whatup']: + services = config.load( + build_config_details( + {valid_name: {'image': 'busybox'}}, + 'tests/fixtures/extends', + 'common.yml')).services + assert services[0]['name'] == valid_name + + def test_config_hint(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': str(VERSION), + 'services': { + 'foo': {'image': 'busybox', 'privilige': 'something'}, + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert "(did you mean 'privileged'?)" in excinfo.exconly() + + def test_load_errors_on_uppercase_with_no_image(self): + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details({ + 'Foo': {'build': '.'}, + }, 'tests/fixtures/build-ctx')) + assert "Service 'Foo' contains uppercase characters" in exc.exconly() + + def test_invalid_config_v1(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': str(VERSION), + 'services': { + 'foo': {'image': 1}, + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert "foo.image contains an invalid type, it should be a string" \ + in excinfo.exconly() + + def test_invalid_config_v2(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': '2', + 'services': { + 'foo': {'image': 1}, + }, + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert "services.foo.image contains an invalid type, it should be a string" \ + in excinfo.exconly() + + def test_invalid_config_build_and_image_specified_v1(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'foo': {'image': 'busybox', 'build': '.'}, + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert "foo has both an image and build path specified." in excinfo.exconly() + + def test_invalid_config_type_should_be_an_array(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': str(VERSION), + 'services': { + 'foo': {'image': 'busybox', 'links': 'an_link'}, + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert "foo.links contains an invalid type, it should be an array" \ + in excinfo.exconly() + + def test_invalid_config_not_a_dictionary(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + ['foo', 'lol'], + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert "Top level object in 'filename.yml' needs to be an object" \ + in excinfo.exconly() + + def test_invalid_config_not_unique_items(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': str(VERSION), + 'services': { + 'web': {'build': '.', 'devices': ['/dev/foo:/dev/foo', '/dev/foo:/dev/foo']} + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert "has non-unique elements" in excinfo.exconly() + + def test_invalid_list_of_strings_format(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': str(VERSION), + 'services': { + 'web': {'build': '.', 'command': [1]} + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert "web.command contains 1, which is an invalid type, it should be a string" \ + in excinfo.exconly() + + def test_load_config_dockerfile_without_build_raises_error_v1(self): + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details({ + 'web': { + 'image': 'busybox', + 'dockerfile': 'Dockerfile.alt' + } + })) + + assert "web has both an image and alternate Dockerfile." in exc.exconly() + + def test_config_extra_hosts_string_raises_validation_error(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': str(VERSION), + 'services': { + 'web': { + 'image': 'busybox', + 'extra_hosts': 'somehost:162.242.195.82'}} + }, + 'working_dir', + 'filename.yml' + ) + ) + + assert "web.extra_hosts contains an invalid type" \ + in excinfo.exconly() + + def test_config_extra_hosts_list_of_dicts_validation_error(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': str(VERSION), + 'services': { + 'web': { + 'image': 'busybox', + 'extra_hosts': [ + {'somehost': '162.242.195.82'}, + {'otherhost': '50.31.209.229'} + ]}} + }, + 'working_dir', + 'filename.yml' + ) + ) + + assert "web.extra_hosts contains {\"somehost\": \"162.242.195.82\"}, " \ + "which is an invalid type, it should be a string" \ + in excinfo.exconly() + + def test_config_ulimits_invalid_keys_validation_error(self): + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details( + { + 'version': str(VERSION), + 'services': { + 'web': { + 'image': 'busybox', + 'ulimits': { + 'nofile': { + "not_soft_or_hard": 100, + "soft": 10000, + "hard": 20000, + } + } + } + } + }, + 'working_dir', + 'filename.yml')) + + assert "web.ulimits.nofile contains unsupported option: 'not_soft_or_hard'" \ + in exc.exconly() + + def test_config_ulimits_required_keys_validation_error(self): + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details( + { + 'version': str(VERSION), + 'services': { + 'web': { + 'image': 'busybox', + 'ulimits': {'nofile': {"soft": 10000}} + } + } + }, + 'working_dir', + 'filename.yml')) + assert "web.ulimits.nofile" in exc.exconly() + assert "'hard' is a required property" in exc.exconly() + + def test_config_ulimits_soft_greater_than_hard_error(self): + expected = "'soft' value can not be greater than 'hard' value" + + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details( + { + 'version': str(VERSION), + 'services': { + 'web': { + 'image': 'busybox', + 'ulimits': { + 'nofile': {"soft": 10000, "hard": 1000} + } + } + } + }, + 'working_dir', + 'filename.yml')) + assert expected in exc.exconly() + + def test_valid_config_which_allows_two_type_definitions(self): + expose_values = [["8000"], [8000]] + for expose in expose_values: + service = config.load( + build_config_details( + { + 'version': str(VERSION), + 'services': { + 'web': { + 'image': 'busybox', + 'expose': expose}}}, + 'working_dir', + 'filename.yml' + ) + ).services + assert service[0]['expose'] == expose + + def test_valid_config_oneof_string_or_list(self): + entrypoint_values = [["sh"], "sh"] + for entrypoint in entrypoint_values: + service = config.load( + build_config_details( + { + 'version': str(VERSION), + 'services': { + 'web': { + 'image': 'busybox', + 'entrypoint': entrypoint}}}, + 'working_dir', + 'filename.yml' + ) + ).services + assert service[0]['entrypoint'] == entrypoint + + def test_logs_warning_for_boolean_in_environment(self): + config_details = build_config_details({ + 'version': str(VERSION), + 'services': { + 'web': { + 'image': 'busybox', + 'environment': {'SHOW_STUFF': True} + } + } + }) + + with pytest.raises(ConfigurationError) as exc: + config.load(config_details) + + assert "contains true, which is an invalid type" in exc.exconly() + + def test_config_valid_environment_dict_key_contains_dashes(self): + services = config.load( + build_config_details( + { + 'version': str(VERSION), + 'services': { + 'web': { + 'image': 'busybox', + 'environment': {'SPRING_JPA_HIBERNATE_DDL-AUTO': 'none'}}}}, + 'working_dir', + 'filename.yml' + ) + ).services + assert services[0]['environment']['SPRING_JPA_HIBERNATE_DDL-AUTO'] == 'none' + + def test_load_yaml_with_yaml_error(self): + tmpdir = tempfile.mkdtemp('invalid_yaml_test') + self.addCleanup(shutil.rmtree, tmpdir) + invalid_yaml_file = os.path.join(tmpdir, 'docker-compose.yml') + with open(invalid_yaml_file, mode="w") as invalid_yaml_file_fh: + invalid_yaml_file_fh.write(""" +web: + this is bogus: ok: what + """) + with pytest.raises(ConfigurationError) as exc: + config.load_yaml(str(invalid_yaml_file)) + + assert 'line 3, column 22' in exc.exconly() + + def test_load_yaml_with_bom(self): + tmpdir = tempfile.mkdtemp('bom_yaml') + self.addCleanup(shutil.rmtree, tmpdir) + bom_yaml = os.path.join(tmpdir, 'docker-compose.yml') + with codecs.open(str(bom_yaml), 'w', encoding='utf-8') as f: + f.write('''\ufeff + version: '2.3' + volumes: + park_bom: + ''') + assert config.load_yaml(str(bom_yaml)) == { + 'version': '2.3', + 'volumes': {'park_bom': None} + } + + def test_validate_extra_hosts_invalid(self): + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details({ + 'version': str(VERSION), + 'services': { + 'web': { + 'image': 'alpine', + 'extra_hosts': "www.example.com: 192.168.0.17", + } + } + })) + assert "web.extra_hosts contains an invalid type" in exc.exconly() + + def test_validate_extra_hosts_invalid_list(self): + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details({ + 'version': str(VERSION), + 'services': { + 'web': { + 'image': 'alpine', + 'extra_hosts': [ + {'www.example.com': '192.168.0.17'}, + {'api.example.com': '192.168.0.18'} + ], + } + } + })) + assert "which is an invalid type" in exc.exconly() + + def test_normalize_dns_options(self): + actual = config.load(build_config_details({ + 'version': str(VERSION), + 'services': { + 'web': { + 'image': 'alpine', + 'dns': '8.8.8.8', + 'dns_search': 'domain.local', + } + } + })) + assert actual.services == [ + { + 'name': 'web', + 'image': 'alpine', + 'dns': ['8.8.8.8'], + 'dns_search': ['domain.local'], + } + ] + + def test_tmpfs_option(self): + actual = config.load(build_config_details({ + 'version': '2', + 'services': { + 'web': { + 'image': 'alpine', + 'tmpfs': '/run', + } + } + })) + assert actual.services == [ + { + 'name': 'web', + 'image': 'alpine', + 'tmpfs': ['/run'], + } + ] + + def test_oom_score_adj_option(self): + + actual = config.load(build_config_details({ + 'version': '2', + 'services': { + 'web': { + 'image': 'alpine', + 'oom_score_adj': 500 + } + } + })) + + assert actual.services == [ + { + 'name': 'web', + 'image': 'alpine', + 'oom_score_adj': 500 + } + ] + + def test_swappiness_option(self): + actual = config.load(build_config_details({ + 'version': '2', + 'services': { + 'web': { + 'image': 'alpine', + 'mem_swappiness': 10, + } + } + })) + assert actual.services == [ + { + 'name': 'web', + 'image': 'alpine', + 'mem_swappiness': 10, + } + ] + + @data( + '2 ', + '3.', + '3.0.0', + '3.0.a', + '3.a', + '3a') + def test_invalid_version_formats(self, version): + content = { + 'version': version, + 'services': { + 'web': { + 'image': 'alpine', + } + } + } + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details(content)) + assert 'Version "{}" in "filename.yml" is invalid.'.format(version) in exc.exconly() + + def test_group_add_option(self): + actual = config.load(build_config_details({ + 'version': '2', + 'services': { + 'web': { + 'image': 'alpine', + 'group_add': ["docker", 777] + } + } + })) + + assert actual.services == [ + { + 'name': 'web', + 'image': 'alpine', + 'group_add': ["docker", 777] + } + ] + + def test_dns_opt_option(self): + actual = config.load(build_config_details({ + 'version': '2', + 'services': { + 'web': { + 'image': 'alpine', + 'dns_opt': ["use-vc", "no-tld-query"] + } + } + })) + + assert actual.services == [ + { + 'name': 'web', + 'image': 'alpine', + 'dns_opt': ["use-vc", "no-tld-query"] + } + ] + + def test_isolation_option(self): + actual = config.load(build_config_details({ + 'services': { + 'web': { + 'image': 'win10', + 'isolation': 'hyperv' + } + } + })) + + assert actual.services == [ + { + 'name': 'web', + 'image': 'win10', + 'isolation': 'hyperv', + } + ] + + def test_runtime_option(self): + actual = config.load(build_config_details({ + 'services': { + 'web': { + 'image': 'nvidia/cuda', + 'runtime': 'nvidia' + } + } + })) + + assert actual.services == [ + { + 'name': 'web', + 'image': 'nvidia/cuda', + 'runtime': 'nvidia', + } + ] + + def test_merge_service_dicts_from_files_with_extends_in_base(self): + base = { + 'volumes': ['.:/app'], + 'extends': {'service': 'app'} + } + override = { + 'image': 'alpine:edge', + } + actual = config.merge_service_dicts_from_files( + base, + override, + DEFAULT_VERSION) + assert actual == { + 'image': 'alpine:edge', + 'volumes': ['.:/app'], + 'extends': {'service': 'app'} + } + + def test_merge_service_dicts_from_files_with_extends_in_override(self): + base = { + 'volumes': ['.:/app'], + 'extends': {'service': 'app'} + } + override = { + 'image': 'alpine:edge', + 'extends': {'service': 'foo'} + } + actual = config.merge_service_dicts_from_files( + base, + override, + DEFAULT_VERSION) + assert actual == { + 'image': 'alpine:edge', + 'volumes': ['.:/app'], + 'extends': {'service': 'foo'} + } + + def test_merge_service_dicts_heterogeneous(self): + base = { + 'volumes': ['.:/app'], + 'ports': ['5432'] + } + override = { + 'image': 'alpine:edge', + 'ports': [5432] + } + actual = config.merge_service_dicts_from_files( + base, + override, + DEFAULT_VERSION) + assert actual == { + 'image': 'alpine:edge', + 'volumes': ['.:/app'], + 'ports': types.ServicePort.parse('5432') + } + + def test_merge_service_dicts_heterogeneous_2(self): + base = { + 'volumes': ['.:/app'], + 'ports': [5432] + } + override = { + 'image': 'alpine:edge', + 'ports': ['5432'] + } + actual = config.merge_service_dicts_from_files( + base, + override, + DEFAULT_VERSION) + assert actual == { + 'image': 'alpine:edge', + 'volumes': ['.:/app'], + 'ports': types.ServicePort.parse('5432') + } + + def test_merge_service_dicts_ports_sorting(self): + base = { + 'ports': [5432] + } + override = { + 'image': 'alpine:edge', + 'ports': ['5432/udp'] + } + actual = config.merge_service_dicts_from_files( + base, + override, + DEFAULT_VERSION) + assert len(actual['ports']) == 2 + assert types.ServicePort.parse('5432')[0] in actual['ports'] + assert types.ServicePort.parse('5432/udp')[0] in actual['ports'] + + def test_merge_service_dicts_heterogeneous_volumes(self): + base = { + 'volumes': ['/a:/b', '/x:/z'], + } + + override = { + 'image': 'alpine:edge', + 'volumes': [ + {'source': '/e', 'target': '/b', 'type': 'bind'}, + {'source': '/c', 'target': '/d', 'type': 'bind'} + ] + } + + actual = config.merge_service_dicts_from_files( + base, override, VERSION + ) + + assert actual['volumes'] == [ + {'source': '/e', 'target': '/b', 'type': 'bind'}, + {'source': '/c', 'target': '/d', 'type': 'bind'}, + '/x:/z' + ] + + def test_merge_logging_v1(self): + base = { + 'image': 'alpine:edge', + 'log_driver': 'something', + 'log_opt': {'foo': 'three'}, + } + override = { + 'image': 'alpine:edge', + 'command': 'true', + } + actual = config.merge_service_dicts(base, override, V1) + assert actual == { + 'image': 'alpine:edge', + 'log_driver': 'something', + 'log_opt': {'foo': 'three'}, + 'command': 'true', + } + + def test_merge_logging_v2(self): + base = { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'json-file', + 'options': { + 'frequency': '2000', + 'timeout': '23' + } + } + } + override = { + 'logging': { + 'options': { + 'timeout': '360', + 'pretty-print': 'on' + } + } + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'json-file', + 'options': { + 'frequency': '2000', + 'timeout': '360', + 'pretty-print': 'on' + } + } + } + + def test_merge_logging_v2_override_driver(self): + base = { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'json-file', + 'options': { + 'frequency': '2000', + 'timeout': '23' + } + } + } + override = { + 'logging': { + 'driver': 'syslog', + 'options': { + 'timeout': '360', + 'pretty-print': 'on' + } + } + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'syslog', + 'options': { + 'timeout': '360', + 'pretty-print': 'on' + } + } + } + + def test_merge_logging_v2_no_base_driver(self): + base = { + 'image': 'alpine:edge', + 'logging': { + 'options': { + 'frequency': '2000', + 'timeout': '23' + } + } + } + override = { + 'logging': { + 'driver': 'json-file', + 'options': { + 'timeout': '360', + 'pretty-print': 'on' + } + } + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'json-file', + 'options': { + 'frequency': '2000', + 'timeout': '360', + 'pretty-print': 'on' + } + } + } + + def test_merge_logging_v2_no_drivers(self): + base = { + 'image': 'alpine:edge', + 'logging': { + 'options': { + 'frequency': '2000', + 'timeout': '23' + } + } + } + override = { + 'logging': { + 'options': { + 'timeout': '360', + 'pretty-print': 'on' + } + } + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == { + 'image': 'alpine:edge', + 'logging': { + 'options': { + 'frequency': '2000', + 'timeout': '360', + 'pretty-print': 'on' + } + } + } + + def test_merge_logging_v2_no_override_options(self): + base = { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'json-file', + 'options': { + 'frequency': '2000', + 'timeout': '23' + } + } + } + override = { + 'logging': { + 'driver': 'syslog' + } + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'syslog', + } + } + + def test_merge_logging_v2_no_base(self): + base = { + 'image': 'alpine:edge' + } + override = { + 'logging': { + 'driver': 'json-file', + 'options': { + 'frequency': '2000' + } + } + } + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'json-file', + 'options': { + 'frequency': '2000' + } + } + } + + def test_merge_logging_v2_no_override(self): + base = { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'syslog', + 'options': { + 'frequency': '2000' + } + } + } + override = {} + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'syslog', + 'options': { + 'frequency': '2000' + } + } + } + + def test_merge_mixed_ports(self): + base = { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'ports': [ + { + 'target': '1245', + 'published': '1245', + 'protocol': 'udp', + } + ] + } + + override = { + 'ports': ['1245:1245/udp'] + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == { + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'command': 'top', + 'ports': [types.ServicePort('1245', '1245', 'udp', None, None)] + } + + def test_merge_depends_on_no_override(self): + base = { + 'image': 'busybox', + 'depends_on': { + 'app1': {'condition': 'service_started'}, + 'app2': {'condition': 'service_healthy'} + } + } + override = {} + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == base + + def test_merge_depends_on_mixed_syntax(self): + base = { + 'image': 'busybox', + 'depends_on': { + 'app1': {'condition': 'service_started'}, + 'app2': {'condition': 'service_healthy'} + } + } + override = { + 'depends_on': ['app3'] + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == { + 'image': 'busybox', + 'depends_on': { + 'app1': {'condition': 'service_started'}, + 'app2': {'condition': 'service_healthy'}, + 'app3': {'condition': 'service_started'} + } + } + + def test_empty_environment_key_allowed(self): + service_dict = config.load( + build_config_details( + { + 'web': { + 'build': '.', + 'environment': { + 'POSTGRES_PASSWORD': '' + }, + }, + }, + '.', + None, + ) + ).services[0] + assert service_dict['environment']['POSTGRES_PASSWORD'] == '' + + def test_merge_pid(self): + # Regression: https://github.com/docker/compose/issues/4184 + base = { + 'image': 'busybox', + 'pid': 'host' + } + + override = { + 'labels': {'com.docker.compose.test': 'yes'} + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == { + 'image': 'busybox', + 'pid': 'host', + 'labels': {'com.docker.compose.test': 'yes'} + } + + def test_merge_different_secrets(self): + base = { + 'image': 'busybox', + 'secrets': [ + {'source': 'src.txt'} + ] + } + override = {'secrets': ['other-src.txt']} + + actual = config.merge_service_dicts(base, override, VERSION) + assert secret_sort(actual['secrets']) == secret_sort([ + {'source': 'src.txt'}, + {'source': 'other-src.txt'} + ]) + + def test_merge_secrets_override(self): + base = { + 'image': 'busybox', + 'secrets': ['src.txt'], + } + override = { + 'secrets': [ + { + 'source': 'src.txt', + 'target': 'data.txt', + 'mode': 0o400 + } + ] + } + actual = config.merge_service_dicts(base, override, VERSION) + assert actual['secrets'] == override['secrets'] + + def test_merge_different_configs(self): + base = { + 'image': 'busybox', + 'configs': [ + {'source': 'src.txt'} + ] + } + override = {'configs': ['other-src.txt']} + + actual = config.merge_service_dicts(base, override, VERSION) + assert secret_sort(actual['configs']) == secret_sort([ + {'source': 'src.txt'}, + {'source': 'other-src.txt'} + ]) + + def test_merge_configs_override(self): + base = { + 'image': 'busybox', + 'configs': ['src.txt'], + } + override = { + 'configs': [ + { + 'source': 'src.txt', + 'target': 'data.txt', + 'mode': 0o400 + } + ] + } + actual = config.merge_service_dicts(base, override, VERSION) + assert actual['configs'] == override['configs'] + + def test_merge_deploy(self): + base = { + 'image': 'busybox', + } + override = { + 'deploy': { + 'mode': 'global', + 'restart_policy': { + 'condition': 'on-failure' + } + } + } + actual = config.merge_service_dicts(base, override, VERSION) + assert actual['deploy'] == override['deploy'] + + def test_merge_deploy_override(self): + base = { + 'deploy': { + 'endpoint_mode': 'vip', + 'labels': ['com.docker.compose.a=1', 'com.docker.compose.b=2'], + 'mode': 'replicated', + 'placement': { + 'max_replicas_per_node': 1, + 'constraints': [ + 'node.role == manager', 'engine.labels.aws == true' + ], + 'preferences': [ + {'spread': 'node.labels.zone'}, {'spread': 'x.d.z'} + ] + }, + 'replicas': 3, + 'resources': { + 'limits': {'cpus': '0.50', 'memory': '50m'}, + 'reservations': { + 'cpus': '0.1', + 'generic_resources': [ + {'discrete_resource_spec': {'kind': 'abc', 'value': 123}} + ], + 'memory': '15m' + } + }, + 'restart_policy': {'condition': 'any', 'delay': '10s'}, + 'update_config': {'delay': '10s', 'max_failure_ratio': 0.3} + }, + 'image': 'hello-world' + } + override = { + 'deploy': { + 'labels': { + 'com.docker.compose.b': '21', 'com.docker.compose.c': '3' + }, + 'placement': { + 'constraints': ['node.role == worker', 'engine.labels.dev == true'], + 'preferences': [{'spread': 'node.labels.zone'}, {'spread': 'x.d.s'}] + }, + 'resources': { + 'limits': {'memory': '200m'}, + 'reservations': { + 'cpus': '0.78', + 'generic_resources': [ + {'discrete_resource_spec': {'kind': 'abc', 'value': 134}}, + {'discrete_resource_spec': {'kind': 'xyz', 'value': 0.1}} + ] + } + }, + 'restart_policy': {'condition': 'on-failure', 'max_attempts': 42}, + 'update_config': {'max_failure_ratio': 0.712, 'parallelism': 4} + } + } + actual = config.merge_service_dicts(base, override, VERSION) + assert actual['deploy'] == { + 'mode': 'replicated', + 'endpoint_mode': 'vip', + 'labels': { + 'com.docker.compose.a': '1', + 'com.docker.compose.b': '21', + 'com.docker.compose.c': '3' + }, + 'placement': { + 'max_replicas_per_node': 1, + 'constraints': [ + 'engine.labels.aws == true', 'engine.labels.dev == true', + 'node.role == manager', 'node.role == worker' + ], + 'preferences': [ + {'spread': 'node.labels.zone'}, {'spread': 'x.d.s'}, {'spread': 'x.d.z'} + ] + }, + 'replicas': 3, + 'resources': { + 'limits': {'cpus': '0.50', 'memory': '200m'}, + 'reservations': { + 'cpus': '0.78', + 'memory': '15m', + 'generic_resources': [ + {'discrete_resource_spec': {'kind': 'abc', 'value': 134}}, + {'discrete_resource_spec': {'kind': 'xyz', 'value': 0.1}}, + ] + } + }, + 'restart_policy': { + 'condition': 'on-failure', + 'delay': '10s', + 'max_attempts': 42, + }, + 'update_config': { + 'max_failure_ratio': 0.712, + 'delay': '10s', + 'parallelism': 4 + } + } + + def test_merge_credential_spec(self): + base = { + 'image': 'bb', + 'credential_spec': { + 'file': '/hello-world', + } + } + + override = { + 'credential_spec': { + 'registry': 'revolution.com', + } + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual['credential_spec'] == override['credential_spec'] + + def test_merge_scale(self): + base = { + 'image': 'bar', + 'scale': 2, + } + + override = { + 'scale': 4, + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == {'image': 'bar', 'scale': 4} + + def test_merge_blkio_config(self): + base = { + 'image': 'bar', + 'blkio_config': { + 'weight': 300, + 'weight_device': [ + {'path': '/dev/sda1', 'weight': 200} + ], + 'device_read_iops': [ + {'path': '/dev/sda1', 'rate': 300} + ], + 'device_write_iops': [ + {'path': '/dev/sda1', 'rate': 1000} + ] + } + } + + override = { + 'blkio_config': { + 'weight': 450, + 'weight_device': [ + {'path': '/dev/sda2', 'weight': 400} + ], + 'device_read_iops': [ + {'path': '/dev/sda1', 'rate': 2000} + ], + 'device_read_bps': [ + {'path': '/dev/sda1', 'rate': 1024} + ] + } + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == { + 'image': 'bar', + 'blkio_config': { + 'weight': override['blkio_config']['weight'], + 'weight_device': ( + base['blkio_config']['weight_device'] + + override['blkio_config']['weight_device'] + ), + 'device_read_iops': override['blkio_config']['device_read_iops'], + 'device_read_bps': override['blkio_config']['device_read_bps'], + 'device_write_iops': base['blkio_config']['device_write_iops'] + } + } + + def test_merge_extra_hosts(self): + base = { + 'image': 'bar', + 'extra_hosts': { + 'foo': '1.2.3.4', + } + } + + override = { + 'extra_hosts': ['bar:5.6.7.8', 'foo:127.0.0.1'] + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual['extra_hosts'] == { + 'foo': '127.0.0.1', + 'bar': '5.6.7.8', + } + + def test_merge_healthcheck_config(self): + base = { + 'image': 'bar', + 'healthcheck': { + 'start_period': 1000, + 'interval': 3000, + 'test': ['true'] + } + } + + override = { + 'healthcheck': { + 'interval': 5000, + 'timeout': 10000, + 'test': ['echo', 'OK'], + } + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual['healthcheck'] == { + 'start_period': base['healthcheck']['start_period'], + 'test': override['healthcheck']['test'], + 'interval': override['healthcheck']['interval'], + 'timeout': override['healthcheck']['timeout'], + } + + def test_merge_healthcheck_override_disables(self): + base = { + 'image': 'bar', + 'healthcheck': { + 'start_period': 1000, + 'interval': 3000, + 'timeout': 2000, + 'retries': 3, + 'test': ['true'] + } + } + + override = { + 'healthcheck': { + 'disabled': True + } + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual['healthcheck'] == {'disabled': True} + + def test_merge_healthcheck_override_enables(self): + base = { + 'image': 'bar', + 'healthcheck': { + 'disabled': True + } + } + + override = { + 'healthcheck': { + 'disabled': False, + 'start_period': 1000, + 'interval': 3000, + 'timeout': 2000, + 'retries': 3, + 'test': ['true'] + } + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual['healthcheck'] == override['healthcheck'] + + def test_merge_device_cgroup_rules(self): + base = { + 'image': 'bar', + 'device_cgroup_rules': ['c 7:128 rwm', 'x 3:244 rw'] + } + + override = { + 'device_cgroup_rules': ['c 7:128 rwm', 'f 0:128 n'] + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert sorted(actual['device_cgroup_rules']) == sorted( + ['c 7:128 rwm', 'x 3:244 rw', 'f 0:128 n'] + ) + + def test_merge_isolation(self): + base = { + 'image': 'bar', + 'isolation': 'default', + } + + override = { + 'isolation': 'hyperv', + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual == { + 'image': 'bar', + 'isolation': 'hyperv', + } + + def test_merge_storage_opt(self): + base = { + 'image': 'bar', + 'storage_opt': { + 'size': '1G', + 'readonly': 'false', + } + } + + override = { + 'storage_opt': { + 'size': '2G', + 'encryption': 'aes', + } + } + + actual = config.merge_service_dicts(base, override, VERSION) + assert actual['storage_opt'] == { + 'size': '2G', + 'readonly': 'false', + 'encryption': 'aes', + } + + def test_external_volume_config(self): + config_details = build_config_details({ + 'version': '2', + 'services': { + 'bogus': {'image': 'busybox'} + }, + 'volumes': { + 'ext': {'external': True}, + 'ext2': {'external': {'name': 'aliased'}} + } + }) + config_result = config.load(config_details) + volumes = config_result.volumes + assert 'ext' in volumes + assert volumes['ext']['external'] is True + assert 'ext2' in volumes + assert volumes['ext2']['external']['name'] == 'aliased' + + def test_external_volume_invalid_config(self): + config_details = build_config_details({ + 'version': '2', + 'services': { + 'bogus': {'image': 'busybox'} + }, + 'volumes': { + 'ext': {'external': True, 'driver': 'foo'} + } + }) + with pytest.raises(ConfigurationError): + config.load(config_details) + + def test_depends_on_orders_services(self): + config_details = build_config_details({ + 'version': '2', + 'services': { + 'one': {'image': 'busybox', 'depends_on': ['three', 'two']}, + 'two': {'image': 'busybox', 'depends_on': ['three']}, + 'three': {'image': 'busybox'}, + }, + }) + actual = config.load(config_details) + assert ( + [service['name'] for service in actual.services] == + ['three', 'two', 'one'] + ) + + def test_depends_on_unknown_service_errors(self): + config_details = build_config_details({ + 'version': '2', + 'services': { + 'one': {'image': 'busybox', 'depends_on': ['three']}, + }, + }) + with pytest.raises(ConfigurationError) as exc: + config.load(config_details) + assert "Service 'one' depends on service 'three'" in exc.exconly() + + def test_linked_service_is_undefined(self): + with pytest.raises(ConfigurationError): + config.load( + build_config_details({ + 'version': '2', + 'services': { + 'web': {'image': 'busybox', 'links': ['db:db']}, + }, + }) + ) + + def test_load_dockerfile_without_context(self): + config_details = build_config_details({ + 'version': '2', + 'services': { + 'one': {'build': {'dockerfile': 'Dockerfile.foo'}}, + }, + }) + with pytest.raises(ConfigurationError) as exc: + config.load(config_details) + assert 'has neither an image nor a build context' in exc.exconly() + + def test_load_secrets(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '3.1', + 'services': { + 'web': { + 'image': 'example/web', + 'secrets': [ + 'one', + { + 'source': 'source', + 'target': 'target', + 'uid': '100', + 'gid': '200', + 'mode': 0o777, + }, + ], + }, + }, + 'secrets': { + 'one': {'file': 'secret.txt'}, + }, + }) + details = config.ConfigDetails('.', [base_file]) + service_dicts = config.load(details).services + expected = [ + { + 'name': 'web', + 'image': 'example/web', + 'secrets': [ + types.ServiceSecret('one', None, None, None, None, None), + types.ServiceSecret('source', 'target', '100', '200', 0o777, None), + ], + }, + ] + assert service_sort(service_dicts) == service_sort(expected) + + def test_load_secrets_multi_file(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '3.1', + 'services': { + 'web': { + 'image': 'example/web', + 'secrets': ['one'], + }, + }, + 'secrets': { + 'one': {'file': 'secret.txt'}, + }, + }) + override_file = config.ConfigFile( + 'base.yaml', + { + 'version': '3.1', + 'services': { + 'web': { + 'secrets': [ + { + 'source': 'source', + 'target': 'target', + 'uid': '100', + 'gid': '200', + 'mode': 0o777, + }, + ], + }, + }, + }) + details = config.ConfigDetails('.', [base_file, override_file]) + service_dicts = config.load(details).services + expected = [ + { + 'name': 'web', + 'image': 'example/web', + 'secrets': [ + types.ServiceSecret('one', None, None, None, None, None), + types.ServiceSecret('source', 'target', '100', '200', 0o777, None), + ], + }, + ] + assert service_sort(service_dicts) == service_sort(expected) + + def test_load_configs(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '3.3', + 'services': { + 'web': { + 'image': 'example/web', + 'configs': [ + 'one', + { + 'source': 'source', + 'target': 'target', + 'uid': '100', + 'gid': '200', + 'mode': 0o777, + }, + ], + }, + }, + 'configs': { + 'one': {'file': 'secret.txt'}, + }, + }) + details = config.ConfigDetails('.', [base_file]) + service_dicts = config.load(details).services + expected = [ + { + 'name': 'web', + 'image': 'example/web', + 'configs': [ + types.ServiceConfig('one', None, None, None, None, None), + types.ServiceConfig('source', 'target', '100', '200', 0o777, None), + ], + }, + ] + assert service_sort(service_dicts) == service_sort(expected) + + def test_load_configs_multi_file(self): + base_file = config.ConfigFile( + 'base.yaml', + { + 'version': '3.3', + 'services': { + 'web': { + 'image': 'example/web', + 'configs': ['one'], + }, + }, + 'configs': { + 'one': {'file': 'secret.txt'}, + }, + }) + override_file = config.ConfigFile( + 'base.yaml', + { + 'version': '3.3', + 'services': { + 'web': { + 'configs': [ + { + 'source': 'source', + 'target': 'target', + 'uid': '100', + 'gid': '200', + 'mode': 0o777, + }, + ], + }, + }, + }) + details = config.ConfigDetails('.', [base_file, override_file]) + service_dicts = config.load(details).services + expected = [ + { + 'name': 'web', + 'image': 'example/web', + 'configs': [ + types.ServiceConfig('one', None, None, None, None, None), + types.ServiceConfig('source', 'target', '100', '200', 0o777, None), + ], + }, + ] + assert service_sort(service_dicts) == service_sort(expected) + + def test_config_convertible_label_types(self): + config_details = build_config_details( + { + 'version': '3.5', + 'services': { + 'web': { + 'build': { + 'labels': {'testbuild': True}, + 'context': os.getcwd() + }, + 'labels': { + "key": 12345 + } + }, + }, + 'networks': { + 'foo': { + 'labels': {'network.ips.max': 1023} + } + }, + 'volumes': { + 'foo': { + 'labels': {'volume.is_readonly': False} + } + }, + 'secrets': { + 'foo': { + 'labels': {'secret.data.expires': 1546282120} + } + }, + 'configs': { + 'foo': { + 'labels': {'config.data.correction.value': -0.1412} + } + } + } + ) + loaded_config = config.load(config_details) + + assert loaded_config.services[0]['build']['labels'] == {'testbuild': 'True'} + assert loaded_config.services[0]['labels'] == {'key': '12345'} + assert loaded_config.networks['foo']['labels']['network.ips.max'] == '1023' + assert loaded_config.volumes['foo']['labels']['volume.is_readonly'] == 'False' + assert loaded_config.secrets['foo']['labels']['secret.data.expires'] == '1546282120' + assert loaded_config.configs['foo']['labels']['config.data.correction.value'] == '-0.1412' + + def test_config_invalid_label_types(self): + config_details = build_config_details({ + 'version': '2.3', + 'volumes': { + 'foo': {'labels': [1, 2, 3]} + } + }) + with pytest.raises(ConfigurationError): + config.load(config_details) + + def test_service_volume_invalid_config(self): + config_details = build_config_details( + { + 'version': '3.2', + 'services': { + 'web': { + 'build': { + 'context': '.', + 'args': None, + }, + 'volumes': [ + { + "type": "volume", + "source": "/data", + "garbage": { + "and": "error" + } + } + ] + } + } + } + ) + with pytest.raises(ConfigurationError) as exc: + config.load(config_details) + + assert "services.web.volumes contains unsupported option: 'garbage'" in exc.exconly() + + def test_config_valid_service_label_validation(self): + config_details = build_config_details( + { + 'version': '3.5', + 'services': { + 'web': { + 'image': 'busybox', + 'labels': { + "key": "string" + } + }, + }, + } + ) + config.load(config_details) + + def test_config_duplicate_mount_points(self): + config1 = build_config_details( + { + 'version': '3.5', + 'services': { + 'web': { + 'image': 'busybox', + 'volumes': ['/tmp/foo:/tmp/foo', '/tmp/foo:/tmp/foo:rw'] + } + } + } + ) + + config2 = build_config_details( + { + 'version': '3.5', + 'services': { + 'web': { + 'image': 'busybox', + 'volumes': ['/x:/y', '/z:/y'] + } + } + } + ) + + with self.assertRaises(ConfigurationError) as e: + config.load(config1) + self.assertEqual(str(e.exception), 'Duplicate mount points: [%s]' % ( + ', '.join(['/tmp/foo:/tmp/foo:rw']*2))) + + with self.assertRaises(ConfigurationError) as e: + config.load(config2) + self.assertEqual(str(e.exception), 'Duplicate mount points: [%s]' % ( + ', '.join(['/x:/y:rw', '/z:/y:rw']))) + + +class NetworkModeTest(unittest.TestCase): + + def test_network_mode_standard(self): + config_data = config.load(build_config_details({ + 'version': '2', + 'services': { + 'web': { + 'image': 'busybox', + 'command': "top", + 'network_mode': 'bridge', + }, + }, + })) + + assert config_data.services[0]['network_mode'] == 'bridge' + + def test_network_mode_standard_v1(self): + config_data = config.load(build_config_details({ + 'web': { + 'image': 'busybox', + 'command': "top", + 'net': 'bridge', + }, + })) + + assert config_data.services[0]['network_mode'] == 'bridge' + assert 'net' not in config_data.services[0] + + def test_network_mode_container(self): + config_data = config.load(build_config_details({ + 'version': '2', + 'services': { + 'web': { + 'image': 'busybox', + 'command': "top", + 'network_mode': 'container:foo', + }, + }, + })) + + assert config_data.services[0]['network_mode'] == 'container:foo' + + def test_network_mode_container_v1(self): + config_data = config.load(build_config_details({ + 'web': { + 'image': 'busybox', + 'command': "top", + 'net': 'container:foo', + }, + })) + + assert config_data.services[0]['network_mode'] == 'container:foo' + + def test_network_mode_service(self): + config_data = config.load(build_config_details({ + 'version': '2', + 'services': { + 'web': { + 'image': 'busybox', + 'command': "top", + 'network_mode': 'service:foo', + }, + 'foo': { + 'image': 'busybox', + 'command': "top", + }, + }, + })) + + assert config_data.services[1]['network_mode'] == 'service:foo' + + def test_network_mode_service_v1(self): + config_data = config.load(build_config_details({ + 'web': { + 'image': 'busybox', + 'command': "top", + 'net': 'container:foo', + }, + 'foo': { + 'image': 'busybox', + 'command': "top", + }, + })) + + assert config_data.services[1]['network_mode'] == 'service:foo' + + def test_network_mode_service_nonexistent(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load(build_config_details({ + 'version': '2', + 'services': { + 'web': { + 'image': 'busybox', + 'command': "top", + 'network_mode': 'service:foo', + }, + }, + })) + + assert "service 'foo' which is undefined" in excinfo.exconly() + + def test_network_mode_plus_networks_is_invalid(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load(build_config_details({ + 'version': '2', + 'services': { + 'web': { + 'image': 'busybox', + 'command': "top", + 'network_mode': 'bridge', + 'networks': ['front'], + }, + }, + 'networks': { + 'front': None, + } + })) + + assert "'network_mode' and 'networks' cannot be combined" in excinfo.exconly() + + +class PortsTest(unittest.TestCase): + INVALID_PORTS_TYPES = [ + {"1": "8000"}, + False, + "8000", + 8000, + ] + + NON_UNIQUE_SINGLE_PORTS = [ + ["8000", "8000"], + ] + + INVALID_PORT_MAPPINGS = [ + ["8000-8004:8000-8002"], + ["4242:4242-4244"], + ] + + VALID_SINGLE_PORTS = [ + ["8000"], + ["8000/tcp"], + ["8000", "9000"], + [8000], + [8000, 9000], + ] + + VALID_PORT_MAPPINGS = [ + ["8000:8050"], + ["49153-49154:3002-3003"], + ] + + def test_config_invalid_ports_type_validation(self): + for invalid_ports in self.INVALID_PORTS_TYPES: + with pytest.raises(ConfigurationError) as exc: + self.check_config({'ports': invalid_ports}) + + assert "contains an invalid type" in exc.value.msg + + def test_config_non_unique_ports_validation(self): + for invalid_ports in self.NON_UNIQUE_SINGLE_PORTS: + with pytest.raises(ConfigurationError) as exc: + self.check_config({'ports': invalid_ports}) + + assert "non-unique" in exc.value.msg + + @pytest.mark.skip(reason="Validator is one_off (generic error)") + def test_config_invalid_ports_format_validation(self): + for invalid_ports in self.INVALID_PORT_MAPPINGS: + with pytest.raises(ConfigurationError) as exc: + self.check_config({'ports': invalid_ports}) + + assert "Port ranges don't match in length" in exc.value.msg + + def test_config_valid_ports_format_validation(self): + for valid_ports in self.VALID_SINGLE_PORTS + self.VALID_PORT_MAPPINGS: + self.check_config({'ports': valid_ports}) + + def test_config_invalid_expose_type_validation(self): + for invalid_expose in self.INVALID_PORTS_TYPES: + with pytest.raises(ConfigurationError) as exc: + self.check_config({'expose': invalid_expose}) + + assert "contains an invalid type" in exc.value.msg + + def test_config_non_unique_expose_validation(self): + for invalid_expose in self.NON_UNIQUE_SINGLE_PORTS: + with pytest.raises(ConfigurationError) as exc: + self.check_config({'expose': invalid_expose}) + + assert "non-unique" in exc.value.msg + + def test_config_invalid_expose_format_validation(self): + # Valid port mappings ARE NOT valid 'expose' entries + for invalid_expose in self.INVALID_PORT_MAPPINGS + self.VALID_PORT_MAPPINGS: + with pytest.raises(ConfigurationError) as exc: + self.check_config({'expose': invalid_expose}) + + assert "should be of the format" in exc.value.msg + + def test_config_valid_expose_format_validation(self): + # Valid single ports ARE valid 'expose' entries + for valid_expose in self.VALID_SINGLE_PORTS: + self.check_config({'expose': valid_expose}) + + def check_config(self, cfg): + config.load( + build_config_details({ + 'version': '2.3', + 'services': { + 'web': dict(image='busybox', **cfg) + }, + }, 'working_dir', 'filename.yml') + ) + + +class SubnetTest(unittest.TestCase): + INVALID_SUBNET_TYPES = [ + None, + False, + 10, + ] + + INVALID_SUBNET_MAPPINGS = [ + "", + "192.168.0.1/sdfsdfs", + "192.168.0.1/", + "192.168.0.1/33", + "192.168.0.1/01", + "192.168.0.1", + "fe80:0000:0000:0000:0204:61ff:fe9d:f156/sdfsdfs", + "fe80:0000:0000:0000:0204:61ff:fe9d:f156/", + "fe80:0000:0000:0000:0204:61ff:fe9d:f156/129", + "fe80:0000:0000:0000:0204:61ff:fe9d:f156/01", + "fe80:0000:0000:0000:0204:61ff:fe9d:f156", + "ge80:0000:0000:0000:0204:61ff:fe9d:f156/128", + "192.168.0.1/31/31", + ] + + VALID_SUBNET_MAPPINGS = [ + "192.168.0.1/0", + "192.168.0.1/32", + "fe80:0000:0000:0000:0204:61ff:fe9d:f156/0", + "fe80:0000:0000:0000:0204:61ff:fe9d:f156/128", + "1:2:3:4:5:6:7:8/0", + "1::/0", + "1:2:3:4:5:6:7::/0", + "1::8/0", + "1:2:3:4:5:6::8/0", + "::/0", + "::8/0", + "::2:3:4:5:6:7:8/0", + "fe80::7:8%eth0/0", + "fe80::7:8%1/0", + "::255.255.255.255/0", + "::ffff:255.255.255.255/0", + "::ffff:0:255.255.255.255/0", + "2001:db8:3:4::192.0.2.33/0", + "64:ff9b::192.0.2.33/0", + ] + + def test_config_invalid_subnet_type_validation(self): + for invalid_subnet in self.INVALID_SUBNET_TYPES: + with pytest.raises(ConfigurationError) as exc: + self.check_config(invalid_subnet) + + assert "contains an invalid type" in exc.value.msg + + def test_config_invalid_subnet_format_validation(self): + for invalid_subnet in self.INVALID_SUBNET_MAPPINGS: + with pytest.raises(ConfigurationError) as exc: + self.check_config(invalid_subnet) + + assert "should use the CIDR format" in exc.value.msg + + def test_config_valid_subnet_format_validation(self): + for valid_subnet in self.VALID_SUBNET_MAPPINGS: + self.check_config(valid_subnet) + + def check_config(self, subnet): + config.load( + build_config_details({ + 'version': '3.5', + 'services': { + 'web': { + 'image': 'busybox' + } + }, + 'networks': { + 'default': { + 'ipam': { + 'config': [ + { + 'subnet': subnet + } + ], + 'driver': 'default' + } + } + } + }) + ) + + +class InterpolationTest(unittest.TestCase): + + @mock.patch.dict(os.environ) + def test_config_file_with_environment_file(self): + project_dir = 'tests/fixtures/default-env-file' + service_dicts = config.load( + config.find( + project_dir, None, Environment.from_env_file(project_dir) + ) + ).services + + assert service_dicts[0] == { + 'name': 'web', + 'image': 'alpine:latest', + 'ports': [ + types.ServicePort.parse('5643')[0], + types.ServicePort.parse('9999')[0] + ], + 'command': 'true' + } + + @mock.patch.dict(os.environ) + def test_config_file_with_options_environment_file(self): + project_dir = 'tests/fixtures/default-env-file' + service_dicts = config.load( + config.find( + project_dir, None, Environment.from_env_file(project_dir, '.env2') + ) + ).services + + assert service_dicts[0] == { + 'name': 'web', + 'image': 'alpine:latest', + 'ports': [ + types.ServicePort.parse('5644')[0], + types.ServicePort.parse('9998')[0] + ], + 'command': 'false' + } + + @mock.patch.dict(os.environ) + def test_config_file_with_environment_variable(self): + project_dir = 'tests/fixtures/environment-interpolation' + os.environ.update( + IMAGE="busybox", + HOST_PORT="80", + LABEL_VALUE="myvalue", + ) + + service_dicts = config.load( + config.find( + project_dir, None, Environment.from_env_file(project_dir) + ) + ).services + + assert service_dicts == [ + { + 'name': 'web', + 'image': 'busybox', + 'ports': types.ServicePort.parse('80:8000'), + 'labels': {'mylabel': 'myvalue'}, + 'hostname': 'host-', + 'command': '${ESCAPED}', + } + ] + + @mock.patch.dict(os.environ) + def test_config_file_with_environment_variable_with_defaults(self): + project_dir = 'tests/fixtures/environment-interpolation-with-defaults' + os.environ.update( + IMAGE="busybox", + ) + + service_dicts = config.load( + config.find( + project_dir, None, Environment.from_env_file(project_dir) + ) + ).services + + assert service_dicts == [ + { + 'name': 'web', + 'image': 'busybox', + 'ports': types.ServicePort.parse('80:8000'), + 'hostname': 'host-', + } + ] + + @mock.patch.dict(os.environ) + def test_unset_variable_produces_warning(self): + os.environ.pop('FOO', None) + os.environ.pop('BAR', None) + config_details = build_config_details( + { + 'web': { + 'image': '${FOO}', + 'command': '${BAR}', + 'container_name': '${BAR}', + }, + }, + '.', + None, + ) + + with mock.patch('compose.config.environment.log') as log: + config.load(config_details) + + assert 2 == log.warning.call_count + warnings = sorted(args[0][0] for args in log.warning.call_args_list) + assert 'BAR' in warnings[0] + assert 'FOO' in warnings[1] + + @pytest.mark.skip(reason='compatibility mode was removed internally') + def test_compatibility_mode_warnings(self): + config_details = build_config_details({ + 'version': '3.5', + 'services': { + 'web': { + 'deploy': { + 'labels': ['abc=def'], + 'endpoint_mode': 'dnsrr', + 'update_config': {'max_failure_ratio': 0.4}, + 'placement': {'constraints': ['node.id==deadbeef']}, + 'resources': { + 'reservations': {'cpus': '0.2'} + }, + 'restart_policy': { + 'delay': '2s', + 'window': '12s' + } + }, + 'image': 'busybox' + } + } + }) + + with mock.patch('compose.config.config.log') as log: + config.load(config_details, compatibility=True) + + assert log.warning.call_count == 1 + warn_message = log.warning.call_args[0][0] + assert warn_message.startswith( + 'The following deploy sub-keys are not supported in compatibility mode' + ) + assert 'labels' in warn_message + assert 'endpoint_mode' in warn_message + assert 'update_config' in warn_message + assert 'resources.reservations.cpus' in warn_message + assert 'restart_policy.delay' in warn_message + assert 'restart_policy.window' in warn_message + + @pytest.mark.skip(reason='compatibility mode was removed internally') + def test_compatibility_mode_load(self): + config_details = build_config_details({ + 'version': '3.5', + 'services': { + 'foo': { + 'image': 'alpine:3.10.1', + 'deploy': { + 'replicas': 3, + 'restart_policy': { + 'condition': 'any', + 'max_attempts': 7, + }, + 'resources': { + 'limits': {'memory': '300M', 'cpus': '0.7'}, + 'reservations': {'memory': '100M'}, + }, + }, + 'credential_spec': { + 'file': 'spec.json' + }, + }, + }, + }) + + with mock.patch('compose.config.config.log') as log: + cfg = config.load(config_details, compatibility=True) + + assert log.warning.call_count == 0 + + service_dict = cfg.services[0] + assert service_dict == { + 'image': 'alpine:3.10.1', + 'scale': 3, + 'restart': {'MaximumRetryCount': 7, 'Name': 'always'}, + 'mem_limit': '300M', + 'mem_reservation': '100M', + 'cpus': 0.7, + 'name': 'foo', + 'security_opt': ['credentialspec=file://spec.json'], + } + + @mock.patch.dict(os.environ) + def test_invalid_interpolation(self): + with pytest.raises(config.ConfigurationError) as cm: + config.load( + build_config_details( + {'web': {'image': '${'}}, + 'working_dir', + 'filename.yml' + ) + ) + + assert 'Invalid' in cm.value.msg + assert 'for "image" option' in cm.value.msg + assert 'in service "web"' in cm.value.msg + assert '"${"' in cm.value.msg + + @mock.patch.dict(os.environ) + def test_interpolation_secrets_section(self): + os.environ['FOO'] = 'baz.bar' + config_dict = config.load(build_config_details({ + 'version': '3.1', + 'secrets': { + 'secretdata': { + 'external': {'name': '$FOO'} + } + } + })) + assert config_dict.secrets == { + 'secretdata': { + 'external': {'name': 'baz.bar'}, + 'name': 'baz.bar' + } + } + + @mock.patch.dict(os.environ) + def test_interpolation_configs_section(self): + os.environ['FOO'] = 'baz.bar' + config_dict = config.load(build_config_details({ + 'version': '3.3', + 'configs': { + 'configdata': { + 'external': {'name': '$FOO'} + } + } + })) + assert config_dict.configs == { + 'configdata': { + 'external': {'name': 'baz.bar'}, + 'name': 'baz.bar' + } + } + + +class VolumeConfigTest(unittest.TestCase): + + def test_no_binding(self): + d = make_service_dict('foo', {'build': '.', 'volumes': ['/data']}, working_dir='.') + assert d['volumes'] == ['/data'] + + @mock.patch.dict(os.environ) + def test_volume_binding_with_environment_variable(self): + os.environ['VOLUME_PATH'] = '/host/path' + + d = config.load( + build_config_details( + {'foo': {'build': '.', 'volumes': ['${VOLUME_PATH}:/container/path']}}, + '.', + None, + ) + ).services[0] + assert d['volumes'] == [VolumeSpec.parse('/host/path:/container/path')] + + @pytest.mark.skipif(IS_WINDOWS_PLATFORM, reason='posix paths') + def test_volumes_order_is_preserved(self): + volumes = ['/{0}:/{0}'.format(i) for i in range(0, 6)] + shuffle(volumes) + cfg = make_service_dict('foo', {'build': '.', 'volumes': volumes}) + assert cfg['volumes'] == volumes + + @pytest.mark.skipif(IS_WINDOWS_PLATFORM, reason='posix paths') + @mock.patch.dict(os.environ) + def test_volume_binding_with_home(self): + os.environ['HOME'] = '/home/user' + d = make_service_dict('foo', {'build': '.', 'volumes': ['~:/container/path']}, working_dir='.') + assert d['volumes'] == ['/home/user:/container/path'] + + def test_name_does_not_expand(self): + d = make_service_dict('foo', {'build': '.', 'volumes': ['mydatavolume:/data']}, working_dir='.') + assert d['volumes'] == ['mydatavolume:/data'] + + def test_absolute_posix_path_does_not_expand(self): + d = make_service_dict('foo', {'build': '.', 'volumes': ['/var/lib/data:/data']}, working_dir='.') + assert d['volumes'] == ['/var/lib/data:/data'] + + def test_absolute_windows_path_does_not_expand(self): + d = make_service_dict('foo', {'build': '.', 'volumes': ['c:\\data:/data']}, working_dir='.') + assert d['volumes'] == ['c:\\data:/data'] + + @pytest.mark.skipif(IS_WINDOWS_PLATFORM, reason='posix paths') + def test_relative_path_does_expand_posix(self): + d = make_service_dict( + 'foo', + {'build': '.', 'volumes': ['./data:/data']}, + working_dir='/home/me/myproject') + assert d['volumes'] == ['/home/me/myproject/data:/data'] + + d = make_service_dict( + 'foo', + {'build': '.', 'volumes': ['.:/data']}, + working_dir='/home/me/myproject') + assert d['volumes'] == ['/home/me/myproject:/data'] + + d = make_service_dict( + 'foo', + {'build': '.', 'volumes': ['../otherproject:/data']}, + working_dir='/home/me/myproject') + assert d['volumes'] == ['/home/me/otherproject:/data'] + + @pytest.mark.skipif(not IS_WINDOWS_PLATFORM, reason='windows paths') + def test_relative_path_does_expand_windows(self): + d = make_service_dict( + 'foo', + {'build': '.', 'volumes': ['./data:/data']}, + working_dir='c:\\Users\\me\\myproject') + assert d['volumes'] == ['c:\\Users\\me\\myproject\\data:/data'] + + d = make_service_dict( + 'foo', + {'build': '.', 'volumes': ['.:/data']}, + working_dir='c:\\Users\\me\\myproject') + assert d['volumes'] == ['c:\\Users\\me\\myproject:/data'] + + d = make_service_dict( + 'foo', + {'build': '.', 'volumes': ['../otherproject:/data']}, + working_dir='c:\\Users\\me\\myproject') + assert d['volumes'] == ['c:\\Users\\me\\otherproject:/data'] + + @mock.patch.dict(os.environ) + def test_home_directory_with_driver_does_not_expand(self): + os.environ['NAME'] = 'surprise!' + d = make_service_dict('foo', { + 'build': '.', + 'volumes': ['~:/data'], + 'volume_driver': 'foodriver', + }, working_dir='.') + assert d['volumes'] == ['~:/data'] + + def test_volume_path_with_non_ascii_directory(self): + volume = '/Füü/data:/data' + container_path = config.resolve_volume_path(".", volume) + assert container_path == volume + + +class MergePathMappingTest: + config_name = "" + + def test_empty(self): + service_dict = config.merge_service_dicts({}, {}, DEFAULT_VERSION) + assert self.config_name not in service_dict + + def test_no_override(self): + service_dict = config.merge_service_dicts( + {self.config_name: ['/foo:/code', '/data']}, + {}, + DEFAULT_VERSION) + assert set(service_dict[self.config_name]) == {'/foo:/code', '/data'} + + def test_no_base(self): + service_dict = config.merge_service_dicts( + {}, + {self.config_name: ['/bar:/code']}, + DEFAULT_VERSION) + assert set(service_dict[self.config_name]) == {'/bar:/code'} + + def test_override_explicit_path(self): + service_dict = config.merge_service_dicts( + {self.config_name: ['/foo:/code', '/data']}, + {self.config_name: ['/bar:/code']}, + DEFAULT_VERSION) + assert set(service_dict[self.config_name]) == {'/bar:/code', '/data'} + + def test_add_explicit_path(self): + service_dict = config.merge_service_dicts( + {self.config_name: ['/foo:/code', '/data']}, + {self.config_name: ['/bar:/code', '/quux:/data']}, + DEFAULT_VERSION) + assert set(service_dict[self.config_name]) == {'/bar:/code', '/quux:/data'} + + def test_remove_explicit_path(self): + service_dict = config.merge_service_dicts( + {self.config_name: ['/foo:/code', '/quux:/data']}, + {self.config_name: ['/bar:/code', '/data']}, + DEFAULT_VERSION) + assert set(service_dict[self.config_name]) == {'/bar:/code', '/data'} + + +class MergeVolumesTest(unittest.TestCase, MergePathMappingTest): + config_name = 'volumes' + + +class MergeDevicesTest(unittest.TestCase, MergePathMappingTest): + config_name = 'devices' + + +class BuildOrImageMergeTest(unittest.TestCase): + + def test_merge_build_or_image_no_override(self): + assert config.merge_service_dicts({'build': '.'}, {}, V1) == {'build': '.'} + + assert config.merge_service_dicts({'image': 'redis'}, {}, V1) == {'image': 'redis'} + + def test_merge_build_or_image_override_with_same(self): + assert config.merge_service_dicts({'build': '.'}, {'build': './web'}, V1) == {'build': './web'} + + assert config.merge_service_dicts({'image': 'redis'}, {'image': 'postgres'}, V1) == { + 'image': 'postgres' + } + + def test_merge_build_or_image_override_with_other(self): + assert config.merge_service_dicts({'build': '.'}, {'image': 'redis'}, V1) == { + 'image': 'redis' + } + + assert config.merge_service_dicts({'image': 'redis'}, {'build': '.'}, V1) == {'build': '.'} + + +class MergeListsTest: + config_name = "" + base_config = [] + override_config = [] + + def merged_config(self): + return set(self.base_config) | set(self.override_config) + + def test_empty(self): + assert self.config_name not in config.merge_service_dicts({}, {}, DEFAULT_VERSION) + + def test_no_override(self): + service_dict = config.merge_service_dicts( + {self.config_name: self.base_config}, + {}, + DEFAULT_VERSION) + assert set(service_dict[self.config_name]) == set(self.base_config) + + def test_no_base(self): + service_dict = config.merge_service_dicts( + {}, + {self.config_name: self.base_config}, + DEFAULT_VERSION) + assert set(service_dict[self.config_name]) == set(self.base_config) + + def test_add_item(self): + service_dict = config.merge_service_dicts( + {self.config_name: self.base_config}, + {self.config_name: self.override_config}, + DEFAULT_VERSION) + assert set(service_dict[self.config_name]) == set(self.merged_config()) + + +class MergePortsTest(unittest.TestCase, MergeListsTest): + config_name = 'ports' + base_config = ['10:8000', '9000'] + override_config = ['20:8000'] + + def merged_config(self): + return self.convert(self.base_config) | self.convert(self.override_config) + + def convert(self, port_config): + return set(config.merge_service_dicts( + {self.config_name: port_config}, + {self.config_name: []}, + DEFAULT_VERSION + )[self.config_name]) + + def test_duplicate_port_mappings(self): + service_dict = config.merge_service_dicts( + {self.config_name: self.base_config}, + {self.config_name: self.base_config}, + DEFAULT_VERSION + ) + assert set(service_dict[self.config_name]) == self.convert(self.base_config) + + def test_no_override(self): + service_dict = config.merge_service_dicts( + {self.config_name: self.base_config}, + {}, + DEFAULT_VERSION) + assert set(service_dict[self.config_name]) == self.convert(self.base_config) + + def test_no_base(self): + service_dict = config.merge_service_dicts( + {}, + {self.config_name: self.base_config}, + DEFAULT_VERSION) + assert set(service_dict[self.config_name]) == self.convert(self.base_config) + + +class MergeNetworksTest(unittest.TestCase, MergeListsTest): + config_name = 'networks' + base_config = {'default': {'aliases': ['foo.bar', 'foo.baz']}} + override_config = {'default': {'ipv4_address': '123.234.123.234'}} + + def test_no_network_overrides(self): + service_dict = config.merge_service_dicts( + {self.config_name: self.base_config}, + {self.config_name: self.override_config}, + DEFAULT_VERSION) + assert service_dict[self.config_name] == { + 'default': { + 'aliases': ['foo.bar', 'foo.baz'], + 'ipv4_address': '123.234.123.234' + } + } + + def test_network_has_none_value(self): + service_dict = config.merge_service_dicts( + {self.config_name: { + 'default': None + }}, + {self.config_name: { + 'default': { + 'aliases': [] + } + }}, + DEFAULT_VERSION) + + assert service_dict[self.config_name] == { + 'default': { + 'aliases': [] + } + } + + def test_all_properties(self): + service_dict = config.merge_service_dicts( + {self.config_name: { + 'default': { + 'aliases': ['foo.bar', 'foo.baz'], + 'link_local_ips': ['192.168.1.10', '192.168.1.11'], + 'ipv4_address': '111.111.111.111', + 'ipv6_address': 'FE80:CD00:0000:0CDE:1257:0000:211E:729C-first' + } + }}, + {self.config_name: { + 'default': { + 'aliases': ['foo.baz', 'foo.baz2'], + 'link_local_ips': ['192.168.1.11', '192.168.1.12'], + 'ipv4_address': '123.234.123.234', + 'ipv6_address': 'FE80:CD00:0000:0CDE:1257:0000:211E:729C-second' + } + }}, + DEFAULT_VERSION) + + assert service_dict[self.config_name] == { + 'default': { + 'aliases': ['foo.bar', 'foo.baz', 'foo.baz2'], + 'link_local_ips': ['192.168.1.10', '192.168.1.11', '192.168.1.12'], + 'ipv4_address': '123.234.123.234', + 'ipv6_address': 'FE80:CD00:0000:0CDE:1257:0000:211E:729C-second' + } + } + + def test_no_network_name_overrides(self): + service_dict = config.merge_service_dicts( + { + self.config_name: { + 'default': { + 'aliases': ['foo.bar', 'foo.baz'], + 'ipv4_address': '123.234.123.234' + } + } + }, + { + self.config_name: { + 'another_network': { + 'ipv4_address': '123.234.123.234' + } + } + }, + DEFAULT_VERSION) + assert service_dict[self.config_name] == { + 'default': { + 'aliases': ['foo.bar', 'foo.baz'], + 'ipv4_address': '123.234.123.234' + }, + 'another_network': { + 'ipv4_address': '123.234.123.234' + } + } + + +class MergeStringsOrListsTest(unittest.TestCase): + + def test_no_override(self): + service_dict = config.merge_service_dicts( + {'dns': '8.8.8.8'}, + {}, + DEFAULT_VERSION) + assert set(service_dict['dns']) == {'8.8.8.8'} + + def test_no_base(self): + service_dict = config.merge_service_dicts( + {}, + {'dns': '8.8.8.8'}, + DEFAULT_VERSION) + assert set(service_dict['dns']) == {'8.8.8.8'} + + def test_add_string(self): + service_dict = config.merge_service_dicts( + {'dns': ['8.8.8.8']}, + {'dns': '9.9.9.9'}, + DEFAULT_VERSION) + assert set(service_dict['dns']) == {'8.8.8.8', '9.9.9.9'} + + def test_add_list(self): + service_dict = config.merge_service_dicts( + {'dns': '8.8.8.8'}, + {'dns': ['9.9.9.9']}, + DEFAULT_VERSION) + assert set(service_dict['dns']) == {'8.8.8.8', '9.9.9.9'} + + +class MergeLabelsTest(unittest.TestCase): + + def test_empty(self): + assert 'labels' not in config.merge_service_dicts({}, {}, DEFAULT_VERSION) + + def test_no_override(self): + service_dict = config.merge_service_dicts( + make_service_dict('foo', {'build': '.', 'labels': ['foo=1', 'bar']}, 'tests/'), + make_service_dict('foo', {'build': '.'}, 'tests/'), + DEFAULT_VERSION) + assert service_dict['labels'] == {'foo': '1', 'bar': ''} + + def test_no_base(self): + service_dict = config.merge_service_dicts( + make_service_dict('foo', {'build': '.'}, 'tests/'), + make_service_dict('foo', {'build': '.', 'labels': ['foo=2']}, 'tests/'), + DEFAULT_VERSION) + assert service_dict['labels'] == {'foo': '2'} + + def test_override_explicit_value(self): + service_dict = config.merge_service_dicts( + make_service_dict('foo', {'build': '.', 'labels': ['foo=1', 'bar']}, 'tests/'), + make_service_dict('foo', {'build': '.', 'labels': ['foo=2']}, 'tests/'), + DEFAULT_VERSION) + assert service_dict['labels'] == {'foo': '2', 'bar': ''} + + def test_add_explicit_value(self): + service_dict = config.merge_service_dicts( + make_service_dict('foo', {'build': '.', 'labels': ['foo=1', 'bar']}, 'tests/'), + make_service_dict('foo', {'build': '.', 'labels': ['bar=2']}, 'tests/'), + DEFAULT_VERSION) + assert service_dict['labels'] == {'foo': '1', 'bar': '2'} + + def test_remove_explicit_value(self): + service_dict = config.merge_service_dicts( + make_service_dict('foo', {'build': '.', 'labels': ['foo=1', 'bar=2']}, 'tests/'), + make_service_dict('foo', {'build': '.', 'labels': ['bar']}, 'tests/'), + DEFAULT_VERSION) + assert service_dict['labels'] == {'foo': '1', 'bar': ''} + + +class MergeBuildTest(unittest.TestCase): + def test_full(self): + base = { + 'context': '.', + 'dockerfile': 'Dockerfile', + 'args': { + 'x': '1', + 'y': '2', + }, + 'cache_from': ['ubuntu'], + 'labels': ['com.docker.compose.test=true'] + } + + override = { + 'context': './prod', + 'dockerfile': 'Dockerfile.prod', + 'args': ['x=12'], + 'cache_from': ['debian'], + 'labels': { + 'com.docker.compose.test': 'false', + 'com.docker.compose.prod': 'true', + } + } + + result = config.merge_build(None, {'build': base}, {'build': override}) + assert result['context'] == override['context'] + assert result['dockerfile'] == override['dockerfile'] + assert result['args'] == {'x': '12', 'y': '2'} + assert set(result['cache_from']) == {'ubuntu', 'debian'} + assert result['labels'] == override['labels'] + + def test_empty_override(self): + base = { + 'context': '.', + 'dockerfile': 'Dockerfile', + 'args': { + 'x': '1', + 'y': '2', + }, + 'cache_from': ['ubuntu'], + 'labels': { + 'com.docker.compose.test': 'true' + } + } + + override = {} + + result = config.merge_build(None, {'build': base}, {'build': override}) + assert result == base + + def test_empty_base(self): + base = {} + + override = { + 'context': './prod', + 'dockerfile': 'Dockerfile.prod', + 'args': {'x': '12'}, + 'cache_from': ['debian'], + 'labels': { + 'com.docker.compose.test': 'false', + 'com.docker.compose.prod': 'true', + } + } + + result = config.merge_build(None, {'build': base}, {'build': override}) + assert result == override + + +class MemoryOptionsTest(unittest.TestCase): + + def test_validation_fails_with_just_memswap_limit(self): + """ + When you set a 'memswap_limit' it is invalid config unless you also set + a mem_limit + """ + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'foo': {'image': 'busybox', 'memswap_limit': 2000000}, + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert "foo.memswap_limit is invalid: when defining " \ + "'memswap_limit' you must set 'mem_limit' as well" \ + in excinfo.exconly() + + def test_validation_with_correct_memswap_values(self): + service_dict = config.load( + build_config_details( + {'foo': {'image': 'busybox', 'mem_limit': 1000000, 'memswap_limit': 2000000}}, + 'tests/fixtures/extends', + 'common.yml' + ) + ).services + assert service_dict[0]['memswap_limit'] == 2000000 + + def test_memswap_can_be_a_string(self): + service_dict = config.load( + build_config_details( + {'foo': {'image': 'busybox', 'mem_limit': "1G", 'memswap_limit': "512M"}}, + 'tests/fixtures/extends', + 'common.yml' + ) + ).services + assert service_dict[0]['memswap_limit'] == "512M" + + +class EnvTest(unittest.TestCase): + + def test_parse_environment_as_list(self): + environment = [ + 'NORMAL=F1', + 'CONTAINS_EQUALS=F=2', + 'TRAILING_EQUALS=', + ] + assert config.parse_environment(environment) == { + 'NORMAL': 'F1', 'CONTAINS_EQUALS': 'F=2', 'TRAILING_EQUALS': '' + } + + def test_parse_environment_as_dict(self): + environment = { + 'NORMAL': 'F1', + 'CONTAINS_EQUALS': 'F=2', + 'TRAILING_EQUALS': None, + } + assert config.parse_environment(environment) == environment + + def test_parse_environment_invalid(self): + with pytest.raises(ConfigurationError): + config.parse_environment('a=b') + + def test_parse_environment_empty(self): + assert config.parse_environment(None) == {} + + @mock.patch.dict(os.environ) + def test_resolve_environment(self): + os.environ['FILE_DEF'] = 'E1' + os.environ['FILE_DEF_EMPTY'] = 'E2' + os.environ['ENV_DEF'] = 'E3' + + service_dict = { + 'build': '.', + 'environment': { + 'FILE_DEF': 'F1', + 'FILE_DEF_EMPTY': '', + 'ENV_DEF': None, + 'NO_DEF': None + }, + } + assert resolve_environment( + service_dict, Environment.from_env_file(None) + ) == {'FILE_DEF': 'F1', 'FILE_DEF_EMPTY': '', 'ENV_DEF': 'E3', 'NO_DEF': None} + + def test_resolve_environment_from_env_file(self): + assert resolve_environment({'env_file': ['tests/fixtures/env/one.env']}) == { + 'ONE': '2', 'TWO': '1', 'THREE': '3', 'FOO': 'bar' + } + + def test_environment_overrides_env_file(self): + assert resolve_environment({ + 'environment': {'FOO': 'baz'}, + 'env_file': ['tests/fixtures/env/one.env'], + }) == {'ONE': '2', 'TWO': '1', 'THREE': '3', 'FOO': 'baz'} + + def test_resolve_environment_with_multiple_env_files(self): + service_dict = { + 'env_file': [ + 'tests/fixtures/env/one.env', + 'tests/fixtures/env/two.env' + ] + } + assert resolve_environment(service_dict) == { + 'ONE': '2', 'TWO': '1', 'THREE': '3', 'FOO': 'baz', 'DOO': 'dah' + } + + def test_resolve_environment_nonexistent_file(self): + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details( + {'foo': {'image': 'example', 'env_file': 'nonexistent.env'}}, + working_dir='tests/fixtures/env')) + + assert 'Couldn\'t find env file' in exc.exconly() + assert 'nonexistent.env' in exc.exconly() + + @mock.patch.dict(os.environ) + def test_resolve_environment_from_env_file_with_empty_values(self): + os.environ['FILE_DEF'] = 'E1' + os.environ['FILE_DEF_EMPTY'] = 'E2' + os.environ['ENV_DEF'] = 'E3' + assert resolve_environment( + {'env_file': ['tests/fixtures/env/resolve.env']}, + Environment.from_env_file(None) + ) == { + 'FILE_DEF': 'bär', + 'FILE_DEF_EMPTY': '', + 'ENV_DEF': 'E3', + 'NO_DEF': None + } + + @mock.patch.dict(os.environ) + def test_resolve_build_args(self): + os.environ['env_arg'] = 'value2' + + build = { + 'context': '.', + 'args': { + 'arg1': 'value1', + 'empty_arg': '', + 'env_arg': None, + 'no_env': None + } + } + assert resolve_build_args(build['args'], Environment.from_env_file(build['context'])) == { + 'arg1': 'value1', 'empty_arg': '', 'env_arg': 'value2', 'no_env': None + } + + @pytest.mark.xfail(IS_WINDOWS_PLATFORM, reason='paths use slash') + @mock.patch.dict(os.environ) + def test_resolve_path(self): + os.environ['HOSTENV'] = '/tmp' + os.environ['CONTAINERENV'] = '/host/tmp' + + service_dict = config.load( + build_config_details( + {'services': { + 'foo': {'build': '.', 'volumes': ['$HOSTENV:$CONTAINERENV']}}}, + "tests/fixtures/env", + ) + ).services[0] + assert set(service_dict['volumes']) == {VolumeSpec.parse('/tmp:/host/tmp')} + + service_dict = config.load( + build_config_details( + {'services': { + 'foo': {'build': '.', 'volumes': ['/opt${HOSTENV}:/opt${CONTAINERENV}']}}}, + "tests/fixtures/env", + ) + ).services[0] + assert set(service_dict['volumes']) == {VolumeSpec.parse('/opt/tmp:/opt/host/tmp')} + + +def load_from_filename(filename, override_dir=None): + return config.load( + config.find('.', [filename], Environment.from_env_file('.'), override_dir=override_dir) + ).services + + +class ExtendsTest(unittest.TestCase): + + def test_extends(self): + service_dicts = load_from_filename('tests/fixtures/extends/docker-compose.yml') + + assert service_sort(service_dicts) == service_sort([ + { + 'name': 'mydb', + 'image': 'busybox', + 'command': 'top', + }, + { + 'name': 'myweb', + 'image': 'busybox', + 'command': 'top', + 'network_mode': 'bridge', + 'links': ['mydb:db'], + 'environment': { + "FOO": "1", + "BAR": "2", + "BAZ": "2", + }, + } + ]) + + def test_merging_env_labels_ulimits(self): + service_dicts = load_from_filename('tests/fixtures/extends/common-env-labels-ulimits.yml') + + assert service_sort(service_dicts) == service_sort([ + { + 'name': 'web', + 'image': 'busybox', + 'command': '/bin/true', + 'network_mode': 'host', + 'environment': { + "FOO": "2", + "BAR": "1", + "BAZ": "3", + }, + 'labels': {'label': 'one'}, + 'ulimits': {'nproc': 65535, 'memlock': {'soft': 1024, 'hard': 2048}} + } + ]) + + def test_nested(self): + service_dicts = load_from_filename('tests/fixtures/extends/nested.yml') + + assert service_dicts == [ + { + 'name': 'myweb', + 'image': 'busybox', + 'command': '/bin/true', + 'network_mode': 'host', + 'environment': { + "FOO": "2", + "BAR": "2", + }, + }, + ] + + def test_self_referencing_file(self): + """ + We specify a 'file' key that is the filename we're already in. + """ + service_dicts = load_from_filename('tests/fixtures/extends/specify-file-as-self.yml') + assert service_sort(service_dicts) == service_sort([ + { + 'environment': + { + 'YEP': '1', 'BAR': '1', 'BAZ': '3' + }, + 'image': 'busybox', + 'name': 'myweb' + }, + { + 'environment': + {'YEP': '1'}, + 'image': 'busybox', + 'name': 'otherweb' + }, + { + 'environment': + {'YEP': '1', 'BAZ': '3'}, + 'image': 'busybox', + 'name': 'web' + } + ]) + + def test_circular(self): + with pytest.raises(config.CircularReference) as exc: + load_from_filename('tests/fixtures/extends/circle-1.yml') + + path = [ + (os.path.basename(filename), service_name) + for (filename, service_name) in exc.value.trail + ] + expected = [ + ('circle-1.yml', 'web'), + ('circle-2.yml', 'other'), + ('circle-1.yml', 'web'), + ] + assert path == expected + + def test_extends_validation_empty_dictionary(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': '3', + 'services': + { + 'web': {'image': 'busybox', 'extends': {}}, + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert 'service' in excinfo.exconly() + + def test_extends_validation_missing_service_key(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': '3', + 'services': + { + 'web': { + 'image': 'busybox', + 'extends': {'file': 'common.yml'} + } + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert "'service' is a required property" in excinfo.exconly() + + def test_extends_validation_invalid_key(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': '3', + 'services': + { + 'web': { + 'image': 'busybox', + 'extends': { + 'file': 'common.yml', + 'service': 'web', + 'rogue_key': 'is not allowed' + } + }, + } + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert "web.extends contains unsupported option: 'rogue_key'" \ + in excinfo.exconly() + + def test_extends_validation_sub_property_key(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details( + { + 'version': '3', + 'services': { + 'web': { + 'image': 'busybox', + 'extends': { + 'file': 1, + 'service': 'web', + } + } + }, + }, + 'tests/fixtures/extends', + 'filename.yml' + ) + ) + + assert "web.extends.file contains 1, which is an invalid type, it should be a string" \ + in excinfo.exconly() + + def test_extends_validation_no_file_key_no_filename_set(self): + dictionary = {'extends': {'service': 'web'}} + + with pytest.raises(ConfigurationError) as excinfo: + make_service_dict('myweb', dictionary, working_dir='tests/fixtures/extends') + + assert 'file' in excinfo.exconly() + + def test_extends_validation_valid_config(self): + service = config.load( + build_config_details( + { + 'web': {'image': 'busybox', 'extends': {'service': 'web', 'file': 'common.yml'}}, + }, + 'tests/fixtures/extends', + 'common.yml' + ) + ).services + + assert len(service) == 1 + assert isinstance(service[0], dict) + assert service[0]['command'] == "/bin/true" + + def test_extended_service_with_invalid_config(self): + with pytest.raises(ConfigurationError) as exc: + load_from_filename('tests/fixtures/extends/service-with-invalid-schema.yml') + assert ( + "myweb has neither an image nor a build context specified" in + exc.exconly() + ) + + def test_extended_service_with_valid_config(self): + service = load_from_filename('tests/fixtures/extends/service-with-valid-composite-extends.yml') + assert service[0]['command'] == "top" + + def test_extends_file_defaults_to_self(self): + """ + Test not specifying a file in our extends options that the + config is valid and correctly extends from itself. + """ + service_dicts = load_from_filename('tests/fixtures/extends/no-file-specified.yml') + assert service_sort(service_dicts) == service_sort([ + { + 'name': 'myweb', + 'image': 'busybox', + 'environment': { + "BAR": "1", + "BAZ": "3", + } + }, + { + 'name': 'web', + 'image': 'busybox', + 'environment': { + "BAZ": "3", + } + } + ]) + + def test_invalid_links_in_extended_service(self): + with pytest.raises(ConfigurationError) as excinfo: + load_from_filename('tests/fixtures/extends/invalid-links.yml') + + assert "services with 'links' cannot be extended" in excinfo.exconly() + + def test_invalid_volumes_from_in_extended_service(self): + with pytest.raises(ConfigurationError) as excinfo: + load_from_filename('tests/fixtures/extends/invalid-volumes.yml') + + assert "services with 'volumes_from' cannot be extended" in excinfo.exconly() + + def test_invalid_net_in_extended_service(self): + with pytest.raises(ConfigurationError) as excinfo: + load_from_filename('tests/fixtures/extends/invalid-net-v2.yml') + + assert 'network_mode: service' in excinfo.exconly() + assert 'cannot be extended' in excinfo.exconly() + + with pytest.raises(ConfigurationError) as excinfo: + load_from_filename('tests/fixtures/extends/invalid-net.yml') + + assert 'net: container' in excinfo.exconly() + assert 'cannot be extended' in excinfo.exconly() + + @mock.patch.dict(os.environ) + def test_load_config_runs_interpolation_in_extended_service(self): + os.environ.update(HOSTNAME_VALUE="penguin") + expected_interpolated_value = "host-penguin" + service_dicts = load_from_filename( + 'tests/fixtures/extends/valid-interpolation.yml') + for service in service_dicts: + assert service['hostname'] == expected_interpolated_value + + @pytest.mark.xfail(IS_WINDOWS_PLATFORM, reason='paths use slash') + def test_volume_path(self): + dicts = load_from_filename('tests/fixtures/volume-path/docker-compose.yml') + + paths = [ + VolumeSpec( + os.path.abspath('tests/fixtures/volume-path/common/foo'), + '/foo', + 'rw'), + VolumeSpec( + os.path.abspath('tests/fixtures/volume-path/bar'), + '/bar', + 'rw') + ] + + assert set(dicts[0]['volumes']) == set(paths) + + def test_parent_build_path_dne(self): + child = load_from_filename('tests/fixtures/extends/nonexistent-path-child.yml') + + assert child == [ + { + 'name': 'dnechild', + 'image': 'busybox', + 'command': '/bin/true', + 'environment': { + "FOO": "1", + "BAR": "2", + }, + }, + ] + + def test_load_throws_error_when_base_service_does_not_exist(self): + with pytest.raises(ConfigurationError) as excinfo: + load_from_filename('tests/fixtures/extends/nonexistent-service.yml') + + assert "Cannot extend service 'foo'" in excinfo.exconly() + assert "Service not found" in excinfo.exconly() + + def test_partial_service_config_in_extends_is_still_valid(self): + dicts = load_from_filename('tests/fixtures/extends/valid-common-config.yml') + assert dicts[0]['environment'] == {'FOO': '1'} + + def test_extended_service_with_verbose_and_shorthand_way(self): + services = load_from_filename('tests/fixtures/extends/verbose-and-shorthand.yml') + assert service_sort(services) == service_sort([ + { + 'name': 'base', + 'image': 'busybox', + 'environment': {'BAR': '1'}, + }, + { + 'name': 'verbose', + 'image': 'busybox', + 'environment': {'BAR': '1', 'FOO': '1'}, + }, + { + 'name': 'shorthand', + 'image': 'busybox', + 'environment': {'BAR': '1', 'FOO': '2'}, + }, + ]) + + @mock.patch.dict(os.environ) + def test_extends_with_environment_and_env_files(self): + tmpdir = tempfile.mkdtemp('test_extends_with_environment') + self.addCleanup(shutil.rmtree, tmpdir) + commondir = os.path.join(tmpdir, 'common') + os.mkdir(commondir) + with open(os.path.join(commondir, 'base.yml'), mode="w") as base_fh: + base_fh.write(""" + app: + image: 'example/app' + env_file: + - 'envs' + environment: + - SECRET + - TEST_ONE=common + - TEST_TWO=common + """) + with open(os.path.join(tmpdir, 'docker-compose.yml'), mode="w") as docker_compose_fh: + docker_compose_fh.write(""" + ext: + extends: + file: common/base.yml + service: app + env_file: + - 'envs' + environment: + - THING + - TEST_ONE=top + """) + with open(os.path.join(commondir, 'envs'), mode="w") as envs_fh: + envs_fh.write(""" + COMMON_ENV_FILE + TEST_ONE=common-env-file + TEST_TWO=common-env-file + TEST_THREE=common-env-file + TEST_FOUR=common-env-file + """) + with open(os.path.join(tmpdir, 'envs'), mode="w") as envs_fh: + envs_fh.write(""" + TOP_ENV_FILE + TEST_ONE=top-env-file + TEST_TWO=top-env-file + TEST_THREE=top-env-file + """) + + expected = [ + { + 'name': 'ext', + 'image': 'example/app', + 'environment': { + 'SECRET': 'secret', + 'TOP_ENV_FILE': 'secret', + 'COMMON_ENV_FILE': 'secret', + 'THING': 'thing', + 'TEST_ONE': 'top', + 'TEST_TWO': 'common', + 'TEST_THREE': 'top-env-file', + 'TEST_FOUR': 'common-env-file', + }, + }, + ] + + os.environ['SECRET'] = 'secret' + os.environ['THING'] = 'thing' + os.environ['COMMON_ENV_FILE'] = 'secret' + os.environ['TOP_ENV_FILE'] = 'secret' + config = load_from_filename(str(os.path.join(tmpdir, 'docker-compose.yml'))) + + assert config == expected + + def test_extends_with_mixed_versions_is_error(self): + tmpdir = tempfile.mkdtemp('test_extends_with_mixed_version') + self.addCleanup(shutil.rmtree, tmpdir) + with open(os.path.join(tmpdir, 'docker-compose.yml'), mode="w") as docker_compose_fh: + docker_compose_fh.write(""" + version: "2" + services: + web: + extends: + file: base.yml + service: base + image: busybox + """) + with open(os.path.join(tmpdir, 'base.yml'), mode="w") as base_fh: + base_fh.write(""" + base: + volumes: ['/foo'] + ports: ['3000:3000'] + """) + + with pytest.raises(ConfigurationError) as exc: + load_from_filename(str(os.path.join(tmpdir, 'docker-compose.yml'))) + assert 'Version mismatch' in exc.exconly() + + def test_extends_with_defined_version_passes(self): + tmpdir = tempfile.mkdtemp('test_extends_with_defined_version') + self.addCleanup(shutil.rmtree, tmpdir) + with open(os.path.join(tmpdir, 'docker-compose.yml'), mode="w") as docker_compose_fh: + docker_compose_fh.write(""" + version: "2" + services: + web: + extends: + file: base.yml + service: base + image: busybox + """) + with open(os.path.join(tmpdir, 'base.yml'), mode="w") as base_fh: + base_fh.write(""" + version: "2" + services: + base: + volumes: ['/foo'] + ports: ['3000:3000'] + command: top + """) + + service = load_from_filename(str(os.path.join(tmpdir, 'docker-compose.yml'))) + assert service[0]['command'] == "top" + + def test_extends_with_depends_on(self): + tmpdir = tempfile.mkdtemp('test_extends_with_depends_on') + self.addCleanup(shutil.rmtree, tmpdir) + with open(os.path.join(tmpdir, 'docker-compose.yml'), mode="w") as docker_compose_fh: + docker_compose_fh.write(""" + version: "2" + services: + base: + image: example + web: + extends: base + image: busybox + depends_on: ['other'] + other: + image: example + """) + services = load_from_filename(str(os.path.join(tmpdir, 'docker-compose.yml'))) + assert service_sort(services)[2]['depends_on'] == { + 'other': {'condition': 'service_started'} + } + + def test_extends_with_healthcheck(self): + service_dicts = load_from_filename('tests/fixtures/extends/healthcheck-2.yml') + assert service_sort(service_dicts) == [{ + 'name': 'demo', + 'image': 'foobar:latest', + 'healthcheck': { + 'test': ['CMD', '/health.sh'], + 'interval': 10000000000, + 'timeout': 5000000000, + 'retries': 36, + } + }] + + def test_extends_with_ports(self): + tmpdir = tempfile.mkdtemp('test_extends_with_ports') + self.addCleanup(shutil.rmtree, tmpdir) + with open(os.path.join(tmpdir, 'docker-compose.yml'), mode="w") as docker_compose_fh: + docker_compose_fh.write(""" + version: '2' + + services: + a: + image: nginx + ports: + - 80 + + b: + extends: + service: a + """) + services = load_from_filename(str(os.path.join(tmpdir, 'docker-compose.yml'))) + + assert len(services) == 2 + for svc in services: + assert svc['ports'] == [types.ServicePort('80', None, None, None, None)] + + def test_extends_with_security_opt(self): + tmpdir = tempfile.mkdtemp('test_extends_with_ports') + self.addCleanup(shutil.rmtree, tmpdir) + with open(os.path.join(tmpdir, 'docker-compose.yml'), mode="w") as docker_compose_fh: + docker_compose_fh.write(""" + version: '2' + + services: + a: + image: nginx + security_opt: + - apparmor:unconfined + - seccomp:unconfined + + b: + extends: + service: a + """) + services = load_from_filename(str(os.path.join(tmpdir, 'docker-compose.yml'))) + assert len(services) == 2 + for svc in services: + assert types.SecurityOpt.parse('apparmor:unconfined') in svc['security_opt'] + assert types.SecurityOpt.parse('seccomp:unconfined') in svc['security_opt'] + + @mock.patch.object(ConfigFile, 'from_filename', wraps=ConfigFile.from_filename) + def test_extends_same_file_optimization(self, from_filename_mock): + load_from_filename('tests/fixtures/extends/no-file-specified.yml') + from_filename_mock.assert_called_once() + + +@pytest.mark.xfail(IS_WINDOWS_PLATFORM, reason='paths use slash') +class ExpandPathTest(unittest.TestCase): + working_dir = '/home/user/somedir' + + def test_expand_path_normal(self): + result = config.expand_path(self.working_dir, 'myfile') + assert result == self.working_dir + '/' + 'myfile' + + def test_expand_path_absolute(self): + abs_path = '/home/user/otherdir/somefile' + result = config.expand_path(self.working_dir, abs_path) + assert result == abs_path + + def test_expand_path_with_tilde(self): + test_path = '~/otherdir/somefile' + with mock.patch.dict(os.environ): + os.environ['HOME'] = user_path = '/home/user/' + result = config.expand_path(self.working_dir, test_path) + + assert result == user_path + 'otherdir/somefile' + + +class VolumePathTest(unittest.TestCase): + + def test_split_path_mapping_with_windows_path(self): + host_path = "c:\\Users\\msamblanet\\Documents\\anvil\\connect\\config" + windows_volume_path = host_path + ":/opt/connect/config:ro" + expected_mapping = ("/opt/connect/config", (host_path, 'ro')) + + mapping = config.split_path_mapping(windows_volume_path) + assert mapping == expected_mapping + + def test_split_path_mapping_with_windows_path_in_container(self): + host_path = 'c:\\Users\\remilia\\data' + container_path = 'c:\\scarletdevil\\data' + expected_mapping = (container_path, (host_path, None)) + + mapping = config.split_path_mapping('{}:{}'.format(host_path, container_path)) + assert mapping == expected_mapping + + def test_split_path_mapping_with_root_mount(self): + host_path = '/' + container_path = '/var/hostroot' + expected_mapping = (container_path, (host_path, None)) + mapping = config.split_path_mapping('{}:{}'.format(host_path, container_path)) + assert mapping == expected_mapping + + +@pytest.mark.xfail(IS_WINDOWS_PLATFORM, reason='paths use slash') +class BuildPathTest(unittest.TestCase): + + def setUp(self): + self.abs_context_path = os.path.join(os.getcwd(), 'tests/fixtures/build-ctx') + + def test_nonexistent_path(self): + with pytest.raises(ConfigurationError): + config.load( + build_config_details( + { + 'foo': {'build': 'nonexistent.path'}, + }, + 'working_dir', + 'filename.yml' + ) + ) + + def test_relative_path(self): + relative_build_path = '../build-ctx/' + service_dict = make_service_dict( + 'relpath', + {'build': relative_build_path}, + working_dir='tests/fixtures/build-path' + ) + assert service_dict['build'] == self.abs_context_path + + def test_absolute_path(self): + service_dict = make_service_dict( + 'abspath', + {'build': self.abs_context_path}, + working_dir='tests/fixtures/build-path' + ) + assert service_dict['build'] == self.abs_context_path + + def test_from_file(self): + service_dict = load_from_filename('tests/fixtures/build-path/docker-compose.yml') + assert service_dict == [{'name': 'foo', 'build': {'context': self.abs_context_path}}] + + def test_from_file_override_dir(self): + override_dir = os.path.join(os.getcwd(), 'tests/fixtures/') + service_dict = load_from_filename( + 'tests/fixtures/build-path-override-dir/docker-compose.yml', override_dir=override_dir) + assert service_dict == [{'name': 'foo', 'build': {'context': self.abs_context_path}}] + + def test_valid_url_in_build_path(self): + valid_urls = [ + 'git://github.com/docker/docker', + 'git@github.com:docker/docker.git', + 'git@bitbucket.org:atlassianlabs/atlassian-docker.git', + 'https://github.com/docker/docker.git', + 'http://github.com/docker/docker.git', + 'github.com/docker/docker.git', + ] + for valid_url in valid_urls: + service_dict = config.load(build_config_details({ + 'validurl': {'build': valid_url}, + }, '.', None)).services + assert service_dict[0]['build'] == {'context': valid_url} + + def test_invalid_url_in_build_path(self): + invalid_urls = [ + 'example.com/bogus', + 'ftp://example.com/', + '/path/does/not/exist', + ] + for invalid_url in invalid_urls: + with pytest.raises(ConfigurationError) as exc: + config.load(build_config_details({ + 'invalidurl': {'build': invalid_url}, + }, '.', None)) + assert 'build path' in exc.exconly() + + +class HealthcheckTest(unittest.TestCase): + def test_healthcheck(self): + config_dict = config.load( + build_config_details({ + 'version': '2.3', + 'services': { + 'test': { + 'image': 'busybox', + 'healthcheck': { + 'test': ['CMD', 'true'], + 'interval': '1s', + 'timeout': '1m', + 'retries': 3, + 'start_period': '10s', + } + } + } + + }) + ) + + serialized_config = yaml.safe_load(serialize_config(config_dict)) + serialized_service = serialized_config['services']['test'] + + assert serialized_service['healthcheck'] == { + 'test': ['CMD', 'true'], + 'interval': '1s', + 'timeout': '1m', + 'retries': 3, + 'start_period': '10s' + } + + def test_disable(self): + config_dict = config.load( + build_config_details({ + 'version': '2.3', + 'services': { + 'test': { + 'image': 'busybox', + 'healthcheck': { + 'disable': True, + } + } + } + + }) + ) + + serialized_config = yaml.safe_load(serialize_config(config_dict)) + serialized_service = serialized_config['services']['test'] + + assert serialized_service['healthcheck'] == { + 'test': ['NONE'], + } + + def test_disable_with_other_config_is_invalid(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details({ + 'version': '2.3', + 'services': { + 'invalid-healthcheck': { + 'image': 'busybox', + 'healthcheck': { + 'disable': True, + 'interval': '1s', + } + } + } + + }) + ) + + assert 'invalid-healthcheck' in excinfo.exconly() + assert '"disable: true" cannot be combined with other options' in excinfo.exconly() + + def test_healthcheck_with_invalid_test(self): + with pytest.raises(ConfigurationError) as excinfo: + config.load( + build_config_details({ + 'version': '2.3', + 'services': { + 'invalid-healthcheck': { + 'image': 'busybox', + 'healthcheck': { + 'test': ['true'], + 'interval': '1s', + 'timeout': '1m', + 'retries': 3, + 'start_period': '10s', + } + } + } + + }) + ) + + assert 'invalid-healthcheck' in excinfo.exconly() + assert 'the first item must be either NONE, CMD or CMD-SHELL' in excinfo.exconly() + + +class GetDefaultConfigFilesTestCase(unittest.TestCase): + + files = [ + 'docker-compose.yml', + 'docker-compose.yaml', + ] + + def test_get_config_path_default_file_in_basedir(self): + for index, filename in enumerate(self.files): + assert filename == get_config_filename_for_files(self.files[index:]) + with pytest.raises(config.ComposeFileNotFound): + get_config_filename_for_files([]) + + def test_get_config_path_default_file_in_parent_dir(self): + """Test with files placed in the subdir""" + + def get_config_in_subdir(files): + return get_config_filename_for_files(files, subdir=True) + + for index, filename in enumerate(self.files): + assert filename == get_config_in_subdir(self.files[index:]) + with pytest.raises(config.ComposeFileNotFound): + get_config_in_subdir([]) + + +def get_config_filename_for_files(filenames, subdir=None): + def make_files(dirname, filenames): + for fname in filenames: + with open(os.path.join(dirname, fname), 'w') as f: + f.write('') + + project_dir = tempfile.mkdtemp() + try: + make_files(project_dir, filenames) + if subdir: + base_dir = tempfile.mkdtemp(dir=project_dir) + else: + base_dir = project_dir + filename, = config.get_default_config_files(base_dir) + return os.path.basename(filename) + finally: + shutil.rmtree(project_dir) + + +class SerializeTest(unittest.TestCase): + def test_denormalize_depends(self): + service_dict = { + 'image': 'busybox', + 'command': 'true', + 'depends_on': { + 'service2': {'condition': 'service_started'}, + 'service3': {'condition': 'service_started'}, + } + } + + assert denormalize_service_dict(service_dict, VERSION) == service_dict + + def test_serialize_time(self): + data = { + 9: '9ns', + 9000: '9us', + 9000000: '9ms', + 90000000: '90ms', + 900000000: '900ms', + 999999999: '999999999ns', + 1000000000: '1s', + 60000000000: '1m', + 60000000001: '60000000001ns', + 9000000000000: '150m', + 90000000000000: '25h', + } + + for k, v in data.items(): + assert serialize_ns_time_value(k) == v + + def test_denormalize_healthcheck(self): + service_dict = { + 'image': 'test', + 'healthcheck': { + 'test': 'exit 1', + 'interval': '1m40s', + 'timeout': '30s', + 'retries': 5, + 'start_period': '2s90ms' + } + } + processed_service = config.process_service(config.ServiceConfig( + '.', 'test', 'test', service_dict + )) + denormalized_service = denormalize_service_dict(processed_service, VERSION) + assert denormalized_service['healthcheck']['interval'] == '100s' + assert denormalized_service['healthcheck']['timeout'] == '30s' + assert denormalized_service['healthcheck']['start_period'] == '2090ms' + + def test_denormalize_image_has_digest(self): + service_dict = { + 'image': 'busybox' + } + image_digest = 'busybox@sha256:abcde' + + assert denormalize_service_dict(service_dict, VERSION, image_digest) == { + 'image': 'busybox@sha256:abcde' + } + + def test_denormalize_image_no_digest(self): + service_dict = { + 'image': 'busybox' + } + + assert denormalize_service_dict(service_dict, VERSION) == { + 'image': 'busybox' + } + + def test_serialize_secrets(self): + service_dict = { + 'image': 'example/web', + 'secrets': [ + {'source': 'one'}, + { + 'source': 'source', + 'target': 'target', + 'uid': '100', + 'gid': '200', + 'mode': 0o777, + } + ] + } + secrets_dict = { + 'one': {'file': '/one.txt'}, + 'source': {'file': '/source.pem'}, + 'two': {'external': True}, + } + config_dict = config.load(build_config_details({ + 'version': '3.1', + 'services': {'web': service_dict}, + 'secrets': secrets_dict + })) + + serialized_config = yaml.safe_load(serialize_config(config_dict)) + serialized_service = serialized_config['services']['web'] + assert secret_sort(serialized_service['secrets']) == secret_sort(service_dict['secrets']) + assert 'secrets' in serialized_config + assert serialized_config['secrets']['two'] == {'external': True, 'name': 'two'} + + def test_serialize_ports(self): + config_dict = config.Config(config_version=VERSION, version=VERSION, services=[ + { + 'ports': [types.ServicePort('80', '8080', None, None, None)], + 'image': 'alpine', + 'name': 'web' + } + ], volumes={}, networks={}, secrets={}, configs={}) + + serialized_config = yaml.safe_load(serialize_config(config_dict)) + assert [{'published': 8080, 'target': 80}] == serialized_config['services']['web']['ports'] + + def test_serialize_ports_v1(self): + config_dict = config.Config(config_version=V1, version=V1, services=[ + { + 'ports': [types.ServicePort('80', '8080', None, None, None)], + 'image': 'alpine', + 'name': 'web' + } + ], volumes={}, networks={}, secrets={}, configs={}) + + serialized_config = yaml.safe_load(serialize_config(config_dict)) + assert ['8080:80/tcp'] == serialized_config['services']['web']['ports'] + + def test_serialize_ports_with_ext_ip(self): + config_dict = config.Config(config_version=VERSION, version=VERSION, services=[ + { + 'ports': [types.ServicePort('80', '8080', None, None, '127.0.0.1')], + 'image': 'alpine', + 'name': 'web' + } + ], volumes={}, networks={}, secrets={}, configs={}) + + serialized_config = yaml.safe_load(serialize_config(config_dict)) + assert '127.0.0.1:8080:80/tcp' in serialized_config['services']['web']['ports'] + + def test_serialize_configs(self): + service_dict = { + 'image': 'example/web', + 'configs': [ + {'source': 'one'}, + { + 'source': 'source', + 'target': 'target', + 'uid': '100', + 'gid': '200', + 'mode': 0o777, + } + ] + } + configs_dict = { + 'one': {'file': '/one.txt'}, + 'source': {'file': '/source.pem'}, + 'two': {'external': True}, + } + config_dict = config.load(build_config_details({ + 'version': '3.3', + 'services': {'web': service_dict}, + 'configs': configs_dict + })) + + serialized_config = yaml.safe_load(serialize_config(config_dict)) + serialized_service = serialized_config['services']['web'] + assert secret_sort(serialized_service['configs']) == secret_sort(service_dict['configs']) + assert 'configs' in serialized_config + assert serialized_config['configs']['two'] == {'external': True, 'name': 'two'} + + def test_serialize_bool_string(self): + cfg = { + 'version': '2.2', + 'services': { + 'web': { + 'image': 'example/web', + 'command': 'true', + 'environment': {'FOO': 'Y', 'BAR': 'on'} + } + } + } + config_dict = config.load(build_config_details(cfg)) + + serialized_config = serialize_config(config_dict) + assert 'command: "true"\n' in serialized_config + assert 'FOO: "Y"\n' in serialized_config + assert 'BAR: "on"\n' in serialized_config + + def test_serialize_escape_dollar_sign(self): + cfg = { + 'version': '2.2', + 'services': { + 'web': { + 'image': 'busybox', + 'command': 'echo $$FOO', + 'environment': { + 'CURRENCY': '$$' + }, + 'entrypoint': ['$$SHELL', '-c'], + } + } + } + config_dict = config.load(build_config_details(cfg)) + + serialized_config = yaml.safe_load(serialize_config(config_dict)) + serialized_service = serialized_config['services']['web'] + assert serialized_service['environment']['CURRENCY'] == '$$' + assert serialized_service['command'] == 'echo $$FOO' + assert serialized_service['entrypoint'][0] == '$$SHELL' + + def test_serialize_escape_dont_interpolate(self): + cfg = { + 'version': '2.2', + 'services': { + 'web': { + 'image': 'busybox', + 'command': 'echo $FOO', + 'environment': { + 'CURRENCY': '$' + }, + 'env_file': ['tests/fixtures/env/three.env'], + 'entrypoint': ['$SHELL', '-c'], + } + } + } + config_dict = config.load(build_config_details(cfg, working_dir='.'), interpolate=False) + + serialized_config = yaml.safe_load(serialize_config(config_dict, escape_dollar=False)) + serialized_service = serialized_config['services']['web'] + assert serialized_service['environment']['CURRENCY'] == '$' + # Values coming from env_files are not allowed to have variables + assert serialized_service['environment']['FOO'] == 'NO $$ENV VAR' + assert serialized_service['environment']['DOO'] == 'NO $${ENV} VAR' + assert serialized_service['command'] == 'echo $FOO' + assert serialized_service['entrypoint'][0] == '$SHELL' + + def test_serialize_unicode_values(self): + cfg = { + 'version': '2.3', + 'services': { + 'web': { + 'image': 'busybox', + 'command': 'echo 十六夜　咲夜' + } + } + } + + config_dict = config.load(build_config_details(cfg)) + + serialized_config = yaml.safe_load(serialize_config(config_dict)) + serialized_service = serialized_config['services']['web'] + assert serialized_service['command'] == 'echo 十六夜　咲夜' + + def test_serialize_external_false(self): + cfg = { + 'version': '3.4', + 'volumes': { + 'test': { + 'name': 'test-false', + 'external': False + } + } + } + + config_dict = config.load(build_config_details(cfg)) + serialized_config = yaml.safe_load(serialize_config(config_dict)) + serialized_volume = serialized_config['volumes']['test'] + assert serialized_volume['external'] is False diff --git a/tests/unit/config/environment_test.py b/tests/unit/config/environment_test.py new file mode 100644 index 00000000000..6a80ff12254 --- /dev/null +++ b/tests/unit/config/environment_test.py @@ -0,0 +1,65 @@ +import codecs +import os +import shutil +import tempfile + +from ddt import data +from ddt import ddt +from ddt import unpack + +from compose.config.environment import env_vars_from_file +from compose.config.environment import Environment +from tests import unittest + + +@ddt +class EnvironmentTest(unittest.TestCase): + @classmethod + def test_get_simple(self): + env = Environment({ + 'FOO': 'bar', + 'BAR': '1', + 'BAZ': '' + }) + + assert env.get('FOO') == 'bar' + assert env.get('BAR') == '1' + assert env.get('BAZ') == '' + + @classmethod + def test_get_undefined(self): + env = Environment({ + 'FOO': 'bar' + }) + assert env.get('FOOBAR') is None + + @classmethod + def test_get_boolean(self): + env = Environment({ + 'FOO': '', + 'BAR': '0', + 'BAZ': 'FALSE', + 'FOOBAR': 'true', + }) + + assert env.get_boolean('FOO') is False + assert env.get_boolean('BAR') is False + assert env.get_boolean('BAZ') is False + assert env.get_boolean('FOOBAR') is True + assert env.get_boolean('UNDEFINED') is False + + @data( + ('unicode exclude test', '\ufeffPARK_BOM=박봄\n', {'PARK_BOM': '박봄'}), + ('export prefixed test', 'export PREFIXED_VARS=yes\n', {"PREFIXED_VARS": "yes"}), + ('quoted vars test', "QUOTED_VARS='yes'\n", {"QUOTED_VARS": "yes"}), + ('double quoted vars test', 'DOUBLE_QUOTED_VARS="yes"\n', {"DOUBLE_QUOTED_VARS": "yes"}), + ('extra spaces test', 'SPACES_VARS = "yes"\n', {"SPACES_VARS": "yes"}), + ) + @unpack + def test_env_vars(self, test_name, content, expected): + tmpdir = tempfile.mkdtemp('env_file') + self.addCleanup(shutil.rmtree, tmpdir) + file_abs_path = str(os.path.join(tmpdir, ".env")) + with codecs.open(file_abs_path, 'w', encoding='utf-8') as f: + f.write(content) + assert env_vars_from_file(file_abs_path) == expected, '"{}" Failed'.format(test_name) diff --git a/tests/unit/config/interpolation_test.py b/tests/unit/config/interpolation_test.py new file mode 100644 index 00000000000..1fd50d60bdd --- /dev/null +++ b/tests/unit/config/interpolation_test.py @@ -0,0 +1,459 @@ +import pytest + +from compose.config.environment import Environment +from compose.config.errors import ConfigurationError +from compose.config.interpolation import interpolate_environment_variables +from compose.config.interpolation import Interpolator +from compose.config.interpolation import InvalidInterpolation +from compose.config.interpolation import TemplateWithDefaults +from compose.config.interpolation import UnsetRequiredSubstitution +from compose.const import COMPOSE_SPEC as VERSION + + +@pytest.fixture +def mock_env(): + return Environment({ + 'USER': 'jenny', + 'FOO': 'bar', + 'TRUE': 'True', + 'FALSE': 'OFF', + 'POSINT': '50', + 'NEGINT': '-200', + 'FLOAT': '0.145', + 'MODE': '0600', + 'BYTES': '512m', + }) + + +@pytest.fixture +def variable_mapping(): + return Environment({'FOO': 'first', 'BAR': ''}) + + +@pytest.fixture +def defaults_interpolator(variable_mapping): + return Interpolator(TemplateWithDefaults, variable_mapping).interpolate + + +def test_interpolate_environment_variables_in_services(mock_env): + services = { + 'servicea': { + 'image': 'example:${USER}', + 'volumes': ['$FOO:/target'], + 'logging': { + 'driver': '${FOO}', + 'options': { + 'user': '$USER', + } + } + } + } + expected = { + 'servicea': { + 'image': 'example:jenny', + 'volumes': ['bar:/target'], + 'logging': { + 'driver': 'bar', + 'options': { + 'user': 'jenny', + } + } + } + } + value = interpolate_environment_variables(VERSION, services, 'service', mock_env) + assert value == expected + + +def test_interpolate_environment_variables_in_volumes(mock_env): + volumes = { + 'data': { + 'driver': '$FOO', + 'driver_opts': { + 'max': 2, + 'user': '${USER}' + } + }, + 'other': None, + } + expected = { + 'data': { + 'driver': 'bar', + 'driver_opts': { + 'max': 2, + 'user': 'jenny' + } + }, + 'other': {}, + } + value = interpolate_environment_variables(VERSION, volumes, 'volume', mock_env) + assert value == expected + + +def test_interpolate_environment_variables_in_secrets(mock_env): + secrets = { + 'secretservice': { + 'file': '$FOO', + 'labels': { + 'max': 2, + 'user': '${USER}' + } + }, + 'other': None, + } + expected = { + 'secretservice': { + 'file': 'bar', + 'labels': { + 'max': '2', + 'user': 'jenny' + } + }, + 'other': {}, + } + value = interpolate_environment_variables(VERSION, secrets, 'secret', mock_env) + assert value == expected + + +def test_interpolate_environment_services_convert_types_v2(mock_env): + entry = { + 'service1': { + 'blkio_config': { + 'weight': '${POSINT}', + 'weight_device': [{'file': '/dev/sda1', 'weight': '${POSINT}'}] + }, + 'cpus': '${FLOAT}', + 'cpu_count': '$POSINT', + 'healthcheck': { + 'retries': '${POSINT:-3}', + 'disable': '${FALSE}', + 'command': 'true' + }, + 'mem_swappiness': '${DEFAULT:-127}', + 'oom_score_adj': '${NEGINT}', + 'scale': '${POSINT}', + 'ulimits': { + 'nproc': '${POSINT}', + 'nofile': { + 'soft': '${POSINT}', + 'hard': '${DEFAULT:-40000}' + }, + }, + 'privileged': '${TRUE}', + 'read_only': '${DEFAULT:-no}', + 'tty': '${DEFAULT:-N}', + 'stdin_open': '${DEFAULT-on}', + 'volumes': [ + {'type': 'tmpfs', 'target': '/target', 'tmpfs': {'size': '$BYTES'}} + ] + } + } + + expected = { + 'service1': { + 'blkio_config': { + 'weight': 50, + 'weight_device': [{'file': '/dev/sda1', 'weight': 50}] + }, + 'cpus': 0.145, + 'cpu_count': 50, + 'healthcheck': { + 'retries': 50, + 'disable': False, + 'command': 'true' + }, + 'mem_swappiness': 127, + 'oom_score_adj': -200, + 'scale': 50, + 'ulimits': { + 'nproc': 50, + 'nofile': { + 'soft': 50, + 'hard': 40000 + }, + }, + 'privileged': True, + 'read_only': False, + 'tty': False, + 'stdin_open': True, + 'volumes': [ + {'type': 'tmpfs', 'target': '/target', 'tmpfs': {'size': 536870912}} + ] + } + } + + value = interpolate_environment_variables(VERSION, entry, 'service', mock_env) + assert value == expected + + +def test_interpolate_environment_services_convert_types_v3(mock_env): + entry = { + 'service1': { + 'healthcheck': { + 'retries': '${POSINT:-3}', + 'disable': '${FALSE}', + 'command': 'true' + }, + 'ulimits': { + 'nproc': '${POSINT}', + 'nofile': { + 'soft': '${POSINT}', + 'hard': '${DEFAULT:-40000}' + }, + }, + 'privileged': '${TRUE}', + 'read_only': '${DEFAULT:-no}', + 'tty': '${DEFAULT:-N}', + 'stdin_open': '${DEFAULT-on}', + 'deploy': { + 'update_config': { + 'parallelism': '${DEFAULT:-2}', + 'max_failure_ratio': '${FLOAT}', + }, + 'restart_policy': { + 'max_attempts': '$POSINT', + }, + 'replicas': '${DEFAULT-3}' + }, + 'ports': [{'target': '${POSINT}', 'published': '${DEFAULT:-5000}'}], + 'configs': [{'mode': '${MODE}', 'source': 'config1'}], + 'secrets': [{'mode': '${MODE}', 'source': 'secret1'}], + } + } + + expected = { + 'service1': { + 'healthcheck': { + 'retries': 50, + 'disable': False, + 'command': 'true' + }, + 'ulimits': { + 'nproc': 50, + 'nofile': { + 'soft': 50, + 'hard': 40000 + }, + }, + 'privileged': True, + 'read_only': False, + 'tty': False, + 'stdin_open': True, + 'deploy': { + 'update_config': { + 'parallelism': 2, + 'max_failure_ratio': 0.145, + }, + 'restart_policy': { + 'max_attempts': 50, + }, + 'replicas': 3 + }, + 'ports': [{'target': 50, 'published': 5000}], + 'configs': [{'mode': 0o600, 'source': 'config1'}], + 'secrets': [{'mode': 0o600, 'source': 'secret1'}], + } + } + + value = interpolate_environment_variables(VERSION, entry, 'service', mock_env) + assert value == expected + + +def test_interpolate_environment_services_convert_types_invalid(mock_env): + entry = {'service1': {'privileged': '${POSINT}'}} + + with pytest.raises(ConfigurationError) as exc: + interpolate_environment_variables(VERSION, entry, 'service', mock_env) + + assert 'Error while attempting to convert service.service1.privileged to '\ + 'appropriate type: "50" is not a valid boolean value' in exc.exconly() + + entry = {'service1': {'cpus': '${TRUE}'}} + with pytest.raises(ConfigurationError) as exc: + interpolate_environment_variables(VERSION, entry, 'service', mock_env) + + assert 'Error while attempting to convert service.service1.cpus to '\ + 'appropriate type: "True" is not a valid float' in exc.exconly() + + entry = {'service1': {'ulimits': {'nproc': '${FLOAT}'}}} + with pytest.raises(ConfigurationError) as exc: + interpolate_environment_variables(VERSION, entry, 'service', mock_env) + + assert 'Error while attempting to convert service.service1.ulimits.nproc to '\ + 'appropriate type: "0.145" is not a valid integer' in exc.exconly() + + +def test_interpolate_environment_network_convert_types(mock_env): + entry = { + 'network1': { + 'external': '${FALSE}', + 'attachable': '${TRUE}', + 'internal': '${DEFAULT:-false}' + } + } + + expected = { + 'network1': { + 'external': False, + 'attachable': True, + 'internal': False, + } + } + + value = interpolate_environment_variables(VERSION, entry, 'network', mock_env) + assert value == expected + + +def test_interpolate_environment_external_resource_convert_types(mock_env): + entry = { + 'resource1': { + 'external': '${TRUE}', + } + } + + expected = { + 'resource1': { + 'external': True, + } + } + + value = interpolate_environment_variables(VERSION, entry, 'network', mock_env) + assert value == expected + value = interpolate_environment_variables(VERSION, entry, 'volume', mock_env) + assert value == expected + value = interpolate_environment_variables(VERSION, entry, 'secret', mock_env) + assert value == expected + value = interpolate_environment_variables(VERSION, entry, 'config', mock_env) + assert value == expected + + +def test_interpolate_service_name_uses_dot(mock_env): + entry = { + 'service.1': { + 'image': 'busybox', + 'ulimits': { + 'nproc': '${POSINT}', + 'nofile': { + 'soft': '${POSINT}', + 'hard': '${DEFAULT:-40000}' + }, + }, + } + } + + expected = { + 'service.1': { + 'image': 'busybox', + 'ulimits': { + 'nproc': 50, + 'nofile': { + 'soft': 50, + 'hard': 40000 + }, + }, + } + } + + value = interpolate_environment_variables(VERSION, entry, 'service', mock_env) + assert value == expected + + +def test_escaped_interpolation(defaults_interpolator): + assert defaults_interpolator('$${foo}') == '${foo}' + + +def test_invalid_interpolation(defaults_interpolator): + with pytest.raises(InvalidInterpolation): + defaults_interpolator('${') + with pytest.raises(InvalidInterpolation): + defaults_interpolator('$}') + with pytest.raises(InvalidInterpolation): + defaults_interpolator('${}') + with pytest.raises(InvalidInterpolation): + defaults_interpolator('${ }') + with pytest.raises(InvalidInterpolation): + defaults_interpolator('${ foo}') + with pytest.raises(InvalidInterpolation): + defaults_interpolator('${foo }') + with pytest.raises(InvalidInterpolation): + defaults_interpolator('${foo!}') + + +def test_interpolate_missing_no_default(defaults_interpolator): + assert defaults_interpolator("This ${missing} var") == "This var" + assert defaults_interpolator("This ${BAR} var") == "This var" + + +def test_interpolate_with_value(defaults_interpolator): + assert defaults_interpolator("This $FOO var") == "This first var" + assert defaults_interpolator("This ${FOO} var") == "This first var" + + +def test_interpolate_missing_with_default(defaults_interpolator): + assert defaults_interpolator("ok ${missing:-def}") == "ok def" + assert defaults_interpolator("ok ${missing-def}") == "ok def" + + +def test_interpolate_with_empty_and_default_value(defaults_interpolator): + assert defaults_interpolator("ok ${BAR:-def}") == "ok def" + assert defaults_interpolator("ok ${BAR-def}") == "ok " + + +def test_interpolate_mandatory_values(defaults_interpolator): + assert defaults_interpolator("ok ${FOO:?bar}") == "ok first" + assert defaults_interpolator("ok ${FOO?bar}") == "ok first" + assert defaults_interpolator("ok ${BAR?bar}") == "ok " + + with pytest.raises(UnsetRequiredSubstitution) as e: + defaults_interpolator("not ok ${BAR:?high bar}") + assert e.value.err == 'high bar' + + with pytest.raises(UnsetRequiredSubstitution) as e: + defaults_interpolator("not ok ${BAZ?dropped the bazz}") + assert e.value.err == 'dropped the bazz' + + +def test_interpolate_mandatory_no_err_msg(defaults_interpolator): + with pytest.raises(UnsetRequiredSubstitution) as e: + defaults_interpolator("not ok ${BAZ?}") + + assert e.value.err == 'BAZ' + + +def test_interpolate_mixed_separators(defaults_interpolator): + assert defaults_interpolator("ok ${BAR:-/non:-alphanumeric}") == "ok /non:-alphanumeric" + assert defaults_interpolator("ok ${BAR:-:?wwegegr??:?}") == "ok :?wwegegr??:?" + assert defaults_interpolator("ok ${BAR-:-hello}") == 'ok ' + + with pytest.raises(UnsetRequiredSubstitution) as e: + defaults_interpolator("not ok ${BAR:?xazz:-redf}") + assert e.value.err == 'xazz:-redf' + + assert defaults_interpolator("ok ${BAR?...:?bar}") == "ok " + + +def test_unbraced_separators(defaults_interpolator): + assert defaults_interpolator("ok $FOO:-bar") == "ok first:-bar" + assert defaults_interpolator("ok $BAZ?error") == "ok ?error" + + +def test_interpolate_unicode_values(): + variable_mapping = { + 'FOO': '十六夜　咲夜'.encode(), + 'BAR': '十六夜　咲夜' + } + interpol = Interpolator(TemplateWithDefaults, variable_mapping).interpolate + + interpol("$FOO") == '十六夜　咲夜' + interpol("${BAR}") == '十六夜　咲夜' + + +def test_interpolate_no_fallthrough(): + # Test regression on docker/compose#5829 + variable_mapping = { + 'TEST:-': 'hello', + 'TEST-': 'hello', + } + interpol = Interpolator(TemplateWithDefaults, variable_mapping).interpolate + + assert interpol('${TEST:-}') == '' + assert interpol('${TEST-}') == '' diff --git a/tests/unit/config/sort_services_test.py b/tests/unit/config/sort_services_test.py new file mode 100644 index 00000000000..508c4bba190 --- /dev/null +++ b/tests/unit/config/sort_services_test.py @@ -0,0 +1,240 @@ +import pytest + +from compose.config.errors import DependencyError +from compose.config.sort_services import sort_service_dicts +from compose.config.types import VolumeFromSpec + + +class TestSortService: + def test_sort_service_dicts_1(self): + services = [ + { + 'links': ['redis'], + 'name': 'web' + }, + { + 'name': 'grunt' + }, + { + 'name': 'redis' + } + ] + + sorted_services = sort_service_dicts(services) + assert len(sorted_services) == 3 + assert sorted_services[0]['name'] == 'grunt' + assert sorted_services[1]['name'] == 'redis' + assert sorted_services[2]['name'] == 'web' + + def test_sort_service_dicts_2(self): + services = [ + { + 'links': ['redis', 'postgres'], + 'name': 'web' + }, + { + 'name': 'postgres', + 'links': ['redis'] + }, + { + 'name': 'redis' + } + ] + + sorted_services = sort_service_dicts(services) + assert len(sorted_services) == 3 + assert sorted_services[0]['name'] == 'redis' + assert sorted_services[1]['name'] == 'postgres' + assert sorted_services[2]['name'] == 'web' + + def test_sort_service_dicts_3(self): + services = [ + { + 'name': 'child' + }, + { + 'name': 'parent', + 'links': ['child'] + }, + { + 'links': ['parent'], + 'name': 'grandparent' + }, + ] + + sorted_services = sort_service_dicts(services) + assert len(sorted_services) == 3 + assert sorted_services[0]['name'] == 'child' + assert sorted_services[1]['name'] == 'parent' + assert sorted_services[2]['name'] == 'grandparent' + + def test_sort_service_dicts_4(self): + services = [ + { + 'name': 'child' + }, + { + 'name': 'parent', + 'volumes_from': [VolumeFromSpec('child', 'rw', 'service')] + }, + { + 'links': ['parent'], + 'name': 'grandparent' + }, + ] + + sorted_services = sort_service_dicts(services) + assert len(sorted_services) == 3 + assert sorted_services[0]['name'] == 'child' + assert sorted_services[1]['name'] == 'parent' + assert sorted_services[2]['name'] == 'grandparent' + + def test_sort_service_dicts_5(self): + services = [ + { + 'links': ['parent'], + 'name': 'grandparent' + }, + { + 'name': 'parent', + 'network_mode': 'service:child' + }, + { + 'name': 'child' + } + ] + + sorted_services = sort_service_dicts(services) + assert len(sorted_services) == 3 + assert sorted_services[0]['name'] == 'child' + assert sorted_services[1]['name'] == 'parent' + assert sorted_services[2]['name'] == 'grandparent' + + def test_sort_service_dicts_6(self): + services = [ + { + 'links': ['parent'], + 'name': 'grandparent' + }, + { + 'name': 'parent', + 'volumes_from': [VolumeFromSpec('child', 'ro', 'service')] + }, + { + 'name': 'child' + } + ] + + sorted_services = sort_service_dicts(services) + assert len(sorted_services) == 3 + assert sorted_services[0]['name'] == 'child' + assert sorted_services[1]['name'] == 'parent' + assert sorted_services[2]['name'] == 'grandparent' + + def test_sort_service_dicts_7(self): + services = [ + { + 'network_mode': 'service:three', + 'name': 'four' + }, + { + 'links': ['two'], + 'name': 'three' + }, + { + 'name': 'two', + 'volumes_from': [VolumeFromSpec('one', 'rw', 'service')] + }, + { + 'name': 'one' + } + ] + + sorted_services = sort_service_dicts(services) + assert len(sorted_services) == 4 + assert sorted_services[0]['name'] == 'one' + assert sorted_services[1]['name'] == 'two' + assert sorted_services[2]['name'] == 'three' + assert sorted_services[3]['name'] == 'four' + + def test_sort_service_dicts_circular_imports(self): + services = [ + { + 'links': ['redis'], + 'name': 'web' + }, + { + 'name': 'redis', + 'links': ['web'] + }, + ] + + with pytest.raises(DependencyError) as exc: + sort_service_dicts(services) + assert 'redis' in exc.exconly() + assert 'web' in exc.exconly() + + def test_sort_service_dicts_circular_imports_2(self): + services = [ + { + 'links': ['postgres', 'redis'], + 'name': 'web' + }, + { + 'name': 'redis', + 'links': ['web'] + }, + { + 'name': 'postgres' + } + ] + + with pytest.raises(DependencyError) as exc: + sort_service_dicts(services) + assert 'redis' in exc.exconly() + assert 'web' in exc.exconly() + + def test_sort_service_dicts_circular_imports_3(self): + services = [ + { + 'links': ['b'], + 'name': 'a' + }, + { + 'name': 'b', + 'links': ['c'] + }, + { + 'name': 'c', + 'links': ['a'] + } + ] + + with pytest.raises(DependencyError) as exc: + sort_service_dicts(services) + assert 'a' in exc.exconly() + assert 'b' in exc.exconly() + + def test_sort_service_dicts_self_imports(self): + services = [ + { + 'links': ['web'], + 'name': 'web' + }, + ] + + with pytest.raises(DependencyError) as exc: + sort_service_dicts(services) + assert 'web' in exc.exconly() + + def test_sort_service_dicts_depends_on_self(self): + services = [ + { + 'depends_on': ['web'], + 'name': 'web' + }, + ] + + with pytest.raises(DependencyError) as exc: + sort_service_dicts(services) + assert 'A service can not depend on itself: web' in exc.exconly() diff --git a/tests/unit/config/types_test.py b/tests/unit/config/types_test.py new file mode 100644 index 00000000000..e5fcde1a621 --- /dev/null +++ b/tests/unit/config/types_test.py @@ -0,0 +1,258 @@ +import pytest + +from compose.config.errors import ConfigurationError +from compose.config.types import parse_extra_hosts +from compose.config.types import ServicePort +from compose.config.types import VolumeFromSpec +from compose.config.types import VolumeSpec +from compose.const import COMPOSE_SPEC as VERSION +from compose.const import COMPOSEFILE_V1 as V1 + + +def test_parse_extra_hosts_list(): + expected = {'www.example.com': '192.168.0.17'} + assert parse_extra_hosts(["www.example.com:192.168.0.17"]) == expected + + expected = {'www.example.com': '192.168.0.17'} + assert parse_extra_hosts(["www.example.com: 192.168.0.17"]) == expected + + assert parse_extra_hosts([ + "www.example.com: 192.168.0.17", + "static.example.com:192.168.0.19", + "api.example.com: 192.168.0.18", + "v6.example.com: ::1" + ]) == { + 'www.example.com': '192.168.0.17', + 'static.example.com': '192.168.0.19', + 'api.example.com': '192.168.0.18', + 'v6.example.com': '::1' + } + + +def test_parse_extra_hosts_dict(): + assert parse_extra_hosts({ + 'www.example.com': '192.168.0.17', + 'api.example.com': '192.168.0.18' + }) == { + 'www.example.com': '192.168.0.17', + 'api.example.com': '192.168.0.18' + } + + +class TestServicePort: + def test_parse_dict(self): + data = { + 'target': 8000, + 'published': 8000, + 'protocol': 'udp', + 'mode': 'global', + } + ports = ServicePort.parse(data) + assert len(ports) == 1 + assert ports[0].repr() == data + + def test_parse_simple_target_port(self): + ports = ServicePort.parse(8000) + assert len(ports) == 1 + assert ports[0].target == 8000 + + def test_parse_complete_port_definition(self): + port_def = '1.1.1.1:3000:3000/udp' + ports = ServicePort.parse(port_def) + assert len(ports) == 1 + assert ports[0].repr() == { + 'target': 3000, + 'published': 3000, + 'external_ip': '1.1.1.1', + 'protocol': 'udp', + } + assert ports[0].legacy_repr() == port_def + + def test_parse_ext_ip_no_published_port(self): + port_def = '1.1.1.1::3000' + ports = ServicePort.parse(port_def) + assert len(ports) == 1 + assert ports[0].legacy_repr() == port_def + '/tcp' + assert ports[0].repr() == { + 'target': 3000, + 'external_ip': '1.1.1.1', + } + + def test_repr_published_port_0(self): + port_def = '0:4000' + ports = ServicePort.parse(port_def) + assert len(ports) == 1 + assert ports[0].legacy_repr() == port_def + '/tcp' + + def test_parse_port_range(self): + ports = ServicePort.parse('25000-25001:4000-4001') + assert len(ports) == 2 + reprs = [p.repr() for p in ports] + assert { + 'target': 4000, + 'published': 25000 + } in reprs + assert { + 'target': 4001, + 'published': 25001 + } in reprs + + def test_parse_port_publish_range(self): + ports = ServicePort.parse('4440-4450:4000') + assert len(ports) == 1 + reprs = [p.repr() for p in ports] + assert { + 'target': 4000, + 'published': '4440-4450' + } in reprs + + def test_parse_invalid_port(self): + port_def = '4000p' + with pytest.raises(ConfigurationError): + ServicePort.parse(port_def) + + def test_parse_invalid_publish_range(self): + port_def = '-4000:4000' + with pytest.raises(ConfigurationError): + ServicePort.parse(port_def) + + port_def = 'asdf:4000' + with pytest.raises(ConfigurationError): + ServicePort.parse(port_def) + + port_def = '1234-12f:4000' + with pytest.raises(ConfigurationError): + ServicePort.parse(port_def) + + port_def = '1234-1235-1239:4000' + with pytest.raises(ConfigurationError): + ServicePort.parse(port_def) + + +class TestVolumeSpec: + + def test_parse_volume_spec_only_one_path(self): + spec = VolumeSpec.parse('/the/volume') + assert spec == (None, '/the/volume', 'rw') + + def test_parse_volume_spec_internal_and_external(self): + spec = VolumeSpec.parse('external:interval') + assert spec == ('external', 'interval', 'rw') + + def test_parse_volume_spec_with_mode(self): + spec = VolumeSpec.parse('external:interval:ro') + assert spec == ('external', 'interval', 'ro') + + spec = VolumeSpec.parse('external:interval:z') + assert spec == ('external', 'interval', 'z') + + def test_parse_volume_spec_too_many_parts(self): + with pytest.raises(ConfigurationError) as exc: + VolumeSpec.parse('one:two:three:four') + assert 'has incorrect format' in exc.exconly() + + def test_parse_volume_windows_absolute_path_normalized(self): + windows_path = "c:\\Users\\me\\Documents\\shiny\\config:/opt/shiny/config:ro" + assert VolumeSpec._parse_win32(windows_path, True) == ( + "/c/Users/me/Documents/shiny/config", + "/opt/shiny/config", + "ro" + ) + + def test_parse_volume_windows_absolute_path_native(self): + windows_path = "c:\\Users\\me\\Documents\\shiny\\config:/opt/shiny/config:ro" + assert VolumeSpec._parse_win32(windows_path, False) == ( + "c:\\Users\\me\\Documents\\shiny\\config", + "/opt/shiny/config", + "ro" + ) + + def test_parse_volume_windows_internal_path_normalized(self): + windows_path = 'C:\\Users\\reimu\\scarlet:C:\\scarlet\\app:ro' + assert VolumeSpec._parse_win32(windows_path, True) == ( + '/c/Users/reimu/scarlet', + 'C:\\scarlet\\app', + 'ro' + ) + + def test_parse_volume_windows_internal_path_native(self): + windows_path = 'C:\\Users\\reimu\\scarlet:C:\\scarlet\\app:ro' + assert VolumeSpec._parse_win32(windows_path, False) == ( + 'C:\\Users\\reimu\\scarlet', + 'C:\\scarlet\\app', + 'ro' + ) + + def test_parse_volume_windows_just_drives_normalized(self): + windows_path = 'E:\\:C:\\:ro' + assert VolumeSpec._parse_win32(windows_path, True) == ( + '/e/', + 'C:\\', + 'ro' + ) + + def test_parse_volume_windows_just_drives_native(self): + windows_path = 'E:\\:C:\\:ro' + assert VolumeSpec._parse_win32(windows_path, False) == ( + 'E:\\', + 'C:\\', + 'ro' + ) + + def test_parse_volume_windows_mixed_notations_normalized(self): + windows_path = 'C:\\Foo:/root/foo' + assert VolumeSpec._parse_win32(windows_path, True) == ( + '/c/Foo', + '/root/foo', + 'rw' + ) + + def test_parse_volume_windows_mixed_notations_native(self): + windows_path = 'C:\\Foo:/root/foo' + assert VolumeSpec._parse_win32(windows_path, False) == ( + 'C:\\Foo', + '/root/foo', + 'rw' + ) + + +class TestVolumesFromSpec: + + services = ['servicea', 'serviceb'] + + def test_parse_v1_from_service(self): + volume_from = VolumeFromSpec.parse('servicea', self.services, V1) + assert volume_from == VolumeFromSpec('servicea', 'rw', 'service') + + def test_parse_v1_from_container(self): + volume_from = VolumeFromSpec.parse('foo:ro', self.services, V1) + assert volume_from == VolumeFromSpec('foo', 'ro', 'container') + + def test_parse_v1_invalid(self): + with pytest.raises(ConfigurationError): + VolumeFromSpec.parse('unknown:format:ro', self.services, V1) + + def test_parse_v2_from_service(self): + volume_from = VolumeFromSpec.parse('servicea', self.services, VERSION) + assert volume_from == VolumeFromSpec('servicea', 'rw', 'service') + + def test_parse_v2_from_service_with_mode(self): + volume_from = VolumeFromSpec.parse('servicea:ro', self.services, VERSION) + assert volume_from == VolumeFromSpec('servicea', 'ro', 'service') + + def test_parse_v2_from_container(self): + volume_from = VolumeFromSpec.parse('container:foo', self.services, VERSION) + assert volume_from == VolumeFromSpec('foo', 'rw', 'container') + + def test_parse_v2_from_container_with_mode(self): + volume_from = VolumeFromSpec.parse('container:foo:ro', self.services, VERSION) + assert volume_from == VolumeFromSpec('foo', 'ro', 'container') + + def test_parse_v2_invalid_type(self): + with pytest.raises(ConfigurationError) as exc: + VolumeFromSpec.parse('bogus:foo:ro', self.services, VERSION) + assert "Unknown volumes_from type 'bogus'" in exc.exconly() + + def test_parse_v2_invalid(self): + with pytest.raises(ConfigurationError): + VolumeFromSpec.parse('unknown:format:ro', self.services, VERSION) diff --git a/tests/unit/container_test.py b/tests/unit/container_test.py new file mode 100644 index 00000000000..452475209c5 --- /dev/null +++ b/tests/unit/container_test.py @@ -0,0 +1,266 @@ +import docker + +from .. import mock +from .. import unittest +from ..helpers import BUSYBOX_IMAGE_WITH_TAG +from compose.const import LABEL_ONE_OFF +from compose.const import LABEL_SLUG +from compose.container import Container +from compose.container import get_container_name + + +class ContainerTest(unittest.TestCase): + + def setUp(self): + self.container_id = "abcabcabcbabc12345" + self.container_dict = { + "Id": self.container_id, + "Image": BUSYBOX_IMAGE_WITH_TAG, + "Command": "top", + "Created": 1387384730, + "Status": "Up 8 seconds", + "Ports": None, + "SizeRw": 0, + "SizeRootFs": 0, + "Names": ["/composetest_db_1", "/composetest_web_1/db"], + "NetworkSettings": { + "Ports": {}, + }, + "Config": { + "Labels": { + "com.docker.compose.project": "composetest", + "com.docker.compose.service": "web", + "com.docker.compose.container-number": "7", + }, + } + } + + def test_from_ps(self): + container = Container.from_ps(None, + self.container_dict, + has_been_inspected=True) + assert container.dictionary == { + "Id": self.container_id, + "Image": BUSYBOX_IMAGE_WITH_TAG, + "Name": "/composetest_db_1", + } + + def test_from_ps_prefixed(self): + self.container_dict['Names'] = [ + '/swarm-host-1' + n for n in self.container_dict['Names'] + ] + + container = Container.from_ps( + None, + self.container_dict, + has_been_inspected=True) + assert container.dictionary == { + "Id": self.container_id, + "Image": BUSYBOX_IMAGE_WITH_TAG, + "Name": "/composetest_db_1", + } + + def test_environment(self): + container = Container(None, { + 'Id': 'abc', + 'Config': { + 'Env': [ + 'FOO=BAR', + 'BAZ=DOGE', + ] + } + }, has_been_inspected=True) + assert container.environment == { + 'FOO': 'BAR', + 'BAZ': 'DOGE', + } + + def test_number(self): + container = Container(None, self.container_dict, has_been_inspected=True) + assert container.number == 7 + + def test_name(self): + container = Container.from_ps(None, + self.container_dict, + has_been_inspected=True) + assert container.name == "composetest_db_1" + + def test_name_without_project(self): + self.container_dict['Name'] = "/composetest_web_7" + container = Container(None, self.container_dict, has_been_inspected=True) + assert container.name_without_project == "web_7" + + def test_name_without_project_custom_container_name(self): + self.container_dict['Name'] = "/custom_name_of_container" + container = Container(None, self.container_dict, has_been_inspected=True) + assert container.name_without_project == "custom_name_of_container" + + def test_name_without_project_one_off(self): + self.container_dict['Name'] = "/composetest_web_092cd63296f" + self.container_dict['Config']['Labels'][LABEL_SLUG] = ( + "092cd63296fdc446ad432d3905dd1fcbe12a2ba6b52" + ) + self.container_dict['Config']['Labels'][LABEL_ONE_OFF] = 'True' + container = Container(None, self.container_dict, has_been_inspected=True) + assert container.name_without_project == 'web_092cd63296fd' + + def test_inspect_if_not_inspected(self): + mock_client = mock.create_autospec(docker.APIClient) + container = Container(mock_client, dict(Id="the_id")) + + container.inspect_if_not_inspected() + mock_client.inspect_container.assert_called_once_with("the_id") + assert container.dictionary == mock_client.inspect_container.return_value + assert container.has_been_inspected + + container.inspect_if_not_inspected() + assert mock_client.inspect_container.call_count == 1 + + def test_human_readable_ports_none(self): + container = Container(None, self.container_dict, has_been_inspected=True) + assert container.human_readable_ports == '' + + def test_human_readable_ports_public_and_private(self): + self.container_dict['NetworkSettings']['Ports'].update({ + "45454/tcp": [{"HostIp": "0.0.0.0", "HostPort": "49197"}], + "45453/tcp": [], + }) + container = Container(None, self.container_dict, has_been_inspected=True) + + expected = "45453/tcp, 0.0.0.0:49197->45454/tcp" + assert container.human_readable_ports == expected + + def test_get_local_port(self): + self.container_dict['NetworkSettings']['Ports'].update({ + "45454/tcp": [{"HostIp": "0.0.0.0", "HostPort": "49197"}], + }) + container = Container(None, self.container_dict, has_been_inspected=True) + + assert container.get_local_port(45454, protocol='tcp') == '0.0.0.0:49197' + + def test_human_readable_states_no_health(self): + container = Container(None, { + "State": { + "Status": "running", + "Running": True, + "Paused": False, + "Restarting": False, + "OOMKilled": False, + "Dead": False, + "Pid": 7623, + "ExitCode": 0, + "Error": "", + "StartedAt": "2018-01-29T00:34:25.2052414Z", + "FinishedAt": "0001-01-01T00:00:00Z" + }, + }, has_been_inspected=True) + expected = "Up" + assert container.human_readable_state == expected + + def test_human_readable_states_starting(self): + container = Container(None, { + "State": { + "Status": "running", + "Running": True, + "Paused": False, + "Restarting": False, + "OOMKilled": False, + "Dead": False, + "Pid": 11744, + "ExitCode": 0, + "Error": "", + "StartedAt": "2018-02-03T07:56:20.3591233Z", + "FinishedAt": "2018-01-31T08:56:11.0505228Z", + "Health": { + "Status": "starting", + "FailingStreak": 0, + "Log": [] + } + } + }, has_been_inspected=True) + expected = "Up (health: starting)" + assert container.human_readable_state == expected + + def test_human_readable_states_healthy(self): + container = Container(None, { + "State": { + "Status": "running", + "Running": True, + "Paused": False, + "Restarting": False, + "OOMKilled": False, + "Dead": False, + "Pid": 5674, + "ExitCode": 0, + "Error": "", + "StartedAt": "2018-02-03T08:32:05.3281831Z", + "FinishedAt": "2018-02-03T08:11:35.7872706Z", + "Health": { + "Status": "healthy", + "FailingStreak": 0, + "Log": [] + } + } + }, has_been_inspected=True) + expected = "Up (healthy)" + assert container.human_readable_state == expected + + def test_get(self): + container = Container(None, { + "Status": "Up 8 seconds", + "HostConfig": { + "VolumesFrom": ["volume_id"] + }, + }, has_been_inspected=True) + + assert container.get('Status') == "Up 8 seconds" + assert container.get('HostConfig.VolumesFrom') == ["volume_id"] + assert container.get('Foo.Bar.DoesNotExist') is None + + def test_short_id(self): + container = Container(None, self.container_dict, has_been_inspected=True) + assert container.short_id == self.container_id[:12] + + def test_has_api_logs(self): + container_dict = { + 'HostConfig': { + 'LogConfig': { + 'Type': 'json-file' + } + } + } + + container = Container(None, container_dict, has_been_inspected=True) + assert container.has_api_logs is True + + container_dict['HostConfig']['LogConfig']['Type'] = 'none' + container = Container(None, container_dict, has_been_inspected=True) + assert container.has_api_logs is False + + container_dict['HostConfig']['LogConfig']['Type'] = 'syslog' + container = Container(None, container_dict, has_been_inspected=True) + assert container.has_api_logs is False + + container_dict['HostConfig']['LogConfig']['Type'] = 'journald' + container = Container(None, container_dict, has_been_inspected=True) + assert container.has_api_logs is True + + container_dict['HostConfig']['LogConfig']['Type'] = 'foobar' + container = Container(None, container_dict, has_been_inspected=True) + assert container.has_api_logs is False + + +class GetContainerNameTestCase(unittest.TestCase): + + def test_get_container_name(self): + assert get_container_name({}) is None + assert get_container_name({'Name': 'myproject_db_1'}) == 'myproject_db_1' + assert get_container_name( + {'Names': ['/myproject_db_1', '/myproject_web_1/db']} + ) == 'myproject_db_1' + assert get_container_name({ + 'Names': [ + '/swarm-host-1/myproject_db_1', + '/swarm-host-1/myproject_web_1/db' + ] + }) == 'myproject_db_1' diff --git a/tests/unit/metrics/__init__.py b/tests/unit/metrics/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/tests/unit/metrics/metrics_test.py b/tests/unit/metrics/metrics_test.py new file mode 100644 index 00000000000..e9f23720a35 --- /dev/null +++ b/tests/unit/metrics/metrics_test.py @@ -0,0 +1,36 @@ +import unittest + +from compose.metrics.client import MetricsCommand +from compose.metrics.client import Status + + +class MetricsTest(unittest.TestCase): + @classmethod + def test_metrics(cls): + assert MetricsCommand('up', 'moby').to_map() == { + 'command': 'compose up', + 'context': 'moby', + 'status': 'success', + 'source': 'docker-compose', + } + + assert MetricsCommand('down', 'local').to_map() == { + 'command': 'compose down', + 'context': 'local', + 'status': 'success', + 'source': 'docker-compose', + } + + assert MetricsCommand('help', 'aci', Status.FAILURE).to_map() == { + 'command': 'compose help', + 'context': 'aci', + 'status': 'failure', + 'source': 'docker-compose', + } + + assert MetricsCommand('run', 'ecs').to_map() == { + 'command': 'compose run', + 'context': 'ecs', + 'status': 'success', + 'source': 'docker-compose', + } diff --git a/tests/unit/network_test.py b/tests/unit/network_test.py new file mode 100644 index 00000000000..ab7ad59cfbd --- /dev/null +++ b/tests/unit/network_test.py @@ -0,0 +1,172 @@ +import pytest + +from .. import mock +from .. import unittest +from compose.network import check_remote_network_config +from compose.network import Network +from compose.network import NetworkConfigChangedError + + +class NetworkTest(unittest.TestCase): + def test_check_remote_network_config_success(self): + options = {'com.docker.network.driver.foo': 'bar'} + ipam_config = { + 'driver': 'default', + 'config': [ + {'subnet': '172.0.0.1/16', }, + { + 'subnet': '156.0.0.1/25', + 'gateway': '156.0.0.1', + 'aux_addresses': ['11.0.0.1', '24.25.26.27'], + 'ip_range': '156.0.0.1-254' + } + ], + 'options': { + 'iface': 'eth0', + } + } + labels = { + 'com.project.tests.istest': 'true', + 'com.project.sound.track': 'way out of here', + } + remote_labels = labels.copy() + remote_labels.update({ + 'com.docker.compose.project': 'compose_test', + 'com.docker.compose.network': 'net1', + }) + net = Network( + None, 'compose_test', 'net1', 'bridge', + options, enable_ipv6=True, ipam=ipam_config, + labels=labels + ) + check_remote_network_config( + { + 'Driver': 'bridge', + 'Options': options, + 'EnableIPv6': True, + 'Internal': False, + 'Attachable': True, + 'IPAM': { + 'Driver': 'default', + 'Config': [{ + 'Subnet': '156.0.0.1/25', + 'Gateway': '156.0.0.1', + 'AuxiliaryAddresses': ['24.25.26.27', '11.0.0.1'], + 'IPRange': '156.0.0.1-254' + }, { + 'Subnet': '172.0.0.1/16', + 'Gateway': '172.0.0.1' + }], + 'Options': { + 'iface': 'eth0', + }, + }, + 'Labels': remote_labels + }, + net + ) + + def test_check_remote_network_config_whitelist(self): + options = {'com.docker.network.driver.foo': 'bar'} + remote_options = { + 'com.docker.network.driver.overlay.vxlanid_list': '257', + 'com.docker.network.driver.foo': 'bar', + 'com.docker.network.windowsshim.hnsid': 'aac3fd4887daaec1e3b', + } + net = Network( + None, 'compose_test', 'net1', 'overlay', + options + ) + check_remote_network_config( + {'Driver': 'overlay', 'Options': remote_options}, net + ) + + @mock.patch('compose.network.Network.true_name', lambda n: n.full_name) + def test_check_remote_network_config_driver_mismatch(self): + net = Network(None, 'compose_test', 'net1', 'overlay') + with pytest.raises(NetworkConfigChangedError) as e: + check_remote_network_config( + {'Driver': 'bridge', 'Options': {}}, net + ) + + assert 'driver has changed' in str(e.value) + + @mock.patch('compose.network.Network.true_name', lambda n: n.full_name) + def test_check_remote_network_config_options_mismatch(self): + net = Network(None, 'compose_test', 'net1', 'overlay') + with pytest.raises(NetworkConfigChangedError) as e: + check_remote_network_config({'Driver': 'overlay', 'Options': { + 'com.docker.network.driver.foo': 'baz' + }}, net) + + assert 'option "com.docker.network.driver.foo" has changed' in str(e.value) + + def test_check_remote_network_config_null_remote(self): + net = Network(None, 'compose_test', 'net1', 'overlay') + check_remote_network_config( + {'Driver': 'overlay', 'Options': None}, net + ) + + def test_check_remote_network_config_null_remote_ipam_options(self): + ipam_config = { + 'driver': 'default', + 'config': [ + {'subnet': '172.0.0.1/16', }, + { + 'subnet': '156.0.0.1/25', + 'gateway': '156.0.0.1', + 'aux_addresses': ['11.0.0.1', '24.25.26.27'], + 'ip_range': '156.0.0.1-254' + } + ] + } + net = Network( + None, 'compose_test', 'net1', 'bridge', ipam=ipam_config, + ) + + check_remote_network_config( + { + 'Driver': 'bridge', + 'Attachable': True, + 'IPAM': { + 'Driver': 'default', + 'Config': [{ + 'Subnet': '156.0.0.1/25', + 'Gateway': '156.0.0.1', + 'AuxiliaryAddresses': ['24.25.26.27', '11.0.0.1'], + 'IPRange': '156.0.0.1-254' + }, { + 'Subnet': '172.0.0.1/16', + 'Gateway': '172.0.0.1' + }], + 'Options': None + }, + }, + net + ) + + @mock.patch('compose.network.Network.true_name', lambda n: n.full_name) + def test_check_remote_network_labels_mismatch(self): + net = Network(None, 'compose_test', 'net1', 'overlay', labels={ + 'com.project.touhou.character': 'sakuya.izayoi' + }) + remote = { + 'Driver': 'overlay', + 'Options': None, + 'Labels': { + 'com.docker.compose.network': 'net1', + 'com.docker.compose.project': 'compose_test', + 'com.project.touhou.character': 'marisa.kirisame', + } + } + with mock.patch('compose.network.log') as mock_log: + check_remote_network_config(remote, net) + + mock_log.warning.assert_called_once_with(mock.ANY) + _, args, kwargs = mock_log.warning.mock_calls[0] + assert 'label "com.project.touhou.character" has changed' in args[0] + + def test_remote_config_labels_none(self): + remote = {'Labels': None} + local = Network(None, 'test_project', 'test_network') + check_remote_network_config(remote, local) diff --git a/tests/unit/parallel_test.py b/tests/unit/parallel_test.py new file mode 100644 index 00000000000..98412f9a259 --- /dev/null +++ b/tests/unit/parallel_test.py @@ -0,0 +1,186 @@ +import unittest +from threading import Lock + +from docker.errors import APIError + +from compose.parallel import GlobalLimit +from compose.parallel import parallel_execute +from compose.parallel import parallel_execute_iter +from compose.parallel import ParallelStreamWriter +from compose.parallel import UpstreamError + + +web = 'web' +db = 'db' +data_volume = 'data_volume' +cache = 'cache' + +objects = [web, db, data_volume, cache] + +deps = { + web: [db, cache], + db: [data_volume], + data_volume: [], + cache: [], +} + + +def get_deps(obj): + return [(dep, None) for dep in deps[obj]] + + +class ParallelTest(unittest.TestCase): + + def test_parallel_execute(self): + results, errors = parallel_execute( + objects=[1, 2, 3, 4, 5], + func=lambda x: x * 2, + get_name=str, + msg="Doubling", + ) + + assert sorted(results) == [2, 4, 6, 8, 10] + assert errors == {} + + def test_parallel_execute_with_limit(self): + limit = 1 + tasks = 20 + lock = Lock() + + def f(obj): + locked = lock.acquire(False) + # we should always get the lock because we're the only thread running + assert locked + lock.release() + return None + + results, errors = parallel_execute( + objects=list(range(tasks)), + func=f, + get_name=str, + msg="Testing", + limit=limit, + ) + + assert results == tasks * [None] + assert errors == {} + + def test_parallel_execute_with_global_limit(self): + GlobalLimit.set_global_limit(1) + self.addCleanup(GlobalLimit.set_global_limit, None) + tasks = 20 + lock = Lock() + + def f(obj): + locked = lock.acquire(False) + # we should always get the lock because we're the only thread running + assert locked + lock.release() + return None + + results, errors = parallel_execute( + objects=list(range(tasks)), + func=f, + get_name=str, + msg="Testing", + ) + + assert results == tasks * [None] + assert errors == {} + + def test_parallel_execute_with_deps(self): + log = [] + + def process(x): + log.append(x) + + parallel_execute( + objects=objects, + func=process, + get_name=lambda obj: obj, + msg="Processing", + get_deps=get_deps, + ) + + assert sorted(log) == sorted(objects) + + assert log.index(data_volume) < log.index(db) + assert log.index(db) < log.index(web) + assert log.index(cache) < log.index(web) + + def test_parallel_execute_with_upstream_errors(self): + log = [] + + def process(x): + if x is data_volume: + raise APIError(None, None, "Something went wrong") + log.append(x) + + parallel_execute( + objects=objects, + func=process, + get_name=lambda obj: obj, + msg="Processing", + get_deps=get_deps, + ) + + assert log == [cache] + + events = [ + (obj, result, type(exception)) + for obj, result, exception + in parallel_execute_iter(objects, process, get_deps, None) + ] + + assert (cache, None, type(None)) in events + assert (data_volume, None, APIError) in events + assert (db, None, UpstreamError) in events + assert (web, None, UpstreamError) in events + + +def test_parallel_execute_alignment(capsys): + ParallelStreamWriter.instance = None + results, errors = parallel_execute( + objects=["short", "a very long name"], + func=lambda x: x, + get_name=str, + msg="Aligning", + ) + + assert errors == {} + + _, err = capsys.readouterr() + a, b = err.split('\n')[:2] + assert a.index('...') == b.index('...') + + +def test_parallel_execute_ansi(capsys): + ParallelStreamWriter.instance = None + ParallelStreamWriter.set_noansi(value=False) + results, errors = parallel_execute( + objects=["something", "something more"], + func=lambda x: x, + get_name=str, + msg="Control characters", + ) + + assert errors == {} + + _, err = capsys.readouterr() + assert "\x1b" in err + + +def test_parallel_execute_noansi(capsys): + ParallelStreamWriter.instance = None + ParallelStreamWriter.set_noansi() + results, errors = parallel_execute( + objects=["something", "something more"], + func=lambda x: x, + get_name=str, + msg="Control characters", + ) + + assert errors == {} + + _, err = capsys.readouterr() + assert "\x1b" not in err diff --git a/tests/unit/progress_stream_test.py b/tests/unit/progress_stream_test.py new file mode 100644 index 00000000000..288c9b6e448 --- /dev/null +++ b/tests/unit/progress_stream_test.py @@ -0,0 +1,114 @@ +import os +import random +import shutil +import tempfile +from io import StringIO + +from compose import progress_stream +from tests import unittest + + +class ProgressStreamTestCase(unittest.TestCase): + def test_stream_output(self): + output = [ + b'{"status": "Downloading", "progressDetail": {"current": ' + b'31019763, "start": 1413653874, "total": 62763875}, ' + b'"progress": "..."}', + ] + events = list(progress_stream.stream_output(output, StringIO())) + assert len(events) == 1 + + def test_stream_output_div_zero(self): + output = [ + b'{"status": "Downloading", "progressDetail": {"current": ' + b'0, "start": 1413653874, "total": 0}, ' + b'"progress": "..."}', + ] + events = list(progress_stream.stream_output(output, StringIO())) + assert len(events) == 1 + + def test_stream_output_null_total(self): + output = [ + b'{"status": "Downloading", "progressDetail": {"current": ' + b'0, "start": 1413653874, "total": null}, ' + b'"progress": "..."}', + ] + events = list(progress_stream.stream_output(output, StringIO())) + assert len(events) == 1 + + def test_stream_output_progress_event_tty(self): + events = [ + b'{"status": "Already exists", "progressDetail": {}, "id": "8d05e3af52b0"}' + ] + + class TTYStringIO(StringIO): + def isatty(self): + return True + + output = TTYStringIO() + events = list(progress_stream.stream_output(events, output)) + assert len(output.getvalue()) > 0 + + def test_stream_output_progress_event_no_tty(self): + events = [ + b'{"status": "Already exists", "progressDetail": {}, "id": "8d05e3af52b0"}' + ] + output = StringIO() + + events = list(progress_stream.stream_output(events, output)) + assert len(output.getvalue()) == 0 + + def test_stream_output_no_progress_event_no_tty(self): + events = [ + b'{"status": "Pulling from library/xy", "id": "latest"}' + ] + output = StringIO() + + events = list(progress_stream.stream_output(events, output)) + assert len(output.getvalue()) > 0 + + def test_mismatched_encoding_stream_write(self): + tmpdir = tempfile.mkdtemp() + self.addCleanup(shutil.rmtree, tmpdir, True) + + def mktempfile(encoding): + fname = os.path.join(tmpdir, hex(random.getrandbits(128))[2:-1]) + return open(fname, mode='w+', encoding=encoding) + + text = '就吃饭' + with mktempfile(encoding='utf-8') as tf: + progress_stream.write_to_stream(text, tf) + tf.seek(0) + assert tf.read() == text + + with mktempfile(encoding='utf-32') as tf: + progress_stream.write_to_stream(text, tf) + tf.seek(0) + assert tf.read() == text + + with mktempfile(encoding='ascii') as tf: + progress_stream.write_to_stream(text, tf) + tf.seek(0) + assert tf.read() == '???' + + def test_get_digest_from_push(self): + digest = "sha256:abcd" + events = [ + {"status": "..."}, + {"status": "..."}, + {"progressDetail": {}, "aux": {"Digest": digest}}, + ] + assert progress_stream.get_digest_from_push(events) == digest + + def test_get_digest_from_pull(self): + events = list() + assert progress_stream.get_digest_from_pull(events) is None + + digest = "sha256:abcd" + events = [ + {"status": "..."}, + {"status": "..."}, + {"status": "Digest: %s" % digest}, + {"status": "..."}, + ] + assert progress_stream.get_digest_from_pull(events) == digest diff --git a/tests/unit/project_test.py b/tests/unit/project_test.py new file mode 100644 index 00000000000..a3ffdb67dae --- /dev/null +++ b/tests/unit/project_test.py @@ -0,0 +1,922 @@ +import datetime +import os +import tempfile + +import docker +import pytest +from docker.errors import NotFound + +from .. import mock +from .. import unittest +from ..helpers import BUSYBOX_IMAGE_WITH_TAG +from compose.config import ConfigurationError +from compose.config.config import Config +from compose.config.types import VolumeFromSpec +from compose.const import COMPOSE_SPEC as VERSION +from compose.const import COMPOSEFILE_V1 as V1 +from compose.const import DEFAULT_TIMEOUT +from compose.const import LABEL_SERVICE +from compose.container import Container +from compose.errors import OperationFailedError +from compose.project import get_secrets +from compose.project import NoSuchService +from compose.project import Project +from compose.project import ProjectError +from compose.service import ImageType +from compose.service import Service + + +def build_config(**kwargs): + return Config( + config_version=kwargs.get('config_version', VERSION), + version=kwargs.get('version', VERSION), + services=kwargs.get('services'), + volumes=kwargs.get('volumes'), + networks=kwargs.get('networks'), + secrets=kwargs.get('secrets'), + configs=kwargs.get('configs'), + ) + + +class ProjectTest(unittest.TestCase): + def setUp(self): + self.mock_client = mock.create_autospec(docker.APIClient) + self.mock_client._general_configs = {} + self.mock_client.api_version = docker.constants.DEFAULT_DOCKER_API_VERSION + + def test_from_config_v1(self): + config = build_config( + version=V1, + services=[ + { + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + }, + { + 'name': 'db', + 'image': BUSYBOX_IMAGE_WITH_TAG, + }, + ], + networks=None, + volumes=None, + secrets=None, + configs=None, + ) + project = Project.from_config( + name='composetest', + config_data=config, + client=None, + ) + assert len(project.services) == 2 + assert project.get_service('web').name == 'web' + assert project.get_service('web').options['image'] == BUSYBOX_IMAGE_WITH_TAG + assert project.get_service('db').name == 'db' + assert project.get_service('db').options['image'] == BUSYBOX_IMAGE_WITH_TAG + assert not project.networks.use_networking + + @mock.patch('compose.network.Network.true_name', lambda n: n.full_name) + def test_from_config_v2(self): + config = build_config( + services=[ + { + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + }, + { + 'name': 'db', + 'image': BUSYBOX_IMAGE_WITH_TAG, + }, + ], + networks=None, + volumes=None, + secrets=None, + configs=None, + ) + project = Project.from_config('composetest', config, None) + assert len(project.services) == 2 + assert project.networks.use_networking + + def test_get_service(self): + web = Service( + project='composetest', + name='web', + client=None, + image=BUSYBOX_IMAGE_WITH_TAG, + ) + project = Project('test', [web], None) + assert project.get_service('web') == web + + def test_get_services_returns_all_services_without_args(self): + web = Service( + project='composetest', + name='web', + image='foo', + ) + console = Service( + project='composetest', + name='console', + image='foo', + ) + project = Project('test', [web, console], None) + assert project.get_services() == [web, console] + + def test_get_services_returns_listed_services_with_args(self): + web = Service( + project='composetest', + name='web', + image='foo', + ) + console = Service( + project='composetest', + name='console', + image='foo', + ) + project = Project('test', [web, console], None) + assert project.get_services(['console']) == [console] + + def test_get_services_with_include_links(self): + db = Service( + project='composetest', + name='db', + image='foo', + ) + web = Service( + project='composetest', + name='web', + image='foo', + links=[(db, 'database')] + ) + cache = Service( + project='composetest', + name='cache', + image='foo' + ) + console = Service( + project='composetest', + name='console', + image='foo', + links=[(web, 'web')] + ) + project = Project('test', [web, db, cache, console], None) + assert project.get_services(['console'], include_deps=True) == [db, web, console] + + def test_get_services_removes_duplicates_following_links(self): + db = Service( + project='composetest', + name='db', + image='foo', + ) + web = Service( + project='composetest', + name='web', + image='foo', + links=[(db, 'database')] + ) + project = Project('test', [web, db], None) + assert project.get_services(['web', 'db'], include_deps=True) == [db, web] + + def test_use_volumes_from_container(self): + container_id = 'aabbccddee' + container_dict = dict(Name='aaa', Id=container_id) + self.mock_client.inspect_container.return_value = container_dict + project = Project.from_config( + name='test', + client=self.mock_client, + config_data=build_config( + services=[{ + 'name': 'test', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes_from': [VolumeFromSpec('aaa', 'rw', 'container')] + }], + networks=None, + volumes=None, + secrets=None, + configs=None, + ), + ) + assert project.get_service('test')._get_volumes_from() == [container_id + ":rw"] + + def test_use_volumes_from_service_no_container(self): + container_name = 'test_vol_1' + self.mock_client.containers.return_value = [ + { + "Name": container_name, + "Names": [container_name], + "Id": container_name, + "Image": BUSYBOX_IMAGE_WITH_TAG + } + ] + project = Project.from_config( + name='test', + client=self.mock_client, + config_data=build_config( + services=[ + { + 'name': 'vol', + 'image': BUSYBOX_IMAGE_WITH_TAG + }, + { + 'name': 'test', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes_from': [VolumeFromSpec('vol', 'rw', 'service')] + } + ], + networks=None, + volumes=None, + secrets=None, + configs=None, + ), + ) + assert project.get_service('test')._get_volumes_from() == [container_name + ":rw"] + + @mock.patch('compose.network.Network.true_name', lambda n: n.full_name) + def test_use_volumes_from_service_container(self): + container_ids = ['aabbccddee', '12345'] + + project = Project.from_config( + name='test', + client=None, + config_data=build_config( + services=[ + { + 'name': 'vol', + 'image': BUSYBOX_IMAGE_WITH_TAG + }, + { + 'name': 'test', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'volumes_from': [VolumeFromSpec('vol', 'rw', 'service')] + } + ], + networks=None, + volumes=None, + secrets=None, + configs=None, + ), + ) + with mock.patch.object(Service, 'containers') as mock_return: + mock_return.return_value = [ + mock.Mock(id=container_id, spec=Container) + for container_id in container_ids] + assert ( + project.get_service('test')._get_volumes_from() == + [container_ids[0] + ':rw'] + ) + + def test_events_legacy(self): + services = [Service(name='web'), Service(name='db')] + project = Project('test', services, self.mock_client) + self.mock_client.api_version = '1.21' + self.mock_client.events.return_value = iter([ + { + 'status': 'create', + 'from': 'example/image', + 'id': 'abcde', + 'time': 1420092061, + 'timeNano': 14200920610000002000, + }, + { + 'status': 'attach', + 'from': 'example/image', + 'id': 'abcde', + 'time': 1420092061, + 'timeNano': 14200920610000003000, + }, + { + 'status': 'create', + 'from': 'example/other', + 'id': 'bdbdbd', + 'time': 1420092061, + 'timeNano': 14200920610000005000, + }, + { + 'status': 'create', + 'from': 'example/db', + 'id': 'ababa', + 'time': 1420092061, + 'timeNano': 14200920610000004000, + }, + { + 'status': 'destroy', + 'from': 'example/db', + 'id': 'eeeee', + 'time': 1420092061, + 'timeNano': 14200920610000004000, + }, + ]) + + def dt_with_microseconds(dt, us): + return datetime.datetime.fromtimestamp(dt).replace(microsecond=us) + + def get_container(cid): + if cid == 'eeeee': + raise NotFound(None, None, "oops") + if cid == 'abcde': + name = 'web' + labels = {LABEL_SERVICE: name} + elif cid == 'ababa': + name = 'db' + labels = {LABEL_SERVICE: name} + else: + labels = {} + name = '' + return { + 'Id': cid, + 'Config': {'Labels': labels}, + 'Name': '/project_%s_1' % name, + } + + self.mock_client.inspect_container.side_effect = get_container + + events = project.events() + + events_list = list(events) + # Assert the return value is a generator + assert not list(events) + assert events_list == [ + { + 'type': 'container', + 'service': 'web', + 'action': 'create', + 'id': 'abcde', + 'attributes': { + 'name': 'project_web_1', + 'image': 'example/image', + }, + 'time': dt_with_microseconds(1420092061, 2), + 'container': Container(None, {'Id': 'abcde'}), + }, + { + 'type': 'container', + 'service': 'web', + 'action': 'attach', + 'id': 'abcde', + 'attributes': { + 'name': 'project_web_1', + 'image': 'example/image', + }, + 'time': dt_with_microseconds(1420092061, 3), + 'container': Container(None, {'Id': 'abcde'}), + }, + { + 'type': 'container', + 'service': 'db', + 'action': 'create', + 'id': 'ababa', + 'attributes': { + 'name': 'project_db_1', + 'image': 'example/db', + }, + 'time': dt_with_microseconds(1420092061, 4), + 'container': Container(None, {'Id': 'ababa'}), + }, + ] + + def test_events(self): + services = [Service(name='web'), Service(name='db')] + project = Project('test', services, self.mock_client) + self.mock_client.api_version = '1.35' + self.mock_client.events.return_value = iter([ + { + 'status': 'create', + 'from': 'example/image', + 'Type': 'container', + 'Actor': { + 'ID': 'abcde', + 'Attributes': { + 'com.docker.compose.project': 'test', + 'com.docker.compose.service': 'web', + 'image': 'example/image', + 'name': 'test_web_1', + } + }, + 'id': 'abcde', + 'time': 1420092061, + 'timeNano': 14200920610000002000, + }, + { + 'status': 'attach', + 'from': 'example/image', + 'Type': 'container', + 'Actor': { + 'ID': 'abcde', + 'Attributes': { + 'com.docker.compose.project': 'test', + 'com.docker.compose.service': 'web', + 'image': 'example/image', + 'name': 'test_web_1', + } + }, + 'id': 'abcde', + 'time': 1420092061, + 'timeNano': 14200920610000003000, + }, + { + 'status': 'create', + 'from': 'example/other', + 'Type': 'container', + 'Actor': { + 'ID': 'bdbdbd', + 'Attributes': { + 'image': 'example/other', + 'name': 'shrewd_einstein', + } + }, + 'id': 'bdbdbd', + 'time': 1420092061, + 'timeNano': 14200920610000005000, + }, + { + 'status': 'create', + 'from': 'example/db', + 'Type': 'container', + 'Actor': { + 'ID': 'ababa', + 'Attributes': { + 'com.docker.compose.project': 'test', + 'com.docker.compose.service': 'db', + 'image': 'example/db', + 'name': 'test_db_1', + } + }, + 'id': 'ababa', + 'time': 1420092061, + 'timeNano': 14200920610000004000, + }, + { + 'status': 'destroy', + 'from': 'example/db', + 'Type': 'container', + 'Actor': { + 'ID': 'eeeee', + 'Attributes': { + 'com.docker.compose.project': 'test', + 'com.docker.compose.service': 'db', + 'image': 'example/db', + 'name': 'test_db_1', + } + }, + 'id': 'eeeee', + 'time': 1420092061, + 'timeNano': 14200920610000004000, + }, + ]) + + def dt_with_microseconds(dt, us): + return datetime.datetime.fromtimestamp(dt).replace(microsecond=us) + + def get_container(cid): + if cid == 'eeeee': + raise NotFound(None, None, "oops") + if cid == 'abcde': + name = 'web' + labels = {LABEL_SERVICE: name} + elif cid == 'ababa': + name = 'db' + labels = {LABEL_SERVICE: name} + else: + labels = {} + name = '' + return { + 'Id': cid, + 'Config': {'Labels': labels}, + 'Name': '/project_%s_1' % name, + } + + self.mock_client.inspect_container.side_effect = get_container + + events = project.events() + + events_list = list(events) + # Assert the return value is a generator + assert not list(events) + assert events_list == [ + { + 'type': 'container', + 'service': 'web', + 'action': 'create', + 'id': 'abcde', + 'attributes': { + 'name': 'test_web_1', + 'image': 'example/image', + }, + 'time': dt_with_microseconds(1420092061, 2), + 'container': Container(None, get_container('abcde')), + }, + { + 'type': 'container', + 'service': 'web', + 'action': 'attach', + 'id': 'abcde', + 'attributes': { + 'name': 'test_web_1', + 'image': 'example/image', + }, + 'time': dt_with_microseconds(1420092061, 3), + 'container': Container(None, get_container('abcde')), + }, + { + 'type': 'container', + 'service': 'db', + 'action': 'create', + 'id': 'ababa', + 'attributes': { + 'name': 'test_db_1', + 'image': 'example/db', + }, + 'time': dt_with_microseconds(1420092061, 4), + 'container': Container(None, get_container('ababa')), + }, + { + 'type': 'container', + 'service': 'db', + 'action': 'destroy', + 'id': 'eeeee', + 'attributes': { + 'name': 'test_db_1', + 'image': 'example/db', + }, + 'time': dt_with_microseconds(1420092061, 4), + 'container': None, + }, + ] + + def test_net_unset(self): + project = Project.from_config( + name='test', + client=self.mock_client, + config_data=build_config( + version=V1, + services=[ + { + 'name': 'test', + 'image': BUSYBOX_IMAGE_WITH_TAG, + } + ], + networks=None, + volumes=None, + secrets=None, + configs=None, + ), + ) + service = project.get_service('test') + assert service.network_mode.id is None + assert 'NetworkMode' not in service._get_container_host_config({}) + + def test_use_net_from_container(self): + container_id = 'aabbccddee' + container_dict = dict(Name='aaa', Id=container_id) + self.mock_client.inspect_container.return_value = container_dict + project = Project.from_config( + name='test', + client=self.mock_client, + config_data=build_config( + services=[ + { + 'name': 'test', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'network_mode': 'container:aaa' + }, + ], + networks=None, + volumes=None, + secrets=None, + configs=None, + ), + ) + service = project.get_service('test') + assert service.network_mode.mode == 'container:' + container_id + + def test_use_net_from_service(self): + container_name = 'test_aaa_1' + self.mock_client.containers.return_value = [ + { + "Name": container_name, + "Names": [container_name], + "Id": container_name, + "Image": BUSYBOX_IMAGE_WITH_TAG + } + ] + project = Project.from_config( + name='test', + client=self.mock_client, + config_data=build_config( + services=[ + { + 'name': 'aaa', + 'image': BUSYBOX_IMAGE_WITH_TAG + }, + { + 'name': 'test', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'network_mode': 'service:aaa' + }, + ], + networks=None, + volumes=None, + secrets=None, + configs=None, + ), + ) + + service = project.get_service('test') + assert service.network_mode.mode == 'container:' + container_name + + def test_uses_default_network_true(self): + project = Project.from_config( + name='test', + client=self.mock_client, + config_data=build_config( + services=[ + { + 'name': 'foo', + 'image': BUSYBOX_IMAGE_WITH_TAG + }, + ], + networks=None, + volumes=None, + secrets=None, + configs=None, + ), + ) + + assert 'default' in project.networks.networks + + def test_uses_default_network_false(self): + project = Project.from_config( + name='test', + client=self.mock_client, + config_data=build_config( + services=[ + { + 'name': 'foo', + 'image': BUSYBOX_IMAGE_WITH_TAG, + 'networks': {'custom': None} + }, + ], + networks={'custom': {}}, + volumes=None, + secrets=None, + configs=None, + ), + ) + + assert 'default' not in project.networks.networks + + def test_container_without_name(self): + self.mock_client.containers.return_value = [ + {'Image': BUSYBOX_IMAGE_WITH_TAG, 'Id': '1', 'Name': '1'}, + {'Image': BUSYBOX_IMAGE_WITH_TAG, 'Id': '2', 'Name': None}, + {'Image': BUSYBOX_IMAGE_WITH_TAG, 'Id': '3'}, + ] + self.mock_client.inspect_container.return_value = { + 'Id': '1', + 'Config': { + 'Labels': { + LABEL_SERVICE: 'web', + }, + }, + } + project = Project.from_config( + name='test', + client=self.mock_client, + config_data=build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + }], + networks=None, + volumes=None, + secrets=None, + configs=None, + ), + ) + assert [c.id for c in project.containers()] == ['1'] + + def test_down_with_no_resources(self): + project = Project.from_config( + name='test', + client=self.mock_client, + config_data=build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + }], + networks={'default': {}}, + volumes={'data': {}}, + secrets=None, + configs=None, + ), + ) + self.mock_client.remove_network.side_effect = NotFound(None, None, 'oops') + self.mock_client.remove_volume.side_effect = NotFound(None, None, 'oops') + + project.down(ImageType.all, True) + self.mock_client.remove_image.assert_called_once_with(BUSYBOX_IMAGE_WITH_TAG) + + def test_no_warning_on_stop(self): + self.mock_client.info.return_value = {'Swarm': {'LocalNodeState': 'active'}} + project = Project('composetest', [], self.mock_client) + + with mock.patch('compose.project.log') as fake_log: + project.stop() + assert fake_log.warn.call_count == 0 + + def test_no_warning_in_normal_mode(self): + self.mock_client.info.return_value = {'Swarm': {'LocalNodeState': 'inactive'}} + project = Project('composetest', [], self.mock_client) + + with mock.patch('compose.project.log') as fake_log: + project.up() + assert fake_log.warn.call_count == 0 + + def test_no_warning_with_no_swarm_info(self): + self.mock_client.info.return_value = {} + project = Project('composetest', [], self.mock_client) + + with mock.patch('compose.project.log') as fake_log: + project.up() + assert fake_log.warn.call_count == 0 + + def test_no_such_service_unicode(self): + assert NoSuchService('十六夜　咲夜'.encode()).msg == 'No such service: 十六夜　咲夜' + assert NoSuchService('十六夜　咲夜').msg == 'No such service: 十六夜　咲夜' + + def test_project_platform_value(self): + service_config = { + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + } + config_data = build_config( + services=[service_config], networks={}, volumes={}, secrets=None, configs=None + ) + + project = Project.from_config(name='test', client=self.mock_client, config_data=config_data) + assert project.get_service('web').platform is None + + project = Project.from_config( + name='test', client=self.mock_client, config_data=config_data, default_platform='windows' + ) + assert project.get_service('web').platform == 'windows' + + service_config['platform'] = 'linux/s390x' + project = Project.from_config(name='test', client=self.mock_client, config_data=config_data) + assert project.get_service('web').platform == 'linux/s390x' + + project = Project.from_config( + name='test', client=self.mock_client, config_data=config_data, default_platform='windows' + ) + assert project.get_service('web').platform == 'linux/s390x' + + def test_build_container_operation_with_timeout_func_does_not_mutate_options_with_timeout(self): + config_data = build_config( + services=[ + {'name': 'web', 'image': BUSYBOX_IMAGE_WITH_TAG}, + {'name': 'db', 'image': BUSYBOX_IMAGE_WITH_TAG, 'stop_grace_period': '1s'}, + ], + networks={}, volumes={}, secrets=None, configs=None, + ) + + project = Project.from_config(name='test', client=self.mock_client, config_data=config_data) + + stop_op = project.build_container_operation_with_timeout_func('stop', options={}) + + web_container = mock.create_autospec(Container, service='web') + db_container = mock.create_autospec(Container, service='db') + + # `stop_grace_period` is not set to 'web' service, + # then it is stopped with the default timeout. + stop_op(web_container) + web_container.stop.assert_called_once_with(timeout=DEFAULT_TIMEOUT) + + # `stop_grace_period` is set to 'db' service, + # then it is stopped with the specified timeout and + # the value is not overridden by the previous function call. + stop_op(db_container) + db_container.stop.assert_called_once_with(timeout=1) + + @mock.patch('compose.parallel.ParallelStreamWriter._write_noansi') + def test_error_parallel_pull(self, mock_write): + project = Project.from_config( + name='test', + client=self.mock_client, + config_data=build_config( + services=[{ + 'name': 'web', + 'image': BUSYBOX_IMAGE_WITH_TAG, + }], + networks=None, + volumes=None, + secrets=None, + configs=None, + ), + ) + + self.mock_client.pull.side_effect = OperationFailedError('pull error') + with pytest.raises(ProjectError): + project.pull(parallel_pull=True) + + self.mock_client.pull.side_effect = OperationFailedError(b'pull error') + with pytest.raises(ProjectError): + project.pull(parallel_pull=True) + + def test_avoid_multiple_push(self): + service_config_latest = {'image': 'busybox:latest', 'build': '.'} + service_config_default = {'image': 'busybox', 'build': '.'} + service_config_sha = { + 'image': 'busybox@sha256:38a203e1986cf79639cfb9b2e1d6e773de84002feea2d4eb006b52004ee8502d', + 'build': '.' + } + svc1 = Service('busy1', **service_config_latest) + svc1_1 = Service('busy11', **service_config_latest) + svc2 = Service('busy2', **service_config_default) + svc2_1 = Service('busy21', **service_config_default) + svc3 = Service('busy3', **service_config_sha) + svc3_1 = Service('busy31', **service_config_sha) + project = Project( + 'composetest', [svc1, svc1_1, svc2, svc2_1, svc3, svc3_1], self.mock_client + ) + with mock.patch('compose.service.Service.push') as fake_push: + project.push() + assert fake_push.call_count == 2 + + def test_get_secrets_no_secret_def(self): + service = 'foo' + secret_source = 'bar' + + secret_defs = mock.Mock() + secret_defs.get.return_value = None + secret = mock.Mock(source=secret_source) + + with self.assertRaises(ConfigurationError): + get_secrets(service, [secret], secret_defs) + + def test_get_secrets_external_warning(self): + service = 'foo' + secret_source = 'bar' + + secret_def = mock.Mock() + secret_def.get.return_value = True + + secret_defs = mock.Mock() + secret_defs.get.side_effect = secret_def + secret = mock.Mock(source=secret_source) + + with mock.patch('compose.project.log') as mock_log: + get_secrets(service, [secret], secret_defs) + + mock_log.warning.assert_called_with("Service \"{service}\" uses secret \"{secret}\" " + "which is external. External secrets are not available" + " to containers created by docker-compose." + .format(service=service, secret=secret_source)) + + def test_get_secrets_uid_gid_mode_warning(self): + service = 'foo' + secret_source = 'bar' + + fd, filename_path = tempfile.mkstemp() + os.close(fd) + self.addCleanup(os.remove, filename_path) + + def mock_get(key): + return {'external': False, 'file': filename_path}[key] + + secret_def = mock.MagicMock() + secret_def.get = mock.MagicMock(side_effect=mock_get) + + secret_defs = mock.Mock() + secret_defs.get.return_value = secret_def + + secret = mock.Mock(uid=True, gid=True, mode=True, source=secret_source) + + with mock.patch('compose.project.log') as mock_log: + get_secrets(service, [secret], secret_defs) + + mock_log.warning.assert_called_with("Service \"{service}\" uses secret \"{secret}\" with uid, " + "gid, or mode. These fields are not supported by this " + "implementation of the Compose file" + .format(service=service, secret=secret_source)) + + def test_get_secrets_secret_file_warning(self): + service = 'foo' + secret_source = 'bar' + not_a_path = 'NOT_A_PATH' + + def mock_get(key): + return {'external': False, 'file': not_a_path}[key] + + secret_def = mock.MagicMock() + secret_def.get = mock.MagicMock(side_effect=mock_get) + + secret_defs = mock.Mock() + secret_defs.get.return_value = secret_def + + secret = mock.Mock(uid=False, gid=False, mode=False, source=secret_source) + + with mock.patch('compose.project.log') as mock_log: + get_secrets(service, [secret], secret_defs) + + mock_log.warning.assert_called_with("Service \"{service}\" uses an undefined secret file " + "\"{secret_file}\", the following file should be created " + "\"{secret_file}\"" + .format(service=service, secret_file=not_a_path)) diff --git a/tests/unit/service_test.py b/tests/unit/service_test.py new file mode 100644 index 00000000000..72cb1d7f9ed --- /dev/null +++ b/tests/unit/service_test.py @@ -0,0 +1,1550 @@ +import docker +import pytest +from docker.constants import DEFAULT_DOCKER_API_VERSION +from docker.errors import APIError +from docker.errors import ImageNotFound +from docker.errors import NotFound + +from .. import mock +from .. import unittest +from compose.config.errors import DependencyError +from compose.config.types import MountSpec +from compose.config.types import ServicePort +from compose.config.types import ServiceSecret +from compose.config.types import VolumeFromSpec +from compose.config.types import VolumeSpec +from compose.const import API_VERSIONS +from compose.const import LABEL_CONFIG_HASH +from compose.const import LABEL_ONE_OFF +from compose.const import LABEL_PROJECT +from compose.const import LABEL_SERVICE +from compose.const import SECRETS_PATH +from compose.const import WINDOWS_LONGPATH_PREFIX +from compose.container import Container +from compose.errors import OperationFailedError +from compose.parallel import ParallelStreamWriter +from compose.project import OneOffFilter +from compose.service import build_ulimits +from compose.service import build_volume_binding +from compose.service import BuildAction +from compose.service import ContainerNetworkMode +from compose.service import format_environment +from compose.service import formatted_ports +from compose.service import get_container_data_volumes +from compose.service import ImageType +from compose.service import merge_volume_bindings +from compose.service import NeedsBuildError +from compose.service import NetworkMode +from compose.service import NoSuchImageError +from compose.service import parse_repository_tag +from compose.service import rewrite_build_path +from compose.service import Service +from compose.service import ServiceNetworkMode +from compose.service import warn_on_masked_volume + + +class ServiceTest(unittest.TestCase): + + def setUp(self): + self.mock_client = mock.create_autospec(docker.APIClient) + self.mock_client.api_version = DEFAULT_DOCKER_API_VERSION + self.mock_client._general_configs = {} + + def test_containers(self): + service = Service('db', self.mock_client, 'myproject', image='foo') + self.mock_client.containers.return_value = [] + assert list(service.containers()) == [] + + def test_containers_with_containers(self): + self.mock_client.containers.return_value = [ + dict(Name=str(i), Image='foo', Id=i) for i in range(3) + ] + service = Service('db', self.mock_client, 'myproject', image='foo') + assert [c.id for c in service.containers()] == list(range(3)) + + expected_labels = [ + '{}=myproject'.format(LABEL_PROJECT), + '{}=db'.format(LABEL_SERVICE), + '{}=False'.format(LABEL_ONE_OFF), + ] + + self.mock_client.containers.assert_called_once_with( + all=False, + filters={'label': expected_labels}) + + def test_container_without_name(self): + self.mock_client.containers.return_value = [ + {'Image': 'foo', 'Id': '1', 'Name': '1'}, + {'Image': 'foo', 'Id': '2', 'Name': None}, + {'Image': 'foo', 'Id': '3'}, + ] + service = Service('db', self.mock_client, 'myproject', image='foo') + + assert [c.id for c in service.containers()] == ['1'] + assert service._next_container_number() == 2 + assert service.get_container(1).id == '1' + + def test_get_volumes_from_container(self): + container_id = 'aabbccddee' + service = Service( + 'test', + image='foo', + volumes_from=[ + VolumeFromSpec( + mock.Mock(id=container_id, spec=Container), + 'rw', + 'container')]) + + assert service._get_volumes_from() == [container_id + ':rw'] + + def test_get_volumes_from_container_read_only(self): + container_id = 'aabbccddee' + service = Service( + 'test', + image='foo', + volumes_from=[ + VolumeFromSpec( + mock.Mock(id=container_id, spec=Container), + 'ro', + 'container')]) + + assert service._get_volumes_from() == [container_id + ':ro'] + + def test_get_volumes_from_service_container_exists(self): + container_ids = ['aabbccddee', '12345'] + from_service = mock.create_autospec(Service) + from_service.containers.return_value = [ + mock.Mock(id=container_id, spec=Container) + for container_id in container_ids + ] + service = Service( + 'test', + volumes_from=[VolumeFromSpec(from_service, 'rw', 'service')], + image='foo') + + assert service._get_volumes_from() == [container_ids[0] + ":rw"] + + def test_get_volumes_from_service_container_exists_with_flags(self): + for mode in ['ro', 'rw', 'z', 'rw,z', 'z,rw']: + container_ids = ['aabbccddee:' + mode, '12345:' + mode] + from_service = mock.create_autospec(Service) + from_service.containers.return_value = [ + mock.Mock(id=container_id.split(':')[0], spec=Container) + for container_id in container_ids + ] + service = Service( + 'test', + volumes_from=[VolumeFromSpec(from_service, mode, 'service')], + image='foo') + + assert service._get_volumes_from() == [container_ids[0]] + + def test_get_volumes_from_service_no_container(self): + container_id = 'abababab' + from_service = mock.create_autospec(Service) + from_service.containers.return_value = [] + from_service.create_container.return_value = mock.Mock( + id=container_id, + spec=Container) + service = Service( + 'test', + image='foo', + volumes_from=[VolumeFromSpec(from_service, 'rw', 'service')]) + + assert service._get_volumes_from() == [container_id + ':rw'] + from_service.create_container.assert_called_once_with() + + def test_memory_swap_limit(self): + self.mock_client.create_host_config.return_value = {} + + service = Service( + name='foo', + image='foo', + hostname='name', + client=self.mock_client, + mem_limit=1000000000, + memswap_limit=2000000000) + service._get_container_create_options({'some': 'overrides'}, 1) + + assert self.mock_client.create_host_config.called + assert self.mock_client.create_host_config.call_args[1]['mem_limit'] == 1000000000 + assert self.mock_client.create_host_config.call_args[1]['memswap_limit'] == 2000000000 + + def test_self_reference_external_link(self): + service = Service( + name='foo', + external_links=['default_foo_1'] + ) + with pytest.raises(DependencyError): + service.get_container_name('foo', 1) + + def test_mem_reservation(self): + self.mock_client.create_host_config.return_value = {} + + service = Service( + name='foo', + image='foo', + hostname='name', + client=self.mock_client, + mem_reservation='512m' + ) + service._get_container_create_options({'some': 'overrides'}, 1) + assert self.mock_client.create_host_config.called is True + assert self.mock_client.create_host_config.call_args[1]['mem_reservation'] == '512m' + + def test_cgroup_parent(self): + self.mock_client.create_host_config.return_value = {} + + service = Service( + name='foo', + image='foo', + hostname='name', + client=self.mock_client, + cgroup_parent='test') + service._get_container_create_options({'some': 'overrides'}, 1) + + assert self.mock_client.create_host_config.called + assert self.mock_client.create_host_config.call_args[1]['cgroup_parent'] == 'test' + + def test_log_opt(self): + self.mock_client.create_host_config.return_value = {} + + log_opt = {'syslog-address': 'tcp://192.168.0.42:123'} + logging = {'driver': 'syslog', 'options': log_opt} + service = Service( + name='foo', + image='foo', + hostname='name', + client=self.mock_client, + log_driver='syslog', + logging=logging) + service._get_container_create_options({'some': 'overrides'}, 1) + + assert self.mock_client.create_host_config.called + assert self.mock_client.create_host_config.call_args[1]['log_config'] == { + 'Type': 'syslog', 'Config': {'syslog-address': 'tcp://192.168.0.42:123'} + } + + def test_stop_grace_period(self): + self.mock_client.api_version = '1.25' + self.mock_client.create_host_config.return_value = {} + service = Service( + 'foo', + image='foo', + client=self.mock_client, + stop_grace_period="1m35s") + opts = service._get_container_create_options({'image': 'foo'}, 1) + assert opts['stop_timeout'] == 95 + + def test_split_domainname_none(self): + service = Service( + 'foo', + image='foo', + hostname='name.domain.tld', + client=self.mock_client) + opts = service._get_container_create_options({'image': 'foo'}, 1) + assert opts['hostname'] == 'name.domain.tld', 'hostname' + assert not ('domainname' in opts), 'domainname' + + def test_split_domainname_fqdn(self): + self.mock_client.api_version = '1.22' + service = Service( + 'foo', + hostname='name.domain.tld', + image='foo', + client=self.mock_client) + opts = service._get_container_create_options({'image': 'foo'}, 1) + assert opts['hostname'] == 'name', 'hostname' + assert opts['domainname'] == 'domain.tld', 'domainname' + + def test_split_domainname_both(self): + self.mock_client.api_version = '1.22' + service = Service( + 'foo', + hostname='name', + image='foo', + domainname='domain.tld', + client=self.mock_client) + opts = service._get_container_create_options({'image': 'foo'}, 1) + assert opts['hostname'] == 'name', 'hostname' + assert opts['domainname'] == 'domain.tld', 'domainname' + + def test_split_domainname_weird(self): + self.mock_client.api_version = '1.22' + service = Service( + 'foo', + hostname='name.sub', + domainname='domain.tld', + image='foo', + client=self.mock_client) + opts = service._get_container_create_options({'image': 'foo'}, 1) + assert opts['hostname'] == 'name.sub', 'hostname' + assert opts['domainname'] == 'domain.tld', 'domainname' + + def test_no_default_hostname_when_not_using_networking(self): + service = Service( + 'foo', + image='foo', + use_networking=False, + client=self.mock_client, + ) + opts = service._get_container_create_options({'image': 'foo'}, 1) + assert opts.get('hostname') is None + + def test_get_container_create_options_with_name_option(self): + service = Service( + 'foo', + image='foo', + client=self.mock_client, + container_name='foo1') + name = 'the_new_name' + opts = service._get_container_create_options( + {'name': name}, + 1, + one_off=OneOffFilter.only) + assert opts['name'] == name + + def test_get_container_create_options_does_not_mutate_options(self): + labels = {'thing': 'real'} + environment = {'also': 'real'} + service = Service( + 'foo', + image='foo', + labels=dict(labels), + client=self.mock_client, + environment=dict(environment), + ) + self.mock_client.inspect_image.return_value = {'Id': 'abcd'} + prev_container = mock.Mock( + id='ababab', + image_config={'ContainerConfig': {}} + ) + prev_container.full_slug = 'abcdefff1234' + prev_container.get.return_value = None + + opts = service._get_container_create_options( + {}, 1, previous_container=prev_container + ) + + assert service.options['labels'] == labels + assert service.options['environment'] == environment + + assert opts['labels'][LABEL_CONFIG_HASH] == \ + '689149e6041a85f6fb4945a2146a497ed43c8a5cbd8991753d875b165f1b4de4' + assert opts['environment'] == ['also=real'] + + def test_get_container_create_options_sets_affinity_with_binds(self): + service = Service( + 'foo', + image='foo', + client=self.mock_client, + ) + self.mock_client.inspect_image.return_value = {'Id': 'abcd'} + prev_container = mock.Mock( + id='ababab', + image_config={'ContainerConfig': {'Volumes': ['/data']}}) + + def container_get(key): + return { + 'Mounts': [ + { + 'Destination': '/data', + 'Source': '/some/path', + 'Name': 'abab1234', + }, + ] + }.get(key, None) + + prev_container.get.side_effect = container_get + prev_container.full_slug = 'abcdefff1234' + + opts = service._get_container_create_options( + {}, + 1, + previous_container=prev_container + ) + + assert opts['environment'] == ['affinity:container==ababab'] + + def test_get_container_create_options_no_affinity_without_binds(self): + service = Service('foo', image='foo', client=self.mock_client) + self.mock_client.inspect_image.return_value = {'Id': 'abcd'} + prev_container = mock.Mock( + id='ababab', + image_config={'ContainerConfig': {}}) + prev_container.get.return_value = None + prev_container.full_slug = 'abcdefff1234' + + opts = service._get_container_create_options( + {}, + 1, + previous_container=prev_container) + assert opts['environment'] == [] + + def test_get_container_not_found(self): + self.mock_client.containers.return_value = [] + service = Service('foo', client=self.mock_client, image='foo') + + with pytest.raises(ValueError): + service.get_container() + + @mock.patch('compose.service.Container', autospec=True) + def test_get_container(self, mock_container_class): + container_dict = dict(Name='default_foo_2_bdfa3ed91e2c') + self.mock_client.containers.return_value = [container_dict] + service = Service('foo', image='foo', client=self.mock_client) + + container = service.get_container(number=2) + assert container == mock_container_class.from_ps.return_value + mock_container_class.from_ps.assert_called_once_with( + self.mock_client, container_dict) + + @mock.patch('compose.service.log', autospec=True) + def test_pull_image(self, mock_log): + service = Service('foo', client=self.mock_client, image='someimage:sometag') + service.pull() + self.mock_client.pull.assert_called_once_with( + 'someimage', + tag='sometag', + stream=True, + platform=None) + mock_log.info.assert_called_once_with('Pulling foo (someimage:sometag)...') + + def test_pull_image_no_tag(self): + service = Service('foo', client=self.mock_client, image='ababab') + service.pull() + self.mock_client.pull.assert_called_once_with( + 'ababab', + tag='latest', + stream=True, + platform=None) + + @mock.patch('compose.service.log', autospec=True) + def test_pull_image_digest(self, mock_log): + service = Service('foo', client=self.mock_client, image='someimage@sha256:1234') + service.pull() + self.mock_client.pull.assert_called_once_with( + 'someimage', + tag='sha256:1234', + stream=True, + platform=None) + mock_log.info.assert_called_once_with('Pulling foo (someimage@sha256:1234)...') + + @mock.patch('compose.service.log', autospec=True) + def test_pull_image_with_platform(self, mock_log): + self.mock_client.api_version = '1.35' + service = Service( + 'foo', client=self.mock_client, image='someimage:sometag', platform='windows/x86_64' + ) + service.pull() + assert self.mock_client.pull.call_count == 1 + call_args = self.mock_client.pull.call_args + assert call_args[1]['platform'] == 'windows/x86_64' + + @mock.patch('compose.service.log', autospec=True) + def test_pull_image_with_platform_unsupported_api(self, mock_log): + self.mock_client.api_version = '1.33' + service = Service( + 'foo', client=self.mock_client, image='someimage:sometag', platform='linux/arm' + ) + with pytest.raises(OperationFailedError): + service.pull() + + def test_pull_image_with_default_platform(self): + self.mock_client.api_version = '1.35' + + service = Service( + 'foo', client=self.mock_client, image='someimage:sometag', + default_platform='linux' + ) + assert service.platform == 'linux' + service.pull() + + assert self.mock_client.pull.call_count == 1 + call_args = self.mock_client.pull.call_args + assert call_args[1]['platform'] == 'linux' + + @mock.patch('compose.service.Container', autospec=True) + def test_recreate_container(self, _): + mock_container = mock.create_autospec(Container) + mock_container.full_slug = 'abcdefff1234' + service = Service('foo', client=self.mock_client, image='someimage') + service.image = lambda: {'Id': 'abc123'} + new_container = service.recreate_container(mock_container) + + mock_container.stop.assert_called_once_with(timeout=10) + mock_container.rename_to_tmp_name.assert_called_once_with() + + new_container.start.assert_called_once_with() + mock_container.remove.assert_called_once_with() + + @mock.patch('compose.service.Container', autospec=True) + def test_recreate_container_with_timeout(self, _): + mock_container = mock.create_autospec(Container) + mock_container.full_slug = 'abcdefff1234' + self.mock_client.inspect_image.return_value = {'Id': 'abc123'} + service = Service('foo', client=self.mock_client, image='someimage') + service.recreate_container(mock_container, timeout=1) + + mock_container.stop.assert_called_once_with(timeout=1) + + def test_parse_repository_tag(self): + assert parse_repository_tag("root") == ("root", "", ":") + assert parse_repository_tag("root:tag") == ("root", "tag", ":") + assert parse_repository_tag("user/repo") == ("user/repo", "", ":") + assert parse_repository_tag("user/repo:tag") == ("user/repo", "tag", ":") + assert parse_repository_tag("url:5000/repo") == ("url:5000/repo", "", ":") + assert parse_repository_tag("url:5000/repo:tag") == ("url:5000/repo", "tag", ":") + assert parse_repository_tag("root@sha256:digest") == ("root", "sha256:digest", "@") + assert parse_repository_tag("user/repo@sha256:digest") == ("user/repo", "sha256:digest", "@") + assert parse_repository_tag("url:5000/repo@sha256:digest") == ( + "url:5000/repo", "sha256:digest", "@" + ) + + def test_create_container(self): + service = Service('foo', client=self.mock_client, build={'context': '.'}) + self.mock_client.inspect_image.side_effect = [ + NoSuchImageError, + {'Id': 'abc123'}, + ] + self.mock_client.build.return_value = [ + '{"stream": "Successfully built abcd"}', + ] + + with mock.patch('compose.service.log', autospec=True) as mock_log: + service.create_container() + assert mock_log.warning.called + _, args, _ = mock_log.warning.mock_calls[0] + assert 'was built because it did not already exist' in args[0] + + assert self.mock_client.build.call_count == 1 + assert self.mock_client.build.call_args[1]['tag'] == 'default_foo' + + def test_create_container_binary_string_error(self): + service = Service('foo', client=self.mock_client, build={'context': '.'}) + service.image = lambda: {'Id': 'abc123'} + + self.mock_client.create_container.side_effect = APIError(None, + None, + b"Test binary string explanation") + with pytest.raises(OperationFailedError) as ex: + service.create_container() + + assert ex.value.msg == "Cannot create container for service foo: Test binary string explanation" + + def test_start_binary_string_error(self): + service = Service('foo', client=self.mock_client) + container = Container(self.mock_client, {'Id': 'abc123'}) + + self.mock_client.start.side_effect = APIError(None, + None, + b"Test binary string explanation with " + b"driver failed programming external " + b"connectivity") + with mock.patch('compose.service.log', autospec=True) as mock_log: + with pytest.raises(OperationFailedError) as ex: + service.start_container(container) + + assert ex.value.msg == "Cannot start service foo: " \ + "Test binary string explanation " \ + "with driver failed programming external connectivity" + mock_log.warn.assert_called_once_with("Host is already in use by another container") + + def test_ensure_image_exists_no_build(self): + service = Service('foo', client=self.mock_client, build={'context': '.'}) + self.mock_client.inspect_image.return_value = {'Id': 'abc123'} + + service.ensure_image_exists(do_build=BuildAction.skip) + assert not self.mock_client.build.called + + def test_ensure_image_exists_no_build_but_needs_build(self): + service = Service('foo', client=self.mock_client, build={'context': '.'}) + self.mock_client.inspect_image.side_effect = NoSuchImageError + with pytest.raises(NeedsBuildError): + service.ensure_image_exists(do_build=BuildAction.skip) + + def test_ensure_image_exists_force_build(self): + service = Service('foo', client=self.mock_client, build={'context': '.'}) + self.mock_client.inspect_image.return_value = {'Id': 'abc123'} + self.mock_client.build.return_value = [ + '{"stream": "Successfully built abcd"}', + ] + + with mock.patch('compose.service.log', autospec=True) as mock_log: + service.ensure_image_exists(do_build=BuildAction.force) + + assert not mock_log.warning.called + assert self.mock_client.build.call_count == 1 + self.mock_client.build.call_args[1]['tag'] == 'default_foo' + + def test_build_does_not_pull(self): + self.mock_client.build.return_value = [ + b'{"stream": "Successfully built 12345"}', + ] + + service = Service('foo', client=self.mock_client, build={'context': '.'}) + service.build() + + assert self.mock_client.build.call_count == 1 + assert not self.mock_client.build.call_args[1]['pull'] + + def test_build_with_platform(self): + self.mock_client.api_version = '1.35' + self.mock_client.build.return_value = [ + b'{"stream": "Successfully built 12345"}', + ] + + service = Service('foo', client=self.mock_client, build={'context': '.'}, platform='linux') + service.build() + + assert self.mock_client.build.call_count == 1 + call_args = self.mock_client.build.call_args + assert call_args[1]['platform'] == 'linux' + + def test_build_with_default_platform(self): + self.mock_client.api_version = '1.35' + self.mock_client.build.return_value = [ + b'{"stream": "Successfully built 12345"}', + ] + + service = Service( + 'foo', client=self.mock_client, build={'context': '.'}, + default_platform='linux' + ) + assert service.platform == 'linux' + service.build() + + assert self.mock_client.build.call_count == 1 + call_args = self.mock_client.build.call_args + assert call_args[1]['platform'] == 'linux' + + def test_service_platform_precedence(self): + self.mock_client.api_version = '1.35' + + service = Service( + 'foo', client=self.mock_client, platform='linux/arm', + default_platform='osx' + ) + assert service.platform == 'linux/arm' + + def test_service_ignore_default_platform_with_unsupported_api(self): + self.mock_client.api_version = '1.32' + self.mock_client.build.return_value = [ + b'{"stream": "Successfully built 12345"}', + ] + + service = Service( + 'foo', client=self.mock_client, default_platform='windows', build={'context': '.'} + ) + assert service.platform is None + service.build() + assert self.mock_client.build.call_count == 1 + call_args = self.mock_client.build.call_args + assert call_args[1]['platform'] is None + + def test_build_with_override_build_args(self): + self.mock_client.build.return_value = [ + b'{"stream": "Successfully built 12345"}', + ] + + build_args = { + 'arg1': 'arg1_new_value', + } + service = Service('foo', client=self.mock_client, + build={'context': '.', 'args': {'arg1': 'arg1', 'arg2': 'arg2'}}) + service.build(build_args_override=build_args) + + called_build_args = self.mock_client.build.call_args[1]['buildargs'] + + assert called_build_args['arg1'] == build_args['arg1'] + assert called_build_args['arg2'] == 'arg2' + + def test_build_with_isolation_from_service_config(self): + self.mock_client.build.return_value = [ + b'{"stream": "Successfully built 12345"}', + ] + + service = Service('foo', client=self.mock_client, build={'context': '.'}, isolation='hyperv') + service.build() + + assert self.mock_client.build.call_count == 1 + called_build_args = self.mock_client.build.call_args[1] + assert called_build_args['isolation'] == 'hyperv' + + def test_build_isolation_from_build_override_service_config(self): + self.mock_client.build.return_value = [ + b'{"stream": "Successfully built 12345"}', + ] + + service = Service( + 'foo', client=self.mock_client, build={'context': '.', 'isolation': 'default'}, + isolation='hyperv' + ) + service.build() + + assert self.mock_client.build.call_count == 1 + called_build_args = self.mock_client.build.call_args[1] + assert called_build_args['isolation'] == 'default' + + def test_config_dict(self): + self.mock_client.inspect_image.return_value = {'Id': 'abcd'} + service = Service( + 'foo', + image='example.com/foo', + client=self.mock_client, + network_mode=ServiceNetworkMode(Service('other')), + networks={'default': None}, + links=[(Service('one'), 'one')], + volumes_from=[VolumeFromSpec(Service('two'), 'rw', 'service')]) + + config_dict = service.config_dict() + expected = { + 'image_id': 'abcd', + 'options': {'image': 'example.com/foo'}, + 'links': [('one', 'one')], + 'net': 'other', + 'secrets': [], + 'networks': {'default': None}, + 'volumes_from': [('two', 'rw')], + } + assert config_dict == expected + + def test_config_dict_with_network_mode_from_container(self): + self.mock_client.inspect_image.return_value = {'Id': 'abcd'} + container = Container( + self.mock_client, + {'Id': 'aaabbb', 'Name': '/foo_1'}) + service = Service( + 'foo', + image='example.com/foo', + client=self.mock_client, + network_mode=ContainerNetworkMode(container)) + + config_dict = service.config_dict() + expected = { + 'image_id': 'abcd', + 'options': {'image': 'example.com/foo'}, + 'links': [], + 'networks': {}, + 'secrets': [], + 'net': 'aaabbb', + 'volumes_from': [], + } + assert config_dict == expected + + def test_config_hash_matches_label(self): + self.mock_client.inspect_image.return_value = {'Id': 'abcd'} + service = Service( + 'foo', + image='example.com/foo', + client=self.mock_client, + network_mode=NetworkMode('bridge'), + networks={'bridge': {}, 'net2': {}}, + links=[(Service('one', client=self.mock_client), 'one')], + volumes_from=[VolumeFromSpec(Service('two', client=self.mock_client), 'rw', 'service')], + volumes=[VolumeSpec('/ext', '/int', 'ro')], + build={'context': 'some/random/path'}, + ) + config_hash = service.config_hash + + for api_version in set(API_VERSIONS.values()): + self.mock_client.api_version = api_version + assert service._get_container_create_options( + {}, 1 + )['labels'][LABEL_CONFIG_HASH] == config_hash + + def test_remove_image_none(self): + web = Service('web', image='example', client=self.mock_client) + assert not web.remove_image(ImageType.none) + assert not self.mock_client.remove_image.called + + def test_remove_image_local_with_image_name_doesnt_remove(self): + web = Service('web', image='example', client=self.mock_client) + assert not web.remove_image(ImageType.local) + assert not self.mock_client.remove_image.called + + def test_remove_image_local_without_image_name_does_remove(self): + web = Service('web', build='.', client=self.mock_client) + assert web.remove_image(ImageType.local) + self.mock_client.remove_image.assert_called_once_with(web.image_name) + + def test_remove_image_all_does_remove(self): + web = Service('web', image='example', client=self.mock_client) + assert web.remove_image(ImageType.all) + self.mock_client.remove_image.assert_called_once_with(web.image_name) + + def test_remove_image_with_error(self): + self.mock_client.remove_image.side_effect = error = APIError( + message="testing", + response={}, + explanation="Boom") + + web = Service('web', image='example', client=self.mock_client) + with mock.patch('compose.service.log', autospec=True) as mock_log: + assert not web.remove_image(ImageType.all) + mock_log.error.assert_called_once_with( + "Failed to remove image for service %s: %s", web.name, error) + + def test_remove_non_existing_image(self): + self.mock_client.remove_image.side_effect = ImageNotFound('image not found') + web = Service('web', image='example', client=self.mock_client) + with mock.patch('compose.service.log', autospec=True) as mock_log: + assert not web.remove_image(ImageType.all) + mock_log.warning.assert_called_once_with("Image %s not found.", web.image_name) + + def test_specifies_host_port_with_no_ports(self): + service = Service( + 'foo', + image='foo') + assert not service.specifies_host_port() + + def test_specifies_host_port_with_container_port(self): + service = Service( + 'foo', + image='foo', + ports=["2000"]) + assert not service.specifies_host_port() + + def test_specifies_host_port_with_host_port(self): + service = Service( + 'foo', + image='foo', + ports=["1000:2000"]) + assert service.specifies_host_port() + + def test_specifies_host_port_with_host_ip_no_port(self): + service = Service( + 'foo', + image='foo', + ports=["127.0.0.1::2000"]) + assert not service.specifies_host_port() + + def test_specifies_host_port_with_host_ip_and_port(self): + service = Service( + 'foo', + image='foo', + ports=["127.0.0.1:1000:2000"]) + assert service.specifies_host_port() + + def test_specifies_host_port_with_container_port_range(self): + service = Service( + 'foo', + image='foo', + ports=["2000-3000"]) + assert not service.specifies_host_port() + + def test_specifies_host_port_with_host_port_range(self): + service = Service( + 'foo', + image='foo', + ports=["1000-2000:2000-3000"]) + assert service.specifies_host_port() + + def test_specifies_host_port_with_host_ip_no_port_range(self): + service = Service( + 'foo', + image='foo', + ports=["127.0.0.1::2000-3000"]) + assert not service.specifies_host_port() + + def test_specifies_host_port_with_host_ip_and_port_range(self): + service = Service( + 'foo', + image='foo', + ports=["127.0.0.1:1000-2000:2000-3000"]) + assert service.specifies_host_port() + + def test_image_name_from_config(self): + image_name = 'example/web:mytag' + service = Service('foo', image=image_name) + assert service.image_name == image_name + + def test_image_name_default(self): + service = Service('foo', project='testing') + assert service.image_name == 'testing_foo' + + @mock.patch('compose.service.log', autospec=True) + def test_only_log_warning_when_host_ports_clash(self, mock_log): + self.mock_client.inspect_image.return_value = {'Id': 'abcd'} + ParallelStreamWriter.instance = None + name = 'foo' + service = Service( + name, + client=self.mock_client, + ports=["8080:80"]) + + service.scale(0) + assert not mock_log.warning.called + + service.scale(1) + assert not mock_log.warning.called + + service.scale(2) + mock_log.warning.assert_called_once_with( + 'The "{}" service specifies a port on the host. If multiple containers ' + 'for this service are created on a single host, the port will clash.'.format(name)) + + def test_parse_proxy_config(self): + default_proxy_config = { + 'httpProxy': 'http://proxy.mycorp.com:3128', + 'httpsProxy': 'https://user:password@proxy.mycorp.com:3129', + 'ftpProxy': 'http://ftpproxy.mycorp.com:21', + 'noProxy': '*.intra.mycorp.com', + } + + self.mock_client.base_url = 'http+docker://localunixsocket' + self.mock_client._general_configs = { + 'proxies': { + 'default': default_proxy_config, + } + } + + service = Service('foo', client=self.mock_client) + + assert service._parse_proxy_config() == { + 'HTTP_PROXY': default_proxy_config['httpProxy'], + 'http_proxy': default_proxy_config['httpProxy'], + 'HTTPS_PROXY': default_proxy_config['httpsProxy'], + 'https_proxy': default_proxy_config['httpsProxy'], + 'FTP_PROXY': default_proxy_config['ftpProxy'], + 'ftp_proxy': default_proxy_config['ftpProxy'], + 'NO_PROXY': default_proxy_config['noProxy'], + 'no_proxy': default_proxy_config['noProxy'], + } + + def test_parse_proxy_config_per_host(self): + default_proxy_config = { + 'httpProxy': 'http://proxy.mycorp.com:3128', + 'httpsProxy': 'https://user:password@proxy.mycorp.com:3129', + 'ftpProxy': 'http://ftpproxy.mycorp.com:21', + 'noProxy': '*.intra.mycorp.com', + } + host_specific_proxy_config = { + 'httpProxy': 'http://proxy.example.com:3128', + 'httpsProxy': 'https://user:password@proxy.example.com:3129', + 'ftpProxy': 'http://ftpproxy.example.com:21', + 'noProxy': '*.intra.example.com' + } + + self.mock_client.base_url = 'http+docker://localunixsocket' + self.mock_client._general_configs = { + 'proxies': { + 'default': default_proxy_config, + 'tcp://example.docker.com:2376': host_specific_proxy_config, + } + } + + service = Service('foo', client=self.mock_client) + + assert service._parse_proxy_config() == { + 'HTTP_PROXY': default_proxy_config['httpProxy'], + 'http_proxy': default_proxy_config['httpProxy'], + 'HTTPS_PROXY': default_proxy_config['httpsProxy'], + 'https_proxy': default_proxy_config['httpsProxy'], + 'FTP_PROXY': default_proxy_config['ftpProxy'], + 'ftp_proxy': default_proxy_config['ftpProxy'], + 'NO_PROXY': default_proxy_config['noProxy'], + 'no_proxy': default_proxy_config['noProxy'], + } + + self.mock_client._original_base_url = 'tcp://example.docker.com:2376' + + assert service._parse_proxy_config() == { + 'HTTP_PROXY': host_specific_proxy_config['httpProxy'], + 'http_proxy': host_specific_proxy_config['httpProxy'], + 'HTTPS_PROXY': host_specific_proxy_config['httpsProxy'], + 'https_proxy': host_specific_proxy_config['httpsProxy'], + 'FTP_PROXY': host_specific_proxy_config['ftpProxy'], + 'ftp_proxy': host_specific_proxy_config['ftpProxy'], + 'NO_PROXY': host_specific_proxy_config['noProxy'], + 'no_proxy': host_specific_proxy_config['noProxy'], + } + + def test_build_service_with_proxy_config(self): + default_proxy_config = { + 'httpProxy': 'http://proxy.mycorp.com:3128', + 'httpsProxy': 'https://user:password@proxy.example.com:3129', + } + buildargs = { + 'HTTPS_PROXY': 'https://rdcf.th08.jp:8911', + 'https_proxy': 'https://rdcf.th08.jp:8911', + } + self.mock_client._general_configs = { + 'proxies': { + 'default': default_proxy_config, + } + } + self.mock_client.base_url = 'http+docker://localunixsocket' + self.mock_client.build.return_value = [ + b'{"stream": "Successfully built 12345"}', + ] + + service = Service('foo', client=self.mock_client, build={'context': '.', 'args': buildargs}) + service.build() + + assert self.mock_client.build.call_count == 1 + assert self.mock_client.build.call_args[1]['buildargs'] == { + 'HTTP_PROXY': default_proxy_config['httpProxy'], + 'http_proxy': default_proxy_config['httpProxy'], + 'HTTPS_PROXY': buildargs['HTTPS_PROXY'], + 'https_proxy': buildargs['HTTPS_PROXY'], + } + + def test_get_create_options_with_proxy_config(self): + default_proxy_config = { + 'httpProxy': 'http://proxy.mycorp.com:3128', + 'httpsProxy': 'https://user:password@proxy.mycorp.com:3129', + 'ftpProxy': 'http://ftpproxy.mycorp.com:21', + } + self.mock_client._general_configs = { + 'proxies': { + 'default': default_proxy_config, + } + } + self.mock_client.base_url = 'http+docker://localunixsocket' + + override_options = { + 'environment': { + 'FTP_PROXY': 'ftp://xdge.exo.au:21', + 'ftp_proxy': 'ftp://xdge.exo.au:21', + } + } + environment = { + 'HTTPS_PROXY': 'https://rdcf.th08.jp:8911', + 'https_proxy': 'https://rdcf.th08.jp:8911', + } + + service = Service('foo', client=self.mock_client, environment=environment) + + create_opts = service._get_container_create_options(override_options, 1) + assert set(create_opts['environment']) == set(format_environment({ + 'HTTP_PROXY': default_proxy_config['httpProxy'], + 'http_proxy': default_proxy_config['httpProxy'], + 'HTTPS_PROXY': environment['HTTPS_PROXY'], + 'https_proxy': environment['HTTPS_PROXY'], + 'FTP_PROXY': override_options['environment']['FTP_PROXY'], + 'ftp_proxy': override_options['environment']['FTP_PROXY'], + })) + + def test_create_when_removed_containers_are_listed(self): + # This is aimed at simulating a race between the API call to list the + # containers, and the ones to inspect each of the listed containers. + # It can happen that a container has been removed after we listed it. + + # containers() returns a container that is about to be removed + self.mock_client.containers.return_value = [ + {'Id': 'rm_cont_id', 'Name': 'rm_cont', 'Image': 'img_id'}, + ] + + # inspect_container() will raise a NotFound when trying to inspect + # rm_cont_id, which at this point has been removed + def inspect(name): + if name == 'rm_cont_id': + raise NotFound(message='Not Found') + + if name == 'new_cont_id': + return {'Id': 'new_cont_id'} + + raise NotImplementedError("incomplete mock") + + self.mock_client.inspect_container.side_effect = inspect + + self.mock_client.inspect_image.return_value = {'Id': 'imageid'} + + self.mock_client.create_container.return_value = {'Id': 'new_cont_id'} + + # We should nonetheless be able to create a new container + service = Service('foo', client=self.mock_client) + + assert service.create_container().id == 'new_cont_id' + + def test_build_volume_options_duplicate_binds(self): + self.mock_client.api_version = '1.29' # Trigger 3.2 format workaround + service = Service('foo', client=self.mock_client) + ctnr_opts, override_opts = service._build_container_volume_options( + previous_container=None, + container_options={ + 'volumes': [ + MountSpec.parse({'source': 'vol', 'target': '/data', 'type': 'volume'}), + VolumeSpec.parse('vol:/data:rw'), + ], + 'environment': {}, + }, + override_options={}, + ) + assert 'binds' in override_opts + assert len(override_opts['binds']) == 1 + assert override_opts['binds'][0] == 'vol:/data:rw' + + def test_volumes_order_is_preserved(self): + service = Service('foo', client=self.mock_client) + volumes = [ + VolumeSpec.parse(cfg) for cfg in [ + '/v{0}:/v{0}:rw'.format(i) for i in range(6) + ] + ] + ctnr_opts, override_opts = service._build_container_volume_options( + previous_container=None, + container_options={ + 'volumes': volumes, + 'environment': {}, + }, + override_options={}, + ) + assert override_opts['binds'] == [vol.repr() for vol in volumes] + + +class TestServiceNetwork(unittest.TestCase): + def setUp(self): + self.mock_client = mock.create_autospec(docker.APIClient) + self.mock_client.api_version = DEFAULT_DOCKER_API_VERSION + self.mock_client._general_configs = {} + + def test_connect_container_to_networks_short_aliase_exists(self): + service = Service( + 'db', + self.mock_client, + 'myproject', + image='foo', + networks={'project_default': {}}) + container = Container( + None, + { + 'Id': 'abcdef', + 'NetworkSettings': { + 'Networks': { + 'project_default': { + 'Aliases': ['analias', 'abcdef'], + }, + }, + }, + }, + True) + service.connect_container_to_networks(container) + + assert not self.mock_client.disconnect_container_from_network.call_count + assert not self.mock_client.connect_container_to_network.call_count + + +def sort_by_name(dictionary_list): + return sorted(dictionary_list, key=lambda k: k['name']) + + +class BuildUlimitsTestCase(unittest.TestCase): + + def test_build_ulimits_with_dict(self): + ulimits = build_ulimits( + { + 'nofile': {'soft': 10000, 'hard': 20000}, + 'nproc': {'soft': 65535, 'hard': 65535} + } + ) + expected = [ + {'name': 'nofile', 'soft': 10000, 'hard': 20000}, + {'name': 'nproc', 'soft': 65535, 'hard': 65535} + ] + assert sort_by_name(ulimits) == sort_by_name(expected) + + def test_build_ulimits_with_ints(self): + ulimits = build_ulimits({'nofile': 20000, 'nproc': 65535}) + expected = [ + {'name': 'nofile', 'soft': 20000, 'hard': 20000}, + {'name': 'nproc', 'soft': 65535, 'hard': 65535} + ] + assert sort_by_name(ulimits) == sort_by_name(expected) + + def test_build_ulimits_with_integers_and_dicts(self): + ulimits = build_ulimits( + { + 'nproc': 65535, + 'nofile': {'soft': 10000, 'hard': 20000} + } + ) + expected = [ + {'name': 'nofile', 'soft': 10000, 'hard': 20000}, + {'name': 'nproc', 'soft': 65535, 'hard': 65535} + ] + assert sort_by_name(ulimits) == sort_by_name(expected) + + +class NetTestCase(unittest.TestCase): + def setUp(self): + self.mock_client = mock.create_autospec(docker.APIClient) + self.mock_client.api_version = DEFAULT_DOCKER_API_VERSION + self.mock_client._general_configs = {} + + def test_network_mode(self): + network_mode = NetworkMode('host') + assert network_mode.id == 'host' + assert network_mode.mode == 'host' + assert network_mode.service_name is None + + def test_network_mode_container(self): + container_id = 'abcd' + network_mode = ContainerNetworkMode(Container(None, {'Id': container_id})) + assert network_mode.id == container_id + assert network_mode.mode == 'container:' + container_id + assert network_mode.service_name is None + + def test_network_mode_service(self): + container_id = 'bbbb' + service_name = 'web' + self.mock_client.containers.return_value = [ + {'Id': container_id, 'Name': container_id, 'Image': 'abcd'}, + ] + + service = Service(name=service_name, client=self.mock_client) + network_mode = ServiceNetworkMode(service) + + assert network_mode.id == service_name + assert network_mode.mode == 'container:' + container_id + assert network_mode.service_name == service_name + + def test_network_mode_service_no_containers(self): + service_name = 'web' + self.mock_client.containers.return_value = [] + + service = Service(name=service_name, client=self.mock_client) + network_mode = ServiceNetworkMode(service) + + assert network_mode.id == service_name + assert network_mode.mode is None + assert network_mode.service_name == service_name + + +class ServicePortsTest(unittest.TestCase): + def test_formatted_ports(self): + ports = [ + '3000', + '0.0.0.0:4025-4030:23000-23005', + ServicePort(6000, None, None, None, None), + ServicePort(8080, 8080, None, None, None), + ServicePort('20000', '20000', 'udp', 'ingress', None), + ServicePort(30000, '30000', 'tcp', None, '127.0.0.1'), + ] + formatted = formatted_ports(ports) + assert ports[0] in formatted + assert ports[1] in formatted + assert '6000/tcp' in formatted + assert '8080:8080/tcp' in formatted + assert '20000:20000/udp' in formatted + assert '127.0.0.1:30000:30000/tcp' in formatted + + +def build_mount(destination, source, mode='rw'): + return {'Source': source, 'Destination': destination, 'Mode': mode} + + +class ServiceVolumesTest(unittest.TestCase): + + def setUp(self): + self.mock_client = mock.create_autospec(docker.APIClient) + self.mock_client.api_version = DEFAULT_DOCKER_API_VERSION + self.mock_client._general_configs = {} + + def test_build_volume_binding(self): + binding = build_volume_binding(VolumeSpec.parse('/outside:/inside', True)) + assert binding == ('/inside', '/outside:/inside:rw') + + def test_get_container_data_volumes(self): + options = [VolumeSpec.parse(v) for v in [ + '/host/volume:/host/volume:ro', + '/new/volume', + '/existing/volume', + 'named:/named/vol', + '/dev/tmpfs' + ]] + + self.mock_client.inspect_image.return_value = { + 'ContainerConfig': { + 'Volumes': { + '/mnt/image/data': {}, + } + } + } + container = Container(self.mock_client, { + 'Image': 'ababab', + 'Mounts': [ + { + 'Source': '/host/volume', + 'Destination': '/host/volume', + 'Mode': '', + 'RW': True, + 'Name': 'hostvolume', + }, { + 'Source': '/var/lib/docker/aaaaaaaa', + 'Destination': '/existing/volume', + 'Mode': '', + 'RW': True, + 'Name': 'existingvolume', + }, { + 'Source': '/var/lib/docker/bbbbbbbb', + 'Destination': '/removed/volume', + 'Mode': '', + 'RW': True, + 'Name': 'removedvolume', + }, { + 'Source': '/var/lib/docker/cccccccc', + 'Destination': '/mnt/image/data', + 'Mode': '', + 'RW': True, + 'Name': 'imagedata', + }, + ] + }, has_been_inspected=True) + + expected = [ + VolumeSpec.parse('existingvolume:/existing/volume:rw'), + VolumeSpec.parse('imagedata:/mnt/image/data:rw'), + ] + + volumes, _ = get_container_data_volumes(container, options, ['/dev/tmpfs'], []) + assert sorted(volumes) == sorted(expected) + + def test_merge_volume_bindings(self): + options = [ + VolumeSpec.parse(v, True) for v in [ + '/host/volume:/host/volume:ro', + '/host/rw/volume:/host/rw/volume', + '/new/volume', + '/existing/volume', + '/dev/tmpfs' + ] + ] + + self.mock_client.inspect_image.return_value = { + 'ContainerConfig': {'Volumes': {}} + } + + previous_container = Container(self.mock_client, { + 'Id': 'cdefab', + 'Image': 'ababab', + 'Mounts': [{ + 'Source': '/var/lib/docker/aaaaaaaa', + 'Destination': '/existing/volume', + 'Mode': '', + 'RW': True, + 'Name': 'existingvolume', + }], + }, has_been_inspected=True) + + expected = [ + '/host/volume:/host/volume:ro', + '/host/rw/volume:/host/rw/volume:rw', + 'existingvolume:/existing/volume:rw', + ] + + binds, affinity = merge_volume_bindings(options, ['/dev/tmpfs'], previous_container, []) + assert sorted(binds) == sorted(expected) + assert affinity == {'affinity:container': '=cdefab'} + + def test_mount_same_host_path_to_two_volumes(self): + service = Service( + 'web', + image='busybox', + volumes=[ + VolumeSpec.parse('/host/path:/data1', True), + VolumeSpec.parse('/host/path:/data2', True), + ], + client=self.mock_client, + ) + + self.mock_client.inspect_image.return_value = { + 'Id': 'ababab', + 'ContainerConfig': { + 'Volumes': {} + } + } + + service._get_container_create_options( + override_options={}, + number=1, + ) + + assert set(self.mock_client.create_host_config.call_args[1]['binds']) == {'/host/path:/data1:rw', + '/host/path:/data2:rw'} + + def test_get_container_create_options_with_different_host_path_in_container_json(self): + service = Service( + 'web', + image='busybox', + volumes=[VolumeSpec.parse('/host/path:/data')], + client=self.mock_client, + ) + volume_name = 'abcdefff1234' + + self.mock_client.inspect_image.return_value = { + 'Id': 'ababab', + 'ContainerConfig': { + 'Volumes': { + '/data': {}, + } + } + } + + self.mock_client.inspect_container.return_value = { + 'Id': '123123123', + 'Image': 'ababab', + 'Mounts': [ + { + 'Destination': '/data', + 'Source': '/mnt/sda1/host/path', + 'Mode': '', + 'RW': True, + 'Driver': 'local', + 'Name': volume_name, + }, + ] + } + + service._get_container_create_options( + override_options={}, + number=1, + previous_container=Container(self.mock_client, {'Id': '123123123'}), + ) + + assert ( + self.mock_client.create_host_config.call_args[1]['binds'] == + ['{}:/data:rw'.format(volume_name)] + ) + + def test_warn_on_masked_volume_no_warning_when_no_container_volumes(self): + volumes_option = [VolumeSpec('/home/user', '/path', 'rw')] + container_volumes = [] + service = 'service_name' + + with mock.patch('compose.service.log', autospec=True) as mock_log: + warn_on_masked_volume(volumes_option, container_volumes, service) + + assert not mock_log.warning.called + + def test_warn_on_masked_volume_when_masked(self): + volumes_option = [VolumeSpec('/home/user', '/path', 'rw')] + container_volumes = [ + VolumeSpec('/var/lib/docker/path', '/path', 'rw'), + VolumeSpec('/var/lib/docker/path', '/other', 'rw'), + ] + service = 'service_name' + + with mock.patch('compose.service.log', autospec=True) as mock_log: + warn_on_masked_volume(volumes_option, container_volumes, service) + + mock_log.warning.assert_called_once_with(mock.ANY) + + def test_warn_on_masked_no_warning_with_same_path(self): + volumes_option = [VolumeSpec('/home/user', '/path', 'rw')] + container_volumes = [VolumeSpec('/home/user', '/path', 'rw')] + service = 'service_name' + + with mock.patch('compose.service.log', autospec=True) as mock_log: + warn_on_masked_volume(volumes_option, container_volumes, service) + + assert not mock_log.warning.called + + def test_warn_on_masked_no_warning_with_container_only_option(self): + volumes_option = [VolumeSpec(None, '/path', 'rw')] + container_volumes = [ + VolumeSpec('/var/lib/docker/volume/path', '/path', 'rw') + ] + service = 'service_name' + + with mock.patch('compose.service.log', autospec=True) as mock_log: + warn_on_masked_volume(volumes_option, container_volumes, service) + + assert not mock_log.warning.called + + def test_create_with_special_volume_mode(self): + self.mock_client.inspect_image.return_value = {'Id': 'imageid'} + + self.mock_client.create_container.return_value = {'Id': 'containerid'} + + volume = '/tmp:/foo:z' + Service( + 'web', + client=self.mock_client, + image='busybox', + volumes=[VolumeSpec.parse(volume, True)], + ).create_container() + + assert self.mock_client.create_container.call_count == 1 + assert self.mock_client.create_host_config.call_args[1]['binds'] == [volume] + + +class ServiceSecretTest(unittest.TestCase): + def setUp(self): + self.mock_client = mock.create_autospec(docker.APIClient) + self.mock_client.api_version = DEFAULT_DOCKER_API_VERSION + self.mock_client._general_configs = {} + + def test_get_secret_volumes(self): + secret1 = { + 'secret': ServiceSecret.parse({'source': 'secret1', 'target': 'b.txt'}), + 'file': 'a.txt' + } + service = Service( + 'web', + client=self.mock_client, + image='busybox', + secrets=[secret1] + ) + volumes = service.get_secret_volumes() + + assert volumes[0].source == secret1['file'] + assert volumes[0].target == '{}/{}'.format(SECRETS_PATH, secret1['secret'].target) + + def test_get_secret_volumes_abspath(self): + secret1 = { + 'secret': ServiceSecret.parse({'source': 'secret1', 'target': '/d.txt'}), + 'file': 'c.txt' + } + service = Service( + 'web', + client=self.mock_client, + image='busybox', + secrets=[secret1] + ) + volumes = service.get_secret_volumes() + + assert volumes[0].source == secret1['file'] + assert volumes[0].target == secret1['secret'].target + + def test_get_secret_volumes_no_target(self): + secret1 = { + 'secret': ServiceSecret.parse({'source': 'secret1'}), + 'file': 'c.txt' + } + service = Service( + 'web', + client=self.mock_client, + image='busybox', + secrets=[secret1] + ) + volumes = service.get_secret_volumes() + + assert volumes[0].source == secret1['file'] + assert volumes[0].target == '{}/{}'.format(SECRETS_PATH, secret1['secret'].source) + + +class RewriteBuildPathTest(unittest.TestCase): + @mock.patch('compose.service.IS_WINDOWS_PLATFORM', True) + def test_rewrite_url_no_prefix(self): + urls = [ + 'http://test.com', + 'https://test.com', + 'git://test.com', + 'github.com/test/test', + 'git@test.com', + ] + for u in urls: + assert rewrite_build_path(u) == u + + @mock.patch('compose.service.IS_WINDOWS_PLATFORM', True) + def test_rewrite_windows_path(self): + assert rewrite_build_path('C:\\context') == WINDOWS_LONGPATH_PREFIX + 'C:\\context' + assert rewrite_build_path( + rewrite_build_path('C:\\context') + ) == rewrite_build_path('C:\\context') + + @mock.patch('compose.service.IS_WINDOWS_PLATFORM', False) + def test_rewrite_unix_path(self): + assert rewrite_build_path('/context') == '/context' diff --git a/tests/unit/split_buffer_test.py b/tests/unit/split_buffer_test.py new file mode 100644 index 00000000000..d6b5b884c87 --- /dev/null +++ b/tests/unit/split_buffer_test.py @@ -0,0 +1,51 @@ +from .. import unittest +from compose.utils import split_buffer + + +class SplitBufferTest(unittest.TestCase): + def test_single_line_chunks(self): + def reader(): + yield b'abc\n' + yield b'def\n' + yield b'ghi\n' + + self.assert_produces(reader, ['abc\n', 'def\n', 'ghi\n']) + + def test_no_end_separator(self): + def reader(): + yield b'abc\n' + yield b'def\n' + yield b'ghi' + + self.assert_produces(reader, ['abc\n', 'def\n', 'ghi']) + + def test_multiple_line_chunk(self): + def reader(): + yield b'abc\ndef\nghi' + + self.assert_produces(reader, ['abc\n', 'def\n', 'ghi']) + + def test_chunked_line(self): + def reader(): + yield b'a' + yield b'b' + yield b'c' + yield b'\n' + yield b'd' + + self.assert_produces(reader, ['abc\n', 'd']) + + def test_preserves_unicode_sequences_within_lines(self): + string = "a\u2022c\n" + + def reader(): + yield string.encode('utf-8') + + self.assert_produces(reader, [string]) + + def assert_produces(self, reader, expectations): + split = split_buffer(reader()) + + for (actual, expected) in zip(split, expectations): + assert type(actual) == type(expected) + assert actual == expected diff --git a/tests/unit/timeparse_test.py b/tests/unit/timeparse_test.py new file mode 100644 index 00000000000..e56595f1893 --- /dev/null +++ b/tests/unit/timeparse_test.py @@ -0,0 +1,53 @@ +from compose import timeparse + + +def test_milli(): + assert timeparse.timeparse('5ms') == 0.005 + + +def test_milli_float(): + assert timeparse.timeparse('50.5ms') == 0.0505 + + +def test_second_milli(): + assert timeparse.timeparse('200s5ms') == 200.005 + + +def test_second_milli_micro(): + assert timeparse.timeparse('200s5ms10us') == 200.00501 + + +def test_second(): + assert timeparse.timeparse('200s') == 200 + + +def test_second_as_float(): + assert timeparse.timeparse('20.5s') == 20.5 + + +def test_minute(): + assert timeparse.timeparse('32m') == 1920 + + +def test_hour_minute(): + assert timeparse.timeparse('2h32m') == 9120 + + +def test_minute_as_float(): + assert timeparse.timeparse('1.5m') == 90 + + +def test_hour_minute_second(): + assert timeparse.timeparse('5h34m56s') == 20096 + + +def test_invalid_with_space(): + assert timeparse.timeparse('5h 34m 56s') is None + + +def test_invalid_with_comma(): + assert timeparse.timeparse('5h,34m,56s') is None + + +def test_invalid_with_empty_string(): + assert timeparse.timeparse('') is None diff --git a/tests/unit/utils_test.py b/tests/unit/utils_test.py new file mode 100644 index 00000000000..3052e4d8644 --- /dev/null +++ b/tests/unit/utils_test.py @@ -0,0 +1,74 @@ +from compose import utils + + +class TestJsonSplitter: + + def test_json_splitter_no_object(self): + data = '{"foo": "bar' + assert utils.json_splitter(data) is None + + def test_json_splitter_with_object(self): + data = '{"foo": "bar"}\n \n{"next": "obj"}' + assert utils.json_splitter(data) == ({'foo': 'bar'}, '{"next": "obj"}') + + def test_json_splitter_leading_whitespace(self): + data = '\n \r{"foo": "bar"}\n\n {"next": "obj"}' + assert utils.json_splitter(data) == ({'foo': 'bar'}, '{"next": "obj"}') + + +class TestStreamAsText: + + def test_stream_with_non_utf_unicode_character(self): + stream = [b'\xed\xf3\xf3'] + output, = utils.stream_as_text(stream) + assert output == '��' + + def test_stream_with_utf_character(self): + stream = ['ěĝ'.encode()] + output, = utils.stream_as_text(stream) + assert output == 'ěĝ' + + +class TestJsonStream: + + def test_with_falsy_entries(self): + stream = [ + '{"one": "two"}\n{}\n', + "[1, 2, 3]\n[]\n", + ] + output = list(utils.json_stream(stream)) + assert output == [ + {'one': 'two'}, + {}, + [1, 2, 3], + [], + ] + + def test_with_leading_whitespace(self): + stream = [ + '\n \r\n {"one": "two"}{"x": 1}', + ' {"three": "four"}\t\t{"x": 2}' + ] + output = list(utils.json_stream(stream)) + assert output == [ + {'one': 'two'}, + {'x': 1}, + {'three': 'four'}, + {'x': 2} + ] + + +class TestParseBytes: + def test_parse_bytes(self): + assert utils.parse_bytes('123kb') == 123 * 1024 + assert utils.parse_bytes(123) == 123 + assert utils.parse_bytes('foobar') is None + assert utils.parse_bytes('123') == 123 + + +class TestMoreItertools: + def test_unique_everseen(self): + unique = utils.unique_everseen + assert list(unique([2, 1, 2, 1])) == [2, 1] + assert list(unique([2, 1, 2, 1], hash)) == [2, 1] + assert list(unique([2, 1, 2, 1], lambda x: 'key_%s' % x)) == [2, 1] diff --git a/tests/unit/volume_test.py b/tests/unit/volume_test.py new file mode 100644 index 00000000000..0dfbfcd40b7 --- /dev/null +++ b/tests/unit/volume_test.py @@ -0,0 +1,23 @@ +import docker +import pytest + +from compose import volume +from tests import mock + + +@pytest.fixture +def mock_client(): + return mock.create_autospec(docker.APIClient) + + +class TestVolume: + + def test_remove_local_volume(self, mock_client): + vol = volume.Volume(mock_client, 'foo', 'project') + vol.remove() + mock_client.remove_volume.assert_called_once_with('foo_project') + + def test_remove_external_volume(self, mock_client): + vol = volume.Volume(mock_client, 'foo', 'project', external=True) + vol.remove() + assert not mock_client.remove_volume.called diff --git a/tox.ini b/tox.ini new file mode 100644 index 00000000000..174fd5ccdc9 --- /dev/null +++ b/tox.ini @@ -0,0 +1,57 @@ +[tox] +envlist = py39,pre-commit + +[testenv] +usedevelop=True +whitelist_externals=mkdir +passenv = + LD_LIBRARY_PATH + DOCKER_HOST + DOCKER_CERT_PATH + DOCKER_TLS_VERIFY + DOCKER_VERSION + SWARM_SKIP_* + SWARM_ASSUME_MULTINODE +setenv = + HOME=/tmp +deps = + -rrequirements-indirect.txt + -rrequirements.txt + -rrequirements-dev.txt +commands = + mkdir -p .coverage-binfiles + py.test -v \ + --cov=compose \ + --cov-report html \ + --cov-report term \ + --cov-config=tox.ini \ + {posargs:tests} + +[testenv:pre-commit] +skip_install = True +deps = + pre-commit +commands = + pre-commit install + pre-commit run --all-files --show-diff-on-failure + +# Coverage configuration +[run] +branch = True +data_file = .coverage-binfiles/.coverage + +[report] +show_missing = true + +[html] +directory = coverage-html +# end coverage configuration + +[flake8] +max-line-length = 105 +# Set this high for now +max-complexity = 11 +exclude = compose/packages + +[pytest] +addopts = --tb=short -rxs