Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

feat: add swarm support to docker acquistion #3744

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 12 commits into from
Jul 31, 2025
Merged

feat: add swarm support to docker acquistion #3744

merged 12 commits into from
Jul 31, 2025

Conversation

LaurenceJJones
Copy link
Contributor

@LaurenceJJones LaurenceJJones commented Jul 22, 2025 •
edited
Loading

fix: #3540

Warning

CrowdSec MUST be running on a manager node for this to work as workers do not have permissions to read service logs.

🚀 Summary

This PR implements Docker Swarm service support for the docker acquisition module. The key improvement addresses the missing service detection and deletion.

📋 Configuration Examples

Service Name/ID Filtering

source: docker
mode: tail
service_name:
  - "web-app"
  - "api-service"
service_id:
  - "abc123xyz"

Service Regex Patterns

source: docker
mode: tail
service_name_regexp:
  - "web-.*"
  - ".*-prod$"
service_id_regexp:
  - "^prod-.*"

Label-based Service Selection

source: docker
mode: tail
use_service_labels: true

Service labels format:

docker service create \
  --label crowdsec.enable=true \
  --label crowdsec.labels.type=nginx \
  nginx:latest

🧪 Testing

1. Unit Tests

# Run all docker acquisition tests
go test ./pkg/acquisition/modules/docker/ -v

# Run only service-specific tests  
go test ./pkg/acquisition/modules/docker/ -v -run "Service"

# Test streaming acquisition (includes service tests)
go test ./pkg/acquisition/modules/docker/ -v -run "TestStreamingAcquisition"

2. Integration Testing

Setup Test Swarm:

# Initialize swarm
docker swarm init

# Create test service
docker service create \
  --name test-nginx \
  --label crowdsec.enable=true \
  --label crowdsec.labels.type=nginx \
  --replicas 2 \
  nginx:latest

Test Configuration:

# /etc/crowdsec/acquis.yaml
source: docker
mode: tail
service_name:
  - "test-nginx"
labels:
  type: nginx

Verify Service Monitoring:

# Check service logs are being processed
sudo crowdsec -t

# Monitor acquisition metrics
curl http://localhost:6060/metrics | grep cs_dockersource_hits_total

3. Manual Testing Scenarios

Service Lifecycle:

# 1. Start service → Should begin monitoring
docker service create --name test-app nginx

# 2. Scale service → Should continue monitoring  
docker service scale test-app=3

# 3. Remove service → Should stop monitoring & cleanup
docker service rm test-app

@github-actions GitHub Actions
Copy link

@LaurenceJJones: There are no 'kind' label on this PR. You need a 'kind' label to generate the release automatically.

  • /kind feature
  • /kind enhancement
  • /kind refactoring
  • /kind fix
  • /kind chore
  • /kind dependencies
Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

@github-actions GitHub Actions
Copy link

@LaurenceJJones: There are no area labels on this PR. You can add as many areas as you see fit.

  • /area agent
  • /area local-api
  • /area cscli
  • /area appsec
  • /area security
  • /area configuration
Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

@codecov Codecov
Copy link

codecov bot commented Jul 22, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 60.66176% with 107 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.68%. Comparing base (aa1abce) to head (02ec7ea).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
pkg/acquisition/modules/docker/docker.go 60.66% 89 Missing and 18 partials ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #3744      +/-   ##
==========================================
+ Coverage   60.60%   60.68%   +0.07%     
==========================================
  Files         390      390              
  Lines       41873    42100     +227     
==========================================
+ Hits        25376    25547     +171     
- Misses      14398    14440      +42     
- Partials     2099     2113      +14
Flag Coverage Δ
bats 44.59% <2.57%> (-0.19%) ⬇️
unit-linux 34.31% <60.66%> (+0.20%) ⬆️
unit-windows 24.41% <60.66%> (+0.22%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mmetc
Copy link
Contributor

mmetc commented Jul 23, 2025 •
edited
Loading

I added a bats test to verify that malformed regexp is not a cause for panic, the configuration is still invalid and should prevent crowdsec from starting but not crash.

We can precompile and store them with DockerSource.

Another thing, are you sure that runningContainerState/runningServiceState don't need a mutex?

@buixor buixor added this to the 1.7.0 milestone Jul 25, 2025
@buixor buixor requested a review from mmetc July 25, 2025 12:41
@LaurenceJJones LaurenceJJones merged commit 57989ea into crowdsecurity:master Jul 31, 2025
20 of 21 checks passed
@LaurenceJJones LaurenceJJones deleted the docker_acquis_swarm_support branch July 31, 2025 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@blotus blotus blotus approved these changes

@mmetc mmetc Awaiting requested review from mmetc

Assignees

No one assigned

Labels

area/agent kind/feature

Projects

None yet

Milestone

1.7.0

Development

Successfully merging this pull request may close these issues.

Add support for Docker Swarm

4 participants

@LaurenceJJones @mmetc @blotus @buixor
Morty Proxy This is a proxified and sanitized view of the page, visit original site.