-
-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Add scoped npm package path #8686
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code looks prett
|
||
// Search for nested packages: node_modules/@scope/package/package.json | ||
// or similar nested structures | ||
boost::filesystem::path pattern2("node_modules/%/%/package.json"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know enough node to know what correct is, but I wonder if this should be node_modules/@%/%/package.json
. Looking at some node modules on my machine, I notice that I have the scoped packages, but also stuff that might be something else:
node_modules/@actions/http-client/package.json
node_modules/@alpinejs/morph/package.json
node_modules/chartkick/chart.js/package.json
node_modules/chartkick/highcharts/package.json
node_modules/date-fns/compareDesc/package.json
node_modules/date-fns/daysToWeeks/package.json
node_modules/date-fns/differenceInBusinessDays/package.json
node_modules/date-fns/differenceInCalendarDays/package.json
node_modules/date-fns/differenceInCalendarISOWeeks/package.json
node_modules/date-fns/differenceInCalendarISOWeekYears/package.json
node_modules/date-fns/differenceInCalendarMonths/package.json
...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on the comments in the header file for resolveFilePattern
* SQL uses '%' as a wildcard matching token, and filesystem globbing uses '*'.
* In osquery-internal methods the filesystem character is used. This helper
* method will perform the correct preg/escape and replace.
This function is where the SQL wildcard %
is translated to filesystem wildcard *
.
The filesystem wildcard *
should include packages starting with @
, and packages starting with letters.
i.e. packages like date-fns/daysToWeeks
should be included in the result since it is also 2 levels under node_modules
. I have added a test case in 8192ae4 to test this. CI is currently failing with docker pull errors:
/usr/bin/docker pull osquery/builder20.04:7e9ee0339
Error response from daemon: unauthorized: authentication required
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
packages like date-fns/daysToWeeks should be included in the result since it is also 2 levels under node_modules
This is what I don't understand. Should this be included? Can it be installed separately?
The docs say scoped packages all start with a @
. I don't understand what the difference is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you advise how packages like date-fns/daysToWeeks
were installed? I tried npm install date-fns --save
, I see a different directory structure:
% ls node_modules/date-fns/
node_modules/date-fns/package.json
node_modules/date-fns/daysToWeeks.js
node_modules/date-fns/compareDesc.js
node_modules/date-fns/differenceInBusinessDays.js
node_modules/date-fns/differenceInCalendarDays.js
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not much of a js person. So I might have some incorrect details:
$ npm why date-fns/daysToWeeks
npm error No dependencies found matching date-fns/daysToWeeks
npm error A complete log of this run can be found in: /Users/seph/.npm/_logs/2025-10-03T12_38_46_306Z-debug-0.log
$ npm why date-fns
date-fns@2.30.0 optional
node_modules/date-fns
optional date-fns@">=2" from chartkick@5.0.1
node_modules/chartkick
chartkick@"^5.0.1" from the root project
peer date-fns@">=2.0.0" from chartjs-adapter-date-fns@3.0.0
node_modules/chartkick/node_modules/chartjs-adapter-date-fns
optional chartjs-adapter-date-fns@">=3" from chartkick@5.0.1
node_modules/chartkick
chartkick@"^5.0.1" from the root project
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we be trying to find nested packages? Or are they an internal implementation detail? I do not know
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AFAICT these do seem to be an implementation detail? I looked at some of the packages in Fleet's node_modules
and none of the nested ones that do not start with @
seem to be installable via NPM. @lichao127 do you have a reason to believe these packages should be included separately? Otherwise I think we should use @directionless' original suggestion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems to be an anti-pattern to publish scoped packages without an @
prefix. I'm ok with finding or ignoring these cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did some further testing with npm
and it seems to be disallowed at least on the client side:
$ npm init
package name: (npm-test) foo/npm-test
Sorry, name can only contain URL-friendly characters.
package name: (npm-test) @foo/npm-test
version: (1.0.0)
Then later in the process it's checked again even if manually edited to remove the @
:
$ npm publish
npm error Invalid name: "foo/npm-test"
The NPM documentation indicates that scopes always use @
.
Given all of this, let's please only include the top level packages and the scoped packages that include @
in the path. The test can be updated to verify nested/package
is NOT returned.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have updated the code and tests accordingly. Package like nested/package
will not be returned.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test is still failing on macOS.
fixed macOS test. |
Closing and reopening to re-trigger CI on |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome, thanks for working with us on this!
This PR patches
npm_packages
to find scoped NPM packages like@anthropic-ai/claude-code
, and adds a unit test.Fixes: #8683