Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Conversation

lichao127
Copy link
Contributor

@lichao127 lichao127 commented Sep 23, 2025

This PR patches npm_packages to find scoped NPM packages like @anthropic-ai/claude-code, and adds a unit test.

Fixes: #8683

@lichao127 lichao127 changed the title add npm package path WIP: add npm package path Sep 23, 2025
@lichao127 lichao127 changed the title WIP: add npm package path Add npm package path Sep 24, 2025
@lichao127 lichao127 marked this pull request as ready for review September 24, 2025 04:09
@lichao127 lichao127 requested review from a team as code owners September 24, 2025 04:09
@lichao127 lichao127 changed the title Add npm package path Add scoped npm package path Sep 24, 2025
Copy link
Member

@directionless directionless left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks prett


// Search for nested packages: node_modules/@scope/package/package.json
// or similar nested structures
boost::filesystem::path pattern2("node_modules/%/%/package.json");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know enough node to know what correct is, but I wonder if this should be node_modules/@%/%/package.json. Looking at some node modules on my machine, I notice that I have the scoped packages, but also stuff that might be something else:

node_modules/@actions/http-client/package.json
node_modules/@alpinejs/morph/package.json
node_modules/chartkick/chart.js/package.json
node_modules/chartkick/highcharts/package.json
node_modules/date-fns/compareDesc/package.json
node_modules/date-fns/daysToWeeks/package.json
node_modules/date-fns/differenceInBusinessDays/package.json
node_modules/date-fns/differenceInCalendarDays/package.json
node_modules/date-fns/differenceInCalendarISOWeeks/package.json
node_modules/date-fns/differenceInCalendarISOWeekYears/package.json
node_modules/date-fns/differenceInCalendarMonths/package.json
...

Copy link
Contributor Author

@lichao127 lichao127 Sep 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the comments in the header file for resolveFilePattern

 * SQL uses '%' as a wildcard matching token, and filesystem globbing uses '*'.
 * In osquery-internal methods the filesystem character is used. This helper
 * method will perform the correct preg/escape and replace.

This function is where the SQL wildcard % is translated to filesystem wildcard *.

The filesystem wildcard * should include packages starting with @, and packages starting with letters.

i.e. packages like date-fns/daysToWeeks should be included in the result since it is also 2 levels under node_modules. I have added a test case in 8192ae4 to test this. CI is currently failing with docker pull errors:

  /usr/bin/docker pull osquery/builder20.04:7e9ee0339
  Error response from daemon: unauthorized: authentication required

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

packages like date-fns/daysToWeeks should be included in the result since it is also 2 levels under node_modules

This is what I don't understand. Should this be included? Can it be installed separately?

The docs say scoped packages all start with a @. I don't understand what the difference is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you advise how packages like date-fns/daysToWeeks were installed? I tried npm install date-fns --save, I see a different directory structure:

% ls node_modules/date-fns/
node_modules/date-fns/package.json
node_modules/date-fns/daysToWeeks.js
node_modules/date-fns/compareDesc.js
node_modules/date-fns/differenceInBusinessDays.js
node_modules/date-fns/differenceInCalendarDays.js

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not much of a js person. So I might have some incorrect details:

$ npm why date-fns/daysToWeeks
npm error No dependencies found matching date-fns/daysToWeeks
npm error A complete log of this run can be found in: /Users/seph/.npm/_logs/2025-10-03T12_38_46_306Z-debug-0.log

$ npm why date-fns            
date-fns@2.30.0 optional
node_modules/date-fns
  optional date-fns@">=2" from chartkick@5.0.1
  node_modules/chartkick
    chartkick@"^5.0.1" from the root project
  peer date-fns@">=2.0.0" from chartjs-adapter-date-fns@3.0.0
  node_modules/chartkick/node_modules/chartjs-adapter-date-fns
    optional chartjs-adapter-date-fns@">=3" from chartkick@5.0.1
    node_modules/chartkick
      chartkick@"^5.0.1" from the root project

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be trying to find nested packages? Or are they an internal implementation detail? I do not know

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICT these do seem to be an implementation detail? I looked at some of the packages in Fleet's node_modules and none of the nested ones that do not start with @ seem to be installable via NPM. @lichao127 do you have a reason to believe these packages should be included separately? Otherwise I think we should use @directionless' original suggestion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be an anti-pattern to publish scoped packages without an @ prefix. I'm ok with finding or ignoring these cases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did some further testing with npm and it seems to be disallowed at least on the client side:

$ npm init
package name: (npm-test) foo/npm-test
Sorry, name can only contain URL-friendly characters.
package name: (npm-test) @foo/npm-test
version: (1.0.0)

Then later in the process it's checked again even if manually edited to remove the @:

$ npm publish
npm error Invalid name: "foo/npm-test"

The NPM documentation indicates that scopes always use @.

Given all of this, let's please only include the top level packages and the scoped packages that include @ in the path. The test can be updated to verify nested/package is NOT returned.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the code and tests accordingly. Package like nested/package will not be returned.

Copy link
Member

@zwass zwass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test is still failing on macOS.

@lichao127
Copy link
Contributor Author

The test is still failing on macOS.

fixed macOS test.
the build_x86 linux test is failing with No space left on device. This is also failing on the main branch.

@lichao127 lichao127 requested a review from zwass September 30, 2025 15:18
@directionless
Copy link
Member

Closing and reopening to re-trigger CI on master

Copy link
Member

@zwass zwass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks for working with us on this!

@zwass zwass merged commit ad99fb5 into osquery:master Oct 9, 2025
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

npm_packages not reporting scoped packages

4 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.