Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Conversation

directionless
Copy link
Member

@directionless directionless commented Aug 2, 2024

Fixes #8391

Modifies the recursive_directory_iterator to the non throwing form

Looking at our code, I notice that we call fs::recursive_directory_iterator in two places. One has a try/catch, one does not. The internet suggests some versions of boost have a bug that causes is_symlink to raise if there's no permissions. They reference https://svn.boost.org/trac/boost/ticket/4494 which doesn't exist any more

@directionless directionless requested review from a team as code owners August 2, 2024 00:28
Copy link
Member

@Smjert Smjert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! It's possible to see the bug here: https://web.archive.org/web/20160804003617/https://svn.boost.org/trac/boost/ticket/4494, but that's for a pretty old version of boost, I would expect for that to have been fixed.

Either way the issue you created and the crash shown there talks about recursive_directory_iterator causing the exception, but we are using all of the boost APIs with their exception throwing variant.
If instead we use the other constructor, it's possible for the API to register errors into a return value; I would use that instead of an exception, since exception throwing is pretty slow, and it's especially not ideal in a loop like this.
You can see the various functions docs here: https://www.boost.org/doc/libs/1_77_0/libs/filesystem/doc/reference.html

Furthermore a Permission Denied is not totally unexpected or "exceptional", so it makes sense to handle it differently.

@directionless
Copy link
Member Author

If instead we use the other constructor, it's possible for the API to register errors into a return value; I would use that instead of an exception, since exception throwing is pretty slow, and it's especially not ideal in a loop like this

I agree, but I'm not much of a c++ programmer. Is there an example I can borrow from?

@directionless
Copy link
Member Author

Cribbing from the internet, if I do this:

    boost::system::error_code ec;

    for (fs::recursive_directory_iterator it{path, ec}, end; it != end;
         it.increment(ec)) {
      LOG(WARNING) << "SEPH: examinging " << it->path() << "\n";
      if (fs::is_symlink(it->path())) {
        results.push_back(it->path().string());
      } else if (fs::is_directory(it->path())) {
        results.push_back(it->path().string());
      }
    }

It stops on the first permission denied error. Obviously not the behavior we want.

@directionless
Copy link
Member Author

As I understand it, there are a couple of things. The first issue, is that we want the non-throw form -- to get this, we pass in an error receiver. The second issue, is that short form of the for loop uses ++ which has causes an exception if the iterate is passed in, thus the 3 part for loop.

I've also opted to pass in fs::directory_options::skip_permission_denied since otherwise the iteration stops on the first permission error.

I opted to ignore the errors, half because I didn't know how to read them, and half because all we'd do is INFO log them.

For my future self, these links had some okay explanations, and some possible helpers if we wanted to make a class for it.

@directionless directionless changed the title Add a try/catch to fs::recursive_directory_iterator Use non-throw version of fs::recursive_directory_iterator Aug 3, 2024
Smjert
Smjert previously requested changes Aug 9, 2024
Copy link
Member

@Smjert Smjert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay, but yes passing the error_code to the constructor so that it can be set is what I meant.

But we have to do it to all the functions, otherwise other can still throw.

osquery/filesystem/filesystem.cpp Outdated Show resolved Hide resolved
osquery/filesystem/filesystem.cpp Outdated Show resolved Hide resolved
osquery/filesystem/filesystem.cpp Outdated Show resolved Hide resolved
osquery/filesystem/filesystem.cpp Outdated Show resolved Hide resolved
osquery/filesystem/filesystem.cpp Outdated Show resolved Hide resolved
@directionless directionless force-pushed the seph/linux-filesystem-error-8391 branch from 42db834 to 8309709 Compare August 13, 2024 16:58
Copy link
Member

@zwass zwass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. By my interpretation we should be getting the same semantics while skipping permission errors and avoiding throws.

@directionless directionless merged commit f2c581e into osquery:master Aug 13, 2024
@directionless directionless deleted the seph/linux-filesystem-error-8391 branch August 13, 2024 23:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Crash in linux file monitoring, somehow related to NFS

3 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.