-
Notifications
You must be signed in to change notification settings - Fork 13.7k
[lldb/Host] Enable inheriting "non-inheritable" FDs #126935
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This is a follow-up to llvm#126935, which enables passing handles to a child process on windows systems. Unlike on unix-like systems, the handles need to be created with the "inheritable" flag because ...
✅ With the latest revision this PR passed the C/C++ code formatter. |
This is a follow-up to llvm#126935, which enables passing handles to a child process on windows systems. Unlike on unix-like systems, the handles need to be created with the "inheritable" flag because ...
This is a follow-up to llvm#126935, which enables passing handles to a child process on windows systems. Unlike on unix-like systems, the handles need to be created with the "inheritable" flag because ...
This is a follow-up to llvm#126935, which enables passing handles to a child process on windows systems. Unlike on unix-like systems, the handles need to be created with the "inheritable" flag because there's to way to change the flag value after it has been created. This is why I don't respect the child_process_inherit flag but rather always set the flag to true. (My next step is to delete the flag entirely.) This does mean that pipe may be created as inheritable even if its not necessary, but I think this is offset by the fact that windows (unlike unixes, which pass all ~O_CLOEXEC descriptors through execve and *all* descriptors through fork) has a way to specify the precise set of handles to pass to a specific child process. If this turns out to be insufficient, instead of a constructor flag, I'd rather go with creating a separate api to create an inheritable copy of a handle (as typically, you only want to inherit one end of the pipe).
Currently we're creating inheritable (`~FD_CLOEXEC`) file descriptors in the (few) cases where we need to pass an FD to a subprocess. The problem with these is that, in a multithreaded application such as lldb, there's essentially no way to prevent them from being leaked into processes other than the intended one. A safer (though still not completely safe) approach is to mark the descriptors as FD_CLOEXEC and only clear this flag in the subprocess. We currently have something that almost does that, which is the ability to add a `DuplicateFileAction` to our `ProcessLaunchInfo` struct (the duplicated file descriptor will be created with the flag cleared). The problem with *that* is that this approach is completely incompatible with Windows. Windows equivalents of file descriptors are `HANDLE`s, but these do not have user controlled values -- applications are expected to work with whatever HANDLE values are assigned by the OS. In unix terms, there is no equivalent to the `dup2` syscall (only `dup`). To find a way out of this conundrum, and create a miniscule API surface that works uniformly across platforms, this PR proposes to extend the `DuplicateFileAction` API to support duplicating a file descriptor onto itself. Currently, this operation does nothing (it leaves the FD_CLOEXEC flag set), because that's how `dup2(fd, fd)` behaves, but I think it's not completely unreasonable to say that this operation should clear the FD_CLOEXEC flag, just like it would do if one was using different fd values. This would enable us to pass a windows HANDLE as itself through the ProcessLaunchInfo API. This PR implements the unix portion of this idea. Macos and non-macos launchers are updated to clear FD_CLOEXEC flag when duplicating a file descriptor onto itself, and I've created a test which enables passing a FD_CLOEXEC file descritor to the subprocess. For the windows portion, please see the follow-up PR.
This is a follow-up to llvm#126935, which enables passing handles to a child process on windows systems. Unlike on unix-like systems, the handles need to be created with the "inheritable" flag because there's to way to change the flag value after it has been created. This is why I don't respect the child_process_inherit flag but rather always set the flag to true. (My next step is to delete the flag entirely.) This does mean that pipe may be created as inheritable even if its not necessary, but I think this is offset by the fact that windows (unlike unixes, which pass all ~O_CLOEXEC descriptors through execve and *all* descriptors through fork) has a way to specify the precise set of handles to pass to a specific child process. If this turns out to be insufficient, instead of a constructor flag, I'd rather go with creating a separate api to create an inheritable copy of a handle (as typically, you only want to inherit one end of the pipe).
@llvm/pr-subscribers-lldb Author: Pavel Labath (labath) ChangesCurrently we're creating inheritable ( A safer (though still not completely safe) approach is to mark the descriptors as FD_CLOEXEC and only clear this flag in the subprocess. We currently have something that almost does that, which is the ability to add a Windows equivalents of file descriptors are To find a way out of this conundrum, and create a miniscule API surface that works uniformly across platforms, this PR proposes to extend the This PR implements the unix portion of this idea. Macos and non-macos launchers are updated to clear FD_CLOEXEC flag when duplicating a file descriptor onto itself, and I've created a test which enables passing a FD_CLOEXEC file descritor to the subprocess. For the windows portion, please see the follow-up PR. Full diff: https://github.com/llvm/llvm-project/pull/126935.diff 3 Files Affected:
diff --git a/lldb/source/Host/macosx/objcxx/Host.mm b/lldb/source/Host/macosx/objcxx/Host.mm
index bb270f6a44e43..e187bf98188ae 100644
--- a/lldb/source/Host/macosx/objcxx/Host.mm
+++ b/lldb/source/Host/macosx/objcxx/Host.mm
@@ -1100,7 +1100,7 @@ static bool AddPosixSpawnFileAction(void *_file_actions, const FileAction *info,
else if (info->GetActionArgument() == -1)
error = Status::FromErrorString(
"invalid duplicate fd for posix_spawn_file_actions_adddup2(...)");
- else {
+ else if (info->GetFD() != info->GetActionArgument()) {
error =
Status(::posix_spawn_file_actions_adddup2(file_actions, info->GetFD(),
info->GetActionArgument()),
@@ -1110,6 +1110,15 @@ static bool AddPosixSpawnFileAction(void *_file_actions, const FileAction *info,
"error: {0}, posix_spawn_file_actions_adddup2 "
"(action={1}, fd={2}, dup_fd={3})",
error, file_actions, info->GetFD(), info->GetActionArgument());
+ } else {
+ error =
+ Status(::posix_spawn_file_actions_addinherit_np(file_actions, info->GetFD()),
+ eErrorTypePOSIX);
+ if (error.Fail())
+ LLDB_LOG(log,
+ "error: {0}, posix_spawn_file_actions_addinherit_np "
+ "(action={1}, fd={2})",
+ error, file_actions, info->GetFD());
}
break;
diff --git a/lldb/source/Host/posix/ProcessLauncherPosixFork.cpp b/lldb/source/Host/posix/ProcessLauncherPosixFork.cpp
index 8c6d503fc7fe2..698524349e16a 100644
--- a/lldb/source/Host/posix/ProcessLauncherPosixFork.cpp
+++ b/lldb/source/Host/posix/ProcessLauncherPosixFork.cpp
@@ -17,6 +17,7 @@
#include "llvm/Support/Errno.h"
#include <climits>
+#include <fcntl.h>
#include <sys/ptrace.h>
#include <sys/wait.h>
#include <unistd.h>
@@ -122,8 +123,14 @@ struct ForkLaunchInfo {
ExitWithError(error_fd, "close");
break;
case FileAction::eFileActionDuplicate:
- if (dup2(action.fd, action.arg) == -1)
- ExitWithError(error_fd, "dup2");
+ if (action.fd != action.arg) {
+ if (dup2(action.fd, action.arg) == -1)
+ ExitWithError(error_fd, "dup2");
+ } else {
+ if (fcntl(action.fd, F_SETFD,
+ fcntl(action.fd, F_GETFD) & ~FD_CLOEXEC) == -1)
+ ExitWithError(error_fd, "fcntl");
+ }
break;
case FileAction::eFileActionOpen:
DupDescriptor(error_fd, action.path.c_str(), action.fd, action.arg);
diff --git a/lldb/unittests/Host/HostTest.cpp b/lldb/unittests/Host/HostTest.cpp
index ed1df6de001ea..222de62ab6697 100644
--- a/lldb/unittests/Host/HostTest.cpp
+++ b/lldb/unittests/Host/HostTest.cpp
@@ -9,8 +9,10 @@
#include "lldb/Host/Host.h"
#include "TestingSupport/SubsystemRAII.h"
#include "lldb/Host/FileSystem.h"
+#include "lldb/Host/Pipe.h"
#include "lldb/Host/ProcessLaunchInfo.h"
#include "lldb/Utility/ProcessInfo.h"
+#include "llvm/ADT/Twine.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/FileSystem.h"
#include "llvm/Testing/Support/Error.h"
@@ -87,3 +89,41 @@ TEST(Host, LaunchProcessSetsArgv0) {
ASSERT_THAT_ERROR(Host::LaunchProcess(info).takeError(), Succeeded());
ASSERT_THAT(exit_status.get_future().get(), 0);
}
+
+#ifdef LLVM_ON_UNIX
+TEST(Host, LaunchProcessDuplicatesHandle) {
+ static constexpr llvm::StringLiteral test_msg("Hello subprocess!");
+
+ SubsystemRAII<FileSystem> subsystems;
+
+ if (test_arg) {
+ Pipe pipe(LLDB_INVALID_PIPE, (lldb::pipe_t)test_arg.getValue());
+ llvm::Expected<size_t> bytes_written =
+ pipe.Write(test_msg.data(), test_msg.size());
+ if (bytes_written && *bytes_written == test_msg.size())
+ exit(0);
+ exit(1);
+ }
+ Pipe pipe;
+ ASSERT_THAT_ERROR(pipe.CreateNew(/*child_process_inherit=*/false).takeError(),
+ llvm::Succeeded());
+ ProcessLaunchInfo info;
+ info.SetExecutableFile(FileSpec(TestMainArgv0),
+ /*add_exe_file_as_first_arg=*/true);
+ info.GetArguments().AppendArgument(
+ "--gtest_filter=Host.LaunchProcessDuplicatesHandle");
+ info.GetArguments().AppendArgument(
+ ("--test-arg=" + llvm::Twine((uint64_t)pipe.GetWritePipe())).str());
+ info.AppendDuplicateFileAction((uint64_t)pipe.GetWritePipe(),
+ (uint64_t)pipe.GetWritePipe());
+ info.SetMonitorProcessCallback(&ProcessLaunchInfo::NoOpMonitorCallback);
+ ASSERT_THAT_ERROR(Host::LaunchProcess(info).takeError(), llvm::Succeeded());
+ pipe.CloseWriteFileDescriptor();
+
+ char msg[100];
+ llvm::Expected<size_t> bytes_read =
+ pipe.Read(msg, sizeof(msg), std::chrono::seconds(10));
+ ASSERT_THAT_EXPECTED(bytes_read, llvm::Succeeded());
+ ASSERT_EQ(llvm::StringRef(msg, *bytes_read), test_msg);
+}
+#endif
|
This is a follow-up to llvm#126935, which enables passing handles to a child process on windows systems. Unlike on unix-like systems, the handles need to be created with the "inheritable" flag because there's to way to change the flag value after it has been created. This is why I don't respect the child_process_inherit flag but rather always set the flag to true. (My next step is to delete the flag entirely.) This does mean that pipe may be created as inheritable even if its not necessary, but I think this is offset by the fact that windows (unlike unixes, which pass all ~O_CLOEXEC descriptors through execve and *all* descriptors through fork) has a way to specify the precise set of handles to pass to a specific child process. If this turns out to be insufficient, instead of a constructor flag, I'd rather go with creating a separate api to create an inheritable copy of a handle (as typically, you only want to inherit one end of the pipe).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently we're creating inheritable (~FD_CLOEXEC) file descriptors in the (few) cases where we need to pass an FD to a subprocess. The problem with these is that, in a multithreaded application such as lldb, there's essentially no way to prevent them from being leaked into processes other than the intended one.
The consequences of this are what exactly?
That the debugee may see differently numbered descriptors when run via lldb vs. normally?
Currently, this is unlikely to impact the processes we launch for debugging for two reasons:
One reason why we that code is debug-only (even though there's no reason why LaunchInfo file descriptor actions should behave differently for debug vs. non-debug launches is that we do have places which rely on this silent passing of file descriptor. This allows us to pass those explicitly, which would in theory allow us to make that cone unconditional (although I'm not entirely sure I want to do that, as I'm not particularly fond of that piece of code). In short, this is not for launches of debugged processes, but launches in general. My main inspiration for that was @slydiman's lldb-server platform refactor. (One?) reason that couldn't be done with threads was due to stray file descriptors. In this case, the problem wasn't just "extra FDs", but actual hard breakage. We had code which was waiting for an pipe EOF, but it never got that due to the FD ending up in an unsuspecting (and unrelated) process. Although that's possible to solve in a different way, I think it's still better to declare all of the FDs passed to a process explicitly. |
Skill issue on my part but I don't understand how a file descriptor "ends up" in another process. My interaction with them has been as handles, so I assumed anyone who has the handle would be able to use them. No need to solve that education problem but that's probably why I am slow to realise the point of this change. I have not dealt with the symptoms before. |
They are, but unless they know about them, the FD is just going to sit there, forever. I love to educate people about FDs, so here's a crash course. Suppose you want to communicate with a child process. Here's one way to do this:
This works in that the other process gets one (and only one) end of the pipe and is made aware of its number (via the cmdline arg). The problem is you run the same code on two threads simultaneously. If the threads run in lockstep, they will first create two pipes and then fork two subprocesses. Each of those processes will get both pipes even though it's expecting only one. A solution to that is to ensure the flag is cleared only inside the child process:
This way, even if both child processes end up with the extraneous pipe, the pipe will be automatically closed due to O_CLOEXEC. Only the one which we expect will have its flag cleared. This patch lets us to that (while accounting for all of the layers that this information needs to step through). (Note that this still means that the FD will exist in the child process until the execve call. This is usually not a problem, since that's only a brief moment, but in some situations it can be, and it's why we have the ETXTBUSY workaround in the launching code (because android's ADB had this issue). Unfortunately, fixing this is a lot trickier. The only way I'm aware of involves using a "fork server" -- forking a new process at the beginning of main (while you're still hopefully single threaded and don't have any extra FDs) and then using this process to fork any new children (communicating with it over a domain socket, and using the socket to pass any FDs that you want to pass). |
I once again attempt to parse your PR description, armed with this understanding.
Because when you You would have to track all the file descriptors you have open and arrange to close all the ones you don't want to be open after the
Which means you can clear the flag only for the descriptors that you want to survive the Right? |
Correct. :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Thank you. |
This is a follow-up to llvm#126935, which enables passing handles to a child process on windows systems. Unlike on unix-like systems, the handles need to be created with the "inheritable" flag because there's to way to change the flag value after it has been created. This is why I don't respect the child_process_inherit flag but rather always set the flag to true. (My next step is to delete the flag entirely.) This does mean that pipe may be created as inheritable even if its not necessary, but I think this is offset by the fact that windows (unlike unixes, which pass all ~O_CLOEXEC descriptors through execve and *all* descriptors through fork) has a way to specify the precise set of handles to pass to a specific child process. If this turns out to be insufficient, instead of a constructor flag, I'd rather go with creating a separate api to create an inheritable copy of a handle (as typically, you only want to inherit one end of the pipe).
…#137978) This is a follow-up to #126935, which enables passing handles to a child process on windows systems. Unlike on unix-like systems, the handles need to be created with the "inheritable" flag because there's to way to change the flag value after it has been created. This is why I don't respect the child_process_inherit flag but rather always set the flag to true. (My next step is to delete the flag entirely.) This does mean that pipe may be created as inheritable even if its not necessary, but I think this is offset by the fact that windows (unlike unixes, which pass all ~O_CLOEXEC descriptors through execve and *all* descriptors through fork) has a way to specify the precise set of handles to pass to a specific child process. If this turns out to be insufficient, instead of a constructor flag, I'd rather go with creating a separate api to create an inheritable copy of a handle (as typically, you only want to inherit one end of the pipe).
… on windows (#137978) This is a follow-up to llvm/llvm-project#126935, which enables passing handles to a child process on windows systems. Unlike on unix-like systems, the handles need to be created with the "inheritable" flag because there's to way to change the flag value after it has been created. This is why I don't respect the child_process_inherit flag but rather always set the flag to true. (My next step is to delete the flag entirely.) This does mean that pipe may be created as inheritable even if its not necessary, but I think this is offset by the fact that windows (unlike unixes, which pass all ~O_CLOEXEC descriptors through execve and *all* descriptors through fork) has a way to specify the precise set of handles to pass to a specific child process. If this turns out to be insufficient, instead of a constructor flag, I'd rather go with creating a separate api to create an inheritable copy of a handle (as typically, you only want to inherit one end of the pipe).
Currently we're creating inheritable (`~FD_CLOEXEC`) file descriptors in the (few) cases where we need to pass an FD to a subprocess. The problem with these is that, in a multithreaded application such as lldb, there's essentially no way to prevent them from being leaked into processes other than the intended one. A safer (though still not completely safe) approach is to mark the descriptors as FD_CLOEXEC and only clear this flag in the subprocess. We currently have something that almost does that, which is the ability to add a `DuplicateFileAction` to our `ProcessLaunchInfo` struct (the duplicated file descriptor will be created with the flag cleared). The problem with *that* is that this approach is completely incompatible with Windows. Windows equivalents of file descriptors are `HANDLE`s, but these do not have user controlled values -- applications are expected to work with whatever HANDLE values are assigned by the OS. In unix terms, there is no equivalent to the `dup2` syscall (only `dup`). To find a way out of this conundrum, and create a miniscule API surface that works uniformly across platforms, this PR proposes to extend the `DuplicateFileAction` API to support duplicating a file descriptor onto itself. Currently, this operation does nothing (it leaves the FD_CLOEXEC flag set), because that's how `dup2(fd, fd)` behaves, but I think it's not completely unreasonable to say that this operation should clear the FD_CLOEXEC flag, just like it would do if one was using different fd values. This would enable us to pass a windows HANDLE as itself through the ProcessLaunchInfo API. This PR implements the unix portion of this idea. Macos and non-macos launchers are updated to clear FD_CLOEXEC flag when duplicating a file descriptor onto itself, and I've created a test which enables passing a FD_CLOEXEC file descritor to the subprocess. For the windows portion, please see the follow-up PR.
…n windows (llvm#137978)" This reverts commit a0260a9, reapplying 7c5f5f3, with a fix that makes *both* pipe handles inheritable. The original commit description was: This is a follow-up to llvm#126935, which enables passing handles to a child process on windows systems. Unlike on unix-like systems, the handles need to be created with the "inheritable" flag because there's to way to change the flag value after it has been created. This is why I don't respect the child_process_inherit flag but rather always set the flag to true. (My next step is to delete the flag entirely.) This does mean that pipe may be created as inheritable even if its not necessary, but I think this is offset by the fact that windows (unlike unixes, which pass all ~O_CLOEXEC descriptors through execve and *all* descriptors through fork) has a way to specify the precise set of handles to pass to a specific child process. If this turns out to be insufficient, instead of a constructor flag, I'd rather go with creating a separate api to create an inheritable copy of a handle (as typically, you only want to inherit one end of the pipe).
…n windows (#137978)" (#138896) This reverts commit a0260a9, reapplying 7c5f5f3, with a fix that makes *both* pipe handles inheritable. The original commit description was: This is a follow-up to #126935, which enables passing handles to a child process on windows systems. Unlike on unix-like systems, the handles need to be created with the "inheritable" flag because there's to way to change the flag value after it has been created. This is why I don't respect the child_process_inherit flag but rather always set the flag to true. (My next step is to delete the flag entirely.) This does mean that pipe may be created as inheritable even if its not necessary, but I think this is offset by the fact that windows (unlike unixes, which pass all ~O_CLOEXEC descriptors through execve and *all* descriptors through fork) has a way to specify the precise set of handles to pass to a specific child process. If this turns out to be insufficient, instead of a constructor flag, I'd rather go with creating a separate api to create an inheritable copy of a handle (as typically, you only want to inherit one end of the pipe).
…) handles on windows (#137978)" (#138896) This reverts commit llvm/llvm-project@a0260a9, reapplying llvm/llvm-project@7c5f5f3, with a fix that makes *both* pipe handles inheritable. The original commit description was: This is a follow-up to llvm/llvm-project#126935, which enables passing handles to a child process on windows systems. Unlike on unix-like systems, the handles need to be created with the "inheritable" flag because there's to way to change the flag value after it has been created. This is why I don't respect the child_process_inherit flag but rather always set the flag to true. (My next step is to delete the flag entirely.) This does mean that pipe may be created as inheritable even if its not necessary, but I think this is offset by the fact that windows (unlike unixes, which pass all ~O_CLOEXEC descriptors through execve and *all* descriptors through fork) has a way to specify the precise set of handles to pass to a specific child process. If this turns out to be insufficient, instead of a constructor flag, I'd rather go with creating a separate api to create an inheritable copy of a handle (as typically, you only want to inherit one end of the pipe).
Currently we're creating inheritable (
~FD_CLOEXEC
) file descriptors in the (few) cases where we need to pass an FD to a subprocess. The problem with these is that, in a multithreaded application such as lldb, there's essentially no way to prevent them from being leaked into processes other than the intended one.A safer (though still not completely safe) approach is to mark the descriptors as FD_CLOEXEC and only clear this flag in the subprocess. We currently have something that almost does that, which is the ability to add a
DuplicateFileAction
to ourProcessLaunchInfo
struct (the duplicated file descriptor will be created with the flag cleared). The problem with that is that this approach is completely incompatible with Windows.Windows equivalents of file descriptors are
HANDLE
s, but these do not have user controlled values -- applications are expected to work with whatever HANDLE values are assigned by the OS. In unix terms, there is no equivalent to thedup2
syscall (onlydup
).To find a way out of this conundrum, and create a miniscule API surface that works uniformly across platforms, this PR proposes to extend the
DuplicateFileAction
API to support duplicating a file descriptor onto itself. Currently, this operation does nothing (it leaves the FD_CLOEXEC flag set), because that's howdup2(fd, fd)
behaves, but I think it's not completely unreasonable to say that this operation should clear the FD_CLOEXEC flag, just like it would do if one was using different fd values. This would enable us to pass a windows HANDLE as itself through the ProcessLaunchInfo API.This PR implements the unix portion of this idea. Macos and non-macos launchers are updated to clear FD_CLOEXEC flag when duplicating a file descriptor onto itself, and I've created a test which enables passing a FD_CLOEXEC file descritor to the subprocess. For the windows portion, please see the follow-up PR.