Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

kubernetes.io/csi: unmounter failed, because of the vol_data.json: no such file or directory #131232

Copy link
Copy link
Open
@Black-max12138

Description

@Black-max12138
Issue body actions

What happened?

When the csi interface is invoked to attach a volume, if the volume fails to be attached, the removeMountDir method is invoked to clear the directory where the volume is attached. If the pod is deleted, the csi volume fails to be attached, and a message is displayed indicating that the volume fails to be detached. The cause is that the vol_data.json file does not exist.
The code for mount a volume is as follows:

err = csi.NodePublishVolume(
ctx,
volumeHandle,
readOnly,
deviceMountPath,
dir,
accessMode,
publishContext,
volAttribs,
nodePublishSecrets,
fsType,
mountOptions,
nodePublishFSGroupArg,
)
if err != nil {
// If operation finished with error then we can remove the mount directory.
if volumetypes.IsOperationFinishedError(err) {
if removeMountDirErr := removeMountDir(c.plugin, dir); removeMountDirErr != nil {
klog.Error(log("mounter.SetupAt failed to remove mount dir after a NodePublish() error [%s]: %v", dir, removeMountDirErr))
}
}
return err
}

The NewUnmounter method is invoked to detach a volume. The vol_data.json file needs to be read. The code is as follows:

data, err := loadVolumeData(dataDir, volDataFileName)
if err != nil {
return nil, errors.New(log("unmounter failed to load volume data file [%s]: %v", dir, err))
}

The log is as follows:
E0120 10:53:49.870240 4083505 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/31c052e2-63f5-4127-95cb-0923db30dd5a-dump podName:31c052e2-63f5-4127-95cb-0923db30dd5a nodeName:}" failed. No retries permitted until 2025-01-20 10:53:50.370182329 +0000 UTC m=+5.022372153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "dump" (UniqueName: "kubernetes.io/csi/31c052e2-63f5-4127-95cb-0923db30dd5a-dump") pod "ndp-spark-history-0" (UID: "31c052e2-63f5-4127-95cb-0923db30dd5a") : rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial unix /var/lib/kubelet/plugins_registry/sop-csi-driver-reg.sock: connect: resource temporarily unavailable"

E0120 13:16:00.518419 4083505 reconciler_common.go:169] "operationExecutor.UnmountVolume failed (controllerAttachDetachEnabled true) for volume "dump" (UniqueName: "kubernetes.io/csi/31c052e2-63f5-4127-95cb-0923db30dd5a-dump") pod "31c052e2-63f5-4127-95cb-0923db30dd5a" (UID: "31c052e2-63f5-4127-95cb-0923db30dd5a") : UnmountVolume.NewUnmounter failed for volume "dump" (UniqueName: "kubernetes.io/csi/31c052e2-63f5-4127-95cb-0923db30dd5a-dump") pod "31c052e2-63f5-4127-95cb-0923db30dd5a" (UID: "31c052e2-63f5-4127-95cb-0923db30dd5a") : kubernetes.io/csi: unmounter failed to load volume data file [/var/lib/kubelet/pods/31c052e2-63f5-4127-95cb-0923db30dd5a/volumes/kubernetes.iocsi/dump/mount]: kubernetes.io/csi: failed to open volume data file [/var/lib/kubelet/pods/31c052e2-63f5-4127-95cb-0923db30dd5a/volumes/kubernetes.iocsi/dump/vol_data.json]: open /var/lib/kubelet/pods/31c052e2-63f5-4127-95cb-0923db30dd5a/volumes/kubernetes.iocsi/dump/vol_data.json: no such file or directory" err="UnmountVolume.NewUnmounter failed for volume "dump" (UniqueName: "kubernetes.io/csi/31c052e2-63f5-4127-95cb-0923db30dd5a-dump") pod "31c052e2-63f5-4127-95cb-0923db30dd5a" (UID: "31c052e2-63f5-4127-95cb-0923db30dd5a") : kubernetes.io/csi: unmounter failed to load volume data file [/var/lib/kubelet/pods/31c052e2-63f5-4127-95cb-0923db30dd5a/volumes/kubernetes.iocsi/dump/mount]: kubernetes.io/csi: failed to open volume data file [/var/lib/kubelet/pods/31c052e2-63f5-4127-95cb-0923db30dd5a/volumes/kubernetes.iocsi/dump/vol_data.json]: open /var/lib/kubelet/pods/31c052e2-63f5-4127-95cb-0923db30dd5a/volumes/kubernetes.iocsi/dump/vol_data.json: no such file or directory"

What did you expect to happen?

Volumes that fail to be mounted can also be unmounted successfully.

How can we reproduce it (as minimally and precisely as possible)?

Simulate a scenario in which a volume fails to be attached to the CSI and the vol_data.json file is deleted. After the volume fails to be mounted, the volume is unmounted.

Anything else we need to know?

No response

Kubernetes version

$ kubectl version
# paste output here

1.31

Cloud provider

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here

Install tools

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.Categorizes issue or PR as related to a bug.needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.Indicates an issue or PR lacks a `triage/foo` label and requires one.sig/storageCategorizes an issue or PR as relevant to SIG Storage.Categorizes an issue or PR as relevant to SIG Storage.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Morty Proxy This is a proxified and sanitized view of the page, visit original site.