Purpose of this PR is to remove Linux's disk_encryption reliance on block_devices. We want a list of all block devices by name and a way to access their parent device, so I walk sysfs for that.

Would this implementation also allow block_devices to work with query context? It's probably worth moving to a helper function and letting both table's generate methods call it.

Purpose of this PR is to remove Linux's disk_encryption reliance on block_devices. We want a list of all block devices by name and a way to access their parent device, so I walk sysfs for that.

Would this implementation also allow block_devices to work with query context? It's probably worth moving to a helper function and letting both table's generate methods call it.

I could definitely look at making that possible.

@Smjert I think you said you could review this PR this week?

@Smjert I think you said you could review this PR this week?

Yes I began looking into it today but other things got my priority; I'll be able to finish my review by eod tomorrow.
I mostly have minor corrections for now.

While I don't think there are major issues, I'm a bit concerned on the support for the various combinations of distros and disk setups.

Would it be possible to know on which distros and which setups has this been tested on?

Very sorry I have taken a long time to get back to this. I've tested these changes on Ubuntu 22.04, CentOS 7, and openSUSE 15.4. For disk setups I've been mostly testing with the below on my openSUSE device:

NAME                        MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINTS
sda                           8:0    0   10G  0 disk
├─sda1                        8:1    0    2M  0 part
├─sda2                        8:2    0   20M  0 part  /boot/efi
└─sda3                        8:3    0   10G  0 part  /
sdb                           8:16   0   10G  0 disk
├─sdb1                        8:17   0    1M  0 part
├─sdb2                        8:18   0    1G  0 part
└─sdb3                        8:19   0    9G  0 part
  └─cryptlvm                254:4    0    9G  0 crypt
    ├─MyVolGroup-root       254:5    0    8G  0 lvm   /mnt/test_drives/lvm_on_encryption
    └─MyVolGroup-home       254:6    0 1016M  0 lvm   /mnt/test_drives/lvm_on_encryption/home
sdc                           8:32   0   10G  0 disk
├─sdc1                        8:33   0    1M  0 part
└─sdc2                        8:34   0   10G  0 part
  ├─LvmEncrypt-lvmcryptroot 254:0    0    5G  0 lvm
  │ └─lvmcrypt              254:7    0    5G  0 crypt /mnt/test_drives/encryption_on_lvm
  ├─LvmEncrypt-lvmcryptswap 254:1    0    1G  0 lvm
  ├─LvmEncrypt-lvmcrypttmp  254:2    0    1G  0 lvm
  └─LvmEncrypt-lvmcrypthome 254:3    0    3G  0 lvm   /mnt/test_drives/encryption_on_lvm/home
sdd                           8:48   0   10G  0 disk
├─sdd1                        8:49   0    1M  0 part
├─sdd2                        8:50   0    1G  0 part
├─sdd3                        8:51   0    5G  0 part
│ └─cryptsetuproot          254:8    0    5G  0 crypt /mnt/test_drives/full_encryption_split_root_home
└─sdd4                        8:52   0    4G  0 part
  └─cryptsetuphome          254:9    0    4G  0 crypt /mnt/test_drives/full_encryption_split_root_home/home

Hey there, Arch Linux has been building osquery with the block_devices table disabled for about 6 months now due to the current implementation's reliance on lvm2app.h which is gone from LVM2. What's broken in Arch today is broken in Ubuntu tomorrow... this implementation is a lot more future proof and it would be really great to see it merged.

I'm working on building an osquery package with this patch and deploying it across my own base of ~15 bare-metal Arch systems. and will report back once I have some useful data.

Update with my results - this PR merged cleanly to the latest master and I was able to produce a build of the Arch Linux package with this PR integrated. It's returning the expected results, although it's not reporting the serial number of my NVMe drive. I'm also observing that Ubuntu 24.04 LTS is shipping lvm2, so osquery can't be built on latest Ubuntu anymore without disabling the block_devices table.

@Smjert - would you mind taking a fresh look at this? In my experience sysfs is pretty stable. If distro compatibility is a priority, I think leaving this PR unmerged is actually doing more harm than good. If it's not already on your radar, @carlsmedstad maintains a patch set to build osquery on Arch, which can be viewed here. I haven't checked each of these patches individually to determine which ones are necessary for building on modern Ubuntu, but at a quick glance it looks like a lot of them are general library compatibility updates. Obviously you'll want to omit the patches that disable libdpkg. :) Separately from Carl's tree there's also a trivial patch to fix building on gcc 15.

The sysfs tree on my system appears as follows. The device link under the NVMe namespace device nvme0n1 links to the device directory for nvme0, and the device link under the physical device nvme0 links to the PCI device node.

/sys/block/nvme0n1 -> /sys/devices/pci0000:00/0000:00:1d.0/0000:6d:00.0/nvme/nvme0/nvme0n1
/sys/block/nvme0n1/device -> /sys/devices/pci0000:00/0000:00:1d.0/0000:6d:00.0/nvme/nvme0
/sys/block/nvme0n1/device/device -> /sys/devices/pci0000:00/0000:00:1d.0/0000:6d:00.0

The serial number can be read from /sys/block/nvme0n1/device/serial.
The PCI vendor ID can be read from /sys/block/nvme0n1/device/device/vendor; it comes back as a hexadecimal number (0x####).

@Micah-Kolide do you have bandwidth to fix serial number and vendor fields using the info above?

If not, I'm happy to rebase your branch, start a new PR and see that through.

@Micah-Kolide do you have bandwidth to fix serial number and vendor fields using the info above?

If not, I'm happy to rebase your branch, start a new PR and see that through.

Hey @fuhry, thank you for taking a look at this PR and testing it as well! I can definitely look at making a commit to fix the serial and vendor fields.