Linux 6.0 adds sysfs-driver-qat entries to read device capabilities:
42e66b1cc3/Documentation/ABI/testing/sysfs-driver-qat
Implement the logic for reading from sysfs and prefer that over debugfs.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Currently, each individual plugin README documents roughly the same
daily development steps to git clone, build, and deploy. Re-purpose
the plugin READMEs more towards cluster admin type of documentation
and start moving all development related documentation to DEVEL.md.
The same is true for e2e testing documentation which is scattered
in places where they don't belong to. Having all day-to-day
development Howtos is good to have in a centralized place.
Finally, the cleanup includes some harmonization to plugins'
table of contents which now follows the pattern:
* [Introduction](#introduction)
(* [Modes and Configuration Options](#modes-and-configuration-options))
* [Installation](#installation)
(* [Prerequisites](#prerequisites))
* [Pre-built Images](#pre-built-images)
* [Verify Plugin Registration](#verify-plugin-registration)
* [Testing and Demos](#testing-and-demos)
* ...
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
QAT_401xx is a derivative of 4xxx. Add support for that device
by including the device IDs (both PF and VF).
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Containers running on QAT Gen4 should be based on qatlib and therefore
kerneldrv is not the right mode. Skip registering 4xxx* devices to
ensure it is not used.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
If the kernel has CONFIG_VFIO_NOIOMMU enabled and the node admin
has explicitly set enable_unsafe_noiommu_mode VFIO parameter,
VFIO taints the kernel and writes "vfio-noiommu" to the IOMMU
group name. If these conditions are true, the /dev/vfio/ devices
are prefixed with "noiommu-".
This use-case is documented for DPDK so we don't want to break
it (as it was before because we added DeviceMounts to
/dev/vfio/<iommugroup> files that did not exist).
See DPDK documentation for further information and warnings.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
The new_id based driver binding is failing on kernels 5.11+ when the
QAT VF is not bound to any driver: attempts to write to new_id with
the same device ID repeatedly error with "file exists".
Move the new_id initialization to the beginning of the startup and
write the enabled device IDs only once.
This commit also fixes an issue where VF devices where not correctly detected
in virtual machines where the VF was not bound any driver.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
After a closer review, it was noticed that some of the QAT dpdkdrv
unit tests need updating:
- "Broken igb_uio DPDKdriver..." is actually testing unknown device ID
and we already have tests for it -> drop.
- "igb_uio DPDKdriver with one kernel bound device (not QAT device)" is
testing something impossible: an unknown VF devID is originated from a
QAT PF -> drop.
- creating files for unbind/new_id etc. is unnecessary because
os.WriteFile() creates them during the tests -> drop these lines to
simplify unit tests maintenance.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
To make QAT plugin deployment consistent with the other plugins
we update the default flags and deploy without the flag settings
provided by the ConfigMap.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
- used the same go version as for the project build
- used verbose output
- fixed gofmt check failures
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
The devices enabled by default are different between the
kustomize and operator based deployments.
This change harmonizes the defaults to c6xxvf and 4xxxvf
in both deployment options.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Add govet-fieldalignment to .golangci.yml
Fix errors that come from adding govet-fieldalignment
- by reordering the fields of structs
- by putting nolint:govet annotations
Signed-off-by: Hyeongju Johannes Lee <hyeongju.lee@intel.com>
<device>/driver symlink does not exist if the device is not bound
to any driver. bindDevice() failed when writing to <device>/driver/unbind
errored but IsNotExist() error is acceptable in case there's no driver
to unbind.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Go 1.16 release notes announced the deprecation of io/ioutil [1]. It's easy
for us to move to use what is was recommended so just do it.
[1] https://golang.org/doc/go1.16#ioutil
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
All but one (VPU) of the published container images can be built with
static binaries which allows us to use distroless/static as the
base image. Moreover, when combined with stripping the plugin binaries,
we can get both build time and image size savings.
This is the part 1 (out of 2) of the rework. Part 2 will finish the
change by making some adjustments to VPU plugin image and moving the
FPGA/SGX/GPU initcontainers to distroless/static too.
Partial: #516
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Replaced multiple instances of master with main.
Reworded line 15 "Verify QAT device plugin is registered" removed 'on master'
and corresponding section heading. Related to pr499.
Signed-off-by: DougTW <doug.martin@intel.com>
Removed device plugin socket check from the documentation as
device plugin support is enabled by default in Kubelet.
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
It looks that for a long time now we have accepted a setup where a valid QAT
device ID is accepted as a QAT device resource even though the device is
not "enabled" via kernelVfDrivers parameter.
Fix device ID validation to skip valid QAT devices that are not
explicitly specified in kernelVfDrivers.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
The updated dp.scan() changes the way how VF devices are detected. The
main reason for the change is to take into account cases where the QAT VF
driver is not present in the system at all but only the PF driver is
loaded (and the SR-IOV devices are are enabled).
The rework also takes into account bare metal and VM deployments and
adds a test case for checking the virtualized environment.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
The plugin now detects/accepts 4xxx and c4xxx devices too
and defaults to those drivers that are part of Linux mainline.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
We have both "path" and "path/filepath" but the latter provides
everything needed so move it completely.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
The code was stripping out "0000:" (bus) and then adding
it back in several places.
That's not necessary so this change simplifies QAT VF addr
handling by operating using full BDF IDs.
Moveover, simplify function calls: use getDpdkDevice() once
for each VF device.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
DPDK uses /sys/class/uio/uioX/device/[control|resource*] and we
had special mounts for the individual uioX paths. However, it turned
out this wasn't working as expected: host's /sys/class/uio/uioX/device/
was mounted to container's /sys/class/uio and DPDK failed to find
uioX/device/[control|resource*] files. Moreover, workloads requesting
more than one QAT resource, still saw only one path.
While cri-o/containerd give sysfs read-only mounts, DPDK needs
device/config RW. Therefore, we need to mount host /sys/class/uio/uioX
to container /sys/class/uio/uioX for each requested device.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
