"unconfined" annotation was needed to get writes to new_id / bind
to succeed on AppArmor enabled OSes.
However, many things have changed:
* new_id should not be used anymore and it was dropped in the plugin.
* QAT initcontainer has assumed the role of HW initialization.
* vfio-pci is the preferred "dpdkDriver" and starting with QAT Gen4, it
is the only available VF driver so unbind isn't necessary.
* k8s AppArmor is "GA" since 1.30 and the annotation is deprecated.
As of now, the initcontainer will take care of binding QAT VFs to vfio-pci
so the plugin does not neeed to set AppArmor at all.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
The documentation needs clarifications to how QAT Gen4 SW differs from
older platfoms:
- only upstream driver is available and due to this, the -mode parameter
is now deprecated
- the QAT VF services are configurable and thus the resource names
differ
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
default qat.conf file does not exist any more. Update it to make
user create own conf file or use --from-literal instead.
Signed-off-by: Hyeongju Johannes Lee <hyeongju.lee@intel.com>
In case there are more than 32 devices and there is no config set,
all scanned devices have the same service. So, we increase the
default value of maxNumDevices to 64 from 32.
Signed-off-by: Hyeongju Johannes Lee <hyeongju.lee@intel.com>
Currently, each individual plugin README documents roughly the same
daily development steps to git clone, build, and deploy. Re-purpose
the plugin READMEs more towards cluster admin type of documentation
and start moving all development related documentation to DEVEL.md.
The same is true for e2e testing documentation which is scattered
in places where they don't belong to. Having all day-to-day
development Howtos is good to have in a centralized place.
Finally, the cleanup includes some harmonization to plugins'
table of contents which now follows the pattern:
* [Introduction](#introduction)
(* [Modes and Configuration Options](#modes-and-configuration-options))
* [Installation](#installation)
(* [Prerequisites](#prerequisites))
* [Pre-built Images](#pre-built-images)
* [Verify Plugin Registration](#verify-plugin-registration)
* [Testing and Demos](#testing-and-demos)
* ...
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
QAT_401xx is a derivative of 4xxx. Add support for that device
by including the device IDs (both PF and VF).
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
To make QAT plugin deployment consistent with the other plugins
we update the default flags and deploy without the flag settings
provided by the ConfigMap.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
The devices enabled by default are different between the
kustomize and operator based deployments.
This change harmonizes the defaults to c6xxvf and 4xxxvf
in both deployment options.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
All but one (VPU) of the published container images can be built with
static binaries which allows us to use distroless/static as the
base image. Moreover, when combined with stripping the plugin binaries,
we can get both build time and image size savings.
This is the part 1 (out of 2) of the rework. Part 2 will finish the
change by making some adjustments to VPU plugin image and moving the
FPGA/SGX/GPU initcontainers to distroless/static too.
Partial: #516
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Replaced multiple instances of master with main.
Reworded line 15 "Verify QAT device plugin is registered" removed 'on master'
and corresponding section heading. Related to pr499.
Signed-off-by: DougTW <doug.martin@intel.com>
Removed device plugin socket check from the documentation as
device plugin support is enabled by default in Kubelet.
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
The plugin now detects/accepts 4xxx and c4xxx devices too
and defaults to those drivers that are part of Linux mainline.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Move the framework, and the qat driver, to use `klog`
for logging and debug.
This has a some noticeable effects:
1) Our default log output gains a bunch of annotation:
From:
QAT device plugin started in 'dpdk' mode
To:
I0312 11:51:02.057728 6053 qat_plugin.go:64] QAT device plugin started in 'dpdk' mode
(there is now a command line option to drop those annotations if
necessary).
2) We gain a bunch of command line parameters from klog for controlling log
levels and output. We go from 5 arguments to 17:
---
Usage of ./cmd/qat_plugin/qat_plugin:
-add_dir_header
If true, adds the file directory to the header
-alsologtostderr
log to standard error as well as files
-debug
enable debug output
-dpdk-driver string
DPDK Device driver for configuring the QAT device (default "vfio-pci")
-kernel-vf-drivers string
Comma separated VF Device Driver of the QuickAssist Devices in the system. Devices supported: DH895xCC,C62x,C3xxx and D15xx (default "dh895xccvf,c6xxvf,c3xxxvf,d15xxvf")
-log_backtrace_at value
when logging hits line file:N, emit a stack trace
-log_dir string
If non-empty, write log files in this directory
-log_file string
If non-empty, use this log file
-log_file_max_size uint
Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
-logtostderr
log to standard error instead of files (default true)
-max-num-devices int
maximum number of QAT devices to be provided to the QuickAssist device plugin (default 32)
-mode string
plugin mode which can be either dpdk (default) or kernel (default "dpdk")
-skip_headers
If true, avoid header prefixes in the log messages
-skip_log_headers
If true, avoid headers when opening log files
-stderrthreshold value
logs at or above this threshold go to stderr (default 2)
-v value
number for the log level verbosity
-vmodule value
comma-separated list of pattern=N settings for file-filtered logging
---
3) Our `-debug` flag is now replaced by the `klog` `-v n` flag.
*NOTE:* This is potentially a minor breaking change. Applying
this debug overlay to any previous (pre-klog edit) images will
cause the container to fail to launch, as it will not recognise
the new `-v` arguments.
We also update the kustomize deployment to move from using
DEBUG env vars to adding a VERBOSITY var that controls both
the log verbosity and now the debug mode enabling.
Signed-off-by: Graham Whaley <graham.whaley@intel.com>
go get'ing does not work due to our k8s.io/kubernetes dependency
so guide users to use git clone to get the code.
Fixes: #290
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
crypto-perf instructions were outdated and hand implicit
assumptions about the environment. More specifically:
Clear Linux builds DPDK libraries as shared so for the
compress and crypto test applications to run, the memory and
QAT PMD libraries must be explicitly preloaded using '-d' parameter.
Also, the test-crypto1 and test-compress1 deployments expect the
cluster is configured with CPU Manager's static policy.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Update the QAT README. Add some descriptions. Add information about
the command line and config options.
Signed-off-by: Graham Whaley <graham.whaley@intel.com>
The default deployment gives rather wide host mounts. We can limit
the mounts only to the subdirectories the plugin needs and mount
them read-only.
Also, add notes that both QAT and GPU plugins can be run as non-root
user.
Fixes: #228
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
We plan to use crypto-perf for simple QAT testing. This commit adds
kustomization to make the deployment easier. The original .yaml is
also moved to deployments/ with some changes.
For instance, it turns out also vfio-pci mode with DPDK needs CAP_SYS_ADMIN
(See PR: #187 which states that only igb_uio would need it).
kustomize is available part of kubectl since kubernetes v1.14.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
-mode kerneldrv comes with no documentation. This patch adds few
notes about it and instructions how to get it build if a user chooses
to have it enabled.
Closes: #197
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Not all QAT chips (e.g, 37c9) are available in pci.ids which makes
"grep QAT" to not show them.
Scan all known VF PCI ids in a loop to ensure all configured devices
are shown.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
