Some Ubuntu systems may run with Apparmor LSM policy enformements making
the default QAT daemonset to fail with (un)bind errors.
This commit adds a sample kustomize overlay to deploy the QAT daemonset with
Apparmor uconfined policy.
Fixes: #381
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
fpga: make AFU resource name 63 char long
webhook: drop mode from README
webhook: extend mappings description
webhook: tighten CRD definitions
webhook: drop mapping to non-existing afuId
explicitly state mappings names can be in any format
use consistent terminology across fpga webhook and plugin
Webhook uses region CRDs even if run in preprogrammed mode.
Adding them to the base configuration should fix this deployment error:
Failed to list *v1.FpgaRegion: the server could not find the requested resource
Fixes: #361
The same fix as previous:
The `-v 1` arg is treated as single word thus klog throws
"flag provided but not defined: -v 1" error.
This time it's in the webhook kustomize base.
Move all the fpga components to using klog for logging
and debug. This includes replacing our homebrew 'fatal()'
with klog.Error().
Modify the deployment files to move from `-debug` to
`-v`, and set their default level to '1' (Info), rather
than full debug mode ('4').
Signed-off-by: Graham Whaley <graham.whaley@intel.com>
Move the framework, and the qat driver, to use `klog`
for logging and debug.
This has a some noticeable effects:
1) Our default log output gains a bunch of annotation:
From:
QAT device plugin started in 'dpdk' mode
To:
I0312 11:51:02.057728 6053 qat_plugin.go:64] QAT device plugin started in 'dpdk' mode
(there is now a command line option to drop those annotations if
necessary).
2) We gain a bunch of command line parameters from klog for controlling log
levels and output. We go from 5 arguments to 17:
---
Usage of ./cmd/qat_plugin/qat_plugin:
-add_dir_header
If true, adds the file directory to the header
-alsologtostderr
log to standard error as well as files
-debug
enable debug output
-dpdk-driver string
DPDK Device driver for configuring the QAT device (default "vfio-pci")
-kernel-vf-drivers string
Comma separated VF Device Driver of the QuickAssist Devices in the system. Devices supported: DH895xCC,C62x,C3xxx and D15xx (default "dh895xccvf,c6xxvf,c3xxxvf,d15xxvf")
-log_backtrace_at value
when logging hits line file:N, emit a stack trace
-log_dir string
If non-empty, write log files in this directory
-log_file string
If non-empty, use this log file
-log_file_max_size uint
Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
-logtostderr
log to standard error instead of files (default true)
-max-num-devices int
maximum number of QAT devices to be provided to the QuickAssist device plugin (default 32)
-mode string
plugin mode which can be either dpdk (default) or kernel (default "dpdk")
-skip_headers
If true, avoid header prefixes in the log messages
-skip_log_headers
If true, avoid headers when opening log files
-stderrthreshold value
logs at or above this threshold go to stderr (default 2)
-v value
number for the log level verbosity
-vmodule value
comma-separated list of pattern=N settings for file-filtered logging
---
3) Our `-debug` flag is now replaced by the `klog` `-v n` flag.
*NOTE:* This is potentially a minor breaking change. Applying
this debug overlay to any previous (pre-klog edit) images will
cause the container to fail to launch, as it will not recognise
the new `-v` arguments.
We also update the kustomize deployment to move from using
DEBUG env vars to adding a VERBOSITY var that controls both
the log verbosity and now the debug mode enabling.
Signed-off-by: Graham Whaley <graham.whaley@intel.com>
Previously, /dev/ion device was just arbitrary string and the plugin
did not need the device for anything. After adding the checks for
topology hints, the device node must be bind mounted in the plugin
container.
Signed-off-by: Alek Du <alek.du@intel.com>
Previously, /dev/vfio/xx devices were just arbitrary strings and the
plugin did not need the devices for anything. After adding the checks
for topology hints, we need to read the devices attached to those so
the device nodes must be bind mounted in the plugin container.
Moreover, be more verbose about any errors coming from the topology code.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
We had securityContext specified twice and the latter was overwriting
readOnlyRootFilesystem=true.
With this commit, the container is properly mounted readonly. However,
we need a tmpfs for DPDK runtime data so an emptyDir volume is added
(NB: see kubernetes/issues/48912 for discussion on emptyDir mount options).
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
runtime uses /dev/termination-log to write container termination
messages. If this file doesn't exist on the host the runtime tries
to create it. As /dev is read-only for intel-fpga-plugin container
attempt to create /dev/termination-log fails with this error:
Warning Failed kubelet, device-plugins-kubernetes-clearlinux-14-4.novalocal Error:
container create failed: container_linux.go:345: starting container process caused
"process_linux.go:430: container init caused \"rootfs_linux.go:58:
mounting \\\"/var/lib/kubelet/pods/d7262db5-e3fc-4b7b-bc2e-da245f600c4b/containers/intel-fpga-plugin/cddd0f76\\\"
to rootfs \\\"/var/lib/containers/storage/overlay/edd75bb94b1b4cf93ae1ea5c064945169fb329d0abdb56b7621cddfc721f6eda/merged\\\"
at \\\"/var/lib/containers/storage/overlay/edd75bb94b1b4cf93ae1ea5c064945169fb329d0abdb56b7621cddfc721f6eda/merged/dev/termination-log\\\"
caused \\\"open /var/lib/containers/storage/overlay/edd75bb94b1b4cf93ae1ea5c064945169fb329d0abdb56b7621cddfc721f6eda/merged/dev/termination-log: read-only file system\\\"\""
Setting terminationMessagePath to rw-mounted file system
/tmp/termination-log for the plugin container should fix this.
Fixes: #259
Since Kubernetes v1.16 release, DaemonSet, Deployment, StatefulSet, and ReplicaSet in the extensions/v1beta1 and apps/v1beta2 API groups have been deprecated. This PR migrates the webhook deployment to use apps/v1 instead of extensions/v1beta1 and add the selector part also required by the migration.
Signed-off-by: Hector Augusto Garcia Baleon <hector.augusto.garcia.baleon@intel.com>
The default deployment gives rather wide host mounts.
Limited sysfs mount only to the subdirectory the plugin
needs.
Mounted sysfs and dev mounts read-only.
Added notes that FPGA plugin can be run as non-root user.
The default deployment gives rather wide host mounts. We can limit
the mounts only to the subdirectories the plugin needs and mount
them read-only.
Also, add notes that both QAT and GPU plugins can be run as non-root
user.
Fixes: #228
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
In preparations to get some of the images to hub.docker.com/intel,
start using intel/ prefix.
Moreover, set the Makefile variables so that the images built
by make [images|demos] can easily be pushed to any registry/org
by 'docker push' (e.g., by Jenkins).
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
- used ENTRYPOINT instead of CMD in plugin and admission webhook
Dockerfiles to avoid duplicating commands in the pod yamls
- fixed path to deploy.sh script in fpga initcontainer Dockerfile
- Ordered collection in DCP release/region/afus order for simpler
maintenance.
- Got rid of ambiguous entries without dcp releases, e.g. Arria10,
Arria10-nlb3 etc.
- For AOCX files, afuId should be set to unique UUID
(can be seen via fpga_tool)
- arria10 now points to DCP 1.2 release
- added mappings for Stratix10 based D5005 PAC card
Clear Linux enables DPDK QAT PMD so we can move to use everything from
there. This saves maintenance efforts and we get more up-to-date DPDK.
The DPDK version in this update gives a tool for compress perf too, for
instance.
The commit also adds kustomize scripts that overlay the original DPDK
demo deployment to run dpdk-test-[compress|crypto|-perf test cases:
$ kubectl apply -k deployments/qat_dpdk_app/test-compress1/
$ kubectl apply -f deployments/qat_dpdk_app/test-crypto1/
New test cases ('ptest's with varying parameters) can be easily added
by following the pattern in test-[crypto|compress]1 directories.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
We plan to use crypto-perf for simple QAT testing. This commit adds
kustomization to make the deployment easier. The original .yaml is
also moved to deployments/ with some changes.
For instance, it turns out also vfio-pci mode with DPDK needs CAP_SYS_ADMIN
(See PR: #187 which states that only igb_uio would need it).
kustomize is available part of kubectl since kubernetes v1.14.
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
