Commit Graph

251 Commits

Author SHA1 Message Date
Mikko Ylinen
f4c33d198e patcher: move ENV validation away from getRequestedResources
This commit adds new function validateContainer() that runs the
same FPGA_* ENV validation checks as before in getRequestedResources().

The restructuring is done in preparations for moving
getRequestedResources() to a separate package.

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
2020-09-10 15:31:26 +03:00
Dmitry Rozhkov
9bdf3a4def
Merge pull request #440 from mythi/ctrl-runtime-062
go.mod: update controller-runtime to v0.6.2
2020-09-03 12:02:06 +03:00
Alexander Kanevskiy
c74cb563dc Implemented SR-IOV Release/Assign ioctl
fpgatool now able to prepare FME via kernel ioctl to release and
assign ports for SR-IOV configurations.
2020-09-02 18:16:53 +03:00
Mikko Ylinen
76aa7b91f0 go.mod: update controller-runtime to v0.6.2
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
2020-09-02 15:16:12 +03:00
Ed Bartosh
ab1612dd7d
Merge pull request #437 from rojkov/linter-checks
lint: enable exportloopref, prealloc and scopelint checks
2020-09-02 11:40:24 +03:00
Mikko Ylinen
d8cd5814d7 operator: regenerate CRDs and small webhook/controller updates
this commits also changes validatePluginImage() to allow
image version as a parameter so that it can be used by by
other webooks too.

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
2020-08-31 11:29:04 +03:00
Dmitry Rozhkov
71075d4478 lint: enable exportloopref, prealloc and scopelint checks 2020-08-31 11:10:51 +03:00
Dmitry Rozhkov
be713f1c8b lint: enable errcheck 2020-08-28 16:14:14 +03:00
Dmitry Rozhkov
7ff08ee874 linter: enable staticcheck 2020-08-25 09:54:59 +03:00
Mikko Ylinen
cd068c797a ci: update tool versions
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
2020-08-21 17:04:04 +03:00
Dmitry Rozhkov
200e2f8181 operator: add simple FPGA operator combined with FPGA webhook 2020-08-18 17:32:23 +03:00
Dmitry Rozhkov
a62c6f7d5e fpga webhook: reimplement to use kubebuilder framework
Simplify upgrade procedure to newer versions of kubernetes by relying on the
kubebuilder framework rather than using codegen directly.

Closes #377
2020-08-17 12:09:03 +03:00
Dmitry Rozhkov
d27b12a925 operator: fix crash when assigning to nil map 2020-06-26 12:35:25 +03:00
Dmitry Rozhkov
6b2fa0a264 operator: initial version with gpu and qat controllers 2020-06-25 13:48:41 +03:00
Dmitry Rozhkov
6f59b868d5 convert syscall.Errno to conventional error 2020-06-18 16:52:18 +03:00
Dmitry Rozhkov
73aea0aa1b linter: enable gosec check 2020-06-11 17:56:24 +03:00
Dmitry Rozhkov
70f862f2aa add golangci linter
In this initial commit the following checks are disabled due to
excessive amount of changes required:
- dupl (duplicate code)
- funlen (function length)
- goerr113 (errors handling expressions)
- gomnd (magic numbers)
- gosec (security)
- nakedret (naked returns)
- wsl (forces to use empty lines)
- errcheck (checking for unchecked errors)
- staticcheck (static analysis)
2020-06-08 14:01:13 +03:00
Dmitry Rozhkov
c63dbf61b8 fpgawebhook: move to v2 API of fpga.intel.com group 2020-05-04 15:43:20 +03:00
Dmitry Rozhkov
99fcb69d33 fpga: compress fpga AF resource names 2020-04-29 11:59:50 +03:00
Dmitry Rozhkov
6c2eacfae5 webhook: remove mode of operation
fpga: make AFU resource name 63 char long

webhook: drop mode from README

webhook: extend mappings description

webhook: tighten CRD definitions

webhook: drop mapping to non-existing afuId

explicitly state mappings names can be in any format

use consistent terminology across fpga webhook and plugin
2020-04-22 13:55:43 +03:00
Mikko Ylinen
96d9e642e1
Merge pull request #365 from rojkov/release-0.18
move to Kubernetes v1.18.2 API
2020-04-17 14:11:54 +03:00
Dmitry Rozhkov
8fc187f4d8 move to k8s v1.18.2 release
Also fix the plugins and e2e tests
2020-04-17 12:40:18 +03:00
Ukri Niemimuukko
39c934b22f topology: avoid unnecessary warning prints
Doing strings.Split with a non-empty separator against a potentially
empty string (hint.NUMAs) may return an empty string in a slice of
size one, which then doesn't parse nicely with ParseInt and results
in a repeating warning.

It is better to check for hint.NUMAs emptiness before trying to
split it.

Signed-off-by: Ukri Niemimuukko <ukri.niemimuukko@intel.com>
2020-04-16 19:11:39 +03:00
Dmitry Rozhkov
6f3302f21d dpapi: update error message 2020-03-30 15:06:06 +03:00
Dmitry Rozhkov
1f807efbc6 deviceplugins: increase unit test coverage 2020-03-30 14:59:05 +03:00
Dmitry Rozhkov
3ff989e4b0 Add interface ContainerPreStarter optionally implemented by device plugins 2020-03-30 14:59:05 +03:00
Mikko Ylinen
d3f6401335
Merge pull request #339 from bart0sh/PR0076-test-coverage-bitstream
fpga bitstream module: increase test coverage
2020-03-25 14:32:46 +02:00
Graham Whaley
e96688bb18 pkg/debug: drop the local debug pkg
Now we have moved all components to use klog for debug
output, we can drop the local debug pkg.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2020-03-24 16:27:24 +00:00
Ed Bartosh
f31946bb98 bitstream: increased test coverage 2020-03-20 11:18:05 +02:00
Ed Bartosh
f5754f490e bitstream: remove unused code 2020-03-19 18:33:58 +02:00
Graham Whaley
f8dbc896a1 devicemanager: qat: use klog for logging and debug
Move the framework, and the qat driver, to use `klog`
for logging and debug.

This has a some noticeable effects:

1) Our default log output gains a bunch of annotation:
From:
    QAT device plugin started in 'dpdk' mode
To:
    I0312 11:51:02.057728    6053 qat_plugin.go:64] QAT device plugin started in 'dpdk' mode

(there is now a command line option to drop those annotations if
necessary).

2) We gain a bunch of command line parameters from klog for controlling log
levels and output. We go from 5 arguments to 17:

---
Usage of ./cmd/qat_plugin/qat_plugin:
  -add_dir_header
        If true, adds the file directory to the header
  -alsologtostderr
        log to standard error as well as files
  -debug
        enable debug output
  -dpdk-driver string
        DPDK Device driver for configuring the QAT device (default "vfio-pci")
  -kernel-vf-drivers string
        Comma separated VF Device Driver of the QuickAssist Devices in the system. Devices supported: DH895xCC,C62x,C3xxx and D15xx (default "dh895xccvf,c6xxvf,c3xxxvf,d15xxvf")
  -log_backtrace_at value
        when logging hits line file:N, emit a stack trace
  -log_dir string
        If non-empty, write log files in this directory
  -log_file string
        If non-empty, use this log file
  -log_file_max_size uint
        Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
  -logtostderr
        log to standard error instead of files (default true)
  -max-num-devices int
        maximum number of QAT devices to be provided to the QuickAssist device plugin (default 32)
  -mode string
        plugin mode which can be either dpdk (default) or kernel (default "dpdk")
  -skip_headers
        If true, avoid header prefixes in the log messages
  -skip_log_headers
        If true, avoid headers when opening log files
  -stderrthreshold value
        logs at or above this threshold go to stderr (default 2)
  -v value
        number for the log level verbosity
  -vmodule value
        comma-separated list of pattern=N settings for file-filtered logging
---

3) Our `-debug` flag is now replaced by the `klog` `-v n` flag.

*NOTE:* This is potentially a minor breaking change. Applying
this debug overlay to any previous (pre-klog edit) images will
cause the container to fail to launch, as it will not recognise
the new `-v` arguments.

We also update the kustomize deployment to move from using
DEBUG env vars to adding a VERBOSITY var that controls both
the log verbosity and now the debug mode enabling.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2020-03-19 11:20:48 +00:00
Mikko Ylinen
1d41852013 qat: mount VFIO devices for topology hints to work
Previously, /dev/vfio/xx devices were just arbitrary strings and the
plugin did not need the devices for anything. After adding the checks
for topology hints, we need to read the devices attached to those so
the device nodes must be bind mounted in the plugin container.

Moreover, be more verbose about any errors coming from the topology code.

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
2020-03-02 08:13:09 +02:00
Ed Bartosh
1bbcbc65da fix cyclomatic check failure
Moved part of the code from NewTopologyHints to getTopologyHint to
decrease cyclomatic complexity.

This should fix this failure:
  $ make checks
  Complexity is over 15 in
  20 topology NewTopologyHints pkg/topology/topology.go:83:1
  Makefile:23: recipe for target 'cyclomatic-check' failed
  make: *** [cyclomatic-check] Error 1
2020-02-11 14:20:44 +02:00
Ed Bartosh
366d07a35e topology: Fix lint warnings
Fixed the following golint warnings:
 $ make checks
  ./pkg/topology/topology.go:44:6: type name will be used as topology.TopologyHint by other packages, and that stutters; consider calling this Hint
  ./pkg/topology/topology.go:52:6: type name will be used as topology.TopologyHints by other packages, and that stutters; consider calling this Hints
  Found 2 lint suggestions; failing.
  Makefile:47: recipe for target 'lint' failed
2020-02-11 14:20:44 +02:00
Ed Bartosh
3f83e2b2eb Copy toplogogy module from cri-resource-manager
Copied topology module to avoid external dependency.
Added GetTopologyInfo function and its test case.
2020-02-11 12:06:25 +02:00
Ed Bartosh
b03812c281 Send topology info to kubelet
Get topology information for the discovered devices
and send it to kubelet.

Fixes: #250
2020-02-10 13:45:34 +02:00
Ed Bartosh
1f4928790f Implement function for DeviceInfo creation
- Made DeviceInfo fields private
- Implement NewDeviceInfo constructor
2020-02-07 15:26:37 +02:00
Dmitry Rozhkov
456c8f3ff1 fpga: fix stutter reported by golint 2020-01-30 15:17:27 +02:00
Dmitry Rozhkov
3a845cfe15 fpga: rename files to make them linux-only 2020-01-29 17:17:06 +02:00
Dmitry Rozhkov
814e2e1a50 bump k8s dependencies up to v1.17.0 2020-01-09 11:19:58 +02:00
Alexander Kanevskiy
2430e204d5 fpga_tool: UX improvements
- user readable output for fpgainfo/fmeinfo/portinfo commands
- new commands: list, list-fme, list-port
- new -q flag to suppres headers, progress and too verbose messages
- install command will now fail if destination file already exist
- new --force flag: allows overwrite files in install command
- removed development and debug output
2019-08-25 02:37:07 +03:00
Alexander Kanevskiy
71bb38f496 Implemented native FPGA flashing
Removed dependency to OPAE libraries
2019-08-25 02:37:01 +03:00
Ed Bartosh
de9df8373e fpga_plugin: support in-tree kernel driver
Extended fpga plugin to support both in-tree(DFL) and
out-of-tree (OPAE) kernel drivers.

- fpga_crihook: move JSON parsing to separate functions
- decreased cyclomatic complexity of the CRI hook main() function
- increased readability
- increased test coverage

Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
2019-08-24 18:27:15 +03:00
Alexander Kanevskiy
186ec6613c FPGA: migrate to ClearLinux environment
- Migrate to OPAE 1.3.2
- Build all the tools from the source
- ignore files in workspace
- minimal fpga_tool utility to check gbs/aocx file parsing and flashing
- implemented kernel IOCTL based flashing of bitstreams
- add PCI and sysfs functions
2019-08-24 02:55:19 +03:00
Paul Hodson
5279dace40 server.Stop() shuts down the socket rather than restarting it. 2019-06-21 11:32:06 -04:00
Dmitry Rozhkov
fef6763927 deviceplugin: fix duplicate device nodes in ContainerResponse
The variable which gets assigned a value from a list element on every
loop iteration is always the same. Getting a pointer to this variable
results in the same pointer on every iteration.

Rather get a pointer to the actual element of the list.
2019-04-04 15:47:01 +03:00
Dmitry Rozhkov
7ff40a9b52 Update codegenerated client code for fpga_admissionwebhook 2019-02-18 17:48:56 +02:00
Dmitry Rozhkov
92f6a547cb Add doc on how to develop new device plugins 2019-02-06 09:56:51 +02:00
Dmitry Rozhkov
54332c5eea announce deviceplugin API public 2019-01-21 17:20:01 +02:00
Dmitry Rozhkov
eccd70c600 replace glog with simpler home-grown debug logging 2018-08-16 17:40:16 +03:00
Dmitry Rozhkov
271bc0d29a webhook: add dynamically configured mappings
Currently we have hardcoded mapping from human readable names of
AFs and FPGA regions like arria10-nlb0 to the resource names
produced by the FPGA device plugin. This is not sustainable
long term solution.

Implement CRD based mappings so that a new mapping can be added or
removed dynamically by cluster admins with CRD resources.
2018-08-08 17:58:18 +03:00