Commit Graph

48 Commits

Author SHA1 Message Date
Michael Henriksen
5195176c16
update to k8s 1.30 libs and controller-runtime 0.18.4 (#3336)
* make deps-update

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* ReourceRequirements -> VolumeResourceRequirements

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix calls to controller.Watch()

controller-runtime changed the API!

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* Fix errors with actual openshift/library-go lib

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* make all works now and everything compiles

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix "make update-codegen" because generate_groups.sh deprecated

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* run "make generate"

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix transfer unittest because of change to controller-runtime

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

---------

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2024-07-14 20:12:50 +02:00
Edu Gómez Escandell
184f2b0e17
Expose Upload Proxy certificate in CDI Config status (#3314)
* Add upload-proxy's certificate authority to CDIConfigStatus

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Update generated files

Update generated files after adding the UploadProxyCA field

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Have the config controller watch the root certificate authority

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Add UploadProxyCA to the reconcile loop

Heavily inspired by the exporter in Kubevirt:
00dfae3a7c/pkg/storage/export/export/links.go (L214-L232)

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Test UploadProxyCA reconciliation

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Reconcile CA when it comes from an Ingress

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Test UploadProxyCA Ingress reconciliation

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

---------

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
2024-06-30 13:10:51 +02:00
Edu Gómez Escandell
cd7c8b14a5
Enable revive linter (#3241)
* Enable revive linter

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Simplify cdi-func-test-proxy

This function had quite a bit of redundant code (caught by the linter).
The workgroup was never Done because all exit paths went through a
log.Fatal.

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Fix 'revive' linter warnings

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Fix tests that asserted on modified error messages

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Run make format

The formatted code has nothing to do with this PR but we may as well
include it.

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Use lower-case variables and use built-in min function in vddk-datasource

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Use contexts in cdi-func-test-proxy

This added quite a bit of boilerplate per call, so I put everything in
a loop.

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

---------

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
2024-05-21 22:35:42 +02:00
Edu Gómez Escandell
0e750262a3
Enable autoformatting linters (#3179)
* Enable gofmt linter

From the docs:

> Gofmt checks whether code was gofmt-ed. By default this tool runs with
> -s option to check for code simplification.

https://golangci-lint.run/usage/linters/#gofmt

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Run gomft on the project

Ran this command after adding the gofmt linter:

	golangci-lint run ./... --fix

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Enable whitespace linter

From the docs:
> Whitespace is a linter that checks for unnecessary newlines at the
> start and end of functions, if, for, etc.

https://golangci-lint.run/usage/linters/#whitespace

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Run whitespace on the project

Ran this command after adding the whitespace linter:

	golangci-lint run ./... --fix

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Enable GCI linter

Per the docs:

> Gci controls Go package import order and makes it always deterministic.

https://golangci-lint.run/usage/linters/#gci

NOTE: I noticed that many files separate their imports in a particular
way, so I set the linter to enforce this standard.

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Run GCI on the project

Ran this command after adding the GCI linter:

	golangci-lint run ./... --fix

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

---------

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
2024-04-24 13:52:22 +02:00
Edu Gómez Escandell
42001ae051
Enable misspell linter and fix spelling errors (#3164)
* Add misspell to list of linters

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Fix spelling errors

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

---------

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
2024-04-09 20:17:55 +02:00
Alex Kalenyuk
31d12e426e
update k8s & related libraries to 1.28 (#3078)
* Bump k8s/OpenShift/ctrl-runtime/lifecycle-sdk & make deps-update

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Operator: adapt for dependency bump

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Controller: adapt watch calls for dependency bump

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Controller: adapt to ctrl-runtime's cache API changes

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Operator: fix unit tests by deleting resources properly in fake client

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Controller: fix unit tests by deleting resources properly in fake client

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Controller: adapt to fake client honoring status subresource

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Fix codegen script & make generate

There are some issues in the new script, so we
will still use the deprecated one.
More context in f4d1a5431b

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Functests: Adapt to NamespacedName now implementing MarshalLog

ns/name -> {"name":"name","namespace":"ns"}

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Functests & API server: address deprecation of wait.PollImmediate

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

---------

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2024-01-23 17:52:05 +01:00
Michael Henriksen
cc8dbc3bae
increase controller cuncurrency and cpu requests (#2862)
This commit ups the cpu request for for all our installed compopents
(cdi-deployment, cdi-apiserver, cdi-uploadproxy, cdi-operator)
for 10m (1% of a core) to 100m (10% of a core).
The main driver of this is BZ: 2216038.
Without this change, it is pretty easy to create a large number of
concurrent clone operations and get token timeout errors.
Upping resource requests and concurrency addresses the issue
in a very direct way.

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2023-08-24 02:48:34 +02:00
Michael Henriksen
f88fab69dc
PVC Clone Populator (#2709)
* touch up zero restoresize snapshot

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* clone populator

only supports PVC source now

snapshot coming soon

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* more unit tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* unit test for clone populator

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* func tests for clone populator

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* move clone populator cleanup function to planner

other review comments

verifier pod should bount readonly

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* add readonly flag to test executor pods

synchronize get hash calls

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* increase linter timeout

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* better/explicit readonly support for test pods

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* check pv for driver info before looking up storageclass as it may not exist

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* addressed review comments

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* chooseStrategy shoud generate more events

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

---------

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2023-05-24 05:11:52 +02:00
garonsky
4e3ee4c44d
Add support for imagePullSecrets (#2589)
* Add support for imagePullSecrets in the CDI CR, to support pulling
images from repositories that require secrets.

The imagePullSecrets is propagated to the following components: cdi-apiserver,
cdi-deployment, and cdi-uploadproxy. The definition of imagePullSecrets in
cdi-operator must be done manually.

Signed-off-by: Gleb Aronsky <gleb.aronsky@windriver.com>

* Modifying code to incorporate review comments.

Signed-off-by: Gleb Aronsky <gleb.aronsky@windriver.com>

---------

Signed-off-by: Gleb Aronsky <gleb.aronsky@windriver.com>
Co-authored-by: Gleb Aronsky <gleb.aronsky@windriver.com>
2023-03-18 01:41:35 +00:00
Arnon Gilboa
bfe30a8964
Split and refactor DV controller (#2483)
- Split the huge DV controller into smaller op-specific DV controllers -
  import, clone, upload
- Add common watch-adding function so each controller watches only its
  relevant DVs
- Refactor the common Reconcile() to use interface DataVolumeReconciler
  implemented by each controller
- Move all functions, structs, consts to the relevant controller
- Split the utests per controller

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
2022-12-22 01:03:15 +00:00
Arnon Gilboa
026785265e
Fix import proxy ConfigMap update (#2474)
* Fix OpenShift cluster-wide proxy config update

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* CR fixes

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Copy import proxy ConfigMap to import namespace

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* CR fixes

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
2022-12-14 03:05:35 +00:00
akalenyu
4d4ad12df5
Only list Ingresses/Routes in CDI namespace instead of cluster level (#2371)
* Only list Ingresses/Routes in CDI namespace instead of cluster level

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Change the way we initialize cache for cdi controller

This gives us flexibility to cache only exactly what we need.
The error that led me to this was that we were attempting to Watch()
Routes/Ingresses which is basically caching all namespaces. We only want to cache the CDI namespace for those.
Source/feature from https://github.com/kubernetes-sigs/controller-runtime/issues/1708

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2022-08-01 22:12:47 +02:00
Alexander Wels
013cb6b62b
Set http(s)_proxy to lower case env variable (#2132)
* Set htpp(s)_proxy to lower case env variable

CURL used by nbdkit doesn't read upper case http(s)_proxy environment
variables, and thus was not using the proxy. Changed the variable to
be lower case.

Added a significant number of tests to test many more variations of
using a proxy. Also added https + auth endpoint to the file-host
container, so we can test https + auth with the proxy.

Added https endpoint to proxy, so we can test an https proxy.

Cleaned up some of the error handling in the import controller for
the proxy, in particular if a trustedCAProxy is defined.

Fixed some of the cluster wide proxy configuration so it works properly
inside an openshift cluster.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Add https proxy support to registry import. Added extra
functional tests to test all registry import combinations

Signed-off-by: Alexander Wels <awels@redhat.com>

* Fixed some tests to work better in Open Shift.

Signed-off-by: Alexander Wels <awels@redhat.com>
2022-02-03 18:09:41 +01:00
Michael Henriksen
b05f768b2f
used uncached client to get cluster proxy configmap (#2055)
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2021-12-09 01:09:48 +01:00
akalenyu
eb639a6ac5
Change some relationship labels on update as well (#2018)
* Update operator-lifecycle-sdk to get fix for labels on upgrade

Update dep to get https://github.com/kubevirt/controller-lifecycle-operator-sdk/pull/19

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Reconcile labels also for CDIConfig

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Reconcile labels on storageprofile

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Reconcile remaining operator resources for updated labels

BZ#2017478

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2021-11-23 16:16:49 +01:00
Arnon Gilboa
7087b57cd2
Add DataImportCron controller (#1949)
* Add DataImportCron controller

-The new controller polls for updates to a registry source container
image, based on a given schedule. When updates to a container image are
detected, the controller imports the content into a new uniquely named
PVC in a golden image namespace.
-For each DataImportCron, the controller manages a corresponding
DataSource to always point to the latest most up-to-date golden
image PVC.
-DataImportCron takes ownership of an existing DataSource (with
controller: false), allowing an admin to opt-in to using auto
delivery/updates later on.
-The controller has PVC garbage collector removing old PVCs.

ToDo:
-status conditions updates
-verify full image streams support
-utests and func tests
-fixmes and commented out code
-doc

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Fix CR comments and fixmes

- isolate imagestream and registry specific code
- fix namespace of CronJob, and its job and pod to CDI namespace
- manage CronJob-DataImportCron ownership relationship with a finalizer,
  handle DataImportCron deletion (CronJob etc.)
- remove CronJob and job pod for ImageStreams, use RequeueAfter and
  cronexpr instead
- add k8s app cdi-source-update-poller executed by CronJob to poll source
  image digest via skopeo inspect for url registry source, and annotate
  the DataImportCron when the image was updated and pending for import based
  on the cron schedule
- add cdi-source-update-poller and skopeo binary to the cdi-importer container
- complete dataimportcron-validate and its tests
- reconcile - use context.Context instead of context.TODO
- remove uncached client
- doc

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Fix ImageStreams watch

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add DataImportCron DV template instead of source

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Fix CR comments

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Split updateSucceeded func

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Improve cdi-source-update-poller cmd logs

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Remove ImageStream reconcile

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Remove ImageStream watch

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Remove unnecessary AnnSourceUpdatePending

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* More CR fixes

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Idempotentify initCron

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Recreate DV in case is't not found

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add DataImportCron spec.importsToKeep and status.currentImports

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add DataImportCron controller functional test

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add insecure TLS support

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Remove finalizers in cluster clean script

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Bound each import to its sha256 digest instead of latest

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add DataImportCron controller utests

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Tests CR fixes

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Minor tests CR fixes

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
2021-11-11 20:09:48 +01:00
Michael Henriksen
aedaf513ec
Move apis to staging, push to containerized-data-importer-api (#1997)
* move apis to new staging area

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* add script to push to staging

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix lint check and api reference

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* push staging to api repo

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2021-10-28 13:40:24 +02:00
akalenyu
2254cf0c1f
Add relationship labels (#1864)
Users don't want 👽 resources in clusters,
and we should also be able to tell if were part of a broader installation.

Note:
- Operator created resources were handled in https://github.com/kubevirt/controller-lifecycle-operator-sdk/pull/18
as these labels will be common to all resources deployed by the HCO.
- Now that the controller is guaranteed to have the labels, we can set env vars
that reference the label values (fieldRef) to spare calling GET on the CR in the controllers.
(thanks mhenriks).

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2021-07-28 20:05:24 +02:00
Alexander Wels
e3eb24650e
Fix crash on ingress without default backend service. (#1870)
Signed-off-by: Alexander Wels <awels@redhat.com>
2021-07-23 18:55:01 +02:00
Alexander Wels
5fedb458e5
Set some reasonable requests/limits for workloads (#1842)
* Set some reasonable requests/limits for workloads

Signed-off-by: Alexander Wels <awels@redhat.com>

* Fix unit tests

Signed-off-by: Alexander Wels <awels@redhat.com>
2021-06-28 20:34:01 +02:00
akalenyu
a602485b0b
Switch deprecated extensions/v1beta1 Ingress (#1826)
Replaced by networking/v1 Ingress
https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-22

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2021-06-08 19:33:31 +02:00
Michael Henriksen
d92c2f459d
update deps and bazel (#1815)
* update deps and bazel

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix apidocs and unit tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix generate-verify

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2021-06-08 01:31:59 +02:00
Marcelo Carneiro do Amaral
8d1721db0a
Add support for proxy in the CDI import pod, reconciling the info from OpenShift cluster wide proxy when available (#1507)
* api: Add new ImportProxy structure to CDIConfig in v1beta1 core api

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* common: Add constants related to configure Import proxy in Import pod and controller reconcile

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* config controller: Add import proxy reconcile, monitoring OCP proxy updates

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* config controller: Add unit test for import proxy reconcile, monitoring OCP proxy updates and creating Secret and ConfigMap

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* import controller: Add unit support of ImportProxy in the tests

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller util: Add support to retrieve an OpenShift cluster wide proxy object as well as its needed configurations

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller util:  Add unit test for the get cluster wide proxy functionality

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller: moved isOpenshift function to utils common and export it

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* importer: Clone http transport to keep the default proxy config such as the usage of environment variables and add support for proxy CA

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* operator: Update CDIConfigCRD with import proxy info

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* operator: Update CDIListCRD with import proxy info

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* operator: Add service account rights to CDI for accessing OpenShift proxy obj

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* operator: Add OpenShift proxy obj scheme support

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: Add new forwarding proxy for testing things that require a proxy

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: Add test to verify the importer pod with proxy

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* doc: Update cdi config doc with the ImportProxy info

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* add support to build the new proxy docker image

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: remove unwanted F parameter from test

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller: fix error handling

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: update the used method to get the pod of the importer pod and proxy

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* fixed comments from the revision

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller config: update the methods to use the log object from the reconcile object

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller: update function GetImportProxyConfig to return errors and include a new unit test

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller: add back a test that was removed by mistake

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* update updateCDIConfigByUpdatingTheClusterWideProxy function

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: decrease the import pod log pooling interval and increase the image size to be able to verify the import pod before it is deleted

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: update the pvc and proxied requests verification

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* tools: add X-Forwarded-For header in the proxy requests  used for testing

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: reset cluster wide proxy with the original values

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: fix proxy update to change spec instead of status and other minor updates

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* doc: update import proxy description

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* update generated files

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
2021-02-16 13:43:01 +01:00
Tomasz Barański
f89ec95f34
Global preallocation setting is not taken into account correctly. (#1565)
* Global preallocation setting is not take into account correctly.

Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1915704

Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>

* Preallocation: copy config spec over to status
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
2021-01-20 21:42:46 +01:00
Michael Henriksen
7c05e8f093
Designate CDI as CDIConfig authority (#1516)
* Formally designate CDI as owner of CDIConfig by adding annotation cdi.kubevirt.io/configAuthority

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* More robust upgrade handling.  No error if beta api not installed yet.

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-12-04 02:52:40 +01:00
Michael Henriksen
86c32e87b8
Add CDIConfig to CDI (#1475)
* Add CDIConfigSpec to CDI

Make CDIConfig singleton mirror data in "active" CDI

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix functional tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-11-11 23:09:54 +01:00
Maya Rashish
b91887e1b7
Reserve overhead when validating that a Filesystem has enough space (#1319)
* When validating disk space, reserve space for filesystem overhead

The amount of available space in a filesystem is not exactly
the advertise amount. Things like indirect blocks or metadata
may use up some of this space. Reserving it to avoid reaching
full capacity by default.

This value is configurable from the CDIConfig object spec,
both globally and per-storageclass.

The default value is 0.055, or "5.5% of the space is
reserved". This value was chosen because some filesystems
reserve 5% of the space as overhead for the root user and
this space doubles as reservation for the worst case
behaviour for unclear space usage. I've chosen a value
that is slightly higher.

This validation is only necessary because we use sparse
images instead of fallocated ones, which was done to have
reasonable alerts regarding space usage from various
storage providers.

---

Update CDIConfig filesystemOverhead status, validate, and
pass the final value to importer/upload pods.

Only the status values controlled by the config controller
are used, and it's filled out for all available storage
classes in the cluster.

Use this value in Validate calls to ensure that some of the
space is reserved for the filesystem overhead to guard from
accidents.

Caveats:

Doesn't use Default: to define the default of 0.055, instead
it is hard-coded in reconcile. It seems like we can't use a
default value.

Validates the per-storageClass values in reconcile, and
doesn't reject bad values.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Use util GetStorageClassByName

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Test filesystem overhead validation against async upload endpoint

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* wait for NFS PVs to be deleted before continuing

Intended to help with flakes, but didn't make a difference.
Probably still worth doing.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Avoid using the uncached client unnecessarily

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Add error handling for the case where even a default SC is not found

Note that this change isn't expected to make a difference, as we
check if the targetStorageClass is nil later on and have the same
behaviour, but this is probably more correct API usage.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Add testing for the validation of filesystem overhead values

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Fix logical error in waiting for NFS PVs.

Wait for all of them, not just the last one.

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2020-10-01 18:31:32 +02:00
Michael Henriksen
75f4fd6f2f
update k8s deps to 18.6 and controller runtime to 0.6.2 (#1330)
* update k8s deps to 1.18.6 and controller runtime to 0.6.2

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* remove building code generators from docker image.  This way the k8s ligray version only has to be updated in go.mod

Do more stuff in the bazel container.  Faster and better interop

Fix unit tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* make format

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* remove unnecessary rsync

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* redo code generator dep management

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* builder uses go modules

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-08-07 14:09:52 +02:00
Bartosz Rybacki
df81550666
Feature gates (#1283)
* Extracted reconcileUploadProxyUrl

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Add FeatureGates to CDIConfig CR

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* POC of FeatureGates

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Remove FeatureGates from spec in CDIConfig, add correct handling of communication errors

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Removed feature gate from DataVolume.

This will be added when actually the first featureGate will be used to toggle a feature.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update to v1beta after rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Create a featuregates package

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Rename first featureGate to HonorWaitForFirstConsumerEnabled

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* return error from feature gates api

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-07-16 05:31:41 +02:00
Michael Henriksen
9e2c79b1e0
move api groups to v1beta1 (#1232)
* move upload.cdi.kubevirt.io API group to v1beta1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* move core api to v1beta1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix os-3.11 cluster sync and add functional tests for alpha api

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* change more occurences of v1alpha1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* updates after rebase

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-07-10 15:47:38 +02:00
Alexander Wels
08ed6f2b39
Clean up client-go left overs after converting to runtime library. (#1163)
* Clean up client-go left overs after converting to runtime library.
Clean unused functions from controller utils.go
Turn util_test.go into gingko test.
Moved functions from util to proper controller if only used by that controller.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Don't export reconciler variables.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-04-02 22:59:54 +02:00
Michael Henriksen
99f8af5b86 k8s client upgrade to 1.16 (#1079)
* initial client upgrade to 1.16

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix Route detection in OpenShift

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-01-14 13:43:17 +01:00
Woohyung Han
2adcc80f95 Fix to create temporary pod with default resource quota (#1055)
If we create a temporary pod in the namespace which has
resource quota, `must specify quota` error occurs. Fix to
create temporary pod with default resource quota.

Co-authored-by: hyoung-90 <ssarang520@gmail.com>
Co-authored-by: Woohyung Han <woohhan@gmail.com>
Signed-off-by: anencore94 <anencore94@kaist.ac.kr>

Co-authored-by: anencore94 <37469330+anencore94@users.noreply.github.com>
Co-authored-by: hyoung-90 <58063867+hyoung-90@users.noreply.github.com>
2020-01-09 17:43:14 +01:00
Alexander Wels
f090ddc2e6 Update Import controller to use runtime controller library. (#1054)
Refactor import related utilities out of util.go into the controller.

Signed-off-by: Alexander Wels <awels@redhat.com>
2019-12-20 19:28:43 +01:00
Alexander Wels
5c34f97ea8 Move CDI config related functions to config controller from utils. (#1057)
Increase code coverage for these functions

Signed-off-by: Alexander Wels <awels@redhat.com>
2019-12-18 22:42:42 +01:00
Alexander Wels
889f231300 Use controller library for config controller (#1034)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-12-09 15:54:25 +01:00
Woohyung Han
d33b9f4db8 Add nil check for ingress http rule (#1049)
Ingress HTTP rule is optional field and can be nil. If we
create a ingress with no http rule nil pointer exception
is occur. Add nil check for ingress http rule

Co-authored-by: anencore94 <ssarang520@gmail.com>
Signed-off-by: Woohyung Han <techhanx@gmail.com>
2019-12-06 23:18:23 +01:00
Alexander Wels
d4a4ece09b Requeue on failure like sample k8s controller (#1015)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-11-12 13:21:15 +01:00
Alexander Wels
630a23ef23 Fix a bunch of go score card issues.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-05-06 16:52:03 -04:00
Irit goihman
71c2ead357 cleaned redundant function
Signed-off-by: Irit goihman <igoihman@redhat.com>
2019-04-26 15:57:13 +03:00
Alexander Wels
0d76a7444b Fix issue with the cdi config controller hanging in k8s
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-03-28 15:32:54 -04:00
Alexander Wels
d2ca78bec3 Use CDIConfig for scratch storage class.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-03-21 15:37:07 -04:00
Y.Horie
0c90e9c901 fix conflict 2019-03-05 23:23:04 +09:00
Y.Horie
09819e8158
Merge branch 'master' into issue-687 2019-03-05 23:05:05 +09:00
Y.Horie
c9104059fb move to klog from glog 2019-03-05 13:33:20 +09:00
Michael Henriksen
49a740d7ba make CDIConfig client cluster scoped and fix occasional crash 2019-03-04 18:54:58 -05:00
Irit goihman
8089d795b9 CDI Config is now owned by the CDI object
Signed-off-by: Irit goihman <igoihman@redhat.com>
2019-02-28 10:29:36 +02:00
Irit Goihman
1691fe1b39 create cdi config with config controller
Signed-off-by: Irit goihman <igoihman@redhat.com>
2019-02-24 10:23:35 +02:00