Commit Graph

24 Commits

Author SHA1 Message Date
Maya Rashish
e3436e0199
Allow specifying nodeSelector, affinity and tolerations for CDI pods (#1346)
* Generate CDI CRD using controller-tools.

This is only done for CDI CRD as it requires the existence of source
code. Other CRDs we create are created by a more bare bones pod.

CDIUninstallStrategy was missing a comment describing it, so add
one. This was spotted manually so there might be more missing.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Allow users to specify which nodes CDI pods will live on.

nodeSelector, affinity and tolerations are possible values.

This is done in the CDI CR (rather than CDIConfig) as we are
interested in having this field be populated by external operators.

Unit tests now require the existence of a CDI CR, so create it.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Add a unit test covering some node placement functions

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Specify that all our pods are linux-only.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Avoid duplicate test, accidental left over.

Pointed out by awels, thanks.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Rename to cdiOperatorDeployment for clarity.

Suggested by awels

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Specify we only run on linux using the CDI CR, no need to embed this
into the code.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Don't dereference workloadPlacement for no reason

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Split off operator test to have its own AfterEach, BeforeEach.

Use even more descriptive function names.

Do all the CDI delete/restore logic in AfterEach, to ensure that
it happens and restores the deployment with the original CR even
if the test fails.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Remove XXX. This is the proper way.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Adapt to latest changes in controller_test.go (renaming import)

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Simplify, not storing intermediate value.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Don't dereference nodeplacement in callers to CreateDeployment

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Remove redundant save & restore. Unit tests do this for us.

Pointed out by awels, thanks.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Split out "find toplevel" to a utility function

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Wait for the CDI CR update to apply before continuing.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Simplify, not storing intermediate value.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Make it clear that the chosen node placement will not be schedulable.

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2020-09-03 22:13:18 +02:00
Michael Henriksen
75f4fd6f2f
update k8s deps to 18.6 and controller runtime to 0.6.2 (#1330)
* update k8s deps to 1.18.6 and controller runtime to 0.6.2

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* remove building code generators from docker image.  This way the k8s ligray version only has to be updated in go.mod

Do more stuff in the bazel container.  Faster and better interop

Fix unit tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* make format

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* remove unnecessary rsync

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* redo code generator dep management

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* builder uses go modules

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-08-07 14:09:52 +02:00
Bartosz Rybacki
ab48911b9b
WaitForFirstConsumer handling for DataVolumes (#1242)
* Set the WaitForFirstConsumer phase on DataVolume when storage uses the WaitForFirstConsumer binding mode and is not bound yet.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Skip PVC if not bound in import|clone|upload controllers.

This is done so the VM pod(not the cdi pod) will be the first consumer, and the PVC can be scheduled on the same location as the pod.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

fixup! Skip PVC if not bound in import|clone|upload controllers.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update importer tests to force bind the PCV by scheduling a pod for pvc, when storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update datavolume tests to force bind the PCV by scheduling a pod for pvc, when storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update upload controller and upload tests to correctly handle force binding the PCV by scheduling a pod for pvc, when storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update clone tests to force bind the PCV by scheduling a pod for pvc when the storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update cloner multi-node tests to force bind the PCV by scheduling a pod for pvc when storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Correct after automerge

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Improve/simplify tests

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Fix error in import test.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update transport_test,operator_test.go

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update rbac_test.go and leaderelection_test.go

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Improve Datavolume and PVC Checks for WFFC.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle wffc only if feature gate is open - import-controller

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* TEST for Handle wffc only if feature gate is open - import-controller - TEST

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle wffc only if feature gate is open - upload-controller with test

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* rename and simplify checks

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* cleanup after rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* update tests after rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* update tests after rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* more cleanups

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Document new WFFC behavior

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Document new HonorWaitForFirstConsumer option

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* update docs according to comments

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* extract common function, cleanup - code review fixes

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* add comment for another pr - 1210, so it can have easier merge/rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* typo

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Simplify getStoragebindingMode - code review comments

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Add FeatureGates interface - code review fix

Additionally pass the features gates instead of the particular feature gate value,
and let shouldReconcilePVC decide what to do with the feature gate. That way shouldReconcilePVC
contains all the logic, and the caller does not need to do additional calls to provide parameters.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update matcher

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-07-22 16:23:44 +02:00
Michael Henriksen
9e2c79b1e0
move api groups to v1beta1 (#1232)
* move upload.cdi.kubevirt.io API group to v1beta1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* move core api to v1beta1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix os-3.11 cluster sync and add functional tests for alpha api

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* change more occurences of v1alpha1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* updates after rebase

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-07-10 15:47:38 +02:00
Michael Henriksen
bd69c350bb
Wait for PVCs to be unused before initiating clone operations (#1263)
* don't create snapshot or clone pods if pvcs in use

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* cleanup pods during functional tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* kill mmore pods blocking clone tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix typos

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-06-30 19:08:50 +02:00
Bartosz Rybacki
ab8b9c025e
Generating label names (#1200)
* Handle labels length correctly

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle service name generation correctly

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Remove not needed labels

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Store import pod name in annotation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Enable long DV name

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle name with dot when creating service/label name

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Test long names on import,  upload and clone

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Store upload pod name in annotation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Store importer scratch pvc name in annotation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Quick fix for tests (need improvements)

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Cleanup handling scratch name

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Ensure pod/service name conflicts are handled

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle client errors when trying to get the import pod

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Style improvements, and other code review fixes.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Store clone source pod name in an annotation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Correct name initialization and tests

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Do not init name if pod already exists. It is not needed.

The situation of having a pod but not name on annotation can happen after the upgrade, when we have a legacy pvc and pod already existing, but clone operation not finished. But when we already have the pod, then in the code (currently) we do not need the name from annotation.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Cleanup scratch name handling

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Use constant for max dv name in validation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Simplify clone source pod name initialization

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-05-29 19:55:32 +02:00
Alexander Wels
c707e781a9
Add conditions to DataVolume and add condition annotation to PVCs (#1194)
* Add conditions to DataVolume and add condition annotation to PVCs

Signed-off-by: Alexander Wels <awels@redhat.com>

* Added reason codes to conditions
Updated functional tests to use reason codes
Fixed broken functional test.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Added last transition time
Added documentation

Signed-off-by: Alexander Wels <awels@redhat.com>

* Switch to conditions slice from conditions pointer slice.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Add heartbeat time for conditions.
Renamed AnnRunningCondition to AnnLastTerminationMessage
Only update reason/message on heartbeat or state change.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Rename heartBeat to heartbeat
Re-factor datavolume-conditions to properly update heartbeat time.

Signed-off-by: Alexander Wels <awels@redhat.com>

* renamed AnnRunningConditionReason to AnnLastTerminationReason

Signed-off-by: Alexander Wels <awels@redhat.com>

* Revert lastTerminatedMessage/Reason rename
Added separate AnnScratchBoundCondition/Message/Reason for scratch space

Signed-off-by: Alexander Wels <awels@redhat.com>

* Update bound condition to take scratch or source PVC status into consideration.
Update running condition to take source pod status into consideration

Signed-off-by: Alexander Wels <awels@redhat.com>

* Addressed comments in review.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Properly get scratch space pvc phase.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-05-12 23:39:23 +02:00
Alexander Wels
08ed6f2b39
Clean up client-go left overs after converting to runtime library. (#1163)
* Clean up client-go left overs after converting to runtime library.
Clean unused functions from controller utils.go
Turn util_test.go into gingko test.
Moved functions from util to proper controller if only used by that controller.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Don't export reconciler variables.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-04-02 22:59:54 +02:00
Bartosz Rybacki
4605cf1dc2
Add a field to DataVolume to track the number of retries/pod restarts (#1155)
* Add a field to DataVolume to track the number of retries/pod restarts

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Add a field to DataVolume to track the number of retries/pod restarts

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Make RESTARTS non-empty on DataVolume (shows as as 0 'zero')

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Test reporting restarts on DataVolume when importing.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Fix tests

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Code review fixes

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Restart Count status test for upload and clone controller

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-03-30 23:17:49 +02:00
Alexander Wels
44b46774d2
Runtime lib smartclone controller (#1134)
* Update datavolume controller to use runtime controller library

Signed-off-by: Alexander Wels <awels@redhat.com>

* Update smart clone controller to use controller runtime library

Signed-off-by: Alexander Wels <awels@redhat.com>

* Mark smart cloned cloned PVCs are k8s.io/CloneOf

Signed-off-by: Alexander Wels <awels@redhat.com>

* Removed unused informer factories.
Fixed naming of controller from import to smart clone.
Ignore not found error on delete.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Only delete snapshot if pvc is bound

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-03-20 17:11:44 +01:00
Alexander Wels
1d48514c2b
Use controller runtime library for upload controller (#1085)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-03-03 15:30:56 +01:00
Alexander Wels
17db2920b1
Update clone controller to use runtime library. (#1075)
Signed-off-by: Alexander Wels <awels@redhat.com>

Co-authored-by: Michael Henriksen <mhenriks@redhat.com>
2020-02-14 23:04:46 +01:00
Michael Henriksen
bd4c4c950b
cert rotation (#1091)
* initial cert rotation controller

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix typo

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-02-03 23:36:58 +01:00
Michael Henriksen
66a1c8c996 Unique client cert per clone source pod (#987)
* unique cert per clone source pod

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* upload controller shouldn't cache certs

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix test variable names

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2019-10-09 15:10:55 -04:00
Michael Henriksen
834b85ecbf Network clone (#897)
* network cloning

* fix clone progress
2019-08-01 16:01:25 -04:00
Michael Henriksen
358709450d fix TestUploadPossible. No longer necessary to wait for pod to be running. Upload proxy handles that case 2019-05-21 08:40:50 -04:00
Artyom Lukianov
40325745d0 Copy triple package from client-go repository
This package droped from client-go 1.13.4, so until
we will have find some good alternatives, we can use local copy.
2019-05-16 09:38:37 +03:00
Alexander Wels
d2ca78bec3 Use CDIConfig for scratch storage class.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-03-21 15:37:07 -04:00
Alexander Wels
1e46004ad2 Scratch space retry framework logic.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-03-07 14:54:08 -05:00
Michael Henriksen
db9e9eb115 force delete upload pod if not completed and pvc deleted 2018-10-18 12:02:55 -04:00
j-griffith
537bcd02b0 Update unit and functional tests 2018-10-12 22:07:43 -06:00
Michael Henriksen
452e30bca7 add verification to functional test and change to shared pod phase annotation 2018-09-19 14:37:19 -04:00
Michael Henriksen
6c8c32fab8 initial apiserver and upload proxy unit tests 2018-09-19 12:30:46 -04:00
Michael Henriksen
b806786da1 refactoring and unit tests 2018-09-19 12:30:46 -04:00