Commit Graph

50 Commits

Author SHA1 Message Date
Michael Henriksen
75f4fd6f2f
update k8s deps to 18.6 and controller runtime to 0.6.2 (#1330)
* update k8s deps to 1.18.6 and controller runtime to 0.6.2

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* remove building code generators from docker image.  This way the k8s ligray version only has to be updated in go.mod

Do more stuff in the bazel container.  Faster and better interop

Fix unit tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* make format

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* remove unnecessary rsync

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* redo code generator dep management

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* builder uses go modules

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-08-07 14:09:52 +02:00
Alexander Wels
c5f8d92d3b
Update external provider to allow for hpp and ceph storage. (#1318)
* Add support for external openshift cluster, in particular CRC.
make cluster-sync, and functional tests should all work.
Added documentation on how to enable CRC to work.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Updates based on review.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Changes based on review comments:
- removed registry in favor of making people use an external registry.
- added ceph for external provider.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Fix review comments

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-08-06 15:41:52 +02:00
Alexander Wels
c7a3610769
Update cdi-builder to use go version 1.14.6 (current latest) (#1310)
* Update cdi-builder to use go version 1.14.6 (current latest)
Fix functional test framework to work with go >= 1.13. We were not
using the ginkgo ordering correctly, and our framework would call
flag.Parse() too soon, and mess with the go testing flag parsing.
As a result we had to alter some of the variables that we use to build
urls to be functions so that when called the required information is
available.
Fixed several tests not checking for errors.
Fixed dumping code being initialized several times messing with the
order in which failures were reported.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Fixed failing tests.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-07-23 23:09:44 +02:00
Bartosz Rybacki
e76edf4868
Fix expectation in test that kills container, executor might be killed -9 by parent. (#1297)
It might happen that `kill 1` is executed so fast that the parent process tries to exit and kills
the child process - `bash` which actually runs the `kill 1` command. In effect our process exits
with code 137. In case of this test it is not a problem, we expect process to kill parent.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-07-22 19:37:44 +02:00
Bartosz Rybacki
ab48911b9b
WaitForFirstConsumer handling for DataVolumes (#1242)
* Set the WaitForFirstConsumer phase on DataVolume when storage uses the WaitForFirstConsumer binding mode and is not bound yet.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Skip PVC if not bound in import|clone|upload controllers.

This is done so the VM pod(not the cdi pod) will be the first consumer, and the PVC can be scheduled on the same location as the pod.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

fixup! Skip PVC if not bound in import|clone|upload controllers.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update importer tests to force bind the PCV by scheduling a pod for pvc, when storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update datavolume tests to force bind the PCV by scheduling a pod for pvc, when storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update upload controller and upload tests to correctly handle force binding the PCV by scheduling a pod for pvc, when storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update clone tests to force bind the PCV by scheduling a pod for pvc when the storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update cloner multi-node tests to force bind the PCV by scheduling a pod for pvc when storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Correct after automerge

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Improve/simplify tests

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Fix error in import test.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update transport_test,operator_test.go

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update rbac_test.go and leaderelection_test.go

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Improve Datavolume and PVC Checks for WFFC.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle wffc only if feature gate is open - import-controller

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* TEST for Handle wffc only if feature gate is open - import-controller - TEST

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle wffc only if feature gate is open - upload-controller with test

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* rename and simplify checks

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* cleanup after rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* update tests after rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* update tests after rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* more cleanups

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Document new WFFC behavior

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Document new HonorWaitForFirstConsumer option

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* update docs according to comments

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* extract common function, cleanup - code review fixes

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* add comment for another pr - 1210, so it can have easier merge/rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* typo

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Simplify getStoragebindingMode - code review comments

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Add FeatureGates interface - code review fix

Additionally pass the features gates instead of the particular feature gate value,
and let shouldReconcilePVC decide what to do with the feature gate. That way shouldReconcilePVC
contains all the logic, and the caller does not need to do additional calls to provide parameters.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update matcher

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-07-22 16:23:44 +02:00
Alexander Wels
4145bf93e6
Run size validation before returning success on upload. This to give immediate feedback to the user when uploading. (#1299)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-07-17 00:57:42 +02:00
Maya Rashish
a8c6cfae75
Non-functional change cleanup: remove unused calls to UploadPodName (#1302)
* Remove unused calls to utils.UploadPodName.

This function has no side effects, it just returns a name.
We don't use the return value, so this is a no-op

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Use utils.UploadPodName rather than hard-coding the name.

No functional change expected.

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2020-07-16 16:29:41 +02:00
Maya Rashish
47b174d0d0
Combine datavolume name tests into a single table. (#1296)
* Combine datavolume name tests into a single table.

They seem to vary only by the DataVolume name.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Fix lint failures

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2020-07-16 00:45:41 +02:00
Michael Henriksen
9e2c79b1e0
move api groups to v1beta1 (#1232)
* move upload.cdi.kubevirt.io API group to v1beta1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* move core api to v1beta1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix os-3.11 cluster sync and add functional tests for alpha api

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* change more occurences of v1alpha1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* updates after rebase

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-07-10 15:47:38 +02:00
Michael Henriksen
621f0b8ca0
Add new endpoint to uploadproxy and uploadserver to support form based uploads (#1288)
They are more efficient from a browser

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-07-09 21:35:38 +02:00
Michael Henriksen
180fac1230
use go package to properly handle CORS requests (#1260)
* use go package to properly handle CORS requests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* add Origin header to upload requests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-06-30 01:20:49 +02:00
Michael Henriksen
9efefdef49
[flaky test] verifier pods hanging around keeps cloning from working (#1261)
* verifier pods hanging around keeping cloning from working

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* GetDiskGroup will wait for pod to actually be deleted if requested

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-06-26 15:08:11 +02:00
Alexander Wels
8d4d89694e
Change permissions to always be 660 for the disk image. (#1231)
Updated functional tests to check for permissions

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-06-17 16:03:42 +02:00
Michael Henriksen
a3cf7051a7
uploadproxy should add Access-Control-Allow-Origin response header (#1229)
see: https://github.com/kubevirt/containerized-data-importer/issues/1224

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-06-10 19:57:38 +02:00
Michael Henriksen
87618df3b9
ember csi with beta snapshot api (#1218)
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-06-02 04:47:35 +02:00
Bartosz Rybacki
ab8b9c025e
Generating label names (#1200)
* Handle labels length correctly

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle service name generation correctly

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Remove not needed labels

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Store import pod name in annotation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Enable long DV name

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle name with dot when creating service/label name

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Test long names on import,  upload and clone

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Store upload pod name in annotation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Store importer scratch pvc name in annotation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Quick fix for tests (need improvements)

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Cleanup handling scratch name

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Ensure pod/service name conflicts are handled

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle client errors when trying to get the import pod

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Style improvements, and other code review fixes.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Store clone source pod name in an annotation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Correct name initialization and tests

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Do not init name if pod already exists. It is not needed.

The situation of having a pod but not name on annotation can happen after the upgrade, when we have a legacy pvc and pod already existing, but clone operation not finished. But when we already have the pod, then in the code (currently) we do not need the name from annotation.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Cleanup scratch name handling

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Use constant for max dv name in validation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Simplify clone source pod name initialization

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-05-29 19:55:32 +02:00
Bartosz Rybacki
6d97be73d3
Add DV restarts test for upload controller (#1186)
* Add import/upload/clone functional tests for DataVolume restarts column

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Cleanup tests tests

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-04-23 21:41:10 +02:00
Alexander Wels
f2657d5927
Set secondary GID of user on import and upload pods to be 107 (qemu) … (#1148)
* Set secondary GID of user on import and upload pods to be 107 (qemu) for kubevirt content type.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Fix broken unit tests on upload.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Add import and clone test.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-03-26 14:33:48 +01:00
maya-r
a3bb6d6262
Restore sparse images as the default (revert #970) (#1140)
(Leaving in some of the checks introduced in that change)

Having sparse disks works better with external storage being
monitored for low capacity. If we allocate the full size at
start, it shows up as a disk capacity alert immediately.

Another motivation to pre-allocate was that cloning would
sometimes fail for capacity reasons.
We now validate available size before cloning, so we don't
expect that problem to return.

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2020-03-12 04:43:58 +01:00
kubevirt-bot
19ee764626
Added functional tests for ResourceRequirements feature. (#1076)
Added documentation for ResourceQuota.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-03-04 23:30:57 +01:00
Michael Henriksen
3376570cb7 uploadproxy should use use pod ready annotation (#996)
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2019-10-28 07:57:01 -04:00
Fred Rolland
fe38c5ac33 Add node selectot for VerifyPvcEmpty pod (#974)
Signed-off-by: Fred Rolland <frolland@redhat.com>
2019-09-26 10:08:53 -04:00
Alexander Wels
2884854375
falloc on resize and blank disk. (#970)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-09-25 07:57:30 -04:00
Alexander Wels
6f350eb0a7
Use right lane k8s-1.15.1, and verify that --enable-ceph is set. (#959)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-09-18 07:21:15 -04:00
Michael Henriksen
d6c30526a4 comment out nonsensical test code that adds instability 2019-07-16 20:23:30 -04:00
Michael Henriksen
019c843586 make clone pods use selinux type spc_t instead of privileged (#875)
* make clone pods use selinux type spc_t instead of privileged

* fix block mode related tests
2019-07-08 13:58:42 -04:00
Alexander Wels
dc4d22dc5d
Allow specifying block storage class. (#815)
Fix node selection functional test

Signed-off-by: Alexander Wels <awels@redhat.com>
2019-06-28 08:22:56 -04:00
Irit goihman
9ef5c3bf91 functional test for upload
Signed-off-by: Irit goihman <igoihman@redhat.com>
2019-06-11 09:59:25 +03:00
Michael Henriksen
f51eacaf05 refactor as result of OLM integration 2019-06-05 16:16:23 -04:00
Alexander Wels
85622de4f4
Update upload_test.go 2019-06-05 09:01:33 -04:00
Denis Ollier
e535ee5fab tests: wait more time for cdi-upload pod to be ready
20s delay seems to be not sufficient on some platforms and upload tests
get flaky.

Signed-off-by: Denis Ollier <dollierp@redhat.com>
2019-06-04 19:50:44 +02:00
Alexander Wels
f40c16b25b Randomize cdi namespace in tests.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-05-14 10:55:42 -04:00
Alexander Wels
05b6a21736 Fix issues with size header in cloning something reading beginning of data.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-04-26 19:47:03 -04:00
tavni
e9b6b4f501 Adding support to upload disk image to a Raw Block PV
Signed-off-by: tavni <tavni@redhat.com>
2019-04-17 10:27:24 +03:00
tavni
e6665945bc Adding support to clone disk image from one Block PV to another
Signed-off-by: tavni <tavni@redhat.com>
2019-04-15 15:33:52 +03:00
Michael Henriksen
c54155d189 import/upload should not look for pods when they may not exist 2019-03-29 10:26:32 -04:00
tavni
d4c7b0be54 Adding support to import disk image into Block PV
Signed-off-by: tavni <tavni@redhat.com>
2019-03-28 11:43:56 +02:00
Alexander Wels
d1b7842a90 Resize on upload.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-03-08 21:51:05 -05:00
Alexander Wels
1e46004ad2 Scratch space retry framework logic.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-03-07 14:54:08 -05:00
Michael Henriksen
ec2736d2c4 ready checks for all upload componets and uploadproxy sync with api server 2019-03-01 10:20:57 -05:00
Michael Henriksen
5a7858498a use port forwarding for upload tests instead of sshing to node01 2019-02-25 15:23:37 -05:00
Nelly Credi
0c6e1ff6ae updated to [attr1][attr2][attr3] struct 2019-01-10 17:09:51 +02:00
Nelly Credi
0149a300c7 Add test attributes
Added rfe id, test id, criticality etc

Based on issue
https://github.com/kubevirt/containerized-data-importer/issues/359
2019-01-10 17:05:57 +02:00
Alexander Wels
8b27e695f0 Fix some functional test issues running locally.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-01-03 10:34:56 -05:00
tavni
f4bb5e531e Adding more functional tests for blank-raw-image, PV allocation on specific
node and upload disk image features.

Signed-off-by: tavni <tavni@redhat.com>
2018-12-20 13:32:54 +02:00
Michael Henriksen
2b6b39361a was incorrectly getting available free space on volume 2018-12-05 11:28:04 -05:00
Alexander Wels
676b4ec133 Fix some flaky tests.
Signed-off-by: Alexander Wels <awels@redhat.com>
2018-11-20 09:23:37 -05:00
j-griffith
537bcd02b0 Update unit and functional tests 2018-10-12 22:07:43 -06:00
Michael Henriksen
452e30bca7 add verification to functional test and change to shared pod phase annotation 2018-09-19 14:37:19 -04:00
Michael Henriksen
be89439734 functional test 2018-09-19 12:30:46 -04:00