Commit Graph

52 Commits

Author SHA1 Message Date
Michael Henriksen
27905cda90
fix crd explain (#1343)
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-08-20 16:57:59 +02:00
Alexander Wels
c5f8d92d3b
Update external provider to allow for hpp and ceph storage. (#1318)
* Add support for external openshift cluster, in particular CRC.
make cluster-sync, and functional tests should all work.
Added documentation on how to enable CRC to work.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Updates based on review.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Changes based on review comments:
- removed registry in favor of making people use an external registry.
- added ceph for external provider.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Fix review comments

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-08-06 15:41:52 +02:00
Maya Rashish
dd0b5f60ee
Update kubevirtci. (#1292)
* Update kubevirtci.

Changes default KUBEVIRT_PROVIDER to k8s-1.18 that works, the previous
CDI-specific change was wrong and got overridden by another
'make generate'.

Removes ocp and okd kubevirtci providers.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* GC okd-4.1 and okd-4.3 cluster-sync logic.

These lanes were removed from kubevirtci.

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2020-07-13 22:37:40 +02:00
Michael Henriksen
9e2c79b1e0
move api groups to v1beta1 (#1232)
* move upload.cdi.kubevirt.io API group to v1beta1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* move core api to v1beta1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix os-3.11 cluster sync and add functional tests for alpha api

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* change more occurences of v1alpha1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* updates after rebase

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-07-10 15:47:38 +02:00
Maya Rashish
588a55ec3a
Remove k8s-1.16 cluster-sync logic (#1280)
* Make the k8s-1.18 lane the default

This is so we can remove the k8s-1.15 lane.
We don't run this lane in CI.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Remove k8s-1.16 cluster-sync support.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Remove k8s-1.15 cluster-sync support.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Update documentation regarding default KUBEVIRT_PROVIDERs

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2020-07-08 08:03:37 +02:00
Maya Rashish
0fe170eb21
Update kubevirtci and add k8s-1.18 lane (#1252)
* Update kubevirtci

This adds a k8s-1.18 lane, among other things.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Add cluster-sync support for k8s-1.18

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2020-06-25 13:28:11 +02:00
Michael Henriksen
87618df3b9
ember csi with beta snapshot api (#1218)
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-06-02 04:47:35 +02:00
Alexander Wels
ac9b487d76
Increase timeout waiting for ceph to be ready, also print ceph namespace pods every 30 seconds (#1209)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-05-22 03:39:29 +02:00
Michael Henriksen
d45574678b
alpha to beta snapshot API (#1206)
* move from alpha to beta snapshot API

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix broken clone tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* don't generate snapshot client

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-05-20 15:25:28 +02:00
maya-r
93565b2c88
Implement local webserver to mimic flawed servers (for functional tests) (#1193)
* Implement local web server to mimic flawed servers

http://cdi-bad-webserver.cdi/flaky/filename will return a 503
"unavailable" error to most requests.

http://cdi-bad-webserver.cdi/forbidden-HEAD/filename will return
a 403 "Forbidden" error on HEAD requests.

In both those cases, we validate that we still eventually succeed
in creating a datavolume.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Add the imageio server to the manual test setup instructions.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Add missing bad-webserver.yaml.in (forgot to git add)

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Make "flaky" bad-webserver reproducibly flaky.

9 failures followed by a success.
Switch to port 9090, os-3.11 disallows using privileged ports,
and we don't have a reason to prefer port 80.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Provide polarion test IDs

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Make flaky test server fail less frequently.

The exponential backoff makes 10 retries very slow.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Tolerate being installed on a namespace that isn't "cdi"

Assume that we are in the same namespace as cdi-file-host, and
use that namespace in the URL.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Use AfterEach to cleanup datavolumes after test.

Suggested by Alexander Wels.

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2020-05-08 00:01:19 +02:00
Michael Henriksen
f4f8e975b6
update kubevirtci (#1198)
* update kubevirtci to 0b941ea5dc647d3aea6f6a6fff95563d6ce0445e

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* make sure SELinux permissions correct for hpp

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix permission issue with clone source pods and k8s-1.17 provider

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-05-07 13:19:19 +02:00
Alexander Wels
bdb079f7bf
Make sure all CDI crds have structural schemas (#1171)
* Make sure all CDI crds have structural schemas

Signed-off-by: Alexander Wels <awels@redhat.com>

* Removed unneeded apiVersion, Kind, Metadata from validation

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-04-15 21:54:02 +02:00
Alexander Wels
753ffc4037
Attempt to unstuck the os-3.11-crio provider when an sdn pod is not ready. (#1176)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-04-14 21:55:59 +02:00
Alexander Wels
044326cb76
Had a bug in the comparison and it would always follow the path even if (#1167)
the provider was not os-3.11-crio

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-04-06 21:05:52 +02:00
Alexander Wels
9131189f6e
Increase time we wait for operator upgrade to complete. It was timing out sometimes even though it was still ongoing (#1166)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-04-03 15:33:51 +02:00
Alexander Wels
74e165bc6e
Change how we wait for CDI to be ready for OS-3.11.0-crio provider. (#1165)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-04-02 18:27:50 +02:00
Alexander Wels
25249c0148
Add ember csi lane (#1143)
* Add loopback device container
Add code to create pv vg and lvm
Add ember csi lvm based provisioner.

Signed-off-by: Alexander Wels <awels@redhat.com>

* import/export

Signed-off-by: Alexander Wels <awels@redhat.com>

* Address PR comments.
Switch deployment to pod for loopback device
Sleep 1 instead of 60 to allow kubernetes easier way of killing pod.
Instead of embercsi master, grab one that works from known repo.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Finish loopback pod instead of looping it.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-03-27 19:13:49 +01:00
Alexander Wels
d756862304
Read timeout from environment variable instead of hard coding it. (#1160)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-03-26 21:49:47 +01:00
Alexander Wels
44b46774d2
Runtime lib smartclone controller (#1134)
* Update datavolume controller to use runtime controller library

Signed-off-by: Alexander Wels <awels@redhat.com>

* Update smart clone controller to use controller runtime library

Signed-off-by: Alexander Wels <awels@redhat.com>

* Mark smart cloned cloned PVCs are k8s.io/CloneOf

Signed-off-by: Alexander Wels <awels@redhat.com>

* Removed unused informer factories.
Fixed naming of controller from import to smart clone.
Ignore not found error on delete.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Only delete snapshot if pvc is bound

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-03-20 17:11:44 +01:00
Piotr Kliczewski
78cee6d166
Imageio - new import data source (#1042)
This PR introduces new import data source to fetch disks from ovirt
and store them in storage available in the cluster. It uses ovirt-sdk
to check whether the this is there, the size of the disk and it obtains
signed ticket which allows to get the disk using ovirt-imageio-proxy.

In order to use it a user needs to create configmap with ovirt api
certificate and secret with ovirt engine user and password. Both are
required and validated by webhook. In order to start import process
we need to post following definition:

---
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
  name: "imageio-dv"
spec:
  source:
      imageio:
         url: "<engine-api-endpoint>"
         secretRef: "<secret-name-containing-engine-credentials>"
         certConfigMap: "<configmap-name-containing-engine-api-certificate>"
         diskId: "<disk-id-which-we-want-to-import>"
  pvc:
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: "64Mi"
---

Signed-off-by: Piotr Kliczewski <piotr.kliczewski@gmail.com>
2020-03-19 19:49:44 +01:00
Michael Henriksen
102ce2e78c
Uninstall strategy and blocking webhook (#1118)
* webhook to block deletion of datavolumes for BlockUninstallIfWorkloadsExist uninstallStrategy

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix apiserver permissions and tighten up cdi delete webhook functional test

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix cdi delete webhook for older k8s versions that don't send the object

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* cleanup webhooks and apiservices on upgrade

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* have to wait for cdi configmap to be garbage collected

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* do dry run deletes for datavolume protection webhook

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-02-21 05:34:50 +01:00
Alexander Wels
4bceec824d
Add NFS as KUBEVIRT_STORAGE to cluster-sync (#1093)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-01-30 22:01:55 +01:00
Alexander Wels
7ace337e97 Allow multiple previous versions to upgrade from (#1086)
Force kill operator during upgrade to test resilience

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-01-21 14:41:49 +01:00
Michael Henriksen
afa4f89ff8 update kubevirci and add support for k8s 1.17.0 (#1083)
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-01-15 17:04:45 +01:00
Alexander Wels
d32b8d9914 Enable hostpath provisioner for OKD 4.3 kubevirtci provider. (#1060)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-12-20 16:04:44 +01:00
Alexander Wels
19a9486199 Setup hostpath provisioner as an option in ci. (#1056)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-12-14 16:58:39 +01:00
Alexander Wels
fa1477f1f8 Added support for OKD-4.3 lane (#1050)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-12-06 18:49:22 +01:00
Alexander Wels
657be041ac Add k8s 1.16.2 provider. (#1041)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-12-02 17:55:45 +01:00
Alexander Wels
74704c72dc Replace current ceph install with operator based rook-ceph. (#1010)
Add reporting of DVs on test failures.

Signed-off-by: Alexander Wels <awels@redhat.com>
2019-11-12 13:25:15 +01:00
Alexander Wels
6b52b29a78
Don't fail immediately when Local is not found when syncing. (#1000)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-10-28 15:21:50 -04:00
Alexander Wels
a67c64c338
initial bazelification of CDI (#986)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-10-15 15:55:44 -04:00
Alexander Wels
4a20e29db3
Remove OLM integration as we aren't using it. (#982)
Removed OLM code since it is not being used in CDI deployment.

Signed-off-by: Alexander Wels <awels@redhat.com>
2019-10-07 18:34:07 -04:00
Alexander Wels
a14916666a
Wait for SC to be available before making it default (#981)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-10-02 14:07:41 -04:00
Alexander Wels
f8dad13db6
Update kubevirtci images to refresh certificates. (#973)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-09-24 19:08:24 -04:00
Alexander Wels
28efefaf0f
Sparsify clone tarring. (#966)
Make xfs default fs for ceph in k8s provider.

Signed-off-by: Alexander Wels <awels@redhat.com>
2019-09-19 19:38:16 -04:00
Alexander Wels
6f350eb0a7
Use right lane k8s-1.15.1, and verify that --enable-ceph is set. (#959)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-09-18 07:21:15 -04:00
Alexander Wels
4af9b3d797
set ceph to be default sc (#956)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-09-12 03:27:49 -04:00
Alexander Wels
399a05dc53
Add upgrade from release version 1.10.4 to current master. (#954)
Switch OKD-4.1 provider from olm install to operator install.

Signed-off-by: Alexander Wels <awels@redhat.com>
2019-09-11 00:35:46 -04:00
Alexander Wels
45eecea14e
Added conditions to match the HCO requirements. (#910)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-08-28 18:36:44 -04:00
Michael Henriksen
412b6e10ca CDI upgrade support (#929)
* * Initial upgrade support
* - Detect from reconcile loop that it is uograde flow
* - Set ObeservedVersion to target when upgrade is finished
* - Delete unused objects at the end of upgrade

* *     opertor controller unit test - detect upgrade
    *  cdi upgrade unit tests
    *  - verify upgrade flow is detected when version is updated
    *  - verify on upgrade objects are updated
    *  - verify on upgrade unused objects are deleted

* * optimize cleanuoUnusedResourses function
* fix logging error

* * CR fixes
* remove unused methods in unit tests
* use reflect.DeepEqual to compare runtime.Objects in unit test
* check DeletionTimeStamp before entering upgrade

* * uit tests - CR is deleted during/before upgrade

* * CR fixes:
* - invoke Deletion callbacks before and after resource deletion on clenaupUnusedResourse function
* - when looking for object to delete - search not only by name but by namespace as well

* * delete unused resources of previous version is CDI CRF is marked for deletion during upgrade
* add unit test for this case

* * should not start upgrade if versions are identical

* * add unit tests to verify there is no upgrade on identical versions

* CR fix - return error

* don't think we have to explicitly cleanup old resources when CDI deleted during upgrade

* refactor code and properly handle deleting resources on upgrade

* reconcile loop now does three way merge to better handle upgrade
2019-08-27 08:43:49 -04:00
Alexander Wels
25135ee46e
Updated kubevirtci images to latest. Rolling OKD 4.1 instead (#937)
of specific versions.
okd provider is now aware of KUBEVIRT_NUM_NODES

Signed-off-by: Alexander Wels <awels@redhat.com>
2019-08-23 15:39:51 -04:00
Alexander Wels
e047b17c95 Update kubevirtci k8s version to 1.15.1
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-08-15 08:24:34 -04:00
Anna Stopel
91eea99747 * bug fix - make OLM manifests be set per provider (#919)
* * bug fix - make OLM manifests be set per provider

* * move all CDI installation related variables to cluster-sync directory
2019-08-13 08:25:44 -04:00
annastopel
03497da071 minor bugfix 2019-07-14 10:29:12 +03:00
annastopel
ed5894cb08 * build fixes 2019-07-14 09:49:36 +03:00
annastopel
b9ecc4cf5a Refactor code of variuos CDI installations:
* cluster-sync/install.sh - contains all supported installation techniques
  and a install_cdi method that derives the technique based on CDI_INSTALL variable
	- operator manifests
        - OLM manifests
* per provider there is cluster-sync/${PROVIDER}/install.sh that has the settting for CDI_INSTALL
  for specific provider
* default technique is CDI installation via operator
2019-07-14 09:48:10 +03:00
annastopel
34bc7e1d94 OLM - ci -test. Create cdi-olm-catalog to be able to deploy cdi via olm in ci. This is due to okd 4.1.
* Add make tagret docker-olm-catalog
  * Add build-olm-catalog script that builds tree expected by operator-registry.

  * add catalogsource manifest to deploy operator registry per provider
    * os
    * k8s
  * update olm documentation with opertor-registry deployment
2019-07-14 09:48:09 +03:00
Alexander Wels
fd92d71543
Switch to default namespace after creating local storage. (#878)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-07-10 08:05:30 -04:00
Alexander Wels
5fecd05927
Updated cluster-up with latest kubevirtci images. (#873)
Now using OKD 4.1.2 instead of 4.1.0.
k8s-1.13.3 and os-3.11.0-crio now using latest version of local volume provisioner:v2.3.2

Signed-off-by: Alexander Wels <awels@redhat.com>
2019-07-03 14:29:23 -04:00
Michael Henriksen
e04684882e fix cluster-sync for okd (#872) 2019-07-03 08:18:13 -04:00