Commit Graph

103 Commits

Author SHA1 Message Date
zhlhahaha
d6172f33b6
e2e, arm64: make the e2e tests works of Arm64 platform (#3577)
* cluster-sync, kind: add setup_hostname_resolution_for_registry func

For the Kind provider, we need to configure hostname
resolution for the local image registry in the CoreDNS
service. This ensures that local container images can
be successfully pulled into Kubernetes pods during
certain e2e tests.

Signed-off-by: howard zhang <howard.zhang@arm.com>

* cluster-sync kind: support kind provider in cluster-sync

Signed-off-by: howard zhang <howard.zhang@arm.com>

* bazel arm64: stop build and deploy vddk on Arm64

VDDK is not supported on Arm64

Signed-off-by: howard zhang <howard.zhang@arm.com>

* cluster-sync: add capabilities for registry-populate in kind

Add SYS_ADMIN and SYS_CHROOT capability to make buildah work
in kind cluster

Signed-off-by: howard zhang <howard.zhang@arm.com>

* e2e: add vddk label

Signed-off-by: howard zhang <howard.zhang@arm.com>

* e2e: add no-kubernetes-in-docker label

This label means the e2e tests are not suitable to test in
containerized k8s env (kind provider).

In this commit, we label two tests because the large size images
would lead to OOM of cdi-uploader. For more detail, please
refer to:
https://github.com/kubevirt/containerized-data-importer/pull/3577#discussion_r1903547758

Signed-off-by: howard zhang <howard.zhang@arm.com>

* e2e: amend the VerifyPermissions strings

The output of ls -ln for a file with 660 permissions can be either
-rw-rw----. or -rw-rw----

Signed-off-by: howard zhang <howard.zhang@arm.com>

* automation, test: add kind provider in automation test

Signed-off-by: howard zhang <howard.zhang@arm.com>

---------

Signed-off-by: howard zhang <howard.zhang@arm.com>
2025-01-15 01:46:40 +01:00
Alex Kalenyuk
745e4df162
Update kubevirtci, 1.31 lanes (#3555)
* Bump kvci to latest

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Switch testing lanes to 1.31/1.30

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Retry seeding images

This fails often since the most recent kvci bump,
so let's retry

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

---------

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2024-12-17 09:34:01 +01:00
Thomas-David Griedel
e4bc183702
Exclude Unsupported Func-Test Images for s390x (#3463)
* exclude unsupported test images

Signed-off-by: Thomas-David Griedel <griedel911@gmail.com>

* switch to env-var

Signed-off-by: Thomas-David Griedel <griedel911@gmail.com>

---------

Signed-off-by: Thomas-David Griedel <griedel911@gmail.com>
2024-10-29 01:22:54 +01:00
Alex Kalenyuk
368ad80a28
Update kubevirtci, 1.30 lanes (#3343)
* Bump kvci to latest

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Switch testing lanes to 1.30/1.29

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Revert "Add W/A for NFS OOMKills"

https://github.com/kubevirt/kubevirt/issues/10822#issuecomment-2220620883

This reverts commit aba780300f.

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

---------

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2024-07-26 06:25:15 +02:00
Edu Gómez Escandell
b887d87943
Fix uploadproxy override in cluster-sync (#3306)
* Fix uploadproxy override in cluster-sync

Compare to kubevirt's:
https://github.com/kubevirt/kubevirt/blob/main/hack/cluster-deploy.sh#L53-L56

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Remove https prefixing of proxy URL in functional tests

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

* Tests prefix the upload proxy URL with https:// when necessary

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>

---------

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
2024-06-23 14:35:34 +02:00
Edu Gómez Escandell
9527df3bdc
Fix cluster-sync wait_cdi_crd_installed timeout check (#3245)
This line was writing to file `./0` instead of performing a greater-than
comparison.

Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
2024-05-07 12:00:14 +02:00
Alex Kalenyuk
fed4d7a604
Update kubevirtci, 1.29 lanes (#3023)
* Update kubevirtci

May help with nfs lane flaking:
https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/pull/kubevirt_containerized-data-importer/3017/pull-containerized-data-importer-e2e-nfs/1731266718799499264
Looks like the nfs server is working but mount commands from nfs csi against it are timing out.

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Use crictl to prepull images

https://github.com/kubevirt/kubevirtci/pull/1100

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Add W/A for NFS OOMKills

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Switch testing lanes to 1.29/1.28

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

---------

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2024-03-25 23:41:20 +01:00
alromeros
fab858e2fa
Prepare CDI testing for the upcoming non-CSI lane (#2730)
* Update functional tests to skip incompatible default storage classes

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Enable the use of non-csi HPP in testing lanes

This commit modifies several scripts to allow the usage of classic HPP as the default SC in tests.

This allows us to test our non-populator flow with a non-csi provisioner.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

---------

Signed-off-by: Alvaro Romero <alromero@redhat.com>
2023-06-06 04:30:42 +02:00
Michael Henriksen
f88fab69dc
PVC Clone Populator (#2709)
* touch up zero restoresize snapshot

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* clone populator

only supports PVC source now

snapshot coming soon

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* more unit tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* unit test for clone populator

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* func tests for clone populator

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* move clone populator cleanup function to planner

other review comments

verifier pod should bount readonly

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* add readonly flag to test executor pods

synchronize get hash calls

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* increase linter timeout

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* better/explicit readonly support for test pods

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* check pv for driver info before looking up storageclass as it may not exist

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* addressed review comments

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* chooseStrategy shoud generate more events

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

---------

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2023-05-24 05:11:52 +02:00
Michael Henriksen
93bcff88ca
let's start testing 1.26 (#2697)
* let's start testing 1.26

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* remove sync-os-ci.sh

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

---------

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2023-04-22 04:47:54 +01:00
akalenyu
6bb139e5f0
Ceph WFFC lane set up (#2644)
* Bump kubevirtci for wffc ceph storage class

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Add infrastructure needed for ceph wffc lane

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Remove unused cluster sync providers

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

---------

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2023-04-06 18:38:53 +01:00
Maya Rashish
a623d6d071
Simplify upgrade testing, and only test v1beta1 DVs (#2618)
Newer releases of CDI no longer contain v1alpha1 support, so we
can't blindly apply v1alpha1 DVs. Any old version DV is good enough,
nothing specific to v1alpha1, so let's remove that variant.

While here, make the DVs for upgrade testing always contain the
annotation to not be garbage collected.
This helps simplify the cluster-sync logic.

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2023-03-06 14:28:07 +00:00
alromeros
2e9a9257f9
Add support for volume populators (#2482)
* Add support for volume populators in CDI

This commit enables the use of volume populators in CDI, so datavolume-owned PVCs can be populated using custom logic.

Volume populators are CRDs used to populate volumes externally, independently of CDI. These CRDs can now be specified using the new DataSourceRef API field in the DataVolume spec.

When a DataVolume is created with a populated DataSourceRef field, the datavolume-controller creates the corresponding PVC accordingly but skips all the population-related steps. Once the PVC is bound, the DV phase changes to succeeded.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Modify CDI test infrastructure to support testing of external populators

This commit introduces several changes to CDI ci to support the testing of DataVolumes with external populators:
* A sample volume populator is now deployed in the test infrastructure, in a similar way as bad-webserver or test-proxy. This populator will be used in functional tests from now on.
* A new test file with external population tests has been introduced in the tests directory

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Update dependencies to include lib-volume-populator library

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Add functional tests for proper coverage of external population of DataVolumes

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Minor fixes on external-population logic for DataVolumes:
* Added comments for exported structs
* Removed non-inclusive language
* Improved error messages in webhooks
* Fixed logic on datavolume-controller

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Improve DataVolume external-population logic when using the old 'DataSource' API

This commit introduces several changes into the datavolume external-population controller to improve its behavior when using the DataSource field.

It also introduces minor fixes on the generic populator logic.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Add unit tests for external-population controller and DV admission

Signed-off-by: Alvaro Romero <alromero@redhat.com>

Signed-off-by: Alvaro Romero <alromero@redhat.com>
2023-01-17 21:56:15 +01:00
akalenyu
9ff9780c85
Add note about public repos when developing on external provider (#2511)
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2023-01-08 21:42:23 +00:00
akalenyu
bae2638386
Make cluster-sync more friendly for external provider (#2480)
The aim of this change is to make setting up a dev environment on external cluster easier,
By doing mainly 2 things:
- Bazel push (CDI containers/e2e testing containers) expects external registry creds to be in
the docker CRI default path. This seems to be half-configurable on the bazel side,
except for the actual file name (which differs for podman).
Now we prompt for the credentials and save them on the expected path.
- Deploy the e2e testing manifests on external provider as well
Not sure why not do that today? probably reason dates way back

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2022-12-13 20:51:35 +00:00
Maya Rashish
52651224da
Bump kubevirtci, bump k8s versions used in CI (#2481)
* Update kubevirtci

Changes default KUBEVIRT_PROVIDER to k8s-1.25.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Bump kubernetes provider versions

1.24 -> 1.25
1.23 -> 1.24

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Add k8s-1.25 provider

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Remove cluster-sync/k8s-* versions that no longer exist in kubevirtci

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Note kubevirtci uses tags, not releases.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Use a namespace that exists to keep test valid

Newer kubernetes first complains about the namespace not existing,
let's make sure we use one that exists rather than modifying the
tests to expect a completely different failure.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2022-11-24 08:08:49 +00:00
Alexander Wels
a7b31b759d
Change how we deploy the upgrade testing DVs (#2434)
* Dump some information if the upgrade testing objects are
not ready in time.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Added new logic for upgrading

Signed-off-by: Alexander Wels <awels@redhat.com>

Signed-off-by: Alexander Wels <awels@redhat.com>
2022-09-28 03:34:08 +01:00
Michael Henriksen
46c6aa994a
Support restricted PSA for worker pods (#2410)
* remove root worker pods

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* remove selinux requirement for worker pods

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* run tests in restricted namespace and required changes

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* handle empty tar

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* add PSA label when running functional tests in OpenShift

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* cannot use restricted PSA with istio (for now)

refactor scc management

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix clean script

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2022-09-14 21:16:23 +01:00
akalenyu
cbb439072b
Switch CI lanes to k8s 1.24 (#2328)
* Update kubevirtci

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Add cluster-sync 1.24 provider

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Increment k8s versions used in CI lanes

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Handle secrets not being auto generated in 1.24

More info in third bullet point
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md#urgent-upgrade-notes

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2022-06-30 01:30:22 +02:00
Alexander Wels
6366ef397f
Increment versions of k8s used in CI (#2278)
* Increment versions of k8s used in CI

Take advantage of newer versions of k8s for testing.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Udpate kubevirtci version to one that includes ceph 1.7.1

Signed-off-by: Alexander Wels <awels@redhat.com>
2022-05-19 19:37:43 +02:00
akalenyu
8b92395ddf
Change the way we fetch latest release from GitHub (#2256)
CI is failing, it looks like gh releases/latest now redirects (302 HTTP).
Also seems like the go to is to use the gh API for this

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2022-05-03 16:52:04 +02:00
Maya Rashish
02e70a5a54
Use centos:stream9 as a base image, cleanup unused code (#1983)
* Retire ember LVM code, unused

(Rationale: avoid having to change more things for changing the
base image)

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Remove unreferenced files from WORKSPACE

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Switch to centos:stream9 as a base image.

It has a significantly longer support cycle than Fedora releases,
and supposedly offers vulnerability scans.

Add a tinyCore.vdi to the repo instead of generating it.
The centos qemu-img has read-only VDI support, so we can't generate
it. Generate it using my system and add to the file-host.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Use full names for pulls from dockerhub

CentOS doesn't like short tags

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Avoid specifying checksum for CentOS images.

They expire faster than we can update checksums, this is unfortunate
but perhaps they will soon publish images at a lower rate allowing
us to keep up.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Bump number of open file limit to avoid bazel crashes

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Update builder to include #2087, builder based on centos stream9

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Update checksums that seem wrong

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Update ovirt links: old ones were removed

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Remove unused RPMs

Noticed due to: duplicated checksum but no problem in testsuite,
lack of aarch64 equivalent.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Put nbdkit-vddk-plugin back for amd64.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Use quay.io instead of dockerhub.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Install util-linux-core for /usr/sbin/blockdev

Needed after #2174

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Update nbdkit/libnbd/nginx/ovirt versions to the latest

The previous version we were using can't be fetched any more

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Generate our own CentOS stream9 image using RPMs

Now updating the dependencies can be done by running `make rpm-deps`
and committing the change, like kubevirt.

This creates a small complication that we need to run update-ca-trust
to trust root CAs. Do this on the pod, using the entrypoint to do so.

Use a single image with all the dependencies for the test tools, we
don't benefit from making them minimal and it saved some trouble in
the conversion.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Fixup imageio test container

Run update-ca-trust and update-crypto-policies before running
ovirt-imageio, to stop error messages.

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2022-04-20 19:37:51 +02:00
akalenyu
2cb09af168
Include namespace on upgrade test manifest/attempt to fix quota test (#2197)
* Include namespace of upgrade testing manifests in yaml

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Don't back out until we verify quota status is set following us POSTing it

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2022-03-29 00:06:05 +02:00
akalenyu
af347cebd9
Use NFS CSI instead of static no provisioner manual PVs (#2166)
* Update kubevirtci to get NFS-CSI

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Use NFS CSI instead of static no provisioner manual PVs

We are creating (and deleting) NFS PVs on our own before each test,
which seems to trigger a race in the deletion, causing flakes.

There is an NFS CSI driver available that we could use instead for dynamic NFS PV provisioning.

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2022-03-07 15:59:46 +01:00
Maya Rashish
f2939fcee8
add PVC claimName to datavolume status (#2060)
* Make it possible to find the underlying PVC name using the DV

Right now a lot of things assume that the underlying PVC has the
same name/namespace, let's make it possible to reach over and not
need to have this implicit knowledge in a lot of places.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Install some artifacts on the old version of CDI during upgrade tests

And use this to test that DataVolume.Status.ClaimName is set after
upgrades.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Bump CDI pod update timeout

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Only check if non-testing CDI pods have updated.

We don't update the testing environment, so it looks like some of the
update fails.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Restore lower timeouts

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* As elsewhere, don't use local registry artifacts with external provider

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2022-01-07 20:28:25 +01:00
Alexander Wels
1e6976e05b
HPP storage class file name changed. (#2065)
Use the new file name when deploying the legacy HPP CR and storage class.

Signed-off-by: Alexander Wels <awels@redhat.com>
2021-12-18 01:26:27 +01:00
Alexander Wels
2b3bbe85fe
Fix hpp setup for tests. (#2046)
With hpp operator at v0.11.0 it now requires cert-manager, and webhook
This PR modifies the hpp setup to include setting up those items.

Signed-off-by: Alexander Wels <awels@redhat.com>
2021-12-03 00:29:04 +01:00
akalenyu
38af724f1c
Add alert for incomplete storage profiles / delete profile when corresponding SC gone (#2027)
* Add alert for incomplete storage profiles

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Run metric tests on both openshift and k8s

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Add functional test for storageprofile metrics

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Delete profile as a follow up to storage class getting deleted

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Address review, alter tests to cover List metric approach

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Address review; individually loop over metric decrement, shorten reconcile.Result{}

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Address review; deletion timestamp not possible when err/teardown in AfterEach

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2021-12-01 21:54:59 +01:00
Arnon Gilboa
7087b57cd2
Add DataImportCron controller (#1949)
* Add DataImportCron controller

-The new controller polls for updates to a registry source container
image, based on a given schedule. When updates to a container image are
detected, the controller imports the content into a new uniquely named
PVC in a golden image namespace.
-For each DataImportCron, the controller manages a corresponding
DataSource to always point to the latest most up-to-date golden
image PVC.
-DataImportCron takes ownership of an existing DataSource (with
controller: false), allowing an admin to opt-in to using auto
delivery/updates later on.
-The controller has PVC garbage collector removing old PVCs.

ToDo:
-status conditions updates
-verify full image streams support
-utests and func tests
-fixmes and commented out code
-doc

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Fix CR comments and fixmes

- isolate imagestream and registry specific code
- fix namespace of CronJob, and its job and pod to CDI namespace
- manage CronJob-DataImportCron ownership relationship with a finalizer,
  handle DataImportCron deletion (CronJob etc.)
- remove CronJob and job pod for ImageStreams, use RequeueAfter and
  cronexpr instead
- add k8s app cdi-source-update-poller executed by CronJob to poll source
  image digest via skopeo inspect for url registry source, and annotate
  the DataImportCron when the image was updated and pending for import based
  on the cron schedule
- add cdi-source-update-poller and skopeo binary to the cdi-importer container
- complete dataimportcron-validate and its tests
- reconcile - use context.Context instead of context.TODO
- remove uncached client
- doc

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Fix ImageStreams watch

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add DataImportCron DV template instead of source

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Fix CR comments

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Split updateSucceeded func

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Improve cdi-source-update-poller cmd logs

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Remove ImageStream reconcile

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Remove ImageStream watch

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Remove unnecessary AnnSourceUpdatePending

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* More CR fixes

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Idempotentify initCron

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Recreate DV in case is't not found

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add DataImportCron spec.importsToKeep and status.currentImports

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add DataImportCron controller functional test

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add insecure TLS support

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Remove finalizers in cluster clean script

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Bound each import to its sha256 digest instead of latest

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add DataImportCron controller utests

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Tests CR fixes

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Minor tests CR fixes

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
2021-11-11 20:09:48 +01:00
Maya Rashish
0f15e8c108
Update kubevirtci, add newer cluster-sync provider logic (#1993)
* Bump kubevirtci

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Add k8s-1.21, k8s-1.22 cluster-sync goo

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Update comments to match reality - we're making local default here

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Test upgrade from newer versions of CDI

You can't install too old CDI on k8s-1.22, as we used APIs removed
in this version of kubernetes.

Try upgrading only from newer versions

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Use a path under /dev for attaching a PVC

After kubernetes 1.20 it seems like / is mounted nodev, so we can't
have "devicePath: /pvc" any more.

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2021-10-26 12:47:30 +02:00
Alexander Wels
af72cfe171
Update default storage to use CSI hpp driver (#1978)
* Update default storage to use CSI hpp driver

Update the default storage for hpp lanes to be the csi driver
instead of the legacy driver.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Add check for csi driver capabilities in particular fsGroup.
Skip check for fsGroup if csi driver reports it cannot accomidate.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Addressed review comment.

Signed-off-by: Alexander Wels <awels@redhat.com>
2021-10-11 23:08:44 +02:00
Michael Henriksen
2889d68766
BugId: 1999571 - fix clone into larger capacity nfs volume (#1939)
* BugId: 1999571 - fix clone into larger capacity nfs volume

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix lint issues

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2021-09-15 20:22:35 +02:00
Arnon Gilboa
fb2048bdff
Replace all NAMESPACE refs with CDI_NAMESPACE (#1912)
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
2021-09-05 15:06:15 +02:00
akalenyu
2b423469fe
Use dedicated ServiceAccount for test manifests (#1905)
Instead of using our production SA with test manifests, let's have a dedicated & minimal one.
Also provide SCC in case test suite is ran on OpenShift (needed to provide capability to `cdi-docker-registry-host`).

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2021-08-25 13:06:36 +02:00
Arnon Gilboa
0e1e910118
Fix cluster-sync PULL_POLICY usage (#1899)
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
2021-08-23 19:07:54 +02:00
akalenyu
ef7634754a
Use NodePort instead of kubectl port-forward for upload tests (#1897)
Follow up for #1886, we now have a kubevirtci version that gives us a free port.
We can use that to make the switch from `kubectl port-forward` which has been known to cause many flakes in our CI.

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2021-08-23 11:18:50 +02:00
Alexander Wels
ace718b293
Move nfs-server images to quay.io (#1828)
Signed-off-by: Alexander Wels <awels@redhat.com>
2021-06-09 00:47:38 +02:00
Michael Henriksen
74a2c86608
use namespace transfer for smart clone (#1763)
* use namespace transfer for smart clone

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* updates from test failures

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* add expansion func tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* add dv phases for expansion and transfer

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* rebase and integrate with storage profiles

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2021-04-30 15:18:43 +02:00
Maya Rashish
798f37501d
Copy k8s-1.18 logic to k8s-1.20, k8s-1.19 (#1743)
Signed-off-by: Maya Rashish <mrashish@redhat.com>
2021-04-16 03:59:01 +02:00
Michael Henriksen
0b45adfb54
update kubevirtci with builtin rook/ceph (#1703)
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2021-03-17 20:14:50 +01:00
Michael Henriksen
3447bb84c7
Cluster scoped DataVolume/PVC namespace transfer API (#1673)
* Cluster-scoped namespace transfer api and controller

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* unit tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* ObjectTransfer webhook

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* new functests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* experiment with termination grace period

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* quota test

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2021-02-24 20:45:24 +01:00
Marcelo Carneiro do Amaral
8d1721db0a
Add support for proxy in the CDI import pod, reconciling the info from OpenShift cluster wide proxy when available (#1507)
* api: Add new ImportProxy structure to CDIConfig in v1beta1 core api

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* common: Add constants related to configure Import proxy in Import pod and controller reconcile

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* config controller: Add import proxy reconcile, monitoring OCP proxy updates

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* config controller: Add unit test for import proxy reconcile, monitoring OCP proxy updates and creating Secret and ConfigMap

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* import controller: Add unit support of ImportProxy in the tests

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller util: Add support to retrieve an OpenShift cluster wide proxy object as well as its needed configurations

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller util:  Add unit test for the get cluster wide proxy functionality

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller: moved isOpenshift function to utils common and export it

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* importer: Clone http transport to keep the default proxy config such as the usage of environment variables and add support for proxy CA

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* operator: Update CDIConfigCRD with import proxy info

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* operator: Update CDIListCRD with import proxy info

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* operator: Add service account rights to CDI for accessing OpenShift proxy obj

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* operator: Add OpenShift proxy obj scheme support

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: Add new forwarding proxy for testing things that require a proxy

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: Add test to verify the importer pod with proxy

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* doc: Update cdi config doc with the ImportProxy info

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* add support to build the new proxy docker image

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: remove unwanted F parameter from test

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller: fix error handling

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: update the used method to get the pod of the importer pod and proxy

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* fixed comments from the revision

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller config: update the methods to use the log object from the reconcile object

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller: update function GetImportProxyConfig to return errors and include a new unit test

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller: add back a test that was removed by mistake

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* update updateCDIConfigByUpdatingTheClusterWideProxy function

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: decrease the import pod log pooling interval and increase the image size to be able to verify the import pod before it is deleted

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: update the pvc and proxied requests verification

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* tools: add X-Forwarded-For header in the proxy requests  used for testing

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: reset cluster wide proxy with the original values

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: fix proxy update to change spec instead of status and other minor updates

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* doc: update import proxy description

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* update generated files

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
2021-02-16 13:43:01 +01:00
Arnon Gilboa
0736f1ab87
Fix make target cluster-sync-cdi, add cluster-clean-cdi & cluster-clean-test-infra (#1503)
* Fix make target cluster-sync-cdi, add cluster-clean-cdi/test-infra

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Update README.md

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
2020-12-02 10:08:39 +01:00
Maya Rashish
09b61a924c
Move configure_storage to test setup. (#1484)
Some of the storage setup involves deploying pods, and they
are labelled with cdi.kubevirt.io/testing for the benefit of
the "CriticalAddonsTolerations" test.

The test setup deleted all the test pods before deploying,
which means that the storage pods were killed, breaking
KUBEVIRT_STORAGE=nfs make cluster-sync

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2020-11-22 11:30:33 +01:00
Bartosz Rybacki
8f49ab2945
Scratch import bug (#1424)
* check if pod used

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Test scratch deletion during import

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle scratch missing just after being created in utility function.

There was a possibility that just after creating the scratch it is deleted, and the next get results in error. This error was ignored - not returned from util - resulting in wrong behavior.
Other fix here is to skip pods with status PodSucceeded and PodFailed in getPodsUsingPVCs function - such Pods are not using PVC, as these pods are already done.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Reschedule reconcile for the pvc until pvc import is complete.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Increase number of NFS volumes, and document the test.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Try another quick fix for verifier pod flakiness.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Cleanup defaults

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-11-03 16:31:50 +01:00
Arnon Gilboa
32792c8546
Add make targets cluster-sync-cdi & cluster-sync-test-infra (#1451)
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
2020-10-22 22:33:43 +02:00
Maya Rashish
958c1d39b0
Make CDI infra deployments as critical addons. (#1361)
* Make CDI infra deployments as critical addons.

While here, mark cdi-operator as linux-only, it doesn't take
its value from the CDI CR.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Add test CDI pods are deployed with critical addon tolerations

Co-authored-by: Daniel Belenky <dbelenky@redhat.com>
Signed-off-by: Daniel Belenky <dbelenky@redhat.com>
Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Turn nfs-server into a deployment

So that it is restarted after the destructive test destroying
CDI testing pods.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Rewrite CDI critical taint test

Now it is simpler and waits for pods to be running before
continuing for the next tests.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Skip tolerations test if the CDI CR has unexpected values

This is necessary for Prow, which installs the CDI CR from the
previous version and updates to the latest code, but might be
useful in other cases too.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Modify functional test to use AfterEach instead of defer

While here, make it more generic so there's a chance this Describe
block will be used by other tests - iterate over all pods instead
of just the CDI test pods.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Skip test if CDI CR isn't named "cdi" (rather than fail)

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* fix typo

Signed-off-by: Maya Rashish <mrashish@redhat.com>

Co-authored-by: Daniel Belenky <dbelenky@redhat.com>
2020-10-15 09:05:39 +02:00
Tomasz Barański
c0798bb1d5
Add files used in OpenShift CI. (#1416)
Added file `cluster-sync/sync-os-ci.sh`, which pushes images to a
registry and deploys CDI. Added missing commands in
`hack/ci/Dockerfile.ci`

Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
2020-10-15 00:03:39 +02:00
Arnon Gilboa
28b393b8f3
Check for expected changes after CDI upgrade (#1417)
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
2020-10-14 02:41:38 +02:00
Alexander Wels
d2da27b41b
Ensure we can specify different CDI CR name. (#1383)
Fix bug where the CR name was hard coded to CDI and it failed in HCO.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-09-22 00:13:28 +02:00