* Google Cloud Storage Importer
This is a Google Cloud Storage importer for CDI
Signed-off-by: Marcelo Parisi <marcelo@feitoza.com.br>
* Fix auto-generated swagger and openapi
Signed-off-by: Marcelo Parisi <marcelo@feitoza.com.br>
* GCS Importer General Fixes
Signed-off-by: Marcelo Parisi <marcelo@feitoza.com.br>
* Moving back gcs-secret.txt
Moving file back to imageDir to fix unit testing.
Signed-off-by: Marcelo Parisi <marcelo@feitoza.com.br>
---------
Signed-off-by: Marcelo Parisi <marcelo@feitoza.com.br>
Co-authored-by: Marcelo Parisi <marcelo@dev-box.corp.feitoza.com.br>
Newer releases of CDI no longer contain v1alpha1 support, so we
can't blindly apply v1alpha1 DVs. Any old version DV is good enough,
nothing specific to v1alpha1, so let's remove that variant.
While here, make the DVs for upgrade testing always contain the
annotation to not be garbage collected.
This helps simplify the cluster-sync logic.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add support for volume populators in CDI
This commit enables the use of volume populators in CDI, so datavolume-owned PVCs can be populated using custom logic.
Volume populators are CRDs used to populate volumes externally, independently of CDI. These CRDs can now be specified using the new DataSourceRef API field in the DataVolume spec.
When a DataVolume is created with a populated DataSourceRef field, the datavolume-controller creates the corresponding PVC accordingly but skips all the population-related steps. Once the PVC is bound, the DV phase changes to succeeded.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Modify CDI test infrastructure to support testing of external populators
This commit introduces several changes to CDI ci to support the testing of DataVolumes with external populators:
* A sample volume populator is now deployed in the test infrastructure, in a similar way as bad-webserver or test-proxy. This populator will be used in functional tests from now on.
* A new test file with external population tests has been introduced in the tests directory
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Update dependencies to include lib-volume-populator library
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Add functional tests for proper coverage of external population of DataVolumes
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Minor fixes on external-population logic for DataVolumes:
* Added comments for exported structs
* Removed non-inclusive language
* Improved error messages in webhooks
* Fixed logic on datavolume-controller
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Improve DataVolume external-population logic when using the old 'DataSource' API
This commit introduces several changes into the datavolume external-population controller to improve its behavior when using the DataSource field.
It also introduces minor fixes on the generic populator logic.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Add unit tests for external-population controller and DV admission
Signed-off-by: Alvaro Romero <alromero@redhat.com>
Signed-off-by: Alvaro Romero <alromero@redhat.com>
The SCC was not applying properly due to some missing required
fields. This PR adds those fields.
Signed-off-by: Alexander Wels <awels@redhat.com>
Signed-off-by: Alexander Wels <awels@redhat.com>
Modified template that generates CDI cr to honor
wait for first consumer for the release manifest
Signed-off-by: Alexander Wels <awels@redhat.com>
Signed-off-by: Alexander Wels <awels@redhat.com>
* Dump some information if the upgrade testing objects are
not ready in time.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Added new logic for upgrading
Signed-off-by: Alexander Wels <awels@redhat.com>
Signed-off-by: Alexander Wels <awels@redhat.com>
* Retire ember LVM code, unused
(Rationale: avoid having to change more things for changing the
base image)
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Remove unreferenced files from WORKSPACE
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Switch to centos:stream9 as a base image.
It has a significantly longer support cycle than Fedora releases,
and supposedly offers vulnerability scans.
Add a tinyCore.vdi to the repo instead of generating it.
The centos qemu-img has read-only VDI support, so we can't generate
it. Generate it using my system and add to the file-host.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Use full names for pulls from dockerhub
CentOS doesn't like short tags
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Avoid specifying checksum for CentOS images.
They expire faster than we can update checksums, this is unfortunate
but perhaps they will soon publish images at a lower rate allowing
us to keep up.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Bump number of open file limit to avoid bazel crashes
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Update builder to include #2087, builder based on centos stream9
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Update checksums that seem wrong
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Update ovirt links: old ones were removed
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Remove unused RPMs
Noticed due to: duplicated checksum but no problem in testsuite,
lack of aarch64 equivalent.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Put nbdkit-vddk-plugin back for amd64.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Use quay.io instead of dockerhub.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Install util-linux-core for /usr/sbin/blockdev
Needed after #2174
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Update nbdkit/libnbd/nginx/ovirt versions to the latest
The previous version we were using can't be fetched any more
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Generate our own CentOS stream9 image using RPMs
Now updating the dependencies can be done by running `make rpm-deps`
and committing the change, like kubevirt.
This creates a small complication that we need to run update-ca-trust
to trust root CAs. Do this on the pod, using the entrypoint to do so.
Use a single image with all the dependencies for the test tools, we
don't benefit from making them minimal and it saved some trouble in
the conversion.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Fixup imageio test container
Run update-ca-trust and update-crypto-policies before running
ovirt-imageio, to stop error messages.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
Sometimes we see that ticket is not there (thx awels), we might want to
try a few times before giving up.
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Include namespace of upgrade testing manifests in yaml
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Don't back out until we verify quota status is set following us POSTing it
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Create imageio container during CDI build.
Instead of using a really old imageio, use bazel to build a new
imageio based on 2.5.0. Update the tests to use the new image
and paths in that new image. This requires a new repo in quay for
us to push the image to.
Also changed the approach of resolving the warm import potential
dead lock (scratch PVC from previous import pod terminating, while
the new pod is trying to create itself). Instead of trying to avoid
in all scenarios, detect the state, and delete the pod so the dead
lock can be resolved.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Populate test images
Signed-off-by: Alexander Wels <awels@redhat.com>
* Enable disabled test, and fix race condition where the import
controller thought it was done, but we were still on the final
import of a warm migration.
Updated the way we create the ticket on the fake imageio
Signed-off-by: Alexander Wels <awels@redhat.com>
* Set htpp(s)_proxy to lower case env variable
CURL used by nbdkit doesn't read upper case http(s)_proxy environment
variables, and thus was not using the proxy. Changed the variable to
be lower case.
Added a significant number of tests to test many more variations of
using a proxy. Also added https + auth endpoint to the file-host
container, so we can test https + auth with the proxy.
Added https endpoint to proxy, so we can test an https proxy.
Cleaned up some of the error handling in the import controller for
the proxy, in particular if a trustedCAProxy is defined.
Fixed some of the cluster wide proxy configuration so it works properly
inside an openshift cluster.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Add https proxy support to registry import. Added extra
functional tests to test all registry import combinations
Signed-off-by: Alexander Wels <awels@redhat.com>
* Fixed some tests to work better in Open Shift.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Make it possible to find the underlying PVC name using the DV
Right now a lot of things assume that the underlying PVC has the
same name/namespace, let's make it possible to reach over and not
need to have this implicit knowledge in a lot of places.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Install some artifacts on the old version of CDI during upgrade tests
And use this to test that DataVolume.Status.ClaimName is set after
upgrades.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Bump CDI pod update timeout
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Only check if non-testing CDI pods have updated.
We don't update the testing environment, so it looks like some of the
update fails.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Restore lower timeouts
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* As elsewhere, don't use local registry artifacts with external provider
Signed-off-by: Maya Rashish <mrashish@redhat.com>
We see a rare flake where our registry test deployment is using a different cert
to what we saved in a CM in our cluster.
As an attempt to mitigate, lets ensure that during rollout old pods are first gone, and only then new ones are introduced.
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
We see an error starting the deployment's server d/s:
```bash
[cnv-qe-jenkins@alex48-451-xzjsk-executor ocs-test]$ oc logs -f -n openshift-cnv imageio-deployment-f9bfff68-clbs2 imageiotest
Using configuration from /etc/pki/tls/openssl.cnf
...
File "/ovirt-imageio/daemon/ovirt_imageio/uhttp.py", line 80, in server_bind
self.socket.bind(self.server_address)
PermissionError: [Errno 13] Permission denied
curl: (7) Couldn't connect to server
```
Running as root seems to solve it, and doesn't harm test integrity since it's a test-helper.
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Add qemu-img rebase and commit operations.
Also only fail images with backing files that do not exist, so that
ImageIO snapshots can be downloaded and applied to a base disk image.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add merge phase to data processor.
This keeps qemu-img details out of the ImageIO data source.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Beef up transfer ticket finalization/cancellation.
Snapshots seem to be more prone to getting locked indefinitely than
disks if not correctly finalized or cancelled, so do this more carefully
than before.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Allow downloading snapshots from ImageIO.
Download the first snapshot as a raw whole-disk image, and download
subsequent snapshots as QCOW images to be committed to that base.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Allow multi-stage fields on ImageIO data sources.
Also avoid removing base disk image when cleaning data directory.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add ImageIO multi-stage functional tests.
Pick up fakeovirt update for stub functionality, so inventory responses
can be changed on the fly for individual tests.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Update multi-stage documentation for ImageIO.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Move if-else test block to functions.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Reset ImageIO inventory for a test I missed.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Clean up from some review comments.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Sort out calls to cleanupTransfer.
Failures during the creation of a transfer ticket call the original
cleanupTransfer in a single location, and any exits after the data
source is created call a wrapper function. The wrapper has a lock and a
'done' flag to make sure it is only called once on exit, even when
interrupted from the goroutine that waits for SIGTERM.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
Instead of using our production SA with test manifests, let's have a dedicated & minimal one.
Also provide SCC in case test suite is ran on OpenShift (needed to provide capability to `cdi-docker-registry-host`).
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
Follow up for #1886, we now have a kubevirtci version that gives us a free port.
We can use that to make the switch from `kubectl port-forward` which has been known to cause many flakes in our CI.
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Bump kubevirtci for uploadproxy port
Bumping kubevirtci to get https://github.com/kubevirt/kubevirtci/pull/653
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Fix local registry test container
We were never supposed to be able to run buildah bud without the SETFCAP capability:
https://www.redhat.com/sysadmin/podman-inside-kubernetes
Building an image using buildah requires a properly installed shadow-utils package. This requires SETFCAP.
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Increase timeout on propagation from cdi.spec->cdiconfig
Seeing some failures on the istio lane to ensure propagation in 10 second
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* api: Add new ImportProxy structure to CDIConfig in v1beta1 core api
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* common: Add constants related to configure Import proxy in Import pod and controller reconcile
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* config controller: Add import proxy reconcile, monitoring OCP proxy updates
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* config controller: Add unit test for import proxy reconcile, monitoring OCP proxy updates and creating Secret and ConfigMap
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* import controller: Add unit support of ImportProxy in the tests
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller util: Add support to retrieve an OpenShift cluster wide proxy object as well as its needed configurations
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller util: Add unit test for the get cluster wide proxy functionality
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller: moved isOpenshift function to utils common and export it
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* importer: Clone http transport to keep the default proxy config such as the usage of environment variables and add support for proxy CA
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* operator: Update CDIConfigCRD with import proxy info
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* operator: Update CDIListCRD with import proxy info
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* operator: Add service account rights to CDI for accessing OpenShift proxy obj
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* operator: Add OpenShift proxy obj scheme support
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: Add new forwarding proxy for testing things that require a proxy
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: Add test to verify the importer pod with proxy
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* doc: Update cdi config doc with the ImportProxy info
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* add support to build the new proxy docker image
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: remove unwanted F parameter from test
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller: fix error handling
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: update the used method to get the pod of the importer pod and proxy
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* fixed comments from the revision
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller config: update the methods to use the log object from the reconcile object
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller: update function GetImportProxyConfig to return errors and include a new unit test
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller: add back a test that was removed by mistake
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* update updateCDIConfigByUpdatingTheClusterWideProxy function
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: decrease the import pod log pooling interval and increase the image size to be able to verify the import pod before it is deleted
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: update the pvc and proxied requests verification
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* tools: add X-Forwarded-For header in the proxy requests used for testing
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: reset cluster wide proxy with the original values
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: fix proxy update to change spec instead of status and other minor updates
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* doc: update import proxy description
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* update generated files
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* Make CDI infra deployments as critical addons.
While here, mark cdi-operator as linux-only, it doesn't take
its value from the CDI CR.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add test CDI pods are deployed with critical addon tolerations
Co-authored-by: Daniel Belenky <dbelenky@redhat.com>
Signed-off-by: Daniel Belenky <dbelenky@redhat.com>
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Turn nfs-server into a deployment
So that it is restarted after the destructive test destroying
CDI testing pods.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Rewrite CDI critical taint test
Now it is simpler and waits for pods to be running before
continuing for the next tests.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Skip tolerations test if the CDI CR has unexpected values
This is necessary for Prow, which installs the CDI CR from the
previous version and updates to the latest code, but might be
useful in other cases too.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Modify functional test to use AfterEach instead of defer
While here, make it more generic so there's a chance this Describe
block will be used by other tests - iterate over all pods instead
of just the CDI test pods.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Skip test if CDI CR isn't named "cdi" (rather than fail)
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* fix typo
Signed-off-by: Maya Rashish <mrashish@redhat.com>
Co-authored-by: Daniel Belenky <dbelenky@redhat.com>
This allows people to run the test outside of the kubevirtci
environment and have it complete successfully.
Added configmap to vcenter.yaml.in that creates the CM that
lets users specify which init container to use. The default
is the test init container
Signed-off-by: Alexander Wels <awels@redhat.com>
* Add support for a VDDK datasource to the controllers
This commit does not implement the VDDK datasource. It simply
adds the DataVolumeSourceVDDK type and adds support to the
datavolume and import controllers for it. The datasource itself
will need to be done in a follow-up.
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Fix tests and clean up lint
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Add DataVolumeSourceVDDK to the v1alpha1 api group
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Install nbdkit.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add govmomi to vendor directory.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Integrate govmomi into build.
Incomplete! This allows 'find' and 'object' imports from govmomi, but
further dependencies may require more bazel tweaks later.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Create a container image just to hold the VDDK.
Mount the VDDK folder to /opt. Also copy the libxcrypt-compat RPM to
/opt for run-time installation as a temporary workaround for my
inability to install it at container creation time. Make VDDK optional
by storing a dummy file in its directory, so an empty VDDK directory
will not break the build.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fill in VDDK data source.
Add the simplest possible VDDK data source: set up an nbdkit server with
the given VMware parameters, and copy data out using qemu-img.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add vddk to resource validation schema.
Stops "error validating data" messages.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Move libxcrypt-compat install out of VDDK image.
Requiring users to add this to the VDDK image is kind of onerous, so
move it straight into the importer image instead.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Generate BUILD.bazel for the rest of govmomi.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Get VDDK sidecar image from v2v-vmware ConfigMap.
Convention from https://github.com/kubevirt/web-ui-components/pull/534
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Reduce nbdkit startup timeout.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Update govmomi dependencies.
Cleans up 'inconsistent vendoring' errors.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fix schema verification test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Change disk.raw to disk.img.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add vCenter simulator for VDDK datasource tests.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Functional test with vcsim and dummy VDDK plugin.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Clean up lint errors.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Check kubevirt-hyperconverged namespace for VDDK.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add VDDK unit tests and API functional test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Read a raw image for VDDK test plugin.
Have the fake VDDK test plugin for nbdkit read a raw image passed in
/opt/updates/nbdtest.img instead of returning fixed data.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add some documentation about VDDK data sources.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove unit test focus and trailing spaces.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Use updated build container image.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add some extra comments about VDDK fields.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Clarify VDDK ConfigMap a little more.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Clarify namespace for vddk-init sidecar.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove NodePort from vCenter simulator manifest.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Re-word explanations on VDDK source types.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add VDDK to comment with types needing URL check.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Rebuild types.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Move AnnThumbprint to storage.import.vddk.thumbprint.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove run-time installation of libxcrypt-compat.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove fixed namespaces for vddk-init reference.
The importer now looks for the v2v-vmware ConfigMap in the current CDI
namespace instead of openshift-cnv/kubevirt-hyperconverged.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Find CDI install namespace from data volume test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
Co-authored-by: Sam Lucidi <slucidi@redhat.com>
* Generate CDI CRD using controller-tools.
This is only done for CDI CRD as it requires the existence of source
code. Other CRDs we create are created by a more bare bones pod.
CDIUninstallStrategy was missing a comment describing it, so add
one. This was spotted manually so there might be more missing.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Allow users to specify which nodes CDI pods will live on.
nodeSelector, affinity and tolerations are possible values.
This is done in the CDI CR (rather than CDIConfig) as we are
interested in having this field be populated by external operators.
Unit tests now require the existence of a CDI CR, so create it.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add a unit test covering some node placement functions
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Specify that all our pods are linux-only.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Avoid duplicate test, accidental left over.
Pointed out by awels, thanks.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Rename to cdiOperatorDeployment for clarity.
Suggested by awels
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Specify we only run on linux using the CDI CR, no need to embed this
into the code.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Don't dereference workloadPlacement for no reason
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Split off operator test to have its own AfterEach, BeforeEach.
Use even more descriptive function names.
Do all the CDI delete/restore logic in AfterEach, to ensure that
it happens and restores the deployment with the original CR even
if the test fails.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Remove XXX. This is the proper way.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Adapt to latest changes in controller_test.go (renaming import)
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Simplify, not storing intermediate value.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Don't dereference nodeplacement in callers to CreateDeployment
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Remove redundant save & restore. Unit tests do this for us.
Pointed out by awels, thanks.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Split out "find toplevel" to a utility function
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Wait for the CDI CR update to apply before continuing.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Simplify, not storing intermediate value.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Make it clear that the chosen node placement will not be schedulable.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Updated dependencies
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Replace skopeo with containers API.
This commit removes dependence on skopeo (binary) and uses containers
API. By doing that we're able to opimize the use of storage (scratch)
space, storage I/O and download bandwith.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Fixing rebase - dependencies kerfuffle.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Handling docker-format images as well as OCI.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Fix for missing code-generator module.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Remove regex, image file in registry images are matched by a path
prefix.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Added nginx proxy in front of docker registry for a rate-limited access.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* move upload.cdi.kubevirt.io API group to v1beta1
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* move core api to v1beta1
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* fix os-3.11 cluster sync and add functional tests for alpha api
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* change more occurences of v1alpha1
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* updates after rebase
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Implement local web server to mimic flawed servers
http://cdi-bad-webserver.cdi/flaky/filename will return a 503
"unavailable" error to most requests.
http://cdi-bad-webserver.cdi/forbidden-HEAD/filename will return
a 403 "Forbidden" error on HEAD requests.
In both those cases, we validate that we still eventually succeed
in creating a datavolume.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add the imageio server to the manual test setup instructions.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add missing bad-webserver.yaml.in (forgot to git add)
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Make "flaky" bad-webserver reproducibly flaky.
9 failures followed by a success.
Switch to port 9090, os-3.11 disallows using privileged ports,
and we don't have a reason to prefer port 80.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Provide polarion test IDs
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Make flaky test server fail less frequently.
The exponential backoff makes 10 retries very slow.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Tolerate being installed on a namespace that isn't "cdi"
Assume that we are in the same namespace as cdi-file-host, and
use that namespace in the URL.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Use AfterEach to cleanup datavolumes after test.
Suggested by Alexander Wels.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
This PR add source of imageio docker images used for functional tests.
Change-Id: Ib9a2a89e69c71a1a059e9a1b2a9b2f80582e520d
Signed-off-by: Ondra Machacek <omachace@redhat.com>
This PR introduces new import data source to fetch disks from ovirt
and store them in storage available in the cluster. It uses ovirt-sdk
to check whether the this is there, the size of the disk and it obtains
signed ticket which allows to get the disk using ovirt-imageio-proxy.
In order to use it a user needs to create configmap with ovirt api
certificate and secret with ovirt engine user and password. Both are
required and validated by webhook. In order to start import process
we need to post following definition:
---
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
name: "imageio-dv"
spec:
source:
imageio:
url: "<engine-api-endpoint>"
secretRef: "<secret-name-containing-engine-credentials>"
certConfigMap: "<configmap-name-containing-engine-api-certificate>"
diskId: "<disk-id-which-we-want-to-import>"
pvc:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: "64Mi"
---
Signed-off-by: Piotr Kliczewski <piotr.kliczewski@gmail.com>
* cut down operator permissions
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* mage sure csv-generator is built but not included in operator image
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* create aggregate cluster roles
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* CDIConfig needs a ClusterRoleBinding so bind to system:authenticated for reading
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* comment out problematic test
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* operator needs upload.cdi.kubevirt.io permission
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* revert commented out tests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Add make tagret docker-olm-catalog
* Add build-olm-catalog script that builds tree expected by operator-registry.
* add catalogsource manifest to deploy operator registry per provider
* os
* k8s
* update olm documentation with opertor-registry deployment
* CDI OLM update support
* Download latest CDI OLM manifest from marketplace to set replaces field
* Prevent from genarting a csv version that already exist in the marketplace
* * Vendor the bellow packages in order to be able to fetch existing CSV
- github.com/operator-framework/operator-marketplace
- github.com/go-openapi/spec
- github.com/go-openapi/validate
- github.com/go-openapi/errors
- github.com/go-openapi/analysis
- github.com/go-openapi/strfmt
- github.com/go-openapi/loads
- github.com/go-openapi/runtime
* lint bug-fix
* CR fixes
* CR typo fixes
* * Split catalogsource/operatorsource/subscription manifests to k8s and os
* Update OLM documentation
* rename bundle-helper.go file
* * change CDI repository name in quay from cdi to cdi-operatorhub
* Update cdi-operator-olm.md
- Generate OLM related manifests for CDI in _out/manifests/release/olm
OLM bundle:
- cdi CSV manifest
- cdi crd manifest
- cdi package manifest
- operatorsource manifest
- subscription manifest
- operatorgroup manifest
- Modify cdi-operator role not to be cluster-admin but more specific
- Move all final manifests to _out/manifests directory and update travis with new manifests location
- Provide API for vendoring CDI OLM manifests generation code
Note:
- OLM CSV update to be supported in a separate PR
- OLM bundle integration in travis is to be supported together with CSV update
