* Deploy alerts infra as part of our installation
Conditionally deploy the infrastructure that is needed to fire alerts for our users
when bad things are happening to CDI.
Testing with `KUBEVIRT_DEPLOY_PROMETHEUS=true`
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Watch and unit test all prometheus related resources
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* add gateway for changing monitoring namespace (rbac purposes)
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* refactor test to check for exact alert name and firing state
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Align pattern of ensuring prometheus resource exists for all
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Remove potential noisy event
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Extract duplicate code to function
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Dont use empty value for prometheus label due to open issue
https://github.com/prometheus-operator/prometheus-operator/issues/4325
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
Added missing tests for change "Explicitly set the storage class name #1936".
Corrected the behavior when storage class is not provided and not available.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
Moves the cloneStrategy up one level from claimPropertySet to
StorageProfile.spec and StorageProfile.status.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Support registry import using node docker cache
The new CRI (container runtime interface) importer pod is created with three containers and a shared emptyDir volume:
-Init container: copies static http server binary to empty dir
-Server container: container image container configured to run the http binary and serve up the image file in /data
-Client container: import.sh uses cdi-import to import from server container, and writes "done" file on emptydir
-Server container sees "done" file and exits
Thanks mhenriks for the PoC!
Done:
-added ImportMethod to DataVolumeSourceRegistry (DataVolume.Spec.Source.Registry, DataImportCron.Spec.Source.Registry).
Import method can be "skopeo" (default), or "cri" for container runtime interface based import
-added cdi-containerimage-server & import.sh to the cdi-importer container
ToDo:
-utests and func tests
-doc
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add tests, fix CR comments
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* CR fixes
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Use deployment docker prefix and tag in func tests
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add OpenShift ImageStreams import support
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add importer pod lookup annotation for image streams
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add pullMethod and imageStream doc
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
When the storage request is specified using the 'storage' API in
a DataVolume, CDI uses Storage Profiles to look up any missing
values that would be required to create the underlying PVC.
If the user omits the storage class parameter in their DV,
CDI assumes the cluster's default storage class will be used.
Since CDI calculates the storage class that should be used in
this case, it should explicitly set that storage class name
in the PVC it creates.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Add long term token (10 years) to pvcs when host assisted cloning between namespaces
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* clone controller should retry if source in use
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* minor refactor if/else -> switch
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Correct the fsOverhead calculation in profile
Calculation needs play well with the actual resize that is done in data-processor
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Properly reverse the calculation for overhead.
Signed-off-by: Alexander Wels <awels@redhat.com>
Co-authored-by: Alexander Wels <awels@redhat.com>
* Avoid trying to get metrics from non-running pod.
If the importer pod is not running, the attempt to get metrics can hit a
30-second HTTP timeout. For multi-stage imports, this causes the
transition to "Paused" to take much longer than it needs to.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Return nil instead of new error.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* CSI Volume Clone for same namespace
CSI Volume Cloning is available on the same namespace and also
works with namespace transfer and volume expansion.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update documentation for CSI Volume Clone
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Cleanup and refactor - extract common code into functions
Remove csi-clone-controller (only set cloneOf annotation)
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Corrects reconcile results
Do not requeue reconciliation loop when not needed.
Mark DV as Failed when the PVC Claim is lost.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Handles PVC recovery from ClaimLost
Make sure that CSI clone continues when target pvc recovers from
ClaimLost to to Bound or Pending.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Code Review improvements
Extracted common code for doCrossNamespaceClone and expandAfterClone, and some updates to comments/cleanups.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
Users don't want 👽 resources in clusters,
and we should also be able to tell if were part of a broader installation.
Note:
- Operator created resources were handled in https://github.com/kubevirt/controller-lifecycle-operator-sdk/pull/18
as these labels will be common to all resources deployed by the HCO.
- Now that the controller is guaranteed to have the labels, we can set env vars
that reference the label values (fieldRef) to spare calling GET on the CR in the controllers.
(thanks mhenriks).
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Refactor: simplify by extracting methods
Prepare for new clone logic - extracted smartClone reconcile functions.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Select clone strategy based on storageProfile
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Changes from CR comments.
A series of small fixes, and cleanups.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Documentation update
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Fix smartclone sometimes not triggering.
Updated tests to use a real image instead of data that is filled.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Refactor getSnapshotClass into two functions
Signed-off-by: Alexander Wels <awels@redhat.com>
* Use constant instead of magic number for size.
Signed-off-by: Alexander Wels <awels@redhat.com>
* force bind for WFFC storage on tests.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Updated based on comments.
Fixed failing functional test.
Signed-off-by: Alexander Wels <awels@redhat.com>
* update deps and bazel
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* fix apidocs and unit tests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* fix generate-verify
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
An error in the logic that updates DV, it would ignore errors
during the update. Also small typo in tests corrected.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* fix synchronization between smart clone and datavolume controller
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* pvc transfer controller should be more aggressive to force binding
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Fix: Compute fs overhead only for fs volumeMode
Correctly compute fs overhead for an effective VolumeMode. Effective, means one that is
computed based on value in storage spec and the storageProfile.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Test: Add more tests for fs overhead
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Add an interface to watch nbdkit logs.
Useful for fishing out various pieces of information. Save VDDK library
version and connected ESX host by appending to the importer pod's
termination message. Turns nbdkit logging up to verbose for VDDK data
sources, so only the last few lines are printed for debugging.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Copy VDDK info from termination message to PVC/DV.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add unit tests for saved VDDK information.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add functional test for VDDK annotations.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fix unit test, forgot to check for nil pvc.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Don't ignore errors updating PVC with VDDK info.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Watch nbdkit with Scanner instead of ReadString.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Move VDDK info test into existing functional test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Make nbdkit stop sequence slightly clearer.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Save VDDK info in regular DV reconciler.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Don't save VDDK info when PVC is being deleted.
Also, piggyback off existing PVC update instead of introducing a new
error handling path.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fix VDDK-info unit tests.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Use scanner for all nbdkit logging.
Also fix up a minor merge mistake.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Try to satisfy complaints from SonarCloud.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* use namespace transfer for smart clone
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* updates from test failures
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* add expansion func tests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* add dv phases for expansion and transfer
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* rebase and integrate with storage profiles
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Create new Storage type
A new Storage type similar to the PVC Spec is now available to use
in the DataVolume Spec. This is more permissive than PVC, and together
with StorageProfile this allows CDI to apply additional logic for
missing or optional fields.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Use the StorageProfile
Handle the StorageProfile recommended params when creating the PVC for
a DataVolume. When parameters like volumeMode or accessModes are
not provided, CDI checks the StorageProfile for a given StorageClass
to set the recommended defaults. This enables user to create DataVolume
without the need to provide all the parameters.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Allow multiple accessModes
CDI allows multiple access modes to be specified in the DataVolume.spec.storage and in the StorageProfile. This now works the same way as in PVC specification.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Handle the storage.size field
The storage.size specifies how much space a user wants to have.
When creating image on the fileSystem storage CDI takes into
account the file system overhead and requests PVC big enough to
fit an image and file system metadata.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Test storage profile with DV
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Document Storage Profiles
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Refactor: Render the effective PVC early
The helper 'render PVC' was moved earlier in the control flow, so
it can be used in more places. Removing the need for if/else logic.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Test handling size on import, upload and clone
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Code Review: Refactor resolving of volumeMode
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Fix: render target pvc spec correctly in smart clone controller
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Cluster-scoped namespace transfer api and controller
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* unit tests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* ObjectTransfer webhook
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* new functests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* experiment with termination grace period
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* quota test
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Have the healthz server check for the uploadserver first.
Uploadserver and healthz run in separate goroutine, so we need
to synchronize them. This seems to be the easiest way.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Make healthz unit test check for failure.
It doesn't appear to be possible to use httptest to test two servers,
and we now tie the health report to the upload server.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* validate source and target pvcs has the same content type - for all clones
* if source and target volume modes are different, validate content type is kubevirt, and fallback to host-assisted cloning
* unit & func tests
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* [WIP] doc: User-facing doc for preallocation support
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* apis: CDI accepts `preallocation` option.
With this commit CDI accepts (but does handle) `preallocation` settings
for DataVolumes and in CDIConfig.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* core: Implementing preallocation
This commit implements preallocation support for import and upload.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* test: Functional tests for preallocation support
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* core: Remove "preallocation for StorageClasses" config
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* test: Removed unused function
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* test: Fix rook-ceph test failures
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Updated dependencies
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* core: Uss PVC annotation to pass preallocation parameters
DataVolume controller now uses a PVC annotation to pass preallocation
configuration to import and update controllers.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Plumb new checkpoint API through to VDDK importer.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add incremental data copy from VDDK.
Create a new data source implementation similar to vddk-datasource, but
only for blocks of data that changed between two snapshots. Also factor
out common things between the two VDDK data sources.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Check block status for warm and cold imports.
Addresses a bunch of runtime issues, but progress tracking isn't right.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Find snapshots correctly.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove separate warm/cold VDDK importers.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Advance through the checkpoint list in the spec.
Move DataVolume to Paused after each checkpoint, and start a new
importer pod for the next available checkpoint. Keep track of which
checkpoints have been copied by adding PVC annotations associating each
checkpoint with the UID of the pod that copied it.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Allow spec updates to drive multi-stage imports.
A multi-stage import can create checkpoints at any time, so CDI needs to
be able to receive updates to the list of checkpoints. Implement this by
allowing spec changes only for fields related to multi-stage imports.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Avoid deleting destination in multi-stage import.
A multi-stage import will have an initial data copy to the destination
file followed by separate copies for individual deltas. The destination
file should not be deleted before starting these delta copies.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Get VDDK data source to pass formatting tests.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Unit tests for multi-stage import admission rules.
Make sure only updates to checkpoint-related fields are accepted.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add warm import unit tests for VDDK data source.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add VDDK warm import functional test.
Put two snapshots in the vCenter simulator inventory, and run them
through a multi-stage import process. Also clean up some issues
reported by test-lint.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add some documentation about multi-stage imports.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Pass existing multi-stage DataVolume unit tests.
Also remove MD5 sum step used for debugging, since it can take a long time.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove tabs from documentation.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Pass failing import-controller unit test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* More unit tests for multi-stage field updates.
Also factor these tests into a DescribeTable.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add nbdkit retry filter.
Available as of Fedora 33 update.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Give correct file name to nbdkit in more cases.
The backing file in the spec might not always match the backing file in
the snapshot, so try harder to match those files by disk ID. May still
need to allow updates to backingFile, depending on how this gets used.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add more unit tests for datavolume-controller.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fix linter error from last commit.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add unit tests for some govmomi API calls.
Move original calls into mock interfaces to make this work.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add an API for disabling smart-cloning.
We used to detect the possibility of smart-cloning and always use it
if it's there. This might not be the desirable behaviour if:
- Snapshots cost more money than a host-assisted clone
- Snapshots are broken
The API is:
kubectl edit cdi
cdi.Spec.cloneStrategyOverride = "copy"
If no value is chosen, we continue with the existing behaviour of
preferring smart clone if possible.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Remove redundant parentheses, don't open code GetActiveCDI
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add const CloneStrategySnapshot to v1alpha1 too
Pointed out by awels, thanks.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add unit tests for getCloneStrategy
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* create DV if PVC is marked as deleted
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* tests/datavolume_test.go
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* Add checkpoints to DataVolume CRD and reconciliation
* Add Previous, Current, and FinalCheckpoint to DataVolume CRD
* Use checkpoints to set annotations on the PVC
* If an importer pod succeeds while checkpoint annotations are set,
then set the DataVolume status to Paused intstead of Succeeded.
* Clear the PVC checkpoint annotations
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Add new fields to DataVolume CRD creation
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Generate updated code for the DataVolume changes
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Add tests for multistage import annotations
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* When validating disk space, reserve space for filesystem overhead
The amount of available space in a filesystem is not exactly
the advertise amount. Things like indirect blocks or metadata
may use up some of this space. Reserving it to avoid reaching
full capacity by default.
This value is configurable from the CDIConfig object spec,
both globally and per-storageclass.
The default value is 0.055, or "5.5% of the space is
reserved". This value was chosen because some filesystems
reserve 5% of the space as overhead for the root user and
this space doubles as reservation for the worst case
behaviour for unclear space usage. I've chosen a value
that is slightly higher.
This validation is only necessary because we use sparse
images instead of fallocated ones, which was done to have
reasonable alerts regarding space usage from various
storage providers.
---
Update CDIConfig filesystemOverhead status, validate, and
pass the final value to importer/upload pods.
Only the status values controlled by the config controller
are used, and it's filled out for all available storage
classes in the cluster.
Use this value in Validate calls to ensure that some of the
space is reserved for the filesystem overhead to guard from
accidents.
Caveats:
Doesn't use Default: to define the default of 0.055, instead
it is hard-coded in reconcile. It seems like we can't use a
default value.
Validates the per-storageClass values in reconcile, and
doesn't reject bad values.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Use util GetStorageClassByName
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Test filesystem overhead validation against async upload endpoint
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* wait for NFS PVs to be deleted before continuing
Intended to help with flakes, but didn't make a difference.
Probably still worth doing.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Avoid using the uncached client unnecessarily
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add error handling for the case where even a default SC is not found
Note that this change isn't expected to make a difference, as we
check if the targetStorageClass is nil later on and have the same
behaviour, but this is probably more correct API usage.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add testing for the validation of filesystem overhead values
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Fix logical error in waiting for NFS PVs.
Wait for all of them, not just the last one.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add library function to determine if a PVC has been populated fully.
The logic is as following:
If PVC has no ownerRef, then we assume something else fully populated it and
will return true
If PVC has an ownerRef and its a DataVolume, then look up the DataVolume
If DV.status.Phase == succeeded, return true, return false otherwise.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Renamed functions to better indicate its purpose.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Add support for a VDDK datasource to the controllers
This commit does not implement the VDDK datasource. It simply
adds the DataVolumeSourceVDDK type and adds support to the
datavolume and import controllers for it. The datasource itself
will need to be done in a follow-up.
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Fix tests and clean up lint
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Add DataVolumeSourceVDDK to the v1alpha1 api group
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Install nbdkit.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add govmomi to vendor directory.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Integrate govmomi into build.
Incomplete! This allows 'find' and 'object' imports from govmomi, but
further dependencies may require more bazel tweaks later.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Create a container image just to hold the VDDK.
Mount the VDDK folder to /opt. Also copy the libxcrypt-compat RPM to
/opt for run-time installation as a temporary workaround for my
inability to install it at container creation time. Make VDDK optional
by storing a dummy file in its directory, so an empty VDDK directory
will not break the build.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fill in VDDK data source.
Add the simplest possible VDDK data source: set up an nbdkit server with
the given VMware parameters, and copy data out using qemu-img.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add vddk to resource validation schema.
Stops "error validating data" messages.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Move libxcrypt-compat install out of VDDK image.
Requiring users to add this to the VDDK image is kind of onerous, so
move it straight into the importer image instead.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Generate BUILD.bazel for the rest of govmomi.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Get VDDK sidecar image from v2v-vmware ConfigMap.
Convention from https://github.com/kubevirt/web-ui-components/pull/534
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Reduce nbdkit startup timeout.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Update govmomi dependencies.
Cleans up 'inconsistent vendoring' errors.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fix schema verification test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Change disk.raw to disk.img.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add vCenter simulator for VDDK datasource tests.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Functional test with vcsim and dummy VDDK plugin.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Clean up lint errors.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Check kubevirt-hyperconverged namespace for VDDK.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add VDDK unit tests and API functional test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Read a raw image for VDDK test plugin.
Have the fake VDDK test plugin for nbdkit read a raw image passed in
/opt/updates/nbdtest.img instead of returning fixed data.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add some documentation about VDDK data sources.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove unit test focus and trailing spaces.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Use updated build container image.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add some extra comments about VDDK fields.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Clarify VDDK ConfigMap a little more.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Clarify namespace for vddk-init sidecar.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove NodePort from vCenter simulator manifest.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Re-word explanations on VDDK source types.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add VDDK to comment with types needing URL check.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Rebuild types.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Move AnnThumbprint to storage.import.vddk.thumbprint.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove run-time installation of libxcrypt-compat.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove fixed namespaces for vddk-init reference.
The importer now looks for the v2v-vmware ConfigMap in the current CDI
namespace instead of openshift-cnv/kubevirt-hyperconverged.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Find CDI install namespace from data volume test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
Co-authored-by: Sam Lucidi <slucidi@redhat.com>
consistent failed should never happen during normal operations,
it can potentially happen if a pvc claim is lost.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Set the WaitForFirstConsumer phase on DataVolume when storage uses the WaitForFirstConsumer binding mode and is not bound yet.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Skip PVC if not bound in import|clone|upload controllers.
This is done so the VM pod(not the cdi pod) will be the first consumer, and the PVC can be scheduled on the same location as the pod.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
fixup! Skip PVC if not bound in import|clone|upload controllers.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update importer tests to force bind the PCV by scheduling a pod for pvc, when storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update datavolume tests to force bind the PCV by scheduling a pod for pvc, when storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update upload controller and upload tests to correctly handle force binding the PCV by scheduling a pod for pvc, when storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update clone tests to force bind the PCV by scheduling a pod for pvc when the storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update cloner multi-node tests to force bind the PCV by scheduling a pod for pvc when storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Correct after automerge
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Improve/simplify tests
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Fix error in import test.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update transport_test,operator_test.go
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update rbac_test.go and leaderelection_test.go
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Improve Datavolume and PVC Checks for WFFC.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Handle wffc only if feature gate is open - import-controller
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* TEST for Handle wffc only if feature gate is open - import-controller - TEST
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Handle wffc only if feature gate is open - upload-controller with test
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* rename and simplify checks
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* cleanup after rebase
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* update tests after rebase
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* update tests after rebase
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* more cleanups
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Document new WFFC behavior
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Document new HonorWaitForFirstConsumer option
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* update docs according to comments
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* extract common function, cleanup - code review fixes
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* add comment for another pr - 1210, so it can have easier merge/rebase
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* typo
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Simplify getStoragebindingMode - code review comments
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Add FeatureGates interface - code review fix
Additionally pass the features gates instead of the particular feature gate value,
and let shouldReconcilePVC decide what to do with the feature gate. That way shouldReconcilePVC
contains all the logic, and the caller does not need to do additional calls to provide parameters.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update matcher
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* move upload.cdi.kubevirt.io API group to v1beta1
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* move core api to v1beta1
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* fix os-3.11 cluster sync and add functional tests for alpha api
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* change more occurences of v1alpha1
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* updates after rebase
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* don't create snapshot or clone pods if pvcs in use
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* cleanup pods during functional tests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* kill mmore pods blocking clone tests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* fix typos
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Handle labels length correctly
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Handle service name generation correctly
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Remove not needed labels
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Store import pod name in annotation
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Enable long DV name
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Handle name with dot when creating service/label name
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Test long names on import, upload and clone
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Store upload pod name in annotation
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Store importer scratch pvc name in annotation
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Quick fix for tests (need improvements)
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Cleanup handling scratch name
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Ensure pod/service name conflicts are handled
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Handle client errors when trying to get the import pod
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Style improvements, and other code review fixes.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Store clone source pod name in an annotation
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Correct name initialization and tests
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Do not init name if pod already exists. It is not needed.
The situation of having a pod but not name on annotation can happen after the upgrade, when we have a legacy pvc and pod already existing, but clone operation not finished. But when we already have the pod, then in the code (currently) we do not need the name from annotation.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Cleanup scratch name handling
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Use constant for max dv name in validation
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Simplify clone source pod name initialization
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* move from alpha to beta snapshot API
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* fix broken clone tests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* don't generate snapshot client
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Add conditions to DataVolume and add condition annotation to PVCs
Signed-off-by: Alexander Wels <awels@redhat.com>
* Added reason codes to conditions
Updated functional tests to use reason codes
Fixed broken functional test.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Added last transition time
Added documentation
Signed-off-by: Alexander Wels <awels@redhat.com>
* Switch to conditions slice from conditions pointer slice.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Add heartbeat time for conditions.
Renamed AnnRunningCondition to AnnLastTerminationMessage
Only update reason/message on heartbeat or state change.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Rename heartBeat to heartbeat
Re-factor datavolume-conditions to properly update heartbeat time.
Signed-off-by: Alexander Wels <awels@redhat.com>
* renamed AnnRunningConditionReason to AnnLastTerminationReason
Signed-off-by: Alexander Wels <awels@redhat.com>
* Revert lastTerminatedMessage/Reason rename
Added separate AnnScratchBoundCondition/Message/Reason for scratch space
Signed-off-by: Alexander Wels <awels@redhat.com>
* Update bound condition to take scratch or source PVC status into consideration.
Update running condition to take source pod status into consideration
Signed-off-by: Alexander Wels <awels@redhat.com>
* Addressed comments in review.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Properly get scratch space pvc phase.
Signed-off-by: Alexander Wels <awels@redhat.com>
