* remove root worker pods
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* remove selinux requirement for worker pods
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* run tests in restricted namespace and required changes
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* handle empty tar
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* add PSA label when running functional tests in OpenShift
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* cannot use restricted PSA with istio (for now)
refactor scc management
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* fix clean script
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Improve the error handling when pod creation fails
When pod creation fails, the error is usually logged without providing additional information to the user. This behavior is especially risky when the user lacks the permits to check the logs, making it unintuitive and almost impossible to find the source of the problem.
This commit improves the error handling of the pod-creation process, so pertinent info about the failure is included in the pod's PVC.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Update functional tests to check for events when pod-creation fails
Since error handling in pod-creation has been improved in our controllers, this commit introduces several changes in the corresponding functional tests to properly cover the new behavior included when pod-creation fails.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Update unit-tests after improving error-handling of pods for proper coverage
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Minor fixes and improvements on error handling for pods
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Modify datavolume-controller to change the running condition of datavolumes when a pod fails
Until this commit, the way of handling pod errors in the datavolume-controller has been to change the affected datavolume's phase to failed, which conflicts with the declarative approach of the controllers.
This commit modifies this behavior so that, when a pod fails, the affected datavolume's running condition is changed to false while the phase remains unchanged.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Add TLS Security Profile API
TLSSecurityProfile is used by operators to apply cluster-wide TLS security settings to operands.
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Update apiserver & uploadproxy server TLS config on CDIConfig TLS knob change
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Propagate TLS config to uploadserver as well
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Add functests for apiserver and upload that ensure value is respected
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Comply with restricted security context in kubernetes
Ensure CDI pods comply with the restricted security context as much as
possible (have to be root for nbdkit and block devices). Also cannot set
SeccompProfile since SCC won't allow us to set it.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Changed path /var/local/all_certs to stay in /var
Signed-off-by: Alexander Wels <awels@redhat.com>
* Allow empty DV size when cloning using storage API
When cloning a Data Volume, the size of the target can be potentially obtainable via the source PVC, which discards the need to explicitly specify it.
Considering that, this commit introduces a change in the correspondent validation webhook to allow omitting the resources.request.storage field when cloning a PVC using the storage API.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Modify datavolume-controller to allow obtaining storage size from source PVC when cloning
When cloning a PVC, if the target's size is not specified, said value can be attainable from the source PVC.
This commit introduces a change in datavolume controller so, in case of detecting an empty storage size, said value can be obtained when performing CSI and Smart cloning.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Update unit tests for datavolume-validation after enabling cloning with empty size
This commit updates the unit testing for the datavolume validation webhook, covering the possibility of cloning a PVC without setting any storage size.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Update unit testing for controller-related functions after enabling cloning with empty size
This commit includes unit tests for the volumeSize() function after enabling creating clones with blank size.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Update the datavolume controller to create a size-detection pod when performing host-assisted clone
When performing a host-assisted clone with empty clone size, simply copying the original PVC size could lead to potential overhead miscalculations if the source's VolumeMode is "filesystem".
When that's the case, an inspection pod will be created in the datavolume controller so it extracts the size of the virtual image using qemu-img.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Include an image-size detection tool to allow cloning with empty DV size
This commit introduces a new tool in charge of collecting the virtual image size when cloning with an empty DV size. In some cases where said value is unattainable from the original PVC's spec, the datavolume controller will create a new pod containing this new tool.
The binary will then run the 'qemu-img' command and handle its results appropriately.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Optimize the clone-size lookup process to avoid creating unnecessary size-detection pods
When performing host-assisted clone with an empty DV size, in some cases, a size-detection pod is used to obtain the required capacity.
This commit tries to optimize this process to keep the collected value as a PVC annotation, that is checked in subsequent clones to avoid creating redundant pods.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Minor fixes and improvements on mechanism for cloning with empty storage size
* Add new optional flag on size-detection binary to enable using a different URI scheme
* Improve the pod-creation mechanism so the pod is not created until the source PVC has finished the import
* Modify size-finlation mechanism to account for possible round-downs when importing the source image
* Improve the size inflation mechanism so only PVCs with filesystem as volume mode are considered
* Minor style corrections
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Modify the clone-controller to allow skipping the clone size validation in some cases
Due to filesystem overhead differences, the target's size can sometimes be smaller than the source's one when obtaining said value with the size-detection pod.
This commit introduces minor changes in the clone-controller so we can skip the size validation in those cases.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Minor changes and improvements in size-detection mechanism following PR review
* Added new UT that covers using empty storage API for non-cloning sources
* Added new watch on datavolume-controller that looks for changes in the size-detection pod
* Removed redundant and unnecessary specs on size-detection pod
* Added error handling when reading the pod's termination message
* Moved general-usage functions to 'util.go' file
* Updated 'datavolumes' documentation to reference the possibility of omitting the storage size when cloning
* Minor style corrections
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Add unit tests that cover the size-detection mechanism in the DataVolume controller
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Include functional tests for cloning without specifying storage size
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Improve error handling in the creation/deletion process of the size-detection pod
This commit introduces additional handling in case of error after and during the size-detection pod is created.
It also updates several related unit tests.
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Minor fixes to improve fsOverhead calculations when cloning with empty storage size
* Modified the size-detection mechanism so we account for fsOverhead when cloning to filesystem volume mode in all cases
* Clean up the code for reconciling when cloning a PVC that is not ready
* Minor fix in functional test so it works when cloning from block to filesystem volume mode
Signed-off-by: Alvaro Romero <alromero@redhat.com>
* Introduce controller-runtime-sdk api package
Split controller-runtime-sdk into the base package and
controller-runtime-sdk/api.
Signed-off-by: Roman Mohr <rmohr@redhat.com>
* go mod vendor
Signed-off-by: Roman Mohr <rmohr@redhat.com>
* Update code references
Signed-off-by: Roman Mohr <rmohr@redhat.com>
* Create imageio container during CDI build.
Instead of using a really old imageio, use bazel to build a new
imageio based on 2.5.0. Update the tests to use the new image
and paths in that new image. This requires a new repo in quay for
us to push the image to.
Also changed the approach of resolving the warm import potential
dead lock (scratch PVC from previous import pod terminating, while
the new pod is trying to create itself). Instead of trying to avoid
in all scenarios, detect the state, and delete the pod so the dead
lock can be resolved.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Populate test images
Signed-off-by: Alexander Wels <awels@redhat.com>
* Enable disabled test, and fix race condition where the import
controller thought it was done, but we were still on the final
import of a warm migration.
Updated the way we create the ticket on the fake imageio
Signed-off-by: Alexander Wels <awels@redhat.com>
* Append checkpoint ID to multi-stage importer pods.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Ignore completed pods for multi-stage imports.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Reset current import pod when checkpoint is done.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Don't prevent pod deletion for scratch space.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Only ignore pod when retainAfterCompletion is set.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fix data volume unit tests.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Tests for checkpoint suffix and completed pods.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Test for retained pods exiting for scratch space.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add functional test for retaining multistage pods.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Clean up lint error.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove scratch handling that is fixed elsewhere.
This is part of shouldDeletePod now.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add unit tests for long PVC/checkpoint names.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Match retainAfterCompletion test to description.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fix failing imageio test
Always enable scratch space for imageio imports.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Only always enable scratch space with warm migration imports.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Set htpp(s)_proxy to lower case env variable
CURL used by nbdkit doesn't read upper case http(s)_proxy environment
variables, and thus was not using the proxy. Changed the variable to
be lower case.
Added a significant number of tests to test many more variations of
using a proxy. Also added https + auth endpoint to the file-host
container, so we can test https + auth with the proxy.
Added https endpoint to proxy, so we can test an https proxy.
Cleaned up some of the error handling in the import controller for
the proxy, in particular if a trustedCAProxy is defined.
Fixed some of the cluster wide proxy configuration so it works properly
inside an openshift cluster.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Add https proxy support to registry import. Added extra
functional tests to test all registry import combinations
Signed-off-by: Alexander Wels <awels@redhat.com>
* Fixed some tests to work better in Open Shift.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Add optional VDDK initImageURL field.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Pass VDDK image URL through to PVC annotation.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Unit tests for per-DV VDDK image URL.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Functional test for VDDK initImageURL field.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Update documentation for VDDK initImageURL.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fix lint error.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Check for absence of AwaitingVDDK in unit test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Update datavolume conditions when quota exceeded when creating pvc
When creating the pvc from the dv the pvc size
can exceed the allowed quota, in such case so far the only
indication was to look in the logs.
Now added indication in the data volume conditions
(when possible) and emitted event.
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Add functional tests to check the new conditons and event
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* tests cosmetics
-use existing functions
-add missing checks on errors
-remove unused code
-etc..
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Update HTTP data source API to allow custom headers.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Implement custom HTTP headers API.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Document custom headers in HTTP data source.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Correct secretExtraHeader comment to reference Secret.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add volume mounts for secret headers.
Replaces environment variables for headers from secrets.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Avoid failing when there are no extra headers.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Redact contents of headers that come from secrets.
Also split up getExtraHeaders to reduce Sonar Cloud complexity.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Ensure all HTTP client requests use extra headers.
Missed redirect check and content length retrieval, both of which might
need the extra headers.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add some unit tests for extra HTTP headers.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Do not quote headers in nbdkit curl arguments.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add functional tests for extra HTTP headers.
Avoids new test server by specifiying basic authorization headers to the
existing file host port that requires it.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Use filepath.Walk to read secrets.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Minor documentation update for secrets.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Re-run 'make generate' for verification failure.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add DataImportCron controller
-The new controller polls for updates to a registry source container
image, based on a given schedule. When updates to a container image are
detected, the controller imports the content into a new uniquely named
PVC in a golden image namespace.
-For each DataImportCron, the controller manages a corresponding
DataSource to always point to the latest most up-to-date golden
image PVC.
-DataImportCron takes ownership of an existing DataSource (with
controller: false), allowing an admin to opt-in to using auto
delivery/updates later on.
-The controller has PVC garbage collector removing old PVCs.
ToDo:
-status conditions updates
-verify full image streams support
-utests and func tests
-fixmes and commented out code
-doc
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Fix CR comments and fixmes
- isolate imagestream and registry specific code
- fix namespace of CronJob, and its job and pod to CDI namespace
- manage CronJob-DataImportCron ownership relationship with a finalizer,
handle DataImportCron deletion (CronJob etc.)
- remove CronJob and job pod for ImageStreams, use RequeueAfter and
cronexpr instead
- add k8s app cdi-source-update-poller executed by CronJob to poll source
image digest via skopeo inspect for url registry source, and annotate
the DataImportCron when the image was updated and pending for import based
on the cron schedule
- add cdi-source-update-poller and skopeo binary to the cdi-importer container
- complete dataimportcron-validate and its tests
- reconcile - use context.Context instead of context.TODO
- remove uncached client
- doc
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Fix ImageStreams watch
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add DataImportCron DV template instead of source
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Fix CR comments
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Split updateSucceeded func
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Improve cdi-source-update-poller cmd logs
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Remove ImageStream reconcile
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Remove ImageStream watch
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Remove unnecessary AnnSourceUpdatePending
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* More CR fixes
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Idempotentify initCron
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Recreate DV in case is't not found
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add DataImportCron spec.importsToKeep and status.currentImports
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add DataImportCron controller functional test
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add insecure TLS support
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Remove finalizers in cluster clean script
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Bound each import to its sha256 digest instead of latest
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add DataImportCron controller utests
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Tests CR fixes
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Minor tests CR fixes
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* move apis to new staging area
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* add script to push to staging
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* fix lint check and api reference
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* push staging to api repo
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Deploy alerts infra as part of our installation
Conditionally deploy the infrastructure that is needed to fire alerts for our users
when bad things are happening to CDI.
Testing with `KUBEVIRT_DEPLOY_PROMETHEUS=true`
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Watch and unit test all prometheus related resources
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* add gateway for changing monitoring namespace (rbac purposes)
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* refactor test to check for exact alert name and firing state
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Align pattern of ensuring prometheus resource exists for all
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Remove potential noisy event
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Extract duplicate code to function
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Dont use empty value for prometheus label due to open issue
https://github.com/prometheus-operator/prometheus-operator/issues/4325
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Support registry import using node docker cache
The new CRI (container runtime interface) importer pod is created with three containers and a shared emptyDir volume:
-Init container: copies static http server binary to empty dir
-Server container: container image container configured to run the http binary and serve up the image file in /data
-Client container: import.sh uses cdi-import to import from server container, and writes "done" file on emptydir
-Server container sees "done" file and exits
Thanks mhenriks for the PoC!
Done:
-added ImportMethod to DataVolumeSourceRegistry (DataVolume.Spec.Source.Registry, DataImportCron.Spec.Source.Registry).
Import method can be "skopeo" (default), or "cri" for container runtime interface based import
-added cdi-containerimage-server & import.sh to the cdi-importer container
ToDo:
-utests and func tests
-doc
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add tests, fix CR comments
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* CR fixes
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Use deployment docker prefix and tag in func tests
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add OpenShift ImageStreams import support
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add importer pod lookup annotation for image streams
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
* Add pullMethod and imageStream doc
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
Users don't want 👽 resources in clusters,
and we should also be able to tell if were part of a broader installation.
Note:
- Operator created resources were handled in https://github.com/kubevirt/controller-lifecycle-operator-sdk/pull/18
as these labels will be common to all resources deployed by the HCO.
- Now that the controller is guaranteed to have the labels, we can set env vars
that reference the label values (fieldRef) to spare calling GET on the CR in the controllers.
(thanks mhenriks).
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Strip newlines when writing a termination message.
Otherwise it isn't visible, at least when viewing in the -o yaml view.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Write down the nbdkit output and add it to the error output
With the added output from nbdkit, we can see the reason for the
non-existence of the nbdkit socket.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Don't set that we're waiting if it's CrashLoopBackOff
It's better to have the reason for the crash (terminate message)
than "backing off 5 minutes"
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Simplify all "image too large to fit" terminate reasons.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Move verifyConditions to utils, no functional change
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Switch test for image too large to test condition and not log
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Remove unused branch
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Rename setConditionFromPodWithPrefix to setAnnotationsFromPodWithPrefix
No functional change. Intended to be followed by some refactoring.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Fold restart count logic into the common setAnnotationsFromPodWithPrefix
Changing to >= rather than > to ensure a zero pod restart count is
always used -- the import controller unit tests request this.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Reduce indentation by returning right away.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Fold check for pod being nil into common code.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* move saveVddkAnnotations into util and make it unconditional
Call it setVddkAnnotations for consistency.
Check for not-terminated inside the function, not outside.
Removes check for source being VDDK (to avoid passing more arguments):
it won't match the regex anyway.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Reduce indentation by bailing on failure.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Reorder parameters to mirror the order in the function name
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Use a named variable for first container state
Yields shorter, more legible lines.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Use a constant for the nbdkit log.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add more information to function description - also logs to file
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Move AwaitingVDDK constant to common.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Copy pending PVC bound condition reason to DV.
Replace the fixed "Pending" string and tweak the unit test that checked
this.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add a functional test for AwaitingVDDK.
Delete v2v-vmware ConfigMap and create a DataVolume, and the bound
condition should have a reason of "AwaitingVDDK".
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Move AwaitingVDDK to its own functional test.
Retain common test code with existing table, but tack on an extra
cleanup step so v2v-vmware ConfigMap can be restored afterward.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* update deps and bazel
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* fix apidocs and unit tests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* fix generate-verify
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Add an interface to watch nbdkit logs.
Useful for fishing out various pieces of information. Save VDDK library
version and connected ESX host by appending to the importer pod's
termination message. Turns nbdkit logging up to verbose for VDDK data
sources, so only the last few lines are printed for debugging.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Copy VDDK info from termination message to PVC/DV.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add unit tests for saved VDDK information.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add functional test for VDDK annotations.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fix unit test, forgot to check for nil pvc.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Don't ignore errors updating PVC with VDDK info.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Watch nbdkit with Scanner instead of ReadString.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Move VDDK info test into existing functional test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Make nbdkit stop sequence slightly clearer.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Save VDDK info in regular DV reconciler.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Don't save VDDK info when PVC is being deleted.
Also, piggyback off existing PVC update instead of introducing a new
error handling path.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fix VDDK-info unit tests.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Use scanner for all nbdkit logging.
Also fix up a minor merge mistake.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Try to satisfy complaints from SonarCloud.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Refactoring - move PreallocationApplied flag definition to common
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Preallocate cloning DataVolumes
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Cluster-scoped namespace transfer api and controller
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* unit tests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* ObjectTransfer webhook
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* new functests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* experiment with termination grace period
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* quota test
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* api: Add new ImportProxy structure to CDIConfig in v1beta1 core api
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* common: Add constants related to configure Import proxy in Import pod and controller reconcile
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* config controller: Add import proxy reconcile, monitoring OCP proxy updates
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* config controller: Add unit test for import proxy reconcile, monitoring OCP proxy updates and creating Secret and ConfigMap
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* import controller: Add unit support of ImportProxy in the tests
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller util: Add support to retrieve an OpenShift cluster wide proxy object as well as its needed configurations
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller util: Add unit test for the get cluster wide proxy functionality
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller: moved isOpenshift function to utils common and export it
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* importer: Clone http transport to keep the default proxy config such as the usage of environment variables and add support for proxy CA
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* operator: Update CDIConfigCRD with import proxy info
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* operator: Update CDIListCRD with import proxy info
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* operator: Add service account rights to CDI for accessing OpenShift proxy obj
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* operator: Add OpenShift proxy obj scheme support
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: Add new forwarding proxy for testing things that require a proxy
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: Add test to verify the importer pod with proxy
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* doc: Update cdi config doc with the ImportProxy info
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* add support to build the new proxy docker image
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: remove unwanted F parameter from test
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller: fix error handling
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: update the used method to get the pod of the importer pod and proxy
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* fixed comments from the revision
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller config: update the methods to use the log object from the reconcile object
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller: update function GetImportProxyConfig to return errors and include a new unit test
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* controller: add back a test that was removed by mistake
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* update updateCDIConfigByUpdatingTheClusterWideProxy function
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: decrease the import pod log pooling interval and increase the image size to be able to verify the import pod before it is deleted
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: update the pvc and proxied requests verification
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* tools: add X-Forwarded-For header in the proxy requests used for testing
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: reset cluster wide proxy with the original values
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* test: fix proxy update to change spec instead of status and other minor updates
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* doc: update import proxy description
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
* update generated files
Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
This PR removes "skipped" condition for preallocation. Importer/uploader
will preallocate to the available size. Filesystem overhead needs to be
taken into account.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Simplify shouldReconcile function arguments.
By having the function itself grab things it needs and are easily
obtained.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Adapt unit tests to simpler shouldReconcilePVC
Don't set any feature gates for WFFC being disabled.
When the second argument is true, pass the immediate binding annotation
to the PVC itself.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
Assists resolution of BZ#1886566. Use existing mechanism to copy certain
PVC annotations into DV status conditions.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add a DV/PVC annotation "storage.bind.immediate.requested"
When the annotation is applied the CDI will force bind the PVC (by scheduling worker pods), ignoring the logic to handle WaitForFirstConsumer binding mode.
This is useful when uploading "template" images to the cluster on local storage with WaitForFirstConsumer binding. In this case the image has to be available somewhere on the cluster, the actual placement of image on specific node is not important, so the CDI worker node can be used as a first consumer.
For storage with immediate binging it is effectively a NOOP.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Test import/upload/clone with annotation "storage.bind.immediate.requested"
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* [WIP] doc: User-facing doc for preallocation support
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* apis: CDI accepts `preallocation` option.
With this commit CDI accepts (but does handle) `preallocation` settings
for DataVolumes and in CDIConfig.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* core: Implementing preallocation
This commit implements preallocation support for import and upload.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* test: Functional tests for preallocation support
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* core: Remove "preallocation for StorageClasses" config
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* test: Removed unused function
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Updated dependencies
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* core: Uss PVC annotation to pass preallocation parameters
DataVolume controller now uses a PVC annotation to pass preallocation
configuration to import and update controllers.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* core: Implementing preallocation
This commit implements preallocation support for import and upload.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* core: Make sure all import/upload paths do preallocation
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* [WIP] doc: User-facing doc for preallocation support
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* apis: CDI accepts `preallocation` option.
With this commit CDI accepts (but does handle) `preallocation` settings
for DataVolumes and in CDIConfig.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* core: Implementing preallocation
This commit implements preallocation support for import and upload.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* test: Functional tests for preallocation support
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* core: Remove "preallocation for StorageClasses" config
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* test: Removed unused function
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* test: Fix rook-ceph test failures
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Updated dependencies
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* core: Uss PVC annotation to pass preallocation parameters
DataVolume controller now uses a PVC annotation to pass preallocation
configuration to import and update controllers.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Plumb new checkpoint API through to VDDK importer.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add incremental data copy from VDDK.
Create a new data source implementation similar to vddk-datasource, but
only for blocks of data that changed between two snapshots. Also factor
out common things between the two VDDK data sources.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Check block status for warm and cold imports.
Addresses a bunch of runtime issues, but progress tracking isn't right.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Find snapshots correctly.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove separate warm/cold VDDK importers.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Advance through the checkpoint list in the spec.
Move DataVolume to Paused after each checkpoint, and start a new
importer pod for the next available checkpoint. Keep track of which
checkpoints have been copied by adding PVC annotations associating each
checkpoint with the UID of the pod that copied it.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Allow spec updates to drive multi-stage imports.
A multi-stage import can create checkpoints at any time, so CDI needs to
be able to receive updates to the list of checkpoints. Implement this by
allowing spec changes only for fields related to multi-stage imports.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Avoid deleting destination in multi-stage import.
A multi-stage import will have an initial data copy to the destination
file followed by separate copies for individual deltas. The destination
file should not be deleted before starting these delta copies.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Get VDDK data source to pass formatting tests.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Unit tests for multi-stage import admission rules.
Make sure only updates to checkpoint-related fields are accepted.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add warm import unit tests for VDDK data source.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add VDDK warm import functional test.
Put two snapshots in the vCenter simulator inventory, and run them
through a multi-stage import process. Also clean up some issues
reported by test-lint.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add some documentation about multi-stage imports.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Pass existing multi-stage DataVolume unit tests.
Also remove MD5 sum step used for debugging, since it can take a long time.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove tabs from documentation.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Pass failing import-controller unit test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* More unit tests for multi-stage field updates.
Also factor these tests into a DescribeTable.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add nbdkit retry filter.
Available as of Fedora 33 update.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Give correct file name to nbdkit in more cases.
The backing file in the spec might not always match the backing file in
the snapshot, so try harder to match those files by disk ID. May still
need to allow updates to backingFile, depending on how this gets used.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add more unit tests for datavolume-controller.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fix linter error from last commit.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add unit tests for some govmomi API calls.
Move original calls into mock interfaces to make this work.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add checkpoints to DataVolume CRD and reconciliation
* Add Previous, Current, and FinalCheckpoint to DataVolume CRD
* Use checkpoints to set annotations on the PVC
* If an importer pod succeeds while checkpoint annotations are set,
then set the DataVolume status to Paused intstead of Succeeded.
* Clear the PVC checkpoint annotations
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Add new fields to DataVolume CRD creation
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Generate updated code for the DataVolume changes
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Add tests for multistage import annotations
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* check if pod used
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Test scratch deletion during import
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Handle scratch missing just after being created in utility function.
There was a possibility that just after creating the scratch it is deleted, and the next get results in error. This error was ignored - not returned from util - resulting in wrong behavior.
Other fix here is to skip pods with status PodSucceeded and PodFailed in getPodsUsingPVCs function - such Pods are not using PVC, as these pods are already done.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Reschedule reconcile for the pvc until pvc import is complete.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Increase number of NFS volumes, and document the test.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Try another quick fix for verifier pod flakiness.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Cleanup defaults
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* When validating disk space, reserve space for filesystem overhead
The amount of available space in a filesystem is not exactly
the advertise amount. Things like indirect blocks or metadata
may use up some of this space. Reserving it to avoid reaching
full capacity by default.
This value is configurable from the CDIConfig object spec,
both globally and per-storageclass.
The default value is 0.055, or "5.5% of the space is
reserved". This value was chosen because some filesystems
reserve 5% of the space as overhead for the root user and
this space doubles as reservation for the worst case
behaviour for unclear space usage. I've chosen a value
that is slightly higher.
This validation is only necessary because we use sparse
images instead of fallocated ones, which was done to have
reasonable alerts regarding space usage from various
storage providers.
---
Update CDIConfig filesystemOverhead status, validate, and
pass the final value to importer/upload pods.
Only the status values controlled by the config controller
are used, and it's filled out for all available storage
classes in the cluster.
Use this value in Validate calls to ensure that some of the
space is reserved for the filesystem overhead to guard from
accidents.
Caveats:
Doesn't use Default: to define the default of 0.055, instead
it is hard-coded in reconcile. It seems like we can't use a
default value.
Validates the per-storageClass values in reconcile, and
doesn't reject bad values.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Use util GetStorageClassByName
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Test filesystem overhead validation against async upload endpoint
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* wait for NFS PVs to be deleted before continuing
Intended to help with flakes, but didn't make a difference.
Probably still worth doing.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Avoid using the uncached client unnecessarily
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add error handling for the case where even a default SC is not found
Note that this change isn't expected to make a difference, as we
check if the targetStorageClass is nil later on and have the same
behaviour, but this is probably more correct API usage.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add testing for the validation of filesystem overhead values
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Fix logical error in waiting for NFS PVs.
Wait for all of them, not just the last one.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add support for a VDDK datasource to the controllers
This commit does not implement the VDDK datasource. It simply
adds the DataVolumeSourceVDDK type and adds support to the
datavolume and import controllers for it. The datasource itself
will need to be done in a follow-up.
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Fix tests and clean up lint
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Add DataVolumeSourceVDDK to the v1alpha1 api group
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
* Install nbdkit.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add govmomi to vendor directory.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Integrate govmomi into build.
Incomplete! This allows 'find' and 'object' imports from govmomi, but
further dependencies may require more bazel tweaks later.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Create a container image just to hold the VDDK.
Mount the VDDK folder to /opt. Also copy the libxcrypt-compat RPM to
/opt for run-time installation as a temporary workaround for my
inability to install it at container creation time. Make VDDK optional
by storing a dummy file in its directory, so an empty VDDK directory
will not break the build.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fill in VDDK data source.
Add the simplest possible VDDK data source: set up an nbdkit server with
the given VMware parameters, and copy data out using qemu-img.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add vddk to resource validation schema.
Stops "error validating data" messages.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Move libxcrypt-compat install out of VDDK image.
Requiring users to add this to the VDDK image is kind of onerous, so
move it straight into the importer image instead.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Generate BUILD.bazel for the rest of govmomi.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Get VDDK sidecar image from v2v-vmware ConfigMap.
Convention from https://github.com/kubevirt/web-ui-components/pull/534
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Reduce nbdkit startup timeout.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Update govmomi dependencies.
Cleans up 'inconsistent vendoring' errors.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Fix schema verification test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Change disk.raw to disk.img.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add vCenter simulator for VDDK datasource tests.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Functional test with vcsim and dummy VDDK plugin.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Clean up lint errors.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Check kubevirt-hyperconverged namespace for VDDK.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add VDDK unit tests and API functional test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Read a raw image for VDDK test plugin.
Have the fake VDDK test plugin for nbdkit read a raw image passed in
/opt/updates/nbdtest.img instead of returning fixed data.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add some documentation about VDDK data sources.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove unit test focus and trailing spaces.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Use updated build container image.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add some extra comments about VDDK fields.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Clarify VDDK ConfigMap a little more.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Clarify namespace for vddk-init sidecar.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove NodePort from vCenter simulator manifest.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Re-word explanations on VDDK source types.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Add VDDK to comment with types needing URL check.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Rebuild types.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Move AnnThumbprint to storage.import.vddk.thumbprint.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove run-time installation of libxcrypt-compat.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Remove fixed namespaces for vddk-init reference.
The importer now looks for the v2v-vmware ConfigMap in the current CDI
namespace instead of openshift-cnv/kubevirt-hyperconverged.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
* Find CDI install namespace from data volume test.
Signed-off-by: Matthew Arnold <marnold@redhat.com>
Co-authored-by: Sam Lucidi <slucidi@redhat.com>
* Generate CDI CRD using controller-tools.
This is only done for CDI CRD as it requires the existence of source
code. Other CRDs we create are created by a more bare bones pod.
CDIUninstallStrategy was missing a comment describing it, so add
one. This was spotted manually so there might be more missing.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Allow users to specify which nodes CDI pods will live on.
nodeSelector, affinity and tolerations are possible values.
This is done in the CDI CR (rather than CDIConfig) as we are
interested in having this field be populated by external operators.
Unit tests now require the existence of a CDI CR, so create it.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Add a unit test covering some node placement functions
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Specify that all our pods are linux-only.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Avoid duplicate test, accidental left over.
Pointed out by awels, thanks.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Rename to cdiOperatorDeployment for clarity.
Suggested by awels
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Specify we only run on linux using the CDI CR, no need to embed this
into the code.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Don't dereference workloadPlacement for no reason
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Split off operator test to have its own AfterEach, BeforeEach.
Use even more descriptive function names.
Do all the CDI delete/restore logic in AfterEach, to ensure that
it happens and restores the deployment with the original CR even
if the test fails.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Remove XXX. This is the proper way.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Adapt to latest changes in controller_test.go (renaming import)
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Simplify, not storing intermediate value.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Don't dereference nodeplacement in callers to CreateDeployment
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Remove redundant save & restore. Unit tests do this for us.
Pointed out by awels, thanks.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Split out "find toplevel" to a utility function
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Wait for the CDI CR update to apply before continuing.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Simplify, not storing intermediate value.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Make it clear that the chosen node placement will not be schedulable.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* update k8s deps to 1.18.6 and controller runtime to 0.6.2
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* remove building code generators from docker image. This way the k8s ligray version only has to be updated in go.mod
Do more stuff in the bazel container. Faster and better interop
Fix unit tests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* make format
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* remove unnecessary rsync
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* redo code generator dep management
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* builder uses go modules
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Set the WaitForFirstConsumer phase on DataVolume when storage uses the WaitForFirstConsumer binding mode and is not bound yet.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Skip PVC if not bound in import|clone|upload controllers.
This is done so the VM pod(not the cdi pod) will be the first consumer, and the PVC can be scheduled on the same location as the pod.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
fixup! Skip PVC if not bound in import|clone|upload controllers.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update importer tests to force bind the PCV by scheduling a pod for pvc, when storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update datavolume tests to force bind the PCV by scheduling a pod for pvc, when storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update upload controller and upload tests to correctly handle force binding the PCV by scheduling a pod for pvc, when storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update clone tests to force bind the PCV by scheduling a pod for pvc when the storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update cloner multi-node tests to force bind the PCV by scheduling a pod for pvc when storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Correct after automerge
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Improve/simplify tests
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Fix error in import test.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update transport_test,operator_test.go
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update rbac_test.go and leaderelection_test.go
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Improve Datavolume and PVC Checks for WFFC.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Handle wffc only if feature gate is open - import-controller
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* TEST for Handle wffc only if feature gate is open - import-controller - TEST
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Handle wffc only if feature gate is open - upload-controller with test
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* rename and simplify checks
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* cleanup after rebase
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* update tests after rebase
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* update tests after rebase
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* more cleanups
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Document new WFFC behavior
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Document new HonorWaitForFirstConsumer option
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* update docs according to comments
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* extract common function, cleanup - code review fixes
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* add comment for another pr - 1210, so it can have easier merge/rebase
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* typo
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Simplify getStoragebindingMode - code review comments
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Add FeatureGates interface - code review fix
Additionally pass the features gates instead of the particular feature gate value,
and let shouldReconcilePVC decide what to do with the feature gate. That way shouldReconcilePVC
contains all the logic, and the caller does not need to do additional calls to provide parameters.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update matcher
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
