containerized-data-importer/tools/cdi-image-size-detection
alromeros 7eebbfaf1c
Make size optional when cloning using the Storage API (#2222)
* Allow empty DV size when cloning using storage API

When cloning a Data Volume, the size of the target can be potentially obtainable via the source PVC, which discards the need to explicitly specify it.

Considering that, this commit introduces a change in the correspondent validation webhook to allow omitting the resources.request.storage field when cloning a PVC using the storage API.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Modify datavolume-controller to allow obtaining storage size from source PVC when cloning

When cloning a PVC, if the target's size is not specified, said value can be attainable from the source PVC.

This commit introduces a change in datavolume controller so, in case of detecting an empty storage size, said value can be obtained when performing CSI and Smart cloning.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Update unit tests for datavolume-validation after enabling cloning with empty size

This commit updates the unit testing for the datavolume validation webhook, covering the possibility of cloning a PVC without setting any storage size.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Update unit testing for controller-related functions after enabling cloning with empty size

This commit includes unit tests for the volumeSize() function after enabling creating clones with blank size.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Update the datavolume controller to create a size-detection pod when performing host-assisted clone

When performing a host-assisted clone with empty clone size, simply copying the original PVC size could lead to potential overhead miscalculations if the source's VolumeMode is "filesystem".

When that's the case, an inspection pod will be created in the datavolume controller so it extracts the size of the virtual image using qemu-img.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Include an image-size detection tool to allow cloning with empty DV size

This commit introduces a new tool in charge of collecting the virtual image size when cloning with an empty DV size. In some cases where said value is unattainable from the original PVC's spec, the datavolume controller will create a new pod containing this new tool.

The binary will then run the 'qemu-img' command and handle its results appropriately.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Optimize the clone-size lookup process to avoid creating unnecessary size-detection pods

When performing host-assisted clone with an empty DV size, in some cases, a size-detection pod is used to obtain the required capacity.

This commit tries to optimize this process to keep the collected value as a PVC annotation, that is checked in subsequent clones to avoid creating redundant pods.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Minor fixes and improvements on mechanism for cloning with empty storage size
* Add new optional flag on size-detection binary to enable using a different URI scheme
* Improve the pod-creation mechanism so the pod is not created until the source PVC has finished the import
* Modify size-finlation mechanism to account for possible round-downs when importing the source image
* Improve the size inflation mechanism so only PVCs with filesystem as volume mode are considered
* Minor style corrections

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Modify the clone-controller to allow skipping the clone size validation in some cases

Due to filesystem overhead differences, the target's size can sometimes be smaller than the source's one when obtaining said value with the size-detection pod.

This commit introduces minor changes in the clone-controller so we can skip the size validation in those cases.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Minor changes and improvements in size-detection mechanism following PR review
* Added new UT that covers using empty storage API for non-cloning sources
* Added new watch on datavolume-controller that looks for changes in the size-detection pod
* Removed redundant and unnecessary specs on size-detection pod
* Added error handling when reading the pod's termination message
* Moved general-usage functions to 'util.go' file
* Updated 'datavolumes' documentation to reference the possibility of omitting the storage size when cloning
* Minor style corrections

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Add unit tests that cover the size-detection mechanism in the DataVolume controller

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Include functional tests for cloning without specifying storage size

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Improve error handling in the creation/deletion process of the size-detection pod

This commit introduces additional handling in case of error after and during the size-detection pod is created.

It also updates several related unit tests.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Minor fixes to improve fsOverhead calculations when cloning with empty storage size
* Modified the size-detection mechanism so we account for fsOverhead when cloning to filesystem volume mode in all cases
* Clean up the code for reconciling when cloning a PVC that is not ready
* Minor fix in functional test so it works when cloning from block to filesystem volume mode

Signed-off-by: Alvaro Romero <alromero@redhat.com>
2022-05-31 17:03:45 +03:00
..
BUILD.bazel Make size optional when cloning using the Storage API (#2222) 2022-05-31 17:03:45 +03:00
main.go Make size optional when cloning using the Storage API (#2222) 2022-05-31 17:03:45 +03:00