* cleanup: Extract label copying logic into common pkg
Extract the label copying logic from populator-base.go into the common
pkg as CopyAllowedLabels func.
Signed-off-by: Felix Matouschek <fmatouschek@redhat.com>
* fix(import-populator): Make copying of labels more robust
Make copying of labels from a prime PVC to the target PVC more robust,
by moving it before rebinding the PV from prime to target. This way we
can ensure the labels are already present once the PVC becomes ready.
Signed-off-by: Felix Matouschek <fmatouschek@redhat.com>
* cleanup: Do not pass labels from DIC to DS anymore
Do not pass labels from a DataImportCron to a DataSource in the
dataimportcron-controller anymore. In the future this will be
handled by the datasource-controller.
Signed-off-by: Felix Matouschek <fmatouschek@redhat.com>
* feat: Copy labels from source to DataSource
Copy labels from the source of a DataSource to the labels of the
DataSource in the datasource-controller.
Signed-off-by: Felix Matouschek <fmatouschek@redhat.com>
* tests: Add e2e tests for copying labels to DataSources
Add e2e tests that cover all scenarios where labels should be copied
from the source of a DataSource to the DataSource itself.
Signed-off-by: Felix Matouschek <fmatouschek@redhat.com>
* feat(dataimportcron-controller): Copy labels to VolumeSnapshots
When using VolumeSnapshots copy the labels found on the source PVC to
the created or an existing VolumeSnapshot.
Signed-off-by: Felix Matouschek <fmatouschek@redhat.com>
---------
Signed-off-by: Felix Matouschek <fmatouschek@redhat.com>
* Remove sourceref validation
The source can be changed and updated, we want the error to be
presented by an event and have the datavolume pending instead
of preventing the creation of the datavolume.
We are already handling the option of the source not existing yet
so lets also handle all the validation in the controller.
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Move pvc clone specific functions to pvc-clone-controller
Now that the validation is no longer done in the webhook
no need for that code to be in the controller common util
file. Moved the UT accordingly.
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Move function to new Describe node
Will add in future commits other tests to that section
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* pvc-clone-controller: make better event and update dv if validation fails
In order to make it more clear that the validation failed
we should add the reason of validation failure to the event
Added an update watch to the clone controller so if something changes
in the source the clone will reconcile. Hence no need to return an
error, the reconcile will be triggered if someting will change in the
source.
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* snap-clone: always validate snapshot source
This commit adds missing validation due to the remove validation
from the dv webhook, but also fixes an existing missing validation
in cases where the clone was created before the source. In such case
the webhook would not validate the source since it didnt exist yet,
and then if the clone happened with populators then we would not
validate the source size at all.
We should validate the snapshot source before continuing with the clone
whether it is populator/legacy clones.
Moved all the validation to one function.
Updated the event to include the reason for the validation failure and
updated dv status accordingly.
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Cloner_test: remove clone to small size test
The test checks the rejection of the webhook
which no longer exists then clone to appropriate
size dv which should succeed which is done in
any other clone test.
Im not adding a test to check the reconcile loop
since it is covered in the UT.
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* tests/datasource: remove wait for pvc
Now since we added the validation also in the populators
path we are not creating the pvc until the source exists.
Signed-off-by: Shelly Kagan <skagan@redhat.com>
---------
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Enable gofmt linter
From the docs:
> Gofmt checks whether code was gofmt-ed. By default this tool runs with
> -s option to check for code simplification.
https://golangci-lint.run/usage/linters/#gofmt
Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
* Run gomft on the project
Ran this command after adding the gofmt linter:
golangci-lint run ./... --fix
Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
* Enable whitespace linter
From the docs:
> Whitespace is a linter that checks for unnecessary newlines at the
> start and end of functions, if, for, etc.
https://golangci-lint.run/usage/linters/#whitespace
Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
* Run whitespace on the project
Ran this command after adding the whitespace linter:
golangci-lint run ./... --fix
Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
* Enable GCI linter
Per the docs:
> Gci controls Go package import order and makes it always deterministic.
https://golangci-lint.run/usage/linters/#gci
NOTE: I noticed that many files separate their imports in a particular
way, so I set the linter to enforce this standard.
Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
* Run GCI on the project
Ran this command after adding the GCI linter:
golangci-lint run ./... --fix
Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
---------
Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
* Add misspell to list of linters
Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
* Fix spelling errors
Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
---------
Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
Joining hostname+port with string concatenation leads to wrong URLs
when the hostame is an IPv6:
HOST PORT Sprintf CORRECT
::1 8000 ::1:8000 [::1]:8000
To avoid this issue, it's best to use net.JoinHostPort. I added a test
that verifies this fix.
This type of issue can be discovered running the following command:
golangci-lint run ./... --enable nosprintfhostport
Signed-off-by: Edu Gómez Escandell <egomez@redhat.com>
* Default virt storage class
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Add alert for multiple default virt storage classes
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Refactor content type funcs to not return strings
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
---------
Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
* Run `make deps-update`
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Update to ginkgo v2
Avoid using table extension to avoid compilation errors
Switch to v2 everywhere
Update qe-tools as well (required)
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Fix/avoid deprecation warnings
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Do not use v1 reporter
For unit tests: stop using custom reporter, unnecessary
For functional tests: borrow code from kubevirt to keep reporting
Avoid deprecated warnings by golangci for using deprecated reporter
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Increase ginkgo timeout to 24h (default in ginkgo v1)
this may seem excessive, but we have a lower timeout in Prow, let's save
ourselves the future trouble of bumping timeouts in two places.
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* use the ginkgo built-in junit reporter
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* Avoid using deprecated --ginkgo.noColor, use --ginkgo.no-color instead
Signed-off-by: Maya Rashish <mrashish@redhat.com>
---------
Signed-off-by: Maya Rashish <mrashish@redhat.com>
* move cleanup out of dv deletion
It seemed off to call cleanup in the prepare function
just because we don't call cleanup unless the dv is deleting.
Instead we check in the clenup function itself if it should be
done: in this 2 specific cases in case of deletion and in case
the dv succeeded.
The cleanup will be used in future commit also for population cleanup
which we also want to happen not only on deletion.
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Use populator if csi storage class exists
Add new datavolume phase PendingPopulation to
indicate wffc when using populators, this new
phase will be used in kubevirt in order to know
that there is no need for dummy pod to pass wffc phase
and that the population will occur once creating the vm.
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Update population targetPVC with pvc prime annotations
The annotations will be used to update dv that uses the
populators.
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Adjust UT with new behavior
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* updates after review
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Fix import populator report progress
The import pod should be taken from pvcprime
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Prevent requeue upload dv when failing to find progress report pod
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Remove size inflation in populators
The populators are handling existing PVCs.
The PVC already has a defined requested size,
inflating the PVC' with fsoverhead will only be
on the PVC' spec and will not reflect on the target
PVC, this seems undesired.
Instead if the populators is using by PVC that the
datavolume controller created the inflation will happen
there if needed.
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Adjust functional tests to handle dvs using populators
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Fix clone test
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* add shouldUpdateProgress variable to know if need to update progress
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* Change update of annotation from denied list to allowed list
Instead if checking if the annotation on pvcPrime is not desired
go over desired list and if the annotation exists add it.
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* fix removing annotations from pv when rebinding
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* More fixes and UT
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* a bit more updates and UTs
Signed-off-by: Shelly Kagan <skagan@redhat.com>
---------
Signed-off-by: Shelly Kagan <skagan@redhat.com>
* touch up zero restoresize snapshot
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* clone populator
only supports PVC source now
snapshot coming soon
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* more unit tests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* unit test for clone populator
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* func tests for clone populator
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* move clone populator cleanup function to planner
other review comments
verifier pod should bount readonly
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* add readonly flag to test executor pods
synchronize get hash calls
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* increase linter timeout
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* better/explicit readonly support for test pods
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* check pv for driver info before looking up storageclass as it may not exist
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* addressed review comments
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* chooseStrategy shoud generate more events
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
---------
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
- Split the huge DV controller into smaller op-specific DV controllers -
import, clone, upload
- Add common watch-adding function so each controller watches only its
relevant DVs
- Refactor the common Reconcile() to use interface DataVolumeReconciler
implemented by each controller
- Move all functions, structs, consts to the relevant controller
- Split the utests per controller
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
