Commit Graph

96 Commits

Author SHA1 Message Date
akalenyu
71522a1f2d
Switch VolumeSnapshot to v1 (#2235)
* Switch VolumeSnapshot to v1

VolumeSnapshot v1beta is being deprecated:
https://kubernetes.io/blog/2022/04/07/upcoming-changes-in-kubernetes-1-24/#api-removals-deprecations-and-other-changes-for-kubernetes-1-24

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Fix unit tests; change version we look for in IsCsiCrdsDeployed

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
2022-04-12 18:50:19 +02:00
Roman Mohr
b9c0684469
Separate sdk api (#2208)
* Introduce controller-runtime-sdk api package

Split controller-runtime-sdk into the base package and
controller-runtime-sdk/api.

Signed-off-by: Roman Mohr <rmohr@redhat.com>

* go mod vendor

Signed-off-by: Roman Mohr <rmohr@redhat.com>

* Update code references

Signed-off-by: Roman Mohr <rmohr@redhat.com>
2022-03-31 21:31:18 +02:00
Alexander Wels
013cb6b62b
Set http(s)_proxy to lower case env variable (#2132)
* Set htpp(s)_proxy to lower case env variable

CURL used by nbdkit doesn't read upper case http(s)_proxy environment
variables, and thus was not using the proxy. Changed the variable to
be lower case.

Added a significant number of tests to test many more variations of
using a proxy. Also added https + auth endpoint to the file-host
container, so we can test https + auth with the proxy.

Added https endpoint to proxy, so we can test an https proxy.

Cleaned up some of the error handling in the import controller for
the proxy, in particular if a trustedCAProxy is defined.

Fixed some of the cluster wide proxy configuration so it works properly
inside an openshift cluster.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Add https proxy support to registry import. Added extra
functional tests to test all registry import combinations

Signed-off-by: Alexander Wels <awels@redhat.com>

* Fixed some tests to work better in Open Shift.

Signed-off-by: Alexander Wels <awels@redhat.com>
2022-02-03 18:09:41 +01:00
Shelly Kagan
e7dd62eb26
Upload archive (#1969)
* Add support for archive upload

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* fix golang errors

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* Change storage profile property set to support more then one set

So far CDI supported only 1 claim propery set. We want to be able
to support more then one so in case the user provides to the
DV storage volumeMode without accessMode or vice versa cdi
will be able to fit to it the most appropriate match.
Added to rook ceph block a second default of filesystem
volume mode with RWO access mode, it will support archive
upload which has default of filesystem mode.

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* CR fix - change to one endpoint for the user

upload proxy will identify if the upload is archive
or not by looking at the content type annotation on
the pvc. If the content type is archive it will route
the uplaod to upload server to a new archive upload uri.

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* Add storage profile and data volume controllers unit tests

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* CR fixes

* add default volume mode to archive content type
* upload server use data processor for archive upload
* tests for volume mode with archive content type
* tests for archive upload of compressed tar

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* Adjust imports acording to new apis dir

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* CR small fixes

Signed-off-by: Shelly Kagan <skagan@redhat.com>
2021-11-03 20:11:47 +01:00
Michael Henriksen
aedaf513ec
Move apis to staging, push to containerized-data-importer-api (#1997)
* move apis to new staging area

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* add script to push to staging

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix lint check and api reference

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* push staging to api repo

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2021-10-28 13:40:24 +02:00
Bartosz Rybacki
a82a0a7a59
Correct the cloneStrategy on StorageProfile (#1951)
Moves the cloneStrategy up one level from claimPropertySet to
StorageProfile.spec and StorageProfile.status.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2021-09-22 18:06:18 +02:00
Bartosz Rybacki
2f21a36545
Preferred Clone strategy selection (#1846)
* Refactor: simplify by extracting methods

Prepare for new clone logic - extracted smartClone reconcile functions.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Select clone strategy based on storageProfile

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Changes from CR comments.

A series of small fixes, and cleanups.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Documentation update

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2021-07-03 01:53:14 +02:00
Maya Rashish
19d109837d
Add a datavolume condition for the image being too large (#1818)
* Strip newlines when writing a termination message.

Otherwise it isn't visible, at least when viewing in the -o yaml view.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Write down the nbdkit output and add it to the error output

With the added output from nbdkit, we can see the reason for the
non-existence of the nbdkit socket.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Don't set that we're waiting if it's CrashLoopBackOff

It's better to have the reason for the crash (terminate message)
than "backing off 5 minutes"

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Simplify all "image too large to fit" terminate reasons.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Move verifyConditions to utils, no functional change

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Switch test for image too large to test condition and not log

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Remove unused branch

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Rename setConditionFromPodWithPrefix to setAnnotationsFromPodWithPrefix

No functional change. Intended to be followed by some refactoring.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Fold restart count logic into the common setAnnotationsFromPodWithPrefix

Changing to >= rather than > to ensure a zero pod restart count is
always used -- the import controller unit tests request this.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Reduce indentation by returning right away.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Fold check for pod being nil into common code.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* move saveVddkAnnotations into util and make it unconditional

Call it setVddkAnnotations for consistency.
Check for not-terminated inside the function, not outside.

Removes check for source being VDDK (to avoid passing more arguments):
it won't match the regex anyway.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Reduce indentation by bailing on failure.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Reorder parameters to mirror the order in the function name

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Use a named variable for first container state

Yields shorter, more legible lines.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Use a constant for the nbdkit log.

Signed-off-by: Maya Rashish <mrashish@redhat.com>

* Add more information to function description - also logs to file

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2021-06-29 12:47:05 +02:00
Alexander Wels
5fedb458e5
Set some reasonable requests/limits for workloads (#1842)
* Set some reasonable requests/limits for workloads

Signed-off-by: Alexander Wels <awels@redhat.com>

* Fix unit tests

Signed-off-by: Alexander Wels <awels@redhat.com>
2021-06-28 20:34:01 +02:00
Arnon Gilboa
13275ce351
OS image poll and update API (#1808)
* Add CRD for DataSource definition

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add optional sourceRef to DataSource in DataVolumeSpec

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add CRD for DataImportCron definition

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add DataSource and DataImportCron generated files

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Code review fixes

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* More code review fixes

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Code genrated after rebase

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Fix DV source reference in utests

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Remove api validation tests for missing data volume source

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>

* Add standard fields to condition structs

Signed-off-by: Arnon Gilboa <agilboa@redhat.com>
2021-06-14 13:58:42 +02:00
Michael Henriksen
74a2c86608
use namespace transfer for smart clone (#1763)
* use namespace transfer for smart clone

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* updates from test failures

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* add expansion func tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* add dv phases for expansion and transfer

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* rebase and integrate with storage profiles

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2021-04-30 15:18:43 +02:00
Bartosz Rybacki
f81ab950fb
Use storage profiles when handling DataVolumes (#1753)
* Create new Storage type

A new Storage type similar to the PVC Spec is now available to use
in the DataVolume Spec. This is more permissive than PVC, and together
with StorageProfile this allows CDI to apply additional logic for
missing or optional fields.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Use the StorageProfile

Handle the StorageProfile recommended params when creating the PVC for
a DataVolume. When parameters like volumeMode or accessModes are
not provided, CDI checks the StorageProfile for a given StorageClass
to set the recommended defaults. This enables user to create DataVolume
without the need to provide all the parameters.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Allow multiple accessModes

CDI allows multiple access modes to be specified in the DataVolume.spec.storage and in the StorageProfile. This now works the same way as in PVC specification.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle the storage.size field

The storage.size specifies how much space a user wants to have.
When creating image on the fileSystem storage CDI takes into
account the file system overhead and requests PVC big enough to
fit an image and file system metadata.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Test storage profile with DV

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Document Storage Profiles

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Refactor: Render the effective PVC early

The helper 'render PVC' was moved earlier in the control flow, so
it can be used in more places. Removing the need for if/else logic.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Test handling size on import, upload and clone

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Code Review: Refactor resolving of volumeMode

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Fix: render target pvc spec correctly in smart clone controller

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2021-04-29 13:10:24 -05:00
Tomasz Barański
84f51f32f6
Preallocate cloning DV (#1719)
* Refactoring - move PreallocationApplied flag definition to common

Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>

* Preallocate cloning DataVolumes

Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
2021-04-27 15:06:30 +02:00
Marcelo Carneiro do Amaral
8d1721db0a
Add support for proxy in the CDI import pod, reconciling the info from OpenShift cluster wide proxy when available (#1507)
* api: Add new ImportProxy structure to CDIConfig in v1beta1 core api

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* common: Add constants related to configure Import proxy in Import pod and controller reconcile

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* config controller: Add import proxy reconcile, monitoring OCP proxy updates

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* config controller: Add unit test for import proxy reconcile, monitoring OCP proxy updates and creating Secret and ConfigMap

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* import controller: Add unit support of ImportProxy in the tests

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller util: Add support to retrieve an OpenShift cluster wide proxy object as well as its needed configurations

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller util:  Add unit test for the get cluster wide proxy functionality

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller: moved isOpenshift function to utils common and export it

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* importer: Clone http transport to keep the default proxy config such as the usage of environment variables and add support for proxy CA

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* operator: Update CDIConfigCRD with import proxy info

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* operator: Update CDIListCRD with import proxy info

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* operator: Add service account rights to CDI for accessing OpenShift proxy obj

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* operator: Add OpenShift proxy obj scheme support

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: Add new forwarding proxy for testing things that require a proxy

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: Add test to verify the importer pod with proxy

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* doc: Update cdi config doc with the ImportProxy info

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* add support to build the new proxy docker image

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: remove unwanted F parameter from test

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller: fix error handling

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: update the used method to get the pod of the importer pod and proxy

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* fixed comments from the revision

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller config: update the methods to use the log object from the reconcile object

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller: update function GetImportProxyConfig to return errors and include a new unit test

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* controller: add back a test that was removed by mistake

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* update updateCDIConfigByUpdatingTheClusterWideProxy function

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: decrease the import pod log pooling interval and increase the image size to be able to verify the import pod before it is deleted

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: update the pvc and proxied requests verification

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* tools: add X-Forwarded-For header in the proxy requests  used for testing

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: reset cluster wide proxy with the original values

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* test: fix proxy update to change spec instead of status and other minor updates

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* doc: update import proxy description

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>

* update generated files

Signed-off-by: Marcelo Amaral <marcelo.amaral1@ibm.com>
2021-02-16 13:43:01 +01:00
Nahshon Unna Tsameret
93627f4fe8
Stop Using Deprecated Packages (#1548)
* sigs.k8s.io/controller-runtime/pkg/runtime/* packages are deprecated, and were moved to new paths.

Trying to upgrade sigs.k8s.io/controller-runtime to version v0.7.0 in HCO created a conflict because in v0.7.0 the deprecated packages were removed and cannot be used.

This PR replaces the deprecated packages with their new paths.

Signed-off-by: Nahshon Unna-Tsameret <nunnatsa@redhat.com>

* Run `make deps-update`

Signed-off-by: Nahshon Unna-Tsameret <nunnatsa@redhat.com>

* fix logger init

Signed-off-by: Nahshon Unna-Tsameret <nunnatsa@redhat.com>

* fix test loggers

Signed-off-by: Nahshon Unna-Tsameret <nunnatsa@redhat.com>
2020-12-24 07:08:50 +01:00
Tomasz Barański
91a15c57d1
Preallocation support (#1498)
* [WIP] doc: User-facing doc for preallocation support

Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>

* apis: CDI accepts `preallocation` option.

With this commit CDI accepts (but does handle) `preallocation` settings
for DataVolumes and in CDIConfig.

Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>

* core: Implementing preallocation

This commit implements preallocation support for import and upload.

Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>

* test: Functional tests for preallocation support

Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>

* core: Remove "preallocation for StorageClasses" config

Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>

* test: Removed unused function

Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>

* test: Fix rook-ceph test failures

Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>

* Updated dependencies
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>

* core: Uss PVC annotation to pass preallocation parameters

DataVolume controller now uses a PVC annotation to pass preallocation
configuration to import and update controllers.

Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
2020-12-18 16:46:16 -05:00
Michael Henriksen
86c32e87b8
Add CDIConfig to CDI (#1475)
* Add CDIConfigSpec to CDI

Make CDIConfig singleton mirror data in "active" CDI

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix functional tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-11-11 23:09:54 +01:00
Jakub Dzon
7f368900de
Updated controller-lifecycle-operator-sdk dependency (#1389)
Signed-off-by: Jakub Dzon <jdzon@redhat.com>
2020-09-24 14:39:29 +02:00
Alexander Wels
d2da27b41b
Ensure we can specify different CDI CR name. (#1383)
Fix bug where the CR name was hard coded to CDI and it failed in HCO.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-09-22 00:13:28 +02:00
Alexander Wels
6dce12f090
Move CRDS from apiextensions v1beta1 to v1. (#1307)
* Move CRDS from apiextensions v1beta1 to v1.
Ensure that our code based schema validation matches the types in the api.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Ran go mod tidy and vendor in attempt to see if we could use newer runtime controller, but our go version too old.
Addressed review comments.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Addressed more review comments and fixed k8s-1.18 functional test failing.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Remove categories 'all' from cluster scoped CRDs

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-08-01 01:01:50 +02:00
Bartosz Rybacki
ab48911b9b
WaitForFirstConsumer handling for DataVolumes (#1242)
* Set the WaitForFirstConsumer phase on DataVolume when storage uses the WaitForFirstConsumer binding mode and is not bound yet.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Skip PVC if not bound in import|clone|upload controllers.

This is done so the VM pod(not the cdi pod) will be the first consumer, and the PVC can be scheduled on the same location as the pod.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

fixup! Skip PVC if not bound in import|clone|upload controllers.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update importer tests to force bind the PCV by scheduling a pod for pvc, when storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update datavolume tests to force bind the PCV by scheduling a pod for pvc, when storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update upload controller and upload tests to correctly handle force binding the PCV by scheduling a pod for pvc, when storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update clone tests to force bind the PCV by scheduling a pod for pvc when the storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update cloner multi-node tests to force bind the PCV by scheduling a pod for pvc when storage class is wffc.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Correct after automerge

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Improve/simplify tests

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Fix error in import test.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update transport_test,operator_test.go

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update rbac_test.go and leaderelection_test.go

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Improve Datavolume and PVC Checks for WFFC.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle wffc only if feature gate is open - import-controller

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* TEST for Handle wffc only if feature gate is open - import-controller - TEST

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle wffc only if feature gate is open - upload-controller with test

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* rename and simplify checks

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* cleanup after rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* update tests after rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* update tests after rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* more cleanups

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Document new WFFC behavior

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Document new HonorWaitForFirstConsumer option

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* update docs according to comments

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* extract common function, cleanup - code review fixes

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* add comment for another pr - 1210, so it can have easier merge/rebase

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* typo

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Simplify getStoragebindingMode - code review comments

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Add FeatureGates interface - code review fix

Additionally pass the features gates instead of the particular feature gate value,
and let shouldReconcilePVC decide what to do with the feature gate. That way shouldReconcilePVC
contains all the logic, and the caller does not need to do additional calls to provide parameters.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Update matcher

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-07-22 16:23:44 +02:00
Michael Henriksen
9e2c79b1e0
move api groups to v1beta1 (#1232)
* move upload.cdi.kubevirt.io API group to v1beta1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* move core api to v1beta1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix os-3.11 cluster sync and add functional tests for alpha api

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* change more occurences of v1alpha1

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* updates after rebase

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-07-10 15:47:38 +02:00
Michael Henriksen
bd69c350bb
Wait for PVCs to be unused before initiating clone operations (#1263)
* don't create snapshot or clone pods if pvcs in use

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* cleanup pods during functional tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* kill mmore pods blocking clone tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix typos

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-06-30 19:08:50 +02:00
Michael Henriksen
d45574678b
alpha to beta snapshot API (#1206)
* move from alpha to beta snapshot API

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix broken clone tests

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* don't generate snapshot client

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-05-20 15:25:28 +02:00
Alexander Wels
c707e781a9
Add conditions to DataVolume and add condition annotation to PVCs (#1194)
* Add conditions to DataVolume and add condition annotation to PVCs

Signed-off-by: Alexander Wels <awels@redhat.com>

* Added reason codes to conditions
Updated functional tests to use reason codes
Fixed broken functional test.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Added last transition time
Added documentation

Signed-off-by: Alexander Wels <awels@redhat.com>

* Switch to conditions slice from conditions pointer slice.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Add heartbeat time for conditions.
Renamed AnnRunningCondition to AnnLastTerminationMessage
Only update reason/message on heartbeat or state change.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Rename heartBeat to heartbeat
Re-factor datavolume-conditions to properly update heartbeat time.

Signed-off-by: Alexander Wels <awels@redhat.com>

* renamed AnnRunningConditionReason to AnnLastTerminationReason

Signed-off-by: Alexander Wels <awels@redhat.com>

* Revert lastTerminatedMessage/Reason rename
Added separate AnnScratchBoundCondition/Message/Reason for scratch space

Signed-off-by: Alexander Wels <awels@redhat.com>

* Update bound condition to take scratch or source PVC status into consideration.
Update running condition to take source pod status into consideration

Signed-off-by: Alexander Wels <awels@redhat.com>

* Addressed comments in review.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Properly get scratch space pvc phase.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-05-12 23:39:23 +02:00
Alexander Wels
08ed6f2b39
Clean up client-go left overs after converting to runtime library. (#1163)
* Clean up client-go left overs after converting to runtime library.
Clean unused functions from controller utils.go
Turn util_test.go into gingko test.
Moved functions from util to proper controller if only used by that controller.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Don't export reconciler variables.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-04-02 22:59:54 +02:00
Alexander Wels
1d48514c2b
Use controller runtime library for upload controller (#1085)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-03-03 15:30:56 +01:00
Alexander Wels
79b144088f
Copy kubevirt.io annotations to scratch space pvc, but NOT cdi.kubevirt.io as those are what triggers import/upload/clone. (#1109)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-02-20 16:50:50 +01:00
Alexander Wels
17db2920b1
Update clone controller to use runtime library. (#1075)
Signed-off-by: Alexander Wels <awels@redhat.com>

Co-authored-by: Michael Henriksen <mhenriks@redhat.com>
2020-02-14 23:04:46 +01:00
Alexander Wels
9a2b514365
Add async endpoint for upload that closes connection immediately after transfer completes and then continues background processing. (#1095)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-02-12 16:17:26 +01:00
Michael Henriksen
bd4c4c950b
cert rotation (#1091)
* initial cert rotation controller

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix typo

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-02-03 23:36:58 +01:00
Michael Henriksen
99f8af5b86 k8s client upgrade to 1.16 (#1079)
* initial client upgrade to 1.16

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix Route detection in OpenShift

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-01-14 13:43:17 +01:00
Woohyung Han
2adcc80f95 Fix to create temporary pod with default resource quota (#1055)
If we create a temporary pod in the namespace which has
resource quota, `must specify quota` error occurs. Fix to
create temporary pod with default resource quota.

Co-authored-by: hyoung-90 <ssarang520@gmail.com>
Co-authored-by: Woohyung Han <woohhan@gmail.com>
Signed-off-by: anencore94 <anencore94@kaist.ac.kr>

Co-authored-by: anencore94 <37469330+anencore94@users.noreply.github.com>
Co-authored-by: hyoung-90 <58063867+hyoung-90@users.noreply.github.com>
2020-01-09 17:43:14 +01:00
Alexander Wels
f090ddc2e6 Update Import controller to use runtime controller library. (#1054)
Refactor import related utilities out of util.go into the controller.

Signed-off-by: Alexander Wels <awels@redhat.com>
2019-12-20 19:28:43 +01:00
Michael Henriksen
66a1c8c996 Unique client cert per clone source pod (#987)
* unique cert per clone source pod

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* upload controller shouldn't cache certs

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix test variable names

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2019-10-09 15:10:55 -04:00
Michael Henriksen
834b85ecbf Network clone (#897)
* network cloning

* fix clone progress
2019-08-01 16:01:25 -04:00
Michael Henriksen
cc99e0c337 token validation broken 2019-07-16 00:20:40 -04:00
Michael Henriksen
ec52c85a25 Validating webhook and token authorization for PVC cloning (#869)
* baseline refactoring of webhook package

* datavolume clone validation webhook

* rename datavolumes/clone-init to datavolumes/source

* add RBAC doc

* updates from review

* make clone permission check exportable function

* force dumb cloning in in functional test
2019-07-09 14:02:31 -04:00
Michael Henriksen
019c843586 make clone pods use selinux type spc_t instead of privileged (#875)
* make clone pods use selinux type spc_t instead of privileged

* fix block mode related tests
2019-07-08 13:58:42 -04:00
Alexander Wels
b538ab3190 Set running user to the one defined in the docker file.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-06-12 16:55:37 -04:00
Michael Henriksen
d22e5c0c6f run upload/import pods as nonroot 2019-06-05 17:28:15 -04:00
Michael Henriksen
4c7566f1b7 RunAsUser causing issue on OpenShift so let's get rid of it 2019-06-05 16:16:23 -04:00
Michael Henriksen
bc240a91d9 dont't run as root 2019-06-05 16:16:22 -04:00
Michael Henriksen
4f87fb368e fix insecure registry unit test 2019-05-30 09:47:17 -04:00
Daniel Erez
5b3109a324 smart-clone
Changes and flow:

* datavolume-controller:
- When clone pvc is requested and smart-clone is applicable -> Create a new VolumeSnapshot.

* Introduced smart-clone-controller:
- Listens to VolumeSnapshot changes and create a new PVC based on the snapshot.
- Upon PVC successful creation, deletes the snapshot for cleaning up.

Change-Id: I369fc92e72edb8a2cf584c8cb916795415f9d6e0
Signed-off-by: Daniel Erez <derez@redhat.com>
Co-authored-by: Fred Rolland <frolland@redhat.com>
2019-05-20 11:04:10 +03:00
Artyom Lukianov
40325745d0 Copy triple package from client-go repository
This package droped from client-go 1.13.4, so until
we will have find some good alternatives, we can use local copy.
2019-05-16 09:38:37 +03:00
Alexander Wels
309a46e72d Verify source and target PVC are correct before proceeding with clone.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-04-18 10:17:34 -04:00
tavni
e6665945bc Adding support to clone disk image from one Block PV to another
Signed-off-by: tavni <tavni@redhat.com>
2019-04-15 15:33:52 +03:00
tavni
e0d4ec62b0 Fix for isue #1690415 - CDI appends 'source pod' to a label and may fail on error that label must not be longer than 63 characters
Signed-off-by: tavni <tavni@redhat.com>
2019-04-03 14:11:42 +03:00
tavni
d4c7b0be54 Adding support to import disk image into Block PV
Signed-off-by: tavni <tavni@redhat.com>
2019-03-28 11:43:56 +02:00