* Updated dependencies
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Replace skopeo with containers API.
This commit removes dependence on skopeo (binary) and uses containers
API. By doing that we're able to opimize the use of storage (scratch)
space, storage I/O and download bandwith.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Fixing rebase - dependencies kerfuffle.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Handling docker-format images as well as OCI.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Fix for missing code-generator module.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Remove regex, image file in registry images are matched by a path
prefix.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* Added nginx proxy in front of docker registry for a rate-limited access.
Signed-off-by: Tomasz Baranski <tbaransk@redhat.com>
* update k8s deps to 1.18.6 and controller runtime to 0.6.2
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* remove building code generators from docker image. This way the k8s ligray version only has to be updated in go.mod
Do more stuff in the bazel container. Faster and better interop
Fix unit tests
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* make format
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* remove unnecessary rsync
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* redo code generator dep management
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* builder uses go modules
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Update cdi-builder to use go version 1.14.6 (current latest)
Fix functional test framework to work with go >= 1.13. We were not
using the ginkgo ordering correctly, and our framework would call
flag.Parse() too soon, and mess with the go testing flag parsing.
As a result we had to alter some of the variables that we use to build
urls to be functions so that when called the required information is
available.
Fixed several tests not checking for errors.
Fixed dumping code being initialized several times messing with the
order in which failures were reported.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Fixed failing tests.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Set the WaitForFirstConsumer phase on DataVolume when storage uses the WaitForFirstConsumer binding mode and is not bound yet.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Skip PVC if not bound in import|clone|upload controllers.
This is done so the VM pod(not the cdi pod) will be the first consumer, and the PVC can be scheduled on the same location as the pod.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
fixup! Skip PVC if not bound in import|clone|upload controllers.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update importer tests to force bind the PCV by scheduling a pod for pvc, when storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update datavolume tests to force bind the PCV by scheduling a pod for pvc, when storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update upload controller and upload tests to correctly handle force binding the PCV by scheduling a pod for pvc, when storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update clone tests to force bind the PCV by scheduling a pod for pvc when the storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update cloner multi-node tests to force bind the PCV by scheduling a pod for pvc when storage class is wffc.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Correct after automerge
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Improve/simplify tests
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Fix error in import test.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update transport_test,operator_test.go
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update rbac_test.go and leaderelection_test.go
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Improve Datavolume and PVC Checks for WFFC.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Handle wffc only if feature gate is open - import-controller
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* TEST for Handle wffc only if feature gate is open - import-controller - TEST
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Handle wffc only if feature gate is open - upload-controller with test
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* rename and simplify checks
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* cleanup after rebase
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* update tests after rebase
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* update tests after rebase
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* more cleanups
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Document new WFFC behavior
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Document new HonorWaitForFirstConsumer option
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* update docs according to comments
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* extract common function, cleanup - code review fixes
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* add comment for another pr - 1210, so it can have easier merge/rebase
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* typo
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Simplify getStoragebindingMode - code review comments
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Add FeatureGates interface - code review fix
Additionally pass the features gates instead of the particular feature gate value,
and let shouldReconcilePVC decide what to do with the feature gate. That way shouldReconcilePVC
contains all the logic, and the caller does not need to do additional calls to provide parameters.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Update matcher
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* move upload.cdi.kubevirt.io API group to v1beta1
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* move core api to v1beta1
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* fix os-3.11 cluster sync and add functional tests for alpha api
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* change more occurences of v1alpha1
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* updates after rebase
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Add test case where we attempt to import from registry without the CA config map existing.
Verify that the pod remains pending for a while until we create the config map and then the import succeeds.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Add error check on phase.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Switch to using Consistently in test.
Signed-off-by: Alexander Wels <awels@redhat.com>
We were reporting the error generated by the command executions, which normally is something like 'exit status 1' or something similar, which is not really helpful in identifying the problem. Instead this will now report the stderr which should have better information on what exactly failed and why.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Add conditions to DataVolume and add condition annotation to PVCs
Signed-off-by: Alexander Wels <awels@redhat.com>
* Added reason codes to conditions
Updated functional tests to use reason codes
Fixed broken functional test.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Added last transition time
Added documentation
Signed-off-by: Alexander Wels <awels@redhat.com>
* Switch to conditions slice from conditions pointer slice.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Add heartbeat time for conditions.
Renamed AnnRunningCondition to AnnLastTerminationMessage
Only update reason/message on heartbeat or state change.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Rename heartBeat to heartbeat
Re-factor datavolume-conditions to properly update heartbeat time.
Signed-off-by: Alexander Wels <awels@redhat.com>
* renamed AnnRunningConditionReason to AnnLastTerminationReason
Signed-off-by: Alexander Wels <awels@redhat.com>
* Revert lastTerminatedMessage/Reason rename
Added separate AnnScratchBoundCondition/Message/Reason for scratch space
Signed-off-by: Alexander Wels <awels@redhat.com>
* Update bound condition to take scratch or source PVC status into consideration.
Update running condition to take source pod status into consideration
Signed-off-by: Alexander Wels <awels@redhat.com>
* Addressed comments in review.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Properly get scratch space pvc phase.
Signed-off-by: Alexander Wels <awels@redhat.com>
These are basically no-op DataVolumes for PVCs that already exist.
I see this mostly as an internal thing to facilitate VM restore from snapshot
Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
* Clean up client-go left overs after converting to runtime library.
Clean unused functions from controller utils.go
Turn util_test.go into gingko test.
Moved functions from util to proper controller if only used by that controller.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Don't export reconciler variables.
Signed-off-by: Alexander Wels <awels@redhat.com>
* Add a field to DataVolume to track the number of retries/pod restarts
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Add a field to DataVolume to track the number of retries/pod restarts
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Make RESTARTS non-empty on DataVolume (shows as as 0 'zero')
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Test reporting restarts on DataVolume when importing.
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Fix tests
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Code review fixes
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
* Restart Count status test for upload and clone controller
Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
This PR introduces new import data source to fetch disks from ovirt
and store them in storage available in the cluster. It uses ovirt-sdk
to check whether the this is there, the size of the disk and it obtains
signed ticket which allows to get the disk using ovirt-imageio-proxy.
In order to use it a user needs to create configmap with ovirt api
certificate and secret with ovirt engine user and password. Both are
required and validated by webhook. In order to start import process
we need to post following definition:
---
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
name: "imageio-dv"
spec:
source:
imageio:
url: "<engine-api-endpoint>"
secretRef: "<secret-name-containing-engine-credentials>"
certConfigMap: "<configmap-name-containing-engine-api-certificate>"
diskId: "<disk-id-which-we-want-to-import>"
pvc:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: "64Mi"
---
Signed-off-by: Piotr Kliczewski <piotr.kliczewski@gmail.com>
Test 2555[1] (on 'Verify DataVolume' tests) is failing on tier1 cnv 2.1 test.
The issue reproduces when using ubi8 cdi-importer container (see bz#1739149[2]).
I.e. when using 'qemu-img info' on the invalid qcow file[3] (using uni8.1, qemu-img 3.1.0):
$ qemu-img info afl1.img
image: afl1.img
file format: raw
virtual size: 9.5K (9728 bytes)
disk size: 12K
$ qemu-img --version
qemu-img version 3.1.0 (qemu-kvm-3.1.0-30.module+el8.0.1+3755+6782b0ed)
Whereas, using fedora 30 (qemu-img 4.1.0):
$ qemu-img info afl1.img
file format: qcow
virtual size: 152 TiB (167125767422464 bytes)
disk size: unavailable
cluster_size: 4096
$ qemu-img --version
qemu-img version 4.1.0 (qemu-4.1.0-4.fc30)
As 'raw' file format is returned on el8, the supported format validation
isn't failing (on 'qemu.go -> isSupportedFormat), so the import is
completed successfully instead of failing.
Hence, removing this test for now as that file can't be validated
consistently on all envs.
[1] test_id:2555 ('fail creating import dv: invalid qcow large size')
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1739149#c2
[3] https://bugs.launchpad.net/ossa/+bug/1449062/+attachment/4385683/+files/afl1.img
Change-Id: Iadc7a2129cb64a97b0f55dc1553f6a03c0a66ffd
Signed-off-by: Daniel Erez <derez@redhat.com>
Signed-off-by: Fred Rolland frolland@redhat.com
The newer version of qemu-img (3.1.0) that ships in Fedora 30 has issues
with streaming conversions whereby it hangs during the conversion
process. Rather than try to force a downgrade of qemu when building
images instead, let's try reverting back to Fedora29 minimal which has
qemu-img version 3.0.0.
This issue has also exposed the fact that we aren't performing any
testing against streaming external images, we're only testing against
images that we've pulled in to our test infrastucture. So add a
functional test that builds a DataVolume directly from the cirros site.
set dv test back to expectin prlimit failure
Added positive and negative functional tests for
importing tar archive files.
Change-Id: I584ab0e2ae4572a6300cb531ce5f9c02c4449d2b
Signed-off-by: Daniel Erez <derez@redhat.com>
Signed-off-by: Fred Rolland <frolland@redhat.com>
Co-authored-by: Fred Rolland <frolland@redhat.com>
Added functional tests to ensure that malicious
qcow2 images are being rejected. I.e. the creation
of import datavolume should fail when converted to raw.
Change-Id: I3372105fdcd28b8e01811d7e6752f04f5dba9e0d
Signed-off-by: Daniel Erez <derez@redhat.com>
Signed-off-by: Fred Rolland <frolland@redhat.com>
Co-authored-by: Fred Rolland <frolland@redhat.com>
1. Add integration test between DataVolume and import from registry features
2. Add fix for OS environment: set max.user_namespaces for all cluster nodes
