Commit Graph

50 Commits

Author SHA1 Message Date
Bartosz Rybacki
ab8b9c025e
Generating label names (#1200)
* Handle labels length correctly

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle service name generation correctly

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Remove not needed labels

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Store import pod name in annotation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Enable long DV name

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle name with dot when creating service/label name

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Test long names on import,  upload and clone

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Store upload pod name in annotation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Store importer scratch pvc name in annotation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Quick fix for tests (need improvements)

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Cleanup handling scratch name

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Ensure pod/service name conflicts are handled

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Handle client errors when trying to get the import pod

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Style improvements, and other code review fixes.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Store clone source pod name in an annotation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Correct name initialization and tests

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Do not init name if pod already exists. It is not needed.

The situation of having a pod but not name on annotation can happen after the upgrade, when we have a legacy pvc and pod already existing, but clone operation not finished. But when we already have the pod, then in the code (currently) we do not need the name from annotation.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Cleanup scratch name handling

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Use constant for max dv name in validation

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Simplify clone source pod name initialization

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-05-29 19:55:32 +02:00
Alexander Wels
c707e781a9
Add conditions to DataVolume and add condition annotation to PVCs (#1194)
* Add conditions to DataVolume and add condition annotation to PVCs

Signed-off-by: Alexander Wels <awels@redhat.com>

* Added reason codes to conditions
Updated functional tests to use reason codes
Fixed broken functional test.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Added last transition time
Added documentation

Signed-off-by: Alexander Wels <awels@redhat.com>

* Switch to conditions slice from conditions pointer slice.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Add heartbeat time for conditions.
Renamed AnnRunningCondition to AnnLastTerminationMessage
Only update reason/message on heartbeat or state change.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Rename heartBeat to heartbeat
Re-factor datavolume-conditions to properly update heartbeat time.

Signed-off-by: Alexander Wels <awels@redhat.com>

* renamed AnnRunningConditionReason to AnnLastTerminationReason

Signed-off-by: Alexander Wels <awels@redhat.com>

* Revert lastTerminatedMessage/Reason rename
Added separate AnnScratchBoundCondition/Message/Reason for scratch space

Signed-off-by: Alexander Wels <awels@redhat.com>

* Update bound condition to take scratch or source PVC status into consideration.
Update running condition to take source pod status into consideration

Signed-off-by: Alexander Wels <awels@redhat.com>

* Addressed comments in review.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Properly get scratch space pvc phase.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-05-12 23:39:23 +02:00
Bartosz Rybacki
bfe63cb61c
Added GetPodName function used to correctly generate pod names in con… (#1175)
* Create corect naming functions for resources

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Use name generator for importer resources

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-05-07 16:31:19 +02:00
Alexander Wels
08ed6f2b39
Clean up client-go left overs after converting to runtime library. (#1163)
* Clean up client-go left overs after converting to runtime library.
Clean unused functions from controller utils.go
Turn util_test.go into gingko test.
Moved functions from util to proper controller if only used by that controller.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Don't export reconciler variables.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-04-02 22:59:54 +02:00
Bartosz Rybacki
4605cf1dc2
Add a field to DataVolume to track the number of retries/pod restarts (#1155)
* Add a field to DataVolume to track the number of retries/pod restarts

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Add a field to DataVolume to track the number of retries/pod restarts

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Make RESTARTS non-empty on DataVolume (shows as as 0 'zero')

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Test reporting restarts on DataVolume when importing.

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Fix tests

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Code review fixes

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>

* Restart Count status test for upload and clone controller

Signed-off-by: Bartosz Rybacki <brybacki@redhat.com>
2020-03-30 23:17:49 +02:00
Alexander Wels
f2657d5927
Set secondary GID of user on import and upload pods to be 107 (qemu) … (#1148)
* Set secondary GID of user on import and upload pods to be 107 (qemu) for kubevirt content type.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Fix broken unit tests on upload.

Signed-off-by: Alexander Wels <awels@redhat.com>

* Add import and clone test.

Signed-off-by: Alexander Wels <awels@redhat.com>
2020-03-26 14:33:48 +01:00
Piotr Kliczewski
78cee6d166
Imageio - new import data source (#1042)
This PR introduces new import data source to fetch disks from ovirt
and store them in storage available in the cluster. It uses ovirt-sdk
to check whether the this is there, the size of the disk and it obtains
signed ticket which allows to get the disk using ovirt-imageio-proxy.

In order to use it a user needs to create configmap with ovirt api
certificate and secret with ovirt engine user and password. Both are
required and validated by webhook. In order to start import process
we need to post following definition:

---
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
  name: "imageio-dv"
spec:
  source:
      imageio:
         url: "<engine-api-endpoint>"
         secretRef: "<secret-name-containing-engine-credentials>"
         certConfigMap: "<configmap-name-containing-engine-api-certificate>"
         diskId: "<disk-id-which-we-want-to-import>"
  pvc:
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: "64Mi"
---

Signed-off-by: Piotr Kliczewski <piotr.kliczewski@gmail.com>
2020-03-19 19:49:44 +01:00
Alexander Wels
79b144088f
Copy kubevirt.io annotations to scratch space pvc, but NOT cdi.kubevirt.io as those are what triggers import/upload/clone. (#1109)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-02-20 16:50:50 +01:00
Alexander Wels
17db2920b1
Update clone controller to use runtime library. (#1075)
Signed-off-by: Alexander Wels <awels@redhat.com>

Co-authored-by: Michael Henriksen <mhenriks@redhat.com>
2020-02-14 23:04:46 +01:00
Alexander Wels
5ed55cb22c
Remove code that potentially let DV get into failed state with RestartOnFailure policy. (#1099)
Signed-off-by: Alexander Wels <awels@redhat.com>
2020-02-10 23:29:15 +01:00
Michael Henriksen
99f8af5b86 k8s client upgrade to 1.16 (#1079)
* initial client upgrade to 1.16

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix Route detection in OpenShift

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2020-01-14 13:43:17 +01:00
Woohyung Han
2adcc80f95 Fix to create temporary pod with default resource quota (#1055)
If we create a temporary pod in the namespace which has
resource quota, `must specify quota` error occurs. Fix to
create temporary pod with default resource quota.

Co-authored-by: hyoung-90 <ssarang520@gmail.com>
Co-authored-by: Woohyung Han <woohhan@gmail.com>
Signed-off-by: anencore94 <anencore94@kaist.ac.kr>

Co-authored-by: anencore94 <37469330+anencore94@users.noreply.github.com>
Co-authored-by: hyoung-90 <58063867+hyoung-90@users.noreply.github.com>
2020-01-09 17:43:14 +01:00
Alexander Wels
f090ddc2e6 Update Import controller to use runtime controller library. (#1054)
Refactor import related utilities out of util.go into the controller.

Signed-off-by: Alexander Wels <awels@redhat.com>
2019-12-20 19:28:43 +01:00
Alexander Wels
d4a4ece09b Requeue on failure like sample k8s controller (#1015)
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-11-12 13:21:15 +01:00
Michael Henriksen
66a1c8c996 Unique client cert per clone source pod (#987)
* unique cert per clone source pod

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* upload controller shouldn't cache certs

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>

* fix test variable names

Signed-off-by: Michael Henriksen <mhenriks@redhat.com>
2019-10-09 15:10:55 -04:00
zvikorn
7dd334121c Creating blank disk on a block device (#962)
Signed-off-by: tavni <tavni@redhat.com>
2019-09-18 19:06:02 -04:00
Michael Henriksen
834b85ecbf Network clone (#897)
* network cloning

* fix clone progress
2019-08-01 16:01:25 -04:00
Alexander Wels
1d534d58d5 Record import failure messages
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-05-30 09:22:58 -04:00
Alexander Wels
630a23ef23 Fix a bunch of go score card issues.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-05-06 16:52:03 -04:00
tavni
d4c7b0be54 Adding support to import disk image into Block PV
Signed-off-by: tavni <tavni@redhat.com>
2019-03-28 11:43:56 +02:00
Alexander Wels
afe77c3514 Allow missing storage class for scratch space, if none exists, create scratch PVC without SC
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-03-27 11:41:15 -04:00
Alexander Wels
d2ca78bec3 Use CDIConfig for scratch storage class.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-03-21 15:37:07 -04:00
Alexander Wels
1e46004ad2 Scratch space retry framework logic.
Signed-off-by: Alexander Wels <awels@redhat.com>
2019-03-07 14:54:08 -05:00
Y.Horie
c9104059fb move to klog from glog 2019-03-05 13:33:20 +09:00
Michael Henriksen
6f1d130d97 tests and review comments 2019-02-25 20:12:56 -05:00
Michael Henriksen
62f1e77f66 change INSECURE_REGISTRY to INSECURE_TLS because may want to have insecure hosts for http import 2019-02-25 20:12:56 -05:00
Michael Henriksen
3892a7310d add configmap for insecure regestries 2019-02-25 20:12:56 -05:00
j-griffith
30c0e1b814 Delete importer pod on pvc delete request
Related to issue #525, we have a similar situation when using PVCs
without DataVolumes.  The user requests a PVC with CDI Importer
annotations, if the Importer POD is in a fail/retry loop and the user
requests deletion of the PVC the PVC will be stuck in terminating
waiting for the user to forcibly delete the running Import POD.

This PR adds a check for PVC delete in the import controller and
will delete the POD explicitly if it sees a PVC delete request.  This
will also be picked up for Import DataVolumes, so we can remove the
extra checks that were previously added in the DV controller.

fixes #649
fixes #525
2019-02-20 13:43:09 -07:00
Michael Henriksen
717ceaa919 add ability for user to specify certs for import from registry 2019-02-18 09:06:46 -05:00
Michael Henriksen
fbaee779f7 don't set pvc pod phase as error when import pod is starting for the first time 2019-02-11 09:57:04 -05:00
tavni
aebc2b3e87 Add ability to create empty disk images #497
Signed-off-by: tavni <tavni@redhat.com>
2018-11-27 16:15:44 +02:00
Alexander Wels
260d4fa8e0 Automatically resize image to fit the requested PVC.
Combined code from PR#489 and PR#490 by
@gites and
@danielerez
Added some tests and rebased on current master.

Signed-off-by: Alexander Wels <awels@redhat.com>
2018-11-20 15:40:13 -05:00
Alexander Wels
70666e5946 Add source and contentType annotations
Signed-off-by: Alexander Wels <awels@redhat.com>
2018-11-14 12:20:26 -05:00
tavni
bc6a7b8474 change the import controller not to have global parameter #504
Signed-off-by: tavni <tavni@redhat.com>
2018-10-25 20:49:04 +03:00
tavni
c7e40ebffd Add structure to hold all CDI importer environment variables #501
Signed-off-by: tavni <tavni@redhat.com>
2018-10-25 13:21:28 +03:00
tavni
4db7a28eb5 Import/upload pvc 'cdi.kubevirt.io/storage.pod.phase' annotation value is wrong when import fails. Issue #494
Signed-off-by: tavni <tavni@redhat.com>
2018-10-18 17:47:57 +03:00
j-griffith
371e5c2ba0 Set pod restart policy to "OnFailure" and cleanup
This patch does two things:
1. We modify the restar policy from never to "OnFailure" for the
    Importer (we already did this for Upload, and we don't want to do it yet
    for the Clone pods)
2. Upon succesful completion delete our worker pods
    We're not doing this on failed pods right now because in general an
    admin will likely want to inspect the pod to see why it failed.

The first question someone will likely ask is "well what about a node
failure", well in the case of a node failure for CDI we have a bigger
outstanding issue remaining to figure out how to deal with volume
connections.  We can't just simply reschedule the job.
2018-10-09 08:25:22 -06:00
tavni
ce22c48834 CDI controller code dedupe
Signed-off-by: tavni <tavni@redhat.com>
2018-09-17 19:44:09 +03:00
John Griffith
c455577cca Fix lint errors on common package (#449)
This change just updates the common package to pass golint.  Of course
that has some reaching implications into other packages with renaming of
constants.

I've intentionally kept this patch set to the bare minimum for the lint
test, I do think that we should put some effort into our use of
constants (especially those in common.go) in the future.

Ideally we'd declare constants where they're used, and we wouldn't
export them if they don't need to be.  There are some in here that
are only used in one or two packages, and that can be cleaned up pretty
easily however it's likely that there might be plans to use them
elsewhere in the future.
2018-09-10 09:14:09 -04:00
David Vossel
7908038b94 Ensure expectation keys are deleted when object is deleted from cluster (#433)
Signed-off-by: David Vossel <davidvossel@gmail.com>
2018-09-07 08:38:28 -04:00
John Griffith
9468ba94c7 Fix golint errors on pkg directory (#414)
Note we are intentionally skipping test files for now, golint
pkg/controller... will skip them, so I'll skip them to.
2018-09-05 12:17:38 -07:00
John Griffith
69709bf863 Update no annotation PVC import test. (#403) (#408)
- Set verbosity on controller to 3 to enable simpler tests.

Signed-off-by: Alexander Wels <awels@redhat.com>
2018-09-05 11:10:24 -07:00
David Vossel
fdb1b27467 Single Queue and Expectations (#249)
* Pull in expectations code from kubevirt

Signed-off-by: David Vossel <davidvossel@gmail.com>

* add expectations to datavolume controller

Signed-off-by: David Vossel <davidvossel@gmail.com>

* Refactor importer pod to use single queue and expectations

Signed-off-by: David Vossel <davidvossel@gmail.com>

* update utils tests to use single queue

Signed-off-by: David Vossel <davidvossel@gmail.com>

* re-introduce import controller test suite

Signed-off-by: David Vossel <davidvossel@gmail.com>

* make clone controllers pass after utils refactor

Signed-off-by: David Vossel <davidvossel@gmail.com>

* Use log level constants in cdi controllers

Signed-off-by: David Vossel <davidvossel@gmail.com>

* Remove useless shadow variables in expectations pkg

Signed-off-by: David Vossel <davidvossel@gmail.com>

* required changes to get owner references working with openshift 1.10

Signed-off-by: David Vossel <davidvossel@gmail.com>

* update functional tests to work with importer controller refactor

Signed-off-by: David Vossel <davidvossel@gmail.com>

* Rename AnnImportPVC to LabelImportPvc in order to reflect its use

Signed-off-by: David Vossel <davidvossel@gmail.com>

* Add comment about expectations code

Signed-off-by: David Vossel <davidvossel@gmail.com>
2018-07-30 12:09:15 -05:00
Jonathan Cope
11841e6d2d Fixed unreachable log line (#263) 2018-07-16 13:10:18 -07:00
David Vossel
7106bf73dc Conform with top level kubevirt github project (#198)
* Place cdi packages under the top level kubevirt.io project path

Signed-off-by: David Vossel <davidvossel@gmail.com>

* Change all references to cdi.io to cdi.kubevirt.io

Signed-off-by: David Vossel <davidvossel@gmail.com>

* Add go_import_path to travis config

Signed-off-by: David Vossel <davidvossel@gmail.com>

* update test package imports

Signed-off-by: David Vossel <davidvossel@gmail.com>
2018-06-21 15:44:52 -07:00
jeff vance
e8233a3786 change cdi anno from to 2018-06-14 13:50:49 -07:00
Jonathan Cope
941e6e8413
Revert "change controller labels from 'kubevirt.io' to 'cdi.io'" 2018-06-14 12:03:27 -05:00
jeff vance
bc3aec2280 change controller labels to 'cdi.io' 2018-06-12 11:16:26 -07:00
David Vossel
9a9f19dcd6 datavolume controller
Signed-off-by: David Vossel <davidvossel@gmail.com>
2018-06-11 21:58:15 -04:00
David Vossel
3e37aead88 Refactor cdi controller in preparation for multiple controllers
Signed-off-by: David Vossel <davidvossel@gmail.com>
2018-06-11 21:58:15 -04:00