Commit Graph

190 Commits

Author SHA1 Message Date
Ettore Di Giacinto
b29c2d8591 bug: Don't fail when config_url is not accessible (#957)
Don't fail when config_url is not accessible

but rather print a Warning.

Fixes #885

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
Co-authored-by: Dimitris Karakasilis <dimitris@karakasilis.me>
2023-02-24 10:09:22 +01:00
Mauro Morales
5c57dcebdf sparkles: Integrate schema validation (#853)
* Change ValidationError to return the actual error

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add validate command

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Warn validation errors when scanning configs

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Lint

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add schema command to print config json schema

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add strict-validations flag

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Lint and remove focus

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Rename command schema to print-schema

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Fix issue by reading originalData

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Lint

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Remove test from removed feature

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add comments to exported functions

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Lint

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add test for validate.go

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Remove focus

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add more tests for root schema

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add more tests

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

---------

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>
Co-authored-by: Itxaka <itxaka.garcia@spectrocloud.com>
2023-02-14 16:15:13 +01:00
Dimitris Karakasilis
a09db09db7 seedling: Don't swallow yaml config errors (#846)
* Don't swallow yaml config errors

because installation continues and there is no indication that something
went wrong

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>

* Apply suggestions from code review

Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Signed-off-by: Dimitris Karakasilis <jimmykarily@gmail.com>

---------

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
Signed-off-by: Dimitris Karakasilis <jimmykarily@gmail.com>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2023-02-08 22:13:13 +02:00
Mauro Morales
84c68ff0b8 seedling: Kairos config validator library (#798)
* Validate yaml

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* lint feedback

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Validate User name with JsonSchema

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* WIP users validation

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add multiple examples for ssh keys

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add example of complex validation with AnyOf

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Better business rule example with P2P

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Test with message for empty network_token

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Split into a file for each sub section

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add install schema validations

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add to main schema

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add more tests for p2p

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add install schema

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Validate fields between new and old schema

It also adds the missing ones

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Lint

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Remove temp debugging functions

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add new fields in old schema

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add documentation for all exported

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Move schemas into a directory of their own

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Add missing dot at end of comment

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Rebase master and add local_file to bundles

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

---------

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>
2023-02-08 11:02:13 +01:00
Santhosh
0a62e316e0 set local_file bundleconfig to true if target prefix is file:// (#823)
set local file bundleconfig to true if target prefix is file://

Signed-off-by: Santhosh <santhosh@spectrocloud.com>
2023-02-07 18:13:03 +05:30
Oz Tiram
b37a34beae sparkles: custom user bind and ephemeral mounts (#692)
*  custom user bind and ephemeral mounts

Users can now specify custom and ephemeral mounts in cloud-init under
the `install` section, e.g.:

```
users:
 - name: kairos
...
install:
  auto: true
  device: "auto"
  bind_mounts:
  - /mnt/bind1
  - /mnt/bind2
  ephemeral_mounts:
  - /mnt/ephemeral
  - /mnt/ephemeral2
...
```
Ephemeral mounts are mounted as RW - but changes are discarded when the
machine is restart.
Bind mounts will persist changes after restarted.

This is a fix for #210

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* Add the custom user mount to /cos/run/cos-layout.env

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* Add docs for custom user mounts in configuration example

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* Reuse test_install function

DRY the code, change how we call the function install_test

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* Enable custom mount tests and install tests

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* Enable tests in CI

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* Remove duplicate if check

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* Remove uneeded fmt.Println ...

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* Use separate label for custom mounts

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* 🔧 Earthfile - DRY ginkgo

We repeat this a more than twice so just extract to own
target...

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* Correct return type for  ContainElements

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* Remove CLOUD_INIT from custom mounts test

This is not needed here.

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* Fix qemu-test-* earthly targets

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* 🔧 Fix CPU passing to VM

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* 🔧 remove apt cache after install qemu

Helps deal with space running out in the CI.

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

* 📖 Document custom mounts

Signed-off-by: Oz Tiram <oz@spectrocloud.com>

---------

Signed-off-by: Oz Tiram <oz@spectrocloud.com>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2023-02-03 11:04:02 +01:00
Ettore Di Giacinto
da44dbb47f seedling: Drop unrequired copy (#683)
Signed-off-by: mudler <mudler@c3os.io>

Signed-off-by: mudler <mudler@c3os.io>
2023-01-20 10:29:35 +01:00
Mauro Morales
1a9104c121 bug: Remove duplicated list of upgradable releases (#688)
* Remove duplicated list of upgradable releases

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

* Make list output consistent

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>

Signed-off-by: Mauro Morales <mauro.morales@spectrocloud.com>
2023-01-19 22:16:13 +01:00
Dimitris Karakasilis
704cff80d2 Merge pull request #665 from kairos-io/399_deeply_merge_configs
Deeply merge yamls in original data
2023-01-18 11:16:05 +02:00
Ettore Di Giacinto
cffbc3f35f seedling: Webui enhancements (#620)
* 🌱 Make sure webui starts on alpine

Also drop to shell when there are no providers

Signed-off-by: mudler <mudler@c3os.io>

* 🌱 Suppress verbose logging on tty

Signed-off-by: mudler <mudler@c3os.io>

* 🌱 Print WebUI address

Signed-off-by: mudler <mudler@c3os.io>

* 🎨 Update banner

Signed-off-by: mudler <mudler@c3os.io>

* 🌱 Refactor, display also interfaces

Signed-off-by: mudler <mudler@c3os.io>

* 🌱 Address feedback from review

Signed-off-by: mudler <mudler@c3os.io>

Signed-off-by: mudler <mudler@c3os.io>
2023-01-08 21:49:23 +01:00
Ettore Di Giacinto
e15014addc seedling: Webui installer (#587)
* 🌱 Add webui

Signed-off-by: mudler <mudler@c3os.io>

* 🌱 Re-read config files after loading bundles

Signed-off-by: mudler <mudler@c3os.io>

* [check-spelling] Update metadata

Update for https://github.com/kairos-io/kairos/actions/runs/3806058276/attempts/1
Accepted in https://github.com/kairos-io/kairos/pull/587#issuecomment-1367859480

Signed-off-by: check-spelling-bot <check-spelling-bot@users.noreply.github.com>
Signed-off-by: mudler <mudler@c3os.io>

* 🎨 Beautify index page

Signed-off-by: mudler <mudler@c3os.io>

* Do not rerun if we were successful or we are already running

Signed-off-by: mudler <mudler@c3os.io>

* Add syntax highlight

Signed-off-by: mudler <mudler@c3os.io>

* Add error message

Signed-off-by: mudler <mudler@c3os.io>

* Add YAML validation and highlight

Signed-off-by: mudler <mudler@c3os.io>

* Fixup terminal output

Signed-off-by: mudler <mudler@c3os.io>

* Fix newlines

Signed-off-by: mudler <mudler@c3os.io>

* fixups

Signed-off-by: mudler <mudler@c3os.io>

* 🎨 Fixup lint issues

Signed-off-by: mudler <mudler@c3os.io>

* Mark dependencies

Signed-off-by: mudler <mudler@c3os.io>

* Let configure the listening address

Signed-off-by: mudler <mudler@c3os.io>

Signed-off-by: mudler <mudler@c3os.io>
Signed-off-by: check-spelling-bot <check-spelling-bot@users.noreply.github.com>
2023-01-05 14:15:05 +01:00
Ettore Di Giacinto
741ac7a385 seedling: Keep current dir when installing runnable bundles (#582)
Signed-off-by: mudler <mudler@c3os.io>

Signed-off-by: mudler <mudler@c3os.io>
2022-12-23 16:51:54 +01:00
Ettore Di Giacinto
a909c44620 seedling: Return configuration data from the agent (#567)
* 🌱 Return configuration data from the agent

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>

* 🤖 Cover query by tests

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2022-12-19 17:01:57 +01:00
Ettore Di Giacinto
2bf4793c41 bug: Process only cloud configs (#550)
Regression introduced in https://github.com/kairos-io/kairos/pull/502.
Now also files like .lst inside the initramfs are globbed inside /oem/90_custom.yaml

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2022-12-10 18:51:25 +01:00
Ettore Di Giacinto
3d92e4ae43 seedling: Glob all configs found (#502)
* 🌱 Glob all configs found

This allows to read all the configs found during scanning - this is especially useful in first boot as it allows
to have separate config file logic split into several files

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>

* 🌱 Simplify config merge logic

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>

* WIP

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>

* 🌱 Make the function work

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Signed-off-by: mudler <mudler@kairos.io>

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
Signed-off-by: mudler <mudler@kairos.io>
Co-authored-by: Dimitris Karakasilis <dimitris@karakasilis.me>
2022-11-28 15:37:45 +01:00
Ettore Di Giacinto
8bf1babcce bug: Setup / as shared mount on alpine (#445)
* 🐛 Setup / as shared mount on alpine

Since this is not handled in the boot process, we do this in our devoted file to setup the rootfs configuration

Signed-off-by: Ettore Di Giacinto <mudler@kairos.io>

* 🐧 Correctly detect alpine flavors

Signed-off-by: Ettore Di Giacinto <mudler@kairos.io>

Signed-off-by: Ettore Di Giacinto <mudler@kairos.io>
2022-11-25 20:38:19 +01:00
Ettore Di Giacinto
ade95362cb bug: Preserve header when fetching config via URL (#492)
Fixes: https://github.com/kairos-io/kairos/issues/491

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2022-11-25 13:54:39 +01:00
Eng Zer Jun
c080dc6220 art: Move from io/ioutil to io and os packages (#470)
refactor: move from io/ioutil to io and os packages

The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

[1]: https://golang.org/doc/go1.16#ioutil
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-11-22 01:11:03 +08:00
Ettore Di Giacinto
ad8fc74813 arrow_up: Bump repository and drop some workarounds (#434)
* ⬆️ Bump Kairos repositories

Signed-off-by: Ettore Di Giacinto <mudler@mocaccino.org>

* 🐧 Find out which grub to call

Signed-off-by: Ettore Di Giacinto <mudler@mocaccino.org>

Signed-off-by: Ettore Di Giacinto <mudler@mocaccino.org>
2022-11-16 11:44:18 +01:00
Santhosh
fd39b2fa79 set configScanDir as package scoped var (#264)
set config.Install.env variables in environment

adding env to config to use across all stages

switch to config.Scan to read config
2022-10-24 16:43:23 +05:30
Ettore Di Giacinto
5177492f57 sparkles: enhancements (#263)
* 🎨 Do not need to prefix '.' on queries

*  Add recovery to stateapi

*  Add cloudconfig SDK

*  Unify post-install/firstboot hooks

This also adds capabilities to add grub option at first boot rather than
after installation as for bundles.

* 🤖 Optimize tests to wait for state to change
2022-10-24 08:34:49 +02:00
Ettore Di Giacinto
07d524e484 sparkles: state api sdk (#262)
*  Add state api

This is related to https://github.com/kairos-io/kairos/issues/34.

Starts to unify the API to retrieve the state in the sdk to have a common place to query system status information.

* 🤖 Add test

* Update go.mod

* ⚙️ Fine-tune detection of partitions

* 🤖 Add more fine-grained tests

* 🎨 Add /dev/ to partition name

* 🤖 Fixup tests

* ⚙️ Remount accessors

*  Add state partition to cloud-init paths

* 📝 Upper case Kairos in motd

* 🎨 Add mounts sdk

* 🎨 Set grub options via SDK

* 🎨 Make it more idiomatic
2022-10-23 20:22:32 +02:00
Ettore Di Giacinto
e0f54f296e sparkles: Minor enhancements (#239)
* 🐛 Fixup grub option quoting

* ⚙️ Copy discovery to oem if found

*  Add environment block to install

* ⚙️ Use /oem for mount in kcrypt post-hook

* 📝 Update docs with installer env reference

* 🤖 Add test deps

* ⚙️ Be consistent and set env also for post-hooks

* ⚙️ propagate env in post-hooks
2022-10-18 07:45:07 +02:00
Ettore Di Giacinto
5d8fd9e044 sparkles: Add kcrypt to images (#195)
*  Add kcrypt to images

This adds the dracut module and the binary to the images.

Related to https://github.com/kairos-io/kairos/issues/184

*  Add Kcrypt post-install hook

Fixes https://github.com/kairos-io/kairos/issues/184
2022-10-07 13:36:32 +02:00
Ettore Di Giacinto
a2d801efe5 sparkles: Add bundles to post-install hooks (#171)
* 🤖 Add bundles and sysext test

*  Exec bundles also after install

* 🤖 Adapt tests

* 🎨 Create dir only if doesn't exist

* 🎨 Return err on mount

* 🎨 Make bundle errors failure as an option

* 🎨 Minor fixups

* debug

* 🤖 Fix spec

* 🤖 Get correct version for bundle test

* 🎨 Fixups

* 🤖 systemd-sysext is available only on opensuse for now
2022-10-03 00:16:01 +02:00
Ettore Di Giacinto
5bee05cccc art: Rebranding (#130)
See: https://github.com/c3os-io/c3os/issues/88
2022-09-17 18:43:51 +02:00
Ettore Di Giacinto
bb73b8164c sparkles: Boot options (#91)
*  Add grub_options to config

Split post-install into hooks

* 🤖 Adapt test to latest releases
2022-09-08 15:39:26 +02:00
Ettore Di Giacinto
8518a3752e art: Move ListOutput under utils 2022-08-18 13:14:12 +00:00
Ettore Di Giacinto
212cf5132a art: Expose to sdk upgrade/list events 2022-08-18 13:12:05 +00:00
mudler
fdad13bd10 robot: Run only relevant tests, push to core-*
From now on images built from this repo are the c3os core images.
c3os core images contains only the base image, framework files and the c3os agent.

This also sets up tests accordingly to run only to cover the c3os-agent.
2022-08-12 15:49:41 +02:00
mudler
21d6393de3 art: Expose interactive install to providers
In this way install can be driven from providers too

Part of: https://github.com/c3os-io/c3os/issues/68
2022-08-12 12:21:12 +02:00
mudler
ba3c157cf8 art: Move out services
Part of: https://github.com/c3os-io/c3os/issues/68
2022-08-12 09:51:46 +02:00
mudler
f2fd0221b5 art: Drop provider from c3os code
Part of: https://github.com/c3os-io/c3os/issues/68
2022-08-10 18:56:07 +02:00
Ettore Di Giacinto
aee6631687 Merge pull request #55 from c3os-io/feature/cluster-provider
Abstract Cluster Provisioning with Cluster Plugins
2022-08-09 08:01:54 +02:00
Ettore Di Giacinto
ea12ef6cd2 robot: Make linter happy 2022-07-25 22:26:10 +00:00
Ettore Di Giacinto
a5437561eb art: Refactor out config sections
Now there is a `install` section in the config that has the fields that previously where in `c3os` but
were actually only used during install phase.

Also the k3s and c3os config were moved to the provider instead that in the global config.
2022-07-16 20:47:55 +00:00
Ettore Di Giacinto
3af29389d3 robot: Try to fixup tests 2022-07-16 14:28:55 +00:00
Ettore Di Giacinto
5f1ec1c03e gear: Add c3os bundles 2022-07-13 22:23:47 +00:00
Ettore Di Giacinto
2e86e483a7 gear: Extract netboot artifacts
This changeset also adds a `config_url` and `options` keyword in the c3os config.
Along with that the config logic is changed so the configuration is taken also from boot commands and merged in the final installed config file.
2022-07-07 16:57:38 +00:00
Ettore Di Giacinto
0310996d7f Split off cli into separate binaries (#37)
* 🎨 Split off cli into separate binaries

This commit splits off the cli into 3 binaries:
- agent
- cli
- provider

The provider now is a separate component that can be tested by itself
and have its own lifecycle. This paves the way to a ligher c3os variant,
HA support and other features that can be provided on runtime.

This is working, but still there are low hanging fruit to care about.

Fixes #14

* 🤖 Add provider bin to releases

* ⚙️ Handle signals

* ⚙️ Reduce buildsize footprint

* 🎨 Scan for providers also in /system/providers

* 🤖 Run goreleaser

* 🎨 Refactoring
2022-07-04 22:39:34 +02:00