Changelog

We aim to make at least 1 new release every month. Croit releases are named after the year and month they are released in: "vYYMM.X", 2 digits for the year, 2 for the month, and an additional revision number that resets to 0 at the start of the next month.

v2411.0

This release introduces experimental support for DM multipathing, along with several improvements for our new UI.

Updated the UI to Svelte 5.
Added tmux to the docker container.
Added experimental support for DM multipathing.
DM multipathing for all disks can now be enabled via the croit setting croit enable dm multipath support. The multipath configuration can be edited in the new multipath-conf template.
We do not recommend to enable this, but offer it for hardware setups that would benefit from it.
Improved the cluster installation wizard.
Among other things, we added a step to configure hostnames before installing the cluster.
Improved the recovery speed setting in the croit UI.
The setting is now a slider again with better explanations on what is being modified.
The CephFS Explorer now properly shows symlinks and their targets.
Improved UI navigation on mobile touch devices.
Fixed some issues in the UI when inputting a size in bytes.
Previously, there were some formatting issues with larger units such as kB, GB, etc.
Fixed an issue where the Pools page would time out because of RBD mirror.
Fixed an issue with RGW sizes and quotas in the UI using the wrong unit conversion (KiB vs kB).
Fixed an issue where RGW bucket and user quotas couldn't be unset in the new UI.
Fixed an issue where pool quotas couldn't be unset in the new UI.
Fixed an issue where files or folders with special characters in their name couldn't be deleted in the CephFS Explorer.
Fixed an issue that caused the cluster IOPS and bandwidth graphs to not show any data.
Fixed an issue where the UI froze during cluster setup.
Fixed an issue where the terminal sometimes broke.
Fixed an issue where the redirect page when croit is down would redirect to itsself.
Several usability improvements.

v2411 croit enterprise image updates

Added xinetd for the checkmk hook script. The hook script has also been updated to reflect this.
Added kernel-firmware-network to include some specific network drivers.
Added gdb and valgrind to the extended image to allow for easier debugging.

v2410.0

This is a smaller release containing some bug fixes.

Updated some dependencies for the UI.
Improved delete protection column on the pools page.
Improved some task advisor messages and buttons.
Fixed an issue where the UI froze when entering an IPv6 address in certain fields.
Fixed an issue where some hook events would unexpectedly trigger when adding the first hook script for these events.
Fixed units of the Read/Write columns on the disks page.
Fixed an unnecessary NVMe-oF configuration check on croit startup.
Fixed an issue where manually refreshing the license would cache an old license.
Fixed units in the recovery graphs.
Remove toggle for IPv6 during cluster setup.

v2410 croit enterprise image updates

General OS updates, no specific changes otherwise.

v2408.2

This is a hotfix release to fix some regressions in the old croit UI, and improve the new CRUSH map view.

Fixed an API issue with starting/stopping/restarting services in the old UI.
Added back the number of PGs in the new CRUSH map view for OSDs that are marked out.

v2408.1

This is a hotfix release to fix an issue with the logs page.

Fixed the UI freezing when accessing the logs page in light mode.

v2408.0

For this release we marked Ceph Reef as stable (using version 18.2.4). We also updated the CRUSH map view and added support for running multiple RGW and MDS per server. You can read more about this release in our blog: https://croit.io/blog/croit-v2408

Marked the upgrade to Ceph Reef as stable.
Updated the CRUSH map view.
The CRUSH map is now more compact to allow displaying more OSDs at once. It also has a search field in the top right, allowing you to quickly search for an OSD or server.
Added support for running multiple RGW or MDS on a single server.
This feature needs to be enabled in the croit settings (see here).
We don't recommend running multiple RGW/MDS per server in general, but in certain hardware setups it may lead to better performance.

Added support for active/passive bonds over Infiniband in the new croit UI.
This was previously blocked in the UI and is now allowed.
Updated behavior of OnHealthDegrade, OnHealthRecover and OnHealthImprove hook events to ignore muted warnings.
Updated and fixed the SNMP hook script.
Updated and fixed the Zabbix hook script.
Enabled the /debug endpoint in the API.
It offers some diagnostic information and functionality that should only be used by croit support engineers.
Added pre-defined time ranges of 5 and 10 minutes for metrics and logging pages.
Added a button to open an IPMI interface.
You can now select and IPMI interface and click on "Go To IPMI" to open the IPMI interface in a new tab. You can also double-click on the interface.
Added some settings to configure croit's CORS behavior.
Fixed the "Turn on LED" button on the CRUSH map view.
This button didn't work as expected and turned on the LED of an incorrect disk. Note that this only applied to the CRUSH map page.
Fixed MDS service state showing "UNKNOWN" sometimes in the new croit UI.
Fixed several issues in the croit UI when editing ceph keys.
Fixed an issue where pools were sometimes created with ceph's default CRUSH rule.
Fixed an issue where the last VLAN on a NIC couldn't be removed.
Fixed an issue where the croit UI would sometimes break when viewing OSD metrics.
Fixed an issue where double-clicking on a service wouldn't link to the correct gateway page.
Fixed some issues with the size unit when editing RBDs.
Removed a harmless error message "Error when mapping server hardware data" that sometimes appeared in the croit logs.
Fixed a bug where a server is assigned to a hook script multiple times.
This bug could only be triggered by manually calling the API, and not via the croit UI.
Fixed an issue for the NVMe-oF UI where subsystems that NVMe-oF disks are attached to wouldn't be listed in the "Subsystems" column.
Fixed some cosmetic issues and typos in the croit UI.

v2408 croit enterprise image updates

Updated reef to 18.2.4.

v2407.2

This release contains some improvements to NVMe-oF and the new croit UI.

Added the IPMI view to the new UI.
Added a button to restart croit.
The button is located under Maintenance > Croit Updates in the new UI, or on the Maintenance page in the old UI.
Fixed an issue where certain croit services took a long time to start.
Some hosts running the croit docker container had high ulimits for open file descriptors, which caused delays when starting services like ntp and carbonapi. Croit now sets a lower limit at the startup to prevent this.
Fixed and improved rolling restart of NVMe-oF services.
Added missing service stop button to the NVMe-oF servers page.
Fixed an issue in the new UI when configuring a ceph key for CephFS limited to a namespace.
Breaking changes for the services API.
The request body for the /services/[start|stop|restart|rolling-restart] endpoints has been changed. The type field is not used anymore, and instead isOsd should be set to true when referring to an OSD.

v2407.1 croit enterprise image updates

Updated SPDK to work for older CPUs.

v2407.1

This is a hotfix release for some UI fixes.

Fixed rolling reboot feature to consider NVMe-oF.
Fixed an issue where the default rbd pool couldn't be selected after creating the first iSCSI gateway.
Fixed an issue while setting up NVMe-oF where listeners were still showing NICs from the previous server after selecting another server.
Allow configuring HA groups on top of VLANs in the new UI.
Cosmetic fixes and improvements.

v2407.0 croit enterprise image updates

Samba has been updated to 4.18.11.
SPDK has been added (for NVMe-oF support).
Added tmux.

v2407.0

This release adds support for NVMe-oF (NVMe over Fabrics) on Ceph Quincy or later.

Added support for NVMe-oF.
NVMe-oF gateways can only be configured in the new croit UI on Ceph Quincy (or later) clusters. Servers need to be booted with the latest croit image (it contains spdk, a requirement for NVMe-oF).
Enabled the new croit UI for new clusters by default.
Existing clusters will continue using the old UI.

Added a new hook event OnHealthImprove has been added.
The new hook is triggered whenever ceph health improves (e.g. some of the warnings disappear). Hook events OnHealthDegrade and OnHealthRecover have also slightly been altered to reduce spam when the ceph health is quickly changing over a short period of time. The behavior of these hook events can be fine-tuned with the croit settings croit health hook delay and croit health hook count cutoff.
All used croit server images are will now be downloaded when restoring from a croit backup.
Updated the Slack hook script.
Fixed some confusing task advisor warnings during and after the ceph upgrade.
Fixed an issue in the new croit UI while the croit app is being restarted.
Removed an incorrect context menu in the new croit UI.
Prevent RGW/MDS being created while OSDs are still being created.
Otherwise, the RGW/MDS creation may fail if not enough PGs are available while setting up a new cluster.
Fixed an issue where finished tasks would disappear early in the new UI.
Fixed an unnecessary error log during croit startup in new croit clusters.
Fixed some errors not being displayed properly in the new UI.
Fixed which hook events were associated with some of our library scripts.
Fixed an issue when booting nodes using a custom iPXE binary.
Removed some unnecessary characters in the collectd configuration files.

v2405.1

This release contains some fixes and improvements for the new croit UI.

Improved the setup procedure in the new UI.
Added better validation for S3 bucket names in the new UI.
Improved I/O graphs for DAOS.
Improved configuring ceph memory options in the new UI to use more fitting units instead of just bytes.
Fixed opening links in a new tab in the new UI.

v2405.0

This is a small release that adds support for DAOS 2.4.2 and fixes some issues in the new UI.

Added support for DAOS v2.4.2.
Fixed an issue in the new UI when setting the Boot IP during cluster installation.
Fixed an issue in the new UI where croit needed to be restarted when cancelling the initial mon creation.
Fixed an issue in the new UI where moving the management IP to a different NIC didn't work under certain conditions.

v2405 croit enterprise image updates

Disabled automatic MD RAID assembly.
If you require MD RAID, please contact croit support at support@croit.io.

Added sdparm to the image.
Added dmidecode to the image.

v2404.2

This is a small hotfix release that contains several fixes for the new UI.

Added more explanations across the new UI.
Fixed default working directory for the terminal in the new UI.
Fixed some cosmetic bugs in the new UI.

v2404.1

This release adds experimental support for Ceph Reef (we don't recommend to upgrade yet) and fixes and improvements for the new croit UI.

Added experimental support for Ceph Reef.
We do not consider Reef stable enough yet to be used in production, and advise against upgrading for now.
Fixed bucket versioning setting showing an incorrect value when editing ACLs in the new UI.
If you edited bucket ACLs using the new UI, croit may have inadvertently enabled versioning on the bucket.

Separate default images for x86_64 and ARM cpu architectures can now be selected on clusters running mixed architectures.
Added a new view to manage ceph crash reports in the new UI.
It's available under the Status menu on the left.
Added a way to download the full ceph configuration file in the new UI.
Added finer control on ceph health mutes in the new UI.
Improved the hook scripts interface in the new UI.
Improved interface for configuring ceph options to make looking for the correct option easier in the new UI.
When comparing system packages of 2 croit images in the new UI, croit now only shows the differences between the images.
Improved the disks page to show OSD IDs and the NICs page to show the MAC by default in the new UI.
Improved performance of metrics graphs in the new UI.
Fixed an incorrect task advisor warning of servers not running their configured image, when said servers are actually offline.
Fixed services showing as running when the server is actually offline.
Fixed RBD creation interface to show warnings about selected RBD image features.
Fixed some cosmetic errors in the new UI.

v2404.0

This is a hotfix release for a bug in the new UI.

Fixed an issue in the new UI where RBDs were created with a size of 0 bytes.
Removed an incorrect error message that is logged before the ceph cluster is installed.

v2403.0

This release contains bug fixes and improvements for the new croit UI.

Added zoomed view for graphs and metrics in the new UI.
Added option to configure scopes requested for OIDC.
Added flat view button to the S3 explorer in the new UI.
Added button to trigger a manual cloud backup in the new UI.
Added MDS rank directly into the table on the services page in the new UI.
Improvements to sorting and filtering in the new UI.
Made some cosmetic improvements in the new UI.
Fixed an issue viewing Ceph health details on older clusters in the new UI.
Fixed not being able to choose Ceph version on setup in the new UI.
Fixed some issues when navigating prefixes in the S3 explorer in the new UI.
Fixed an issue where the Details button on the Settings page didn't work as expected in the new UI.
Fixed not being able to set manual roles for OIDC users in the new UI.
Fixed NFS/iSCSI/SMB buttons on the status page not redirecting correctly to their respective service pages.
Fixed several issues on the SMB page in the new UI.
Fixed an issue with some buttons not updating on the CRUSH page in the new UI.
Fixed an issue where server selection is ignored when opening a terminal in the new UI.
Fixed an issue where creating hooks wasn't possible in the new UI.

v2402.1

This release fixes exploitable XSS vulnerablities in the croit UI. It is highly recommended to update ASAP.

Fixed exploitable XSS vulnerabilities.

Added more information and warnings to the Max MDS setting.
Fixed an issue where croit failed to start up if IPv6 was disabled on the host.
Fixed an issue with the ceph-crash service. Croit now correctly distributes ceph keys for this service to work as intended.
Improved croitd logging. All systemctl changes are now logged.
Added a new dependencies view in the new croit UI.
Various bug fixes for the new croit UI.

v2402.0

This release contains improvements and fixes for the new UI and the ceph upgrade procedure. The task advisor and the gateway views in particular have been updated in the new UI. We also added a full-text search feature to all tables.

Improved ceph upgrade procedure.
The upgrade procedure uses a new strategy for updating ceph to the new release. The old strategy can still be used by setting croit upgrade update strategy to package.
Redesigned the task advisor in the new UI.
Redesigned the gateway views in the new UI.
Added full-text search to all tables in the new UI.
Added netcat and socat to the OS image.
Added a task advisor warning when servers aren't running their configured image.
Added a new warnings for iSCSI images using the kernel backend, that still emulate legacy capacity.
Moved enabled CPU architecture settings from the config.yml to the images page.
Improved Rolling Reboot feature to handle concurrent reboots better.
Fixed an issue with the disk benchmarking utility when running openSUSE.
Fixed an issue where some tasks couldn't be cancelled.
Added default sorting to all pages in the new UI.
Fixed some graph views in the new UI to properly round values.
Fixed a typo in the S3 view for the new UI.
Fixed an incorrect warning when editing and saving a template in the new UI.
Fixed an issue where NFS exports couldn't be edited in the new UI.
Fixed an issue when filtering disks by size in the new UI.
Fixed an issue where removing a server from SMB may time out and fail.
Fixed some other smaller bugs in the new UI.

v2312.0

Fixed regression that caused disks stats to not show on the disks page.
Fixed changing OSD reweight in the new UI.
Added more UI feedback when creating an NFS service in the new UI.
Added a button to add disks on the iSCSI hosts and groups pages in the new UI.
Improved formatting in some dialogs in the new UI.
Added rounding to some statistics in the new UI.
Fixed filter dropdown not being visible on the server logs page in the new UI.
Fixed showing the changes when editing a template in the new UI. Before, this was hidden when opening an already edited template until it was changed again.
Added indicator on how many items are selected in a table in the new UI (e.g. 7 selected).
Fixed an issue where pagination breaks when the number of entries changes because of filtering in the new UI.
Fixed managing bonds without LACP in the new UI.
Updated the DAOS configuration templates to use IPs instead of hostnames.
Fixed and improved some calls to DAOS.
Added tier stats (free/total space for SCMs/NVMes) for DAOS pools.
Improved validation for invalid object classes for DAOS containers.
Added capacity overhead when creating Ceph pools or DAOS containers in the new UI.
Fixed an issue after deleting the DAOS pool where the pool UUID was displayed in the filter.
Fixed showing 2 tasks when reintegrating a DAOS pool, despite only 1 task running.
Fixed DAOS cluster creation to only prepare the selected storage.
Improved disk view for DAOS in the new UI.
Fixed an issue with serial numbers for PMEMs for DAOS.
Updated daos-config-script template.
Changed DAOS configuration to only run on boot of nodes, instead of running it on any change in the config.
Fixed config page for DAOS clusters showing ceph configuration.
Added more variables to the network-setup template.
Fixed an issue where the network setup runs after changing a servers network configuration, and then changing the network-setup template.
Made user backstore for iSCSI images default again.
Added more logging during croit updates.

v2311.3

Improvements and fixes for DAOS.
Fixed an issue in the CephFS explorer in the new croit UI when folder names starts with a hashtag.
Updated support info package to use better filenames.

v2311.2

Improvements and fixes for the new croit UI.

v2311.1

Improvements and fixes for the new croit UI.
Fixed an issue with the network configuration for servers.
The network setup will now never run on croit updates.

v2311.0

Added experimental DAOS support.
Please contact our sales team to learn more about DAOS. DAOS support is only available in the new croit UI.

Improvements and fixes for the new croit UI.
In particular, we improved the image view to show image information more clearly.
Fixed an issue that caused IPMI interfaces to not be matched with servers.

v2310.0

Fixed CVE-2023-43040.
We recommend everyone using RGW to boot the latest croit image.
croitd will now log certain configuration files whenever they are written.
This feature can be disabled with the croit setting croit disable config file logging.
Fixed an issue when restoring an older croit backup in a newer croit version.
Added iputils-ping and dnsutils to the docker container.
Updated kernel commandline template to include systemd.hostname.
Fixed an incorrect warning about the sshd config failing to apply on servers running older debian images.
Improvements and fixes for the new UI.

v2309.0

Added new options to collect cluster information for support.
Support info can be downloaded under Maintenance > Download support info in the old UI, or via the Support menu in the new UI. When opening a ticket via the croit UI, you now also have the option to attach support info.
Several improvements and fixes for the new croit UI.
Added a template to configure the ssh server on the servers in the cluster.
Note that you need to use the latest croit v2309 image on your servers for this template to take effect.
Added more logging when creating SMB/NFS/iSCSI services.
Added the ceph cluster ID to the backup archive to more easily identify which cluster a backup belongs to.
Moved the CPU idle states setting to the Settings page.
Changed network configuration on servers to run one-by-one instead of all at once.
Fixed an issue during OSD creation that caused stray OSDs sometimes.
We now also retry OSD creation should it fail.
Fixed the description of some variables for SMB templates.

v2308.0

Added beta preview for the new UI for croit.
Croit now ships with a new UI that can be enabled via the Settings page. To disable it again, go to the Config page in the new UI and click on Croit to see the toggle option. The new UI comes with many feature and performance improvements. Please note that the new UI is still in development. Feel free to send us any feedback or issues you come across.
Added list of packages installed in a croit image, viewable via the croit UI.
This is only available in the new croit UI for new croit images.

Improved performance of some pages.
The Tasks page and the RBD view in the new croit UI in particular were improved.
Added earlyoom to our croit image.
This service will help when nodes run out of memory (e.g. because of an MDS memory leak).
Added cephfs-mirror and cephfs-shell to our croit image.
Improved auto-updater to retry if too many concurrent updates happen.
Fixed an issue where the default route was removed on booting croit nodes in certain setups.
Fixed RBD QoS being limited to 32-bit integers.
Fixed an issue in the ceph upgrade when no standby MDS were configured.
Fixed an issue where the croit UI didn't have access to S3 buckets when subusers were configured.
Fixed an issue in the S3 API when providing an empty list of ACLs.
Fixed an incorrect error message of MDS not being configured.
Fixed an issue of suggesting too many DB/WAL partitions.
Under certain configurations, the croit app may have suggested more than our recommended maximum of 16 partitions.
Fixed an off-by-one error in an error message when changing the number of active MDS.
Updated journald configuration on the cluster nodes.
Upgraded some dependencies used by croit.

v2307.1

Fixed an issue with the kernel cmdline template.

v2307.0

Improved performance of the Services page.
Fixed configuring RBD IO limits for images in a namespace.
Also improved the performance of fetching the configuration.
Fixed an issue where manually running a hook script on many servers would result in a database error.
Fixed an issue when creating an iSCSI service where the wrong gateway IP was used.
Improvements to tasks logging.
Displaying server hostnames, IDs and disks has been made more consistent across tasks.
Improvements to the ceph upgrade.
Any questions during the upgrade are now asked at the beginning. The list of croit images is now also refreshed before checking for upgrade images.
Added netdata and telegraf to the extended image.
Changes to the services API.
There are multiple ways services can be started, stopped or deleted via the API. All endpoints for that now behave the same.

v2306.1

Added option to create persistent API tokens.
Under Settings > User Management, you can now also manage API tokens. API tokens have the same permissions as users, but cannot access any API endpoints for managing users. A new setting croit session token cache was also added. If you plan on doing many API requests in a short time, enabling caching will improve performance.
Several fixes and improvements to the RBD mirroring feature.
Made some improvements when sorting on the disks or servers page in the UI.
Improved performance of the disks page in the croit UI.
Improved an error message when trying to access RGW on an old ceph cluster with an incompatible croit container.
Improved stability and performance of the ceph upgrade.
Parts of the upgrade have been parallelized. A new croit setting croit upgrade num jobs can be configured to control this.
Added nosnaptrim flag to the Maintenance page.
Fixed an issue where tasks sometimes weren't properly cancelled.
Small improvements to migration to Bullseye.

v2306.0

Updated croit container to Debian Bullseye.
Updating croit may take longer than usual, a separate migration container will also be started by croit. Downgrading from this release to older croit versions is not supported.

Marked Octopus as deprecated.
We recommend everyone to upgrade to Quincy.
The croit backup now excludes /config/.excluded/ from the backup.

v2305.0

Fixed an issue when creating S3 buckets with versioning enabled.
Versioning wasn't enabled when creating the bucket, only when editing an existing bucket. Please verify that versioning is enabled on the buckets where you expect it to be enabled.
Improved warnings when switching to a 17.2.6+ image.
Changed croitd to leave write cache enabled by default.
If you still want to disable write cache on older hardware, you can create a file /opt/croit/disableWriteCache using a hook script on the servers that should disable it for their disks.
Fixed some logging errors with NFS ganesha.
Some unnecessary errors were logged when setting up NFS Ganesha without exports. Logrotate also complained about permissions on the (unused) NFS Ganesha log directory.
Fixed an issue with pools not having the correct application for crush rule when initially setting up multiple S3 gateways at the same time.
Improvements to container switching between ceph versions.

v2304.1

Improved some error messages when connecting to the croit VPN.
Fixed croitd to only start rpcbind if nfs-ganesha is configured.

v2304.0

Added option to connect to the croit support VPN via the croit UI.
Using the support menu in the bottom left, you can now directly connect your cluster to the croit VPN to receive support.
Reworked the log view.
Rewritten to use xterm.js for stable performance with lots of lines on display and added a log download feature.
Deprecated support for Samba AD with sssd.
The new croit OS image won't have sssd installed.

Added docker binary to the croit container.
If the docker socket is mounted, this allows users to easily run docker commands from within the Ceph Admin Shell in croit.
Included Zabbix sender package in the Zabbix hook script.
Added warning when switching to a different Samba or NFS Ganesha version.
Fixed an issues where the same disks were detected with different serial numbers when booting an image with a different OS.
Fixed an issue with the screenshot tool when sending feedback on high-DPI displays.
Fixed an unuseful log message when deleting a pool.
Added task advisor warning for deprecated croit images.
Added croit setting croit disable default pool creation to prevent croit from creating the default ceph pools when adding an MDS or RGW service.
Fixed an issue where RBD mirror settings were shown on EC pools despite not being supported.

v2303.0

Fixed an issue where DB/WAL partitions weren't correctly linked with OSDs in the croit UI.
Fixed an issue where the croitd daemon tried to start the rsyslog service even though it's not needed.
Fixed an issue where selecting RBDs in various dialogs of the UI was slow if the cluster has many RBDs.
Fixed an issue where the croit UI wouldn't display the correct number of entries per page on the first page of RBDs.
Fixed croitd starting keepalived despite no HA groups being defined.
Fixed an issue with resetting the croit auto-updater settings.
Fixed an issue when deleting RBD images.
Fixed an issue when creating ceph keys with permissions on a pool that contains a space in its name.
Improved image verification when changing the default image or a server's image.
Updated Checkmk and SNMP scripts in the hook script library for openSUSE.
Added a config croit session expiry to configure when unused login sessions expire.
This replaces the setting sessionExpiryTime in the config.yml. The setting is automatically migrated.

v2302.0

Added missing memtest binary for UEFI PXE booting.
We also updated memtest to the latest version (Memtest86+ v6.10).
Added full name when displaying OIDC users.
Improved tasks view to show usernames instead of subject for OIDC users.
Improved performance by reducing frequency of SMART health polling.
We added 2 new croit settings croit smart health interval and croit smart details interval to further configure SMART health polling.
Fixed an issue where RBD statistics weren't displayed for monitored RBDs in a namespace.
Fixed an occasional error when fetching RBD namespaces.
Fixed flat view in the S3 explorer to use the current prefix.
Fixed an issue in the S3 explorer when using the backspace key to change the number of entries per page.
Fixed an issue in the S3 explorer where the entries per page setting wasn't displayed.
Fixed an issue where the Edit dialog for bond nics would apply the incorrect LACP setting.
Fixed an issue with the Ceph Upgrade when servers were running outdated images.

v2301.0

Added OpenID Connect (OIDC) as login option.
You can now configure OIDC providers as login for croit. For more information, see https://croit.io/software/docs/master/production-setup/setting-up-external-auth#openid.
Okta Login is now deprecated.
We recommend to set up OIDC instead.
Removed warning about Ceph Quincy being unstable.
Ceph Quincy is now the default release for new croit clusters. We recommend that users on Octopus skip Pacific and upgrade to Quincy directly instead.

Improved some error messages during the Ceph Upgrade procedure.
Fixed some unnecessary error messages when the ceph cluster is not yet set up.
Changed clog_to_monitors ceph config to default to false.
Disabled PG autoscaler for new clusters and clusters upgrading to Octopus.
Added croit setting croit iscsi api user to configure the username for the iSCSI gateway API.
The iscsi-gateway-cfg template needs to be the latest version for this setting to work.
Added croit setting croit command timeout to configure the minimum timeout used for various commands in the croit app.
This setting replaces the defaultCommandTimeout setting in the config.yml. Croit will migrate the setting automatically, no changes to config.yml need to be made.
Fixed an incorrect task advisor warning about an OS image update showing on octopus clusters.
Added a script to set up Zabbix on openSUSE servers to the hook script library.
Improved performance of fetching S3 users in the croit UI.
Fixed an issue when executing hook scripts manually.
Fixed an issue where users weren't logged out properly when deleting them from croit.
Fixed an issue where the admin account could be deleted when it's the only local admin account in croit.
Fixed an issue where bond nics show as using LACP despite not using it.
Fixed croitd daemon to set up network on servers first before starting OSDs.

v2212.0

Improve Ceph Upgrade procedure for OpenSUSE systems.
Fixed a bug when updating CRUSH map while creating OSDs.
Fixed warning about even number of mons showing up while still creating mons.
Fixed a rare bug causing unnecessary error messages when creating multiple mons on certain systems.

v2211.0

Added option to configure the CephFS filesystem name.
Clusters imported into croit may already have CephFS set up with a different filesystem name, this name can now be configured. The setting is available under the name croit cephfs name on the Settings page.
Added option to configure the base URI of the management node for download links.
The setting is available under the name croit base uri on the Settings page.
Fixed an issue when wiping disks taken from other ceph clusters.
Fixed an issue with the disk LED button not working.
Updated the template for Samba with AD to include the NetBIOS names of the servers running Samba gateways.
Fixed an issue where the wrong systemd service is started on nodes after changing their hostname.
Fixed an error message in the croit logs when a server doesn't have any disks.
Increased logging output limit for tasks.
Some tasks (e.g. creating multiple OSDs) reached the logging output limit in croit before.
Fixed a rare and incorrect warning about data loss when creating many OSDs.
Fixed an issues where HA groups with no servers associated with them wouldn't be listed.
Fixed an issue where RGW pools were created when navigating to the S3 User page, despite S3 not being configured on the cluster.
Added dialog to ask again for confirmation when simultaneously rebooting servers.
Added experimental support to upload iSCSI gateway certificates when configuring the iSCSI API in secure mode.
To enable secure mode, set croit iscsi api secure to true on the Settings page. The certificate files (named iscsi-gateway.crt and iscsi-gateway.key) need to be put into /config/, and will be automatically uploaded to the iSCSI nodes. This feature is still experimental, and may be improved in the future.
Improved performance when configuring RBD snapshot schedules with many RBDs.

v2209.0

Croitd daemon now uses HTTPS when connecting to the management node.
Please note that the croit daemon currently doesn't verify the certificate. We recommend also adding your own certificate, otherwise traffic can easily be decrypted.

Fixed an issue with the network latency statistics not being collected.
Added option to select iSCSI target IQN in case multiple targets are configured.

v2208.1

Fixed an issue when deleting OSDs in the Crush Map view.
Fixed iSCSI gateways showing as inaccessible/unauthorized in the terminal view on the iSCSI service page.

v2208.0

Added template for the iscsi gateway configuration file.
We don't recommend editing this template unless instructed by the croit support team.
Changed daemon to write CephFS key in a file in /etc/ceph/ for easier remounting of /persistent in hook scripts.
Added a check if /config/config.yml contains any syntax errors.
If it does, we fall back to the last working configuration, or the croit defaults.
Added configuration option for the repository URL to use for ceph upgrades (for openSUSE).
This is mostly meant to be used for debugging, or when an upgrade to a specific ceph version is required.
Added configuration option to change target IQN for iSCSI.
This is only meant to be used by the croit support team to manage specific iSCSI setups.

v2207.1

Added hook scripts events OnCroitStartup and OnCroitAsyncStartup to run a hook script on the management node during croit startup.
Added option to keep waiting without asking again to rolling reboot of multiple servers in case the timeout is reached.
This can also be made default by adding rollingRebootNoTimeout: true to the /config/config.yml file, in which case the rolling reboot will never time out.
Improved handling of task advisor warnings after completing a ceph upgrade.

v2207.0

Fixed a configuration error with ctdb (used for samba).
Rolling reboot will now ask the user if they want to keep waiting or stop the rolling reboot once the timeout is reached.
Removed unnecessary validation of S3 bucket policies.
Fixed handling of S3 subuser without any access permissions.
Multiple performance improvements.
In particular, we fixed an issue introduced in croit v2201.1.

v2206.2

Fixed an issue parsing the SMART health status.
Added endpoint to list S3 subusers directly under GET s3/users/{uid}/subusers.

v2206.1

Fixed an issue with disks (in particular NVMes with namespaces) not being updated due to longer than expected WWNs.

v2206.0

Fixed an issue where DB/WAL partitions weren't correctly associated with OSDs in our UI.
Fixed a timeout issue when turning a large disk into a mon disk.
mkfs.xfs may take longer than expected on large disks, so the timeout for that was increased.
Small improvements to auto-updater.

v2205.0

The croit Pacific (and later) container is now used as the default croit/croit:latest container. Users running an older (pre-pacific) ceph version should update to the croit/croit:latest.octopus container instead. The croit app will give you the option to switch to the correct container in case the Pacific container was started on a pre-pacific ceph cluster.
Added option to upgrade to Ceph Quincy.
Warning: We still don't recommend migrating to Quincy (nor Pacific), due to stability issues!

Added Discord notification script to hook script library.
Added support for specific DB/WAL setups using LVM instead of partitions.
Added a small improvement to the Ceph Pacific upgrade procedure.
Fixed a memory corruption issue in the croitd daemon.
Fixed the utilization on the status page to display the maximum utilization per OSD for each device class, instead of an average over OSDs.
Fixed an issue where joining a samba domain failed when the password contains special bash characters.
Fixed an issue where NVMes with multiple namespaces weren't detected as separate disks.
Fixed a compatibility issue with old debian stretch images.
Fixed an issue when doing a minor upgrade to Ceph 16.2.7.
Fixed an issue of the OSD state statistics not being collected in Ceph Octopus or later.
Fixed s3:* not being allowed when configuring bucket policies.
Removed ceph configuration defaults setting active hours for osd scrubs (osd scrub begin/end hour).

v2202.0

Fixed not being able to add more iSCSI gateways on clusters where the iSCSI services take too long to start.
Adding more gateways may have failed on such clusters if the other gateways weren't healthy fast enough.
Fixed an issue that caused a pool .rgw.root to be created despite not using RGW.
Fixed a missing animation when creating OSDs.
Improved performance of the pools page.
Added some small improvements to the Ceph Pacific upgrade procedure.

v2201.1

Added an authenticated endpoint to access file on the management node.
Similar to the existing endpoint serving files in /config/extra-files (inside the croit docker container), the new endpoint serves files from the /config/extra-files-secure folder. The API endpoint is reachable under /extra-files-secure/path-to-file similar to the /extra-files/... endpoint, but needs to be authenticated against using a croit user account with the Secure folder account role (via HTTP Basic authentication).
Fixed an issue that caused some MDS daemons to crash.
The last croit update enabled the messenger v2 protocol for MDS. This can result in the MDS daemons crashing. Messenger v2 is disabled for now if all MDS daemons are not already using v2.
Fixed S3 user stats missing for users that contain '@' in their user ID.
Fixed S3 stats missing for custom zones and placement rules.
The croit-rgw-monitoring ceph key will now be updated to include any necessary RGW pools according to zones and placement rules. If you want to prevent croit from updating this key, and instead manage it manually, add disableRgwMonitoringKeyUpdate: true to your /config/config.yaml inside the croit docker container.
Fixed not being able to edit a user's roles without also changing his password.
Added an option to configure the default command timeout.
To do so, add defaultCommandTimeout: 8 (here for example 8 seconds, the current default) to your /config/config.yaml inside the croit docker container. Generally speaking, this shouldn't be necessary and if you see many timeout warnings in the frontend, please contact croit support (support@croit.io) first.
Added some improvements to the Pacific upgrade to make it more stable.

v2201.0

Added option to upgrade to Ceph Pacific.
Warning: We still don't recommend migrating to Pacific, due to stability issues! Due to compatibility issues between Pacific and older ceph versions (in particular with RGW), we now offer a separate croit container for Pacific clusters. The new container is currently tagged as latest.pacific, and will in the future replace the latest container.
The Pacific upgrade procedure will try to switch to the new container for you. Should that fail, you can also simply update the croit container to latest.pacific manually.
If you intend to stay on an older ceph release, please use the latest.octopus tag from now on, until you decide to upgrade to Pacific. Don't worry if you start the Pacific container by accident, certain features will be blocked, and you will be given the option to switch to the Octopus container again.
Starting with Pacific, our default OS is now openSUSE instead of Debian. After the upgrade, the latest openSUSE image will be selected as new default for your servers. Please update your hook scripts and fix OS-specific commands (e.g. apt-get).
If you have any issues or questions, or need help with any of the upgrade steps, please contact croit support (support@croit.io) for further assistance.
Fixed an issue that caused scheduled jobs (such as automatic updates and backups or scheduled snapshots) to not run anymore.
For users affected by this issue, it also caused log files not being cleaned properly in the croit container, filling up the disk on the management node.
As a temporary workaround for this issue in older croit versions, you can simply restart the croit docker container.

Added toggle for base2 filesizes to RBD view.
Added option for viewer users to change their own password.
Added an extra warning when changing RBD image name or features and improved some error messages when deleting RBD images.
Fixed RGW user stats not being collected and missing from the UI.
Fixed an issue where servers incorrectly report being out-of-sync with the management node clock and having large clock drifts.
Fixed setting the server-specific variables (e.g. hostname) of the Samba AD domain join templates.
The variables now have the correct values for the server joining the domain.
Fixed default ceph.conf options to have messenger v2 (msgr2) enabled.
Fixed rolling reboot for servers when switching between Debian and openSUSE.
Fixed an issue where servers would show as offline in the croit UI when they had broken NFS connections.
Fixed an issue where a warning about failed backups incorrectly showed up.

v2111.3

Added a template for the Samba AD domain join.
This template can be edited to customize the commands used to join the domain.
Added a warning in case the backup on mon disks fails.
Fixed an issue in the croitd daemon when reporting server state on nodes that have no disks.
Fixed an issue when adding or changing NFS exports.
Our daemon will now tell NFS Ganesha to reload the new configuration.
Fixed an issue when monitoring RBD images not in the default rbd pool.
Fixed not being able to create an RBD image with no features enabled, and removing all features on existing RBD images.
Fixed exclusive lock being enabled when adding RBD images as disks to iSCSI on suse clusters.
Fixed sorting by Usage Objects in the S3 explorer.
Fixed an issue with mismatching timezones for certain tasks.
Changing SSH access keys will now not also try to update the IPMI password on servers if the password wasn't changed.
Small improvements to the upgrade procedure for ceph.
Fixed an issue with the network setup script and multiple NICs with the same name on the same node.
Fixed default gateways of VLANs not being set up correctly.

v2111.2

Fixed an issue in the croitd daemon when detecting the model name of NVMe disks.

v2111.1

Fixed an issue when detecting different disks with the same WWN in the croitd daemon.
Some disks may have the same WWN with different vendor extensions. The daemon handled this incorrectly before.

v2111.0

Switched to a new network configuration script to set up the network on the cluster nodes.
This setup script is available as the template network-setup and can be edited for custom networks configurations. Should you experience any network issues after upgrading to the new version, please contact croit support. You can disable the new script by adding useNetworkSetupScript: false to the configuration file /config/config.yml inside the docker container.

Improved performance of disk detection by parallelizing it.
Fixed upgrade procedure for MDS services when migrating to a new ceph version.
Fixed not being able to select an EC pool as data pool when configuring a S3 placement rule.
Fixed an issue in the configuration dialog for bond interfaces.
The LACP checkbox wasn't pre-selected correctly according to current settings.
Fixed an incorrect warning about long hardware detection times.
Improved stability in the new daemon while the management node is down.

v2110.0

Replaced the daemon running on croit nodes with a newer version.
Our croit daemon running on the nodes was rewritten for the future croit platform. If you notice any problems, please let us know.

Added experimental support for IPoIB interfaces.
MAC addresses of such interfaces are 20 bytes long. We use the last 8 bytes for identification, as the first 12 bytes can change over time.
Added the option to edit S3 keys of a user.
Small changes were made to the S3 subuser API (they are put under a /subusers resource to prevent naming conflicts).
Fixed a timeout issue when deleting or flattening large RBD images.
Added a new option disableDhcp in config.yml.
When set to true, the DHCP server in the docker container will be disabled.
Fixed an issue where SMB/NFS services that aren't associated with any server weren't displayed and couldn't be deleted or modified.
Fixed an issue with pagination in the S3 explorer.
Fixed rgw pools not using the correct crush rule when created.
Fixed displaying the ssh key in the server access dialog.
Fixed debian update failing because of testing name change.
Added a check when creating a bond for the length of its name (max. 15 characters).
Fixed an issue when doing a rolling restart of SMB services.
Removed LXC container support.

v2108.0

Fixed an issue when wiping encrypted OSDs that also use a DB/WAL device.
We urge anyone with OSDs with separate DB/WAL devices that used to be encrypted (or still are) to update to the latest croit version without rebooting any servers. The new update will display a warning if your OSDs are affected by this issue. If so, please contact croit support and do not reboot any server.
Added a new page for managing S3 placement rules.
The new page can be accessed under the S3 Explorer tab. Here you can configure new placement rules and add storage classes to these rules. The default placement for individual users can also be configured on the users page. Please note that creating buckets in the croit UI will either use the specified placement rule, or the default rule of the owner.
The API for s3 placements has also changed. If you used this API before, please update your requests or contact croit support for help.

Added task advisor warning for pre-Octopus EOL.
Added a template for DNS configuration (/etc/resolv.conf).
This fixes an issue where the DNS configuration wasn't properly applied to the croit nodes.
Added a template for the keepalived configuration (used for HA groups).

v2106.1

Removed milliseconds from scheduled CephFs and RBD snapshots.
Scheduled snapshots were postfixed with the current time in the format "yyyy-MM-dd.HH:mm:ss.SSS". We removed the milliseconds (SSS) from this format for better compatibility with shadow copies in Samba.

Added descriptions to template variables.
Added a task advisor warning when multiple default gateways are configured.

v2106.0

Users with the Viewer role are now allowed to view the CephFS explorer.
Viewers are only allowed to browse and download files, they can't upload or change any settings.
Updated the nfs-ganesha template for better performance.
We suggest using the updated default template for the ganesha config.

Added option to configure index and extra pool for RGW placement rules.
Removed ctdb-samba-stretch template.
Fixed an issue where some pools weren't protected when created.
Fixed an issue where trying to change compression settings of a pool failed with a 404 error.
Added a warning when switching between openSUSE and debian images.
Added a task advisor warning when disks show SMART health issues.

v2105.0

Fixed an issue where a rebooting server wasn't recognized and added as a new server instead.
Fixed an issue when managing the ceph dashboard.
Fixed an issue when cloning RBD snapshot on Mimic.
Fixed an issue when enabling RBD mirroring.

v2104.1

Fixed an issue where nodelete flag was set too often on pools.

v2104.0

The kernel commandline template has been updated.
Any previous changes made to the template are stored as a new preset "custom-old". The new template is required for openSUSE images.

Added support for scrubbing pools.
When selecting a pool, you can now use the "Scrub" button to start a scrub operation on the pool.
Added configurable timezones for snapshot schedules.
The timezone for schedules can be configured in the snapshot schedule tab.
Added new template for the apt proxy configuration.
Added support for S3 bucket versioning.
Please note that once versioning is enabled for a bucket, it can't be disabled anymore, only suspended.
Fixed handling of symlink files in CephFS explorer.
Support for symlink files is still being improved.

v2103.0

Experimental support for openSUSE.
Added support for our croit images based on openSUSE instead of debian. These images are currently only available as nightly images. Debian and openSUSE images should not be mixed on the same cluster.
Removed hook script parameter for Pre/PostNetworkConfig hooks.
Due to changes with our network setup procedure, the parameters for changed NICs has been removed.

Added option for multiple active MDS services.
When selecting an MDS service, click on Active MDS to choose the number of MDS services that ceph should keep active.
Added support to configure per-pool BlueStore compression.
BlueStore compression can be configured by selecting a pool and clicking on Compression.
Added warning when deleting RGW pools with existing placement rules.
Added warning when choosing a default network for automatic ping metrics that doesn't contain all servers.

v2102.2

Support separate DB and WAL devices.
When creating OSDs, DB and WAL devices can now be specified separately. Previously, croit only supported specifying a single device for both; it would use partition size to determine whether to store only the WAL or both DB and WAL on the device.
Remove kernel commandline dracut template.
croit doesn't provide CentOS images anymore; this removes support for booting CentOS entirely.

Fall back to iproute2 if no MAC is found.
This change allows InfiniBand NICs in IPoIB mode to work with croit.
Make bash history upgrade proof and add timestamps to bash history.
Add /api/audit API.

v2102.1

v2102.0

Wipe disks on all servers in parallel.
When wiping disks in multiple servers, each server can now wipe a disk at the same time.
Fix LDAP query for account name.
Add Task Advisor warning for non-highly-available MDS setups.
croit will now warn when there are too few standby/standby-replay MDS.
Prevent removing admin role from only admin account.
croit will prevent users from locking themselves out of their croit instance. To restore the initial admin account (username and password admin) use the /root/restore_admin.sh script.
Add destroy flag to /api/disks/wipe.
This allows reusing an OSD ID when wiping an OSD.

v2101.1

Make LDAP search for groups configurable.
There are new groupClassName and memberAttributeName options in config.yml.
Fix LDAP search request for OpenLDAP.
Add initial hosts to CRUSH map.
croit will now add hosts to the CRUSH map during cluster creation.

v2101.0

Refactor MDS output in cluster status.

Set default ceph release to Octopus.
Detect current ceph release when importing cluster.
Fix rolling service restart for OSDs.
Monitor OSD fragmentation in Task Advisor and disk view.
Add API support for ceph osd pool deep-scrub <pool>.

v2012.1

Add option for disabling CPU idle states.
Add "OSD Statistics" view.
This view replaces the read/write/subop latency and data balance graphs for improved performance on large clusters.
Add experimental LXC container support.
Set pg_num_min when creating a pool.
Fix memory hungry CRUSH map view on large clusters.
Fix invalid PW/username for iSCSI client.
Make Samba virtual IPs work without creating a network in croit.
Fix old ceph configs still containing underscores in their name.
Only activate OSDs 120s after boot.
Add OSD pause flag in Maintenance view.
Show management node in ping metrics.

v2012.0

Fix storing bond MAC addresses.
Previously bond MACs were stored improperly allowing only one of the bonded NICs to be used for booting.
Support Samba with sssd.
Restart nfs service when changing filesystem path of an export.
Check for OSDs running ShallowFSCK when upgrading ceph.
Remove support for S3 over NFS.
Allow editing MAC addresses.
This feature can be used when replacing NICs in a server while the server is turned off.
Store backup on MONs.
croit will now store a daily, unencrypted backup on all MON hosts in /var/lib/ceph/mon/.croit-backup.zip.
Restart dhcpd when DHCP template is updated.
Add view for rbd perf image iostat.
Implement S3 bucket policy management.
Clean graphite stats after two weeks.
Recreate default CRUSH rule after creating OSDs.
Set MDS to Active/Hot/Cold configuration by default.
Fix dmidecode RAM parsing.
Sort Task Advisor tasks by severity.
Add OnRbdSnapshotComplete hook.
Experimental support for podman.
Fix iSCSI exports in non-default network.
Show targets column in Snapshot Schedule view.
Make templates available during cluster setup.