OSDs Imbalanced

When your OSDs are imbalanced here are some steps to debug:

  1. Evaluate PG numbers.
  2. If you applied any PG number changes, wait for all data movement to finish. Then, If it's still imbalanced, continue.
  3. Make sure the balancer is up and running:
ceph balancer status
  1. Make the balancer more aggressive:
ceph config set mgr mgr/balancer/upmap_max_deviation 1 # The default is 5, meaning it tries to keep PG deviation within +- 5 PGs
  1. Wait for all data movement to finish.