Fix docs and references on mayastore storage

[blmhemu]

Pull Request

What? (description)

Incorrect / outdated docs on mayastore.

Why? (reasoning)

For future users to benefit from the updated docs.

Acceptance

Please use the following checklist:

• you linked an issue (if applicable)
• you included tests (if applicable)
• you ran conformance (make conformance)
• you formatted your code (make fmt)
• you linted your code (make lint)
• you generated documentation (make docs)
• you ran unit-tests (make unit-tests)

See make help for a description of the available targets.

[blmhemu]

While mayastore recommends /var/lib/local i think /var/lib/openebs might be better.

[smira]

it should match I guess something in the installed helm chart?

[blmhemu]

I am suggesting for the source. Lets leave destination as it is.

[smira]

there's something broken here.

I guess there's somewhere hostPath volume mount for /var/lib/local. So on the host path is /var/lib/local. If the path on the kubelet mount is different (like in this PR), this won't work well, as for the kubelet /var/lib/local would not exist.

[smira]

is the mount path difference /var/local vs. /var/lib/local intentional?

[blmhemu]

Openebs suggests /var/local - https://openebs.io/docs/Solutioning/openebs-on-kubernetes-platforms/talos

[smira]

I mean that mounting under different paths is an unusual thing, as usually you want kubelet to see same path structure as the host, otherwise some features won't work

[blmhemu]

I see this one of the templates

---
{{ if and .Values.etcd.persistence.enabled (eq .Values.etcd.persistence.storageClass "manual") }}
{{- range $index, $end := until (.Values.etcd.replicaCount | int) }}
apiVersion: v1
kind: PersistentVolume
metadata:
  name: etcd-volume-{{ $index }}
  labels:
    statefulset.kubernetes.io/pod-name: {{ print $.Release.Name }}-etcd-{{ $index }}
spec:
  storageClassName: manual
  # You must also delete the hostpath on the node
  persistentVolumeReclaimPolicy: {{ $.Values.etcd.persistence.reclaimPolicy }}
  capacity:
    storage: {{ $.Values.etcd.persistence.size | quote }}
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/var/local/{{ $.Release.Name }}/etcd/pod-{{ $index }}"
---
{{- end }}
{{- end }}

[blmhemu]

All instances (of /var/local) use /var/local/{{ $.Release.Name }} prefix - so i guess we can safely make it /var/local/openebs ?

[blmhemu]

Also, FWIW, here is the configuration for my single node cluster which works - note the io_engine.

mayastor: {
  obs: { callhome: { enabled: false } },
  'loki-stack': { enabled: false },
  etcd: { replicaCount: 1 },
  nats: { cluster: { enabled: false } },
  // https://www.talos.dev/v1.8/kubernetes-guides/configuration/storage/#deploy-mayastor
  csi: { node: { initContainers: { enabled: false } } },
  // Without this, io-engine crashes
  io_engine: { envcontext: 'iova-mode=va' },
},