nats-js-kv service registry only knows about one instance of a service #9535
Comments
|
Regression in 5.0.5? We had no changes on the stable branch. |
I found it in 5.0.5 |
|
5.0.0 is also already affected EDIT: from what I remember, |
|
The natsjsregistry implementation just replaces the existing entry: // Register adds a service to the registry
func (n *storeregistry) Register(s *registry.Service, _ ...registry.RegisterOption) error {
n.lock.RLock()
defer n.lock.RUnlock()
if s == nil {
return errors.New("wont store nil service")
}
b, err := json.Marshal(s)
if err != nil {
return err
}
return n.store.Write(&store.Record{
Key: s.Name,
Value: b,
Expiry: n.expiry,
})
}The memory implementation takes into account node name and version to actually add a node. |
Appending sounds also hard in this case, because the expiry is per key. Actually each service instance would need to add it's own key with a instance dependent suffix!? And the service discovery would work by looking at the key prefix!? |
|
That indeed is what the memory implementation does. Pasting here for comparison: func (m *Registry) Register(s *registry.Service, opts ...registry.RegisterOption) error {
m.Lock()
defer m.Unlock()
log := m.options.Logger
var options registry.RegisterOptions
for _, o := range opts {
o(&options)
}
r := serviceToRecord(s, options.TTL)
if _, ok := m.records[s.Name]; !ok {
m.records[s.Name] = make(map[string]*record)
}
if _, ok := m.records[s.Name][s.Version]; !ok {
m.records[s.Name][s.Version] = r
log.Logf(logger.DebugLevel, "Registry added new service: %s, version: %s", s.Name, s.Version)
go m.sendEvent(®istry.Result{Action: "update", Service: s})
return nil
}
addedNodes := false
for _, n := range s.Nodes {
if _, ok := m.records[s.Name][s.Version].Nodes[n.Id]; !ok {
addedNodes = true
metadata := make(map[string]string)
for k, v := range n.Metadata {
metadata[k] = v
m.records[s.Name][s.Version].Nodes[n.Id] = &node{
Node: ®istry.Node{
Id: n.Id,
Address: n.Address,
Metadata: metadata,
},
TTL: options.TTL,
LastSeen: time.Now(),
}
}
}
}
if addedNodes {
log.Logf(logger.DebugLevel, "Registry added new node to service: %s, version: %s", s.Name, s.Version)
go m.sendEvent(®istry.Result{Action: "update", Service: s})
return nil
}
// refresh TTL and timestamp
for _, n := range s.Nodes {
log.Logf(logger.DebugLevel, "Updated registration for service: %s, version: %s", s.Name, s.Version)
m.records[s.Name][s.Version].Nodes[n.Id].TTL = options.TTL
m.records[s.Name][s.Version].Nodes[n.Id].LastSeen = time.Now()
}
return nil
} |
This is the correct way imho. |
Wouldn't it be all entries? Otherwise you need a roundtrip to the registry, if you call next() ? |
Don't get you. Shouldn't |
|
What I meant and probably you too: The service client should cache the state of the registry for a particular service. It may include more than one registered service instance. Thus all registered service instances should be cached. This means we have exactly one request to the registry every seconds. There may be exceptions when no one of the service instances is reachable (early invalidation??) or the cache is empty. |
|
Ah, you are about caching. Sure we can add a cache into the |
|
no ... the registries are wrapped in a cache registry which uses the watch function to invalidate entries. |
Describe the bug
nats-js-kv service registry only knows about one instance of a service
Steps to reproduce
kubectl -n ocis get deployment gateway, it'll have only one replica (=1 pod)kubectl exec -n ocis-nats deployments/nats-box -- nats kv get service-registry ONSXE5TJMNSS24TFM5UXG5DSPFPWG33NFZXXO3TDNRXXKZBOMFYGSLTHMF2GK53BPE====== --raw | jq .data | tr -d '"' | base64 --decode | jq .:{ "name": "com.owncloud.api.gateway", "version": "5.0.5", "metadata": null, "endpoints": [], "nodes": [ { "metadata": { "protocol": "grpc", "registry": "cache", "server": "grpc", "transport": "grpc" }, "id": "com.owncloud.api.gateway-a476b641-1a02-44af-9a56-ad992e1a5376", "address": "10.244.4.205:9142" } ]}
NAME READY UP-TO-DATE AVAILABLE AGE
gateway 10/10 10 10 12m
kubectl -n ocis exec deployments/gateway -- ocis gateway versionExpected behavior
I'd expect to see 10 gateway services.
Actual behavior
I only see the first gateway service.
Setup
Using the ocis-nats deployment example of the oCIS Helm Chart.
oCIS version: owncloud/ocis:5.0.5
Additional context
The text was updated successfully, but these errors were encountered: