profile
viewpoint
Waldemar Quevedo wallyqs Synadia Communications, Inc. San Francisco NATS.io, Distributed Systems, Protocols, Web Operations, Logging

nats-io/nats-streaming-server 2069

NATS Streaming System Server

nats-io/nats.rb 846

Ruby client for NATS, the cloud native messaging system.

nats-io/nats-operator 443

NATS Operator

nats-io/nats.py 342

Python3 client for NATS.io

nats-io/nats-top 197

A top-like tool for monitoring NATS servers.

bdewey/org-ruby 167

This gem contains Ruby routines for parsing org-mode files. The most significant thing this library does today is convert org-mode files to HTML or textile.

nats-io/nats-streaming-operator 156

NATS Streaming Operator

nats-io/prometheus-nats-exporter 154

A Prometheus exporter for NATS metrics

nats-io/k8s 82

NATS on Kubernetes ☸️

nats-io/stan.py 82

Python Asyncio NATS Streaming Client

PR opened nats-io/nats-server

Fix dynamic configuration of Jetstream mem / disk store

Both max_mem and max_file had to be declared to be able to set the max file storage, otherwise it would get ignored.

jetstream: { 
  # max_mem: 1GB
  max_file: 2GB
}

Before this change:

[6225] 2020/10/26 17:18:22.284930 [INF] ----------- JETSTREAM -----------
[6225] 2020/10/26 17:18:22.284958 [INF]   Max Memory:      6.00 GB
[6225] 2020/10/26 17:18:22.284972 [INF]   Max Storage:     1.00 TB

After this change:

[6245] 2020/10/26 17:20:24.952320 [INF] ----------- JETSTREAM -----------
[6245] 2020/10/26 17:20:24.952327 [INF]   Max Memory:      6.00 GB
[6245] 2020/10/26 17:20:24.952334 [INF]   Max Storage:     2.00 GB
  • [ ] Link to issue, e.g. Resolves #NNN
  • [ ] Tests added
  • [ ] Branch rebased on top of current master (git pull --rebase origin master)
  • [ ] Changes squashed to a single commit (described here)
  • [ ] Build is green in Travis CI
  • [ ] You have certified that the contribution is your original work and that you license the work to the project under the Apache 2 license

/cc @nats-io/core

Signed-off-by: Waldemar Quevedo wally@synadia.com

+31 -18

0 comment

1 changed file

pr created time in 14 hours

create barnchwallyqs/nats-server

branch : max-store

created branch time in 14 hours

startedgrafana/tempo

started time in a day

startedintel/GrapheneSGX-Golang-Support-and-Enhancement

started time in 2 days

startedtweag/nickel

started time in 3 days

PR closed nats-io/nats-operator

testing ignore me

testing ignore me

+293 -31

0 comment

16 changed files

variadico

pr closed time in 3 days

issue closednats-io/k8s

Add operators helm chart to helm hub

Add to this repo and include in next release.

closed time in 4 days

wallyqs

issue commentnats-io/k8s

Add operators helm chart to helm hub

Fixed via #144

wallyqs

comment created time in 4 days

PullRequestReviewEvent

delete branch nats-io/nats-box

delete branch : default-nats-url

delete time in 4 days

push eventnats-io/nats-box

Jaime Piña

commit sha bbf01ab93cdaa4d3a9e5fd1b7b2f31324e5f941e

Change default NATS URL to localhost

view details

Waldemar Quevedo

commit sha 8130af2348fd518dd534f1e66b1002e7df80e943

Merge pull request #12 from nats-io/default-nats-url Change default NATS URL to localhost

view details

push time in 4 days

PullRequestReviewEvent

Pull request review commentnats-io/k8s

Add NATS Operator chart

+apiVersion: apiextensions.k8s.io/v1beta1

think we need to update this one for non beta (newer clusters like 1.18)?

variadico

comment created time in 4 days

PullRequestReviewEvent

push eventnats-io/kubecon2020

Waldemar Quevedo

commit sha ff90c8681f6b844fcf7b38f7563efd79a913d6b1

Update README.org

view details

push time in 4 days

pull request commentnats-io/k8s

Add Jetstream file storage options with persistent volume to NATS server helm chart

Thanks for the PR! I will take a look and make a release tomorrow with the changes.

eoneal89

comment created time in 4 days

issue openednats-io/k8s

[stan] Prevent enabling both ft and cluster mode

It should prevent being able to activate both:

stan:
  image: nats-streaming:alpine
  replicas: 3
store:
  type: file
  ft:
    group: stan-group
  volume:
    enabled: true
    mount: /data/stan
    persistentVolumeClaim:
      claimName: stan-efs
  cluster:
    enabled: true
    logPath: /data/stan/log

created time in 4 days

push eventnats-io/kubecon2020

Waldemar Quevedo

commit sha b440e3fb496e3766b2d00274bb3c17f0650c78f1

Update README.org

view details

push time in 5 days

delete branch nats-io/kubecon2020

delete branch : k8s

delete time in 5 days

delete branch nats-io/kubecon2020

delete branch : revoke

delete time in 5 days

delete branch nats-io/kubecon2020

delete branch : rebased

delete time in 5 days

delete branch nats-io/kubecon2020

delete branch : revocation

delete time in 5 days

delete branch nats-io/kubecon2020

delete branch : with-revoke

delete time in 5 days

delete branch nats-io/k8s

delete branch : ws

delete time in 5 days

push eventnats-io/k8s

Waldemar Quevedo

commit sha 1ae9de80a76f70b5f0c0c6e8c3c781bcd8f29d9c

Add websocket support Signed-off-by: Waldemar Quevedo <wally@synadia.com>

view details

Waldemar Quevedo

commit sha 6606dd62389ace6a69f0551fd915dd2267f44b24

Merge pull request #140 from nats-io/ws Add websocket support

view details

push time in 5 days

PR merged nats-io/k8s

Add websocket support

Add support for websocket and embedded NATS Account server (full)

nats:
  image: synadia/nats-server:nightly

  # Bind a host port from the host for each one of the pods.
  externalAccess: true

  logging:
    debug: false
    trace: false

  tls:
    secret:
      name: nats-tls
    cert: "fullchain.pem"
    key: "privkey.pem"

cluster:
  enabled: true

auth:
  enabled: true

  resolver:
    # Use the NATS Account embedded server.
    type: full

    configMap:
      name: nats-accounts
      key: resolver.conf

natsbox:
  enabled: false

websocket:
  enabled: true
  port: 443

  tls:
    secret:
      name: nats-tls
    cert: "fullchain.pem"
    key: "privkey.pem"

Signed-off-by: Waldemar Quevedo wally@synadia.com

+65 -1

0 comment

3 changed files

wallyqs

pr closed time in 5 days

PR opened nats-io/k8s

Reviewers
Add websocket support

Add support for websocket and embedded NATS Account server (full)

nats:
  image: synadia/nats-server:nightly

  # Bind a host port from the host for each one of the pods.
  externalAccess: true

  logging:
    debug: false
    trace: false

  tls:
    secret:
      name: nats-tls
    cert: "fullchain.pem"
    key: "privkey.pem"

cluster:
  enabled: true

auth:
  enabled: true

  resolver:
    # Use the NATS Account embedded server.
    type: full

    configMap:
      name: nats-accounts
      key: resolver.conf

natsbox:
  enabled: false

websocket:
  enabled: true
  port: 443

  tls:
    secret:
      name: nats-tls
    cert: "fullchain.pem"
    key: "privkey.pem"

Signed-off-by: Waldemar Quevedo wally@synadia.com

+65 -1

0 comment

3 changed files

pr created time in 5 days

create barnchnats-io/k8s

branch : ws

created branch time in 5 days

PullRequestReviewEvent

delete branch nats-io/nats-server

delete branch : jetstream-omitempty

delete time in 6 days

startedkevinwatson/rails-microservices-book

started time in 7 days

startedkonstellation-io/kre

started time in 7 days

push eventnats-io/nsc

Matthias Hanel

commit sha 55f56f5681ca249df9a120f589501b6ecdbec0b4

[Added] cmds to push/pull/generate to/from/for nats based resolver (#306) * [Added] cmds to push/pull/generate to/from/for nats based resolver Signed-off-by: Matthias Hanel <mh@synadia.com>

view details

push time in 7 days

PR merged nats-io/nsc

Reviewers
[Added] cmds to push/pull/generate to/from/for nats based resolver

Signed-off-by: Matthias Hanel mh@synadia.com

nsc push --all --prune
[ OK ] push to nats-server "nats://localhost:4222" using system account "SYS" user "sys":
       [ OK ] push SYS to nats-server with nats account resolver:
              [ OK ] pushed "SYS" to nats-server NAXAMXP3UVE6SL54O4OJU2PALLGARGDI7SJAHMDUC6YIIQEM4I735ERO: jwt updated
              [ OK ] pushed to a total of 1 nats-server
       [ OK ] push testme to nats-server with nats account resolver:
              [ OK ] pushed "testme" to nats-server NAXAMXP3UVE6SL54O4OJU2PALLGARGDI7SJAHMDUC6YIIQEM4I735ERO: jwt updated
              [ OK ] pushed to a total of 1 nats-server
[ OK ] prune nats-server with nats account resolver:
       [ OK ] list 4 accounts from nats-server NAXAMXP3UVE6SL54O4OJU2PALLGARGDI7SJAHMDUC6YIIQEM4I735ERO:
              [ OK ] account AAWAJFLSTKK2X42G3IDOLFRBZST3BNQROCVNQT3K2MTJY4MM7BQ3OE3C only exists in server
              [ OK ] account AAYQZDUGIR5LHCJKRXHD3GP5Z3SPJ25GZ7VZA43EIHKMZAR7RZQR3SQX only exists in server
              [ OK ] account ABE47SN2CCVHD3CYSPDJ57YXXXCOWYTAHIYSPS7OTFDGRTQMW3EYSGPC named testme exists
              [ OK ] account AACTUIFJ3D2HCUCPAW2I3D44OM4JIG2HQSHV6GFI5ZM7RSKK2YN2PCGO named SYS exists
       [ OK ] listed accounts from a total of 1 nats-server
       [ OK ] pruning 2 accounts
       [ OK ] pruned nats-server NAXAMXP3UVE6SL54O4OJU2PALLGARGDI7SJAHMDUC6YIIQEM4I735ERO: deleted 2 accounts
all jobs succeeded
nsc generate config --nats-resolver
# Operator named testme
operator: eyJ0eXAiOiJqd3QiLCJhbGciOiJlZDI1NTE5In0.eyJqdGkiOiJFQlBMNUdVVkxMNERPRTdCWUJVSjRDRk1RQ0lOVUJZNzVHUllOMzM0VUxNRUtOWU9UTFJRIiwiaWF0IjoxNjAyMjg0NTAxLCJpc3MiOiJPQ0xBNFZZV0FGVUpFNEJDWUkyU0kyNzdXNlNYWDdGQ0lGWEM2SlpVVU4zMzNUTVRZQ1hGQ1NVSCIsIm5hbWUiOiJ0ZXN0bWUiLCJzdWIiOiJPQ0xBNFZZV0FGVUpFNEJDWUkyU0kyNzdXNlNYWDdGQ0lGWEM2SlpVVU4zMzNUTVRZQ1hGQ1NVSCIsInR5cGUiOiJvcGVyYXRvciIsIm5hdHMiOnsib3BlcmF0b3Jfc2VydmljZV91cmxzIjpbIm5hdHM6Ly9sb2NhbGhvc3Q6NDIyMiJdLCJzeXN0ZW1fYWNjb3VudCI6IkFBQ1RVSUZKM0QySENVQ1BBVzJJM0Q0NE9NNEpJRzJIUVNIVjZHRkk1Wk03UlNLSzJZTjJQQ0dPIn19.OUuStu3XNMEl1lCCObjSQRNBqoR--qDFRaA6rsTRli-PPGulzJcSmT6_or-ZPqkmy7aep0vGJWr-5uuBbNhuCQ
# System Account named SYS
system_account: AACTUIFJ3D2HCUCPAW2I3D44OM4JIG2HQSHV6GFI5ZM7RSKK2YN2PCGO

# configuration of the nats based resolver
resolver {
    type: full
    # Directory in which the account jwt will be stored
    dir: "./jwt"
    # Interval at which a nats-server with a nats based account resolver will compare
    # it's state with one random nats based account resolver in the cluster and if needed,
    # exchange jwt and converge on the same set of jwt.
    interval: "2m"
}

# Preload the nats based resolver with the system account jwt.
# This is not necessary but avoids a bootstrapping system account.
# This only applies to the system account. Therefore other account jwt are not included here.
# To populate the resolver:
# 1) make sure that your operator has Service URLs set and account server URL unset.
#        nsc edit operator --account-jwt-server-url "" --service-url <nats-url1> --service-url <nats-url...>
# 2) push your accounts using: nsc push --all
# To remove jwt: nsc push --prune
# -u is optional if your operator has Service URLs set.
# Later changes to the system account take precedence over the system account jwt listed here.
resolver_preload: {
	AACTUIFJ3D2HCUCPAW2I3D44OM4JIG2HQSHV6GFI5ZM7RSKK2YN2PCGO:eyJ0eXAiOiJqd3QiLCJhbGciOiJlZDI1NTE5In0.eyJqdGkiOiJaSk1aUExCTkREM1FGUE9TUjdMNkFRTTZTUklETUFINUFZTzVEWDZQSkc0WlVLNjRIN0ZBIiwiaWF0IjoxNjAyMTkzODYyLCJpc3MiOiJPQ0xBNFZZV0FGVUpFNEJDWUkyU0kyNzdXNlNYWDdGQ0lGWEM2SlpVVU4zMzNUTVRZQ1hGQ1NVSCIsIm5hbWUiOiJTWVMiLCJzdWIiOiJBQUNUVUlGSjNEMkhDVUNQQVcySTNENDRPTTRKSUcySFFTSFY2R0ZJNVpNN1JTS0syWU4yUENHTyIsInR5cGUiOiJhY2NvdW50IiwibmF0cyI6eyJsaW1pdHMiOnsic3VicyI6LTEsImNvbm4iOi0xLCJsZWFmIjotMSwiaW1wb3J0cyI6LTEsImV4cG9ydHMiOi0xLCJkYXRhIjotMSwicGF5bG9hZCI6LTEsIndpbGRjYXJkcyI6dHJ1ZX19fQ.tQGKr0UdCZ9h3_ZdtE0yyBJ72aGRNKgnuIIRWkpIi1Z5Z_XhkD1lSbuA1WXKNqVyb8Y7fcFln2Uek1qd2_uaAg,
}
+71486 -17191

4 comments

452 changed files

matthiashanel

pr closed time in 7 days

issue closednats-io/nats-server

NATS Account Embedded Server fails to start when dir not present

Defect

Make sure that these boxes are checked before submitting your issue -- thank you!

  • [X] Included nats-server -DV output
  • [X] Included a [Minimal, Complete, and Verifiable example] (https://stackoverflow.com/help/mcve)

Versions of nats-server and affected client libraries used:

OS/Container environment:

Steps or code to reproduce the issue:

Use some config like:

echo '
# Operator named KO
operator: eyJ0eXAiOiJqd3QiLCJhbGciOiJlZDI1NTE5In0.eyJqdGkiOiIyRFZVUlJZTlRFSkJIT0JHRDdFVU0zRk9EN1FHUDZTNFRDNzZSWjRNSkhSSEpSV0JESUVBIiwiaWF0IjoxNjAzMTgxOTYxLCJpc3MiOiJPRERMTkNYWU9STllFWVZISFFLS09GUFJER0lSQ1NVVUY1WEUyUkRTUU1ISEpEM1FPMkxFSFJZUSIsIm5hbWUiOiJLTyIsInN1YiI6Ik9ERExOQ1hZT1JOWUVZVkhIUUtLT0ZQUkRHSVJDU1VVRjVYRTJSRFNRTUhISkQzUU8yTEVIUllRIiwidHlwZSI6Im9wZXJhdG9yIiwibmF0cyI6eyJvcGVyYXRvcl9zZXJ2aWNlX3VybHMiOlsibmF0czovLzEyNy4wLjAuMTo0MjIyIl19fQ.bxtcrruPCubA4eSmsoYb-KvFW9SQiqlaGf5F9MCDxr2hPJzy7SacKSDncWibkiXq7cU-QCkjDrLQsinyrChBAQ
# System Account named SYS
system_account: ABV6BEZIMRYM6WKTVUIVEILAOGKO4BM5KX6F22A74YYYMMOCYR7NU7ST

# configuration of the nats based resolver
resolver {
    type: full
    # Directory in which the account jwt will be stored
    dir: './jwt'
    # Interval at which a nats-server with a nats based account resolver will compare
    # it's state with one random nats based account resolver in the cluster and if needed, 
    # exchange jwt and converge on the same set of jwt.
    interval: "2m"
}
' > resolver.conf
nats-server -c resolver.conf  -DV -m 8222

Expected result:

After creating the folder, it would start ok:

mkdir jwt
nats-server -c conf/resolver.conf  -DV -m 8222
nats-server -T=false -c conf/resolver.conf  -DV -m 8222 
[9387] [INF] Starting nats-server version 2.2.0-beta.24
[9387] [DBG] Go build version go1.15.2
[9387] [INF] Git commit [not set]
[9387] [INF] Using configuration file: conf/resolver.conf
[9387] [INF] Trusted Operators
[9387] [INF]   System  : ""
[9387] [INF]   Operator: "KO"
[9387] [INF]   Issued  : 2020-10-20 01:19:21 -0700 PDT
[9387] [INF]   Expires : 1969-12-31 16:00:00 -0800 PST
[9387] [INF] Managing all jwt in exclusive directory /Users/wallyqs/repos/nats-dev/src/github.com/nats-io/kubecon2020/jwt
[9387] [INF] Starting http monitor on 0.0.0.0:8222
[9387] [INF] Listening for client connections on 0.0.0.0:4222
[9387] [INF] Server id is NCISK3EUGETHBB4OHXWQ3EPBE2P6FKX6Y7IW36GLNS3A4MSWCNDRBGWL
[9387] [INF] Server is ready

Actual result:

Without the jwt folder from the conf, it fails:

nats-server -c resolver.conf  -DV -m 8222
nats-server: resolver.conf:14:5: runtime error: invalid memory address or nil pointer dereference

closed time in 7 days

wallyqs

issue commentnats-io/nats-server

NATS Account Embedded Server fails to start when dir not present

Fixed by @matthiashanel already 👍

wallyqs

comment created time in 7 days

Pull request review commentnats-io/nsc

[Added] cmds to push/pull/generate to/from/for nats based resolver

 func (p *PushCmdParams) Run(ctx ActionCtx) (store.Status, error) { 	if err != nil { 		return nil, err 	}- 	r := store.NewDetailedReport(true)-	for _, v := range p.targeted {-		sub := store.NewReport(store.OK, "push %s to account server", v)-		sub.Opt = store.DetailsOnErrorOrWarning-		r.Add(sub)-		ps, err := p.pushAccount(v, ctx)-		if ps != nil {-			sub.Add(store.HoistChildren(ps)...)+	if p.sysAccUser == "" {+		for _, v := range p.targeted {+			sub := store.NewReport(store.OK, "push %s to account server", v)+			sub.Opt = store.DetailsOnErrorOrWarning+			r.Add(sub)+			ps, err := p.pushAccount(v, ctx)+			if ps != nil {+				sub.Add(store.HoistChildren(ps)...)+			}+			if err != nil {+				sub.AddError("failed to push account %q: %v", v, err)+			}+			if sub.OK() {+				sub.Label = fmt.Sprintf("pushed %q to account server", v)+			} 		}+	} else {+		sysAcc, sysAccUser, opt, err := systemAccountUser(ctx, p.sysAccUser) 		if err != nil {-			sub.AddError("failed to push account %q: %v", v, err)+			r.AddError("error obtaining system account user: %v", err)+			return r, nil 		}-		if sub.OK() {-			sub.Label = fmt.Sprintf("pushed %q to account server", v)+		nc, err := nats.Connect(p.ASU, opt, nats.Name("nsc-client"))+		if err != nil {+			r.AddError("failed to connect: %v", err)+			return r, nil+		}+		defer nc.Close()+		if len(p.targeted) != 0 {+			sub := store.NewReport(store.OK, `push to nats-server "%s" using system account "%s" user "%s"`,+				p.ASU, sysAcc, sysAccUser)+			r.Add(sub)+			for _, v := range p.targeted {+				subAcc := store.NewReport(store.OK, "push %s to nats-server with nats account resolver", v)+				sub.Add(subAcc)+				if raw, err := ctx.StoreCtx().Store.Read(store.Accounts, v, store.JwtName(v)); err != nil {+					subAcc.AddError("failed to read account %q: %v", v, err)+				} else {+					resp := multiRequest(nc, subAcc, "push account", "$SYS.REQ.CLAIMS.UPDATE", raw,

ok thanks, let me know if I can help

matthiashanel

comment created time in 7 days

PullRequestReviewEvent

issue commentnats-io/nats-server

Merging Configuration Blocks

Have you considered using accounts instead as a workaround to separate static users from dynamic users?

system_account = SYS

accounts {
  STATIC {
    users = []
  }
  DYNAMIC {
    users = []
  }
  SYS {
    users = []
  }
}

which then could become something like this using includes and environment variables:

system_account = SYS

accounts {
  STATIC {
    users = []
  }
  DYN {
    include 'dyn.conf'
    users = $DYNAMIC_USERS
  }
  SYS {
    users = []
  }
}

where dyn.con is:

DYNAMIC_USERS = [
  {user: "foo", pass: "bar" }
]
mikehardenize

comment created time in 7 days

issue openednats-io/nats-server

NATS Account Embedded Server fails to start when dir not present

Defect

Make sure that these boxes are checked before submitting your issue -- thank you!

  • [X] Included nats-server -DV output
  • [X] Included a [Minimal, Complete, and Verifiable example] (https://stackoverflow.com/help/mcve)

Versions of nats-server and affected client libraries used:

OS/Container environment:

Steps or code to reproduce the issue:

Use some config like:

echo '
# Operator named KO
operator: eyJ0eXAiOiJqd3QiLCJhbGciOiJlZDI1NTE5In0.eyJqdGkiOiIyRFZVUlJZTlRFSkJIT0JHRDdFVU0zRk9EN1FHUDZTNFRDNzZSWjRNSkhSSEpSV0JESUVBIiwiaWF0IjoxNjAzMTgxOTYxLCJpc3MiOiJPRERMTkNYWU9STllFWVZISFFLS09GUFJER0lSQ1NVVUY1WEUyUkRTUU1ISEpEM1FPMkxFSFJZUSIsIm5hbWUiOiJLTyIsInN1YiI6Ik9ERExOQ1hZT1JOWUVZVkhIUUtLT0ZQUkRHSVJDU1VVRjVYRTJSRFNRTUhISkQzUU8yTEVIUllRIiwidHlwZSI6Im9wZXJhdG9yIiwibmF0cyI6eyJvcGVyYXRvcl9zZXJ2aWNlX3VybHMiOlsibmF0czovLzEyNy4wLjAuMTo0MjIyIl19fQ.bxtcrruPCubA4eSmsoYb-KvFW9SQiqlaGf5F9MCDxr2hPJzy7SacKSDncWibkiXq7cU-QCkjDrLQsinyrChBAQ
# System Account named SYS
system_account: ABV6BEZIMRYM6WKTVUIVEILAOGKO4BM5KX6F22A74YYYMMOCYR7NU7ST

# configuration of the nats based resolver
resolver {
    type: full
    # Directory in which the account jwt will be stored
    dir: './jwt'
    # Interval at which a nats-server with a nats based account resolver will compare
    # it's state with one random nats based account resolver in the cluster and if needed, 
    # exchange jwt and converge on the same set of jwt.
    interval: "2m"
}
' > resolver.conf
nats-server -c resolver.conf  -DV -m 8222

Expected result:

After creating the folder, it would start ok:

mkdir jwt
nats-server -c conf/resolver.conf  -DV -m 8222
nats-server -T=false -c conf/resolver.conf  -DV -m 8222 
[9387] [INF] Starting nats-server version 2.2.0-beta.24
[9387] [DBG] Go build version go1.15.2
[9387] [INF] Git commit [not set]
[9387] [INF] Using configuration file: conf/resolver.conf
[9387] [INF] Trusted Operators
[9387] [INF]   System  : ""
[9387] [INF]   Operator: "KO"
[9387] [INF]   Issued  : 2020-10-20 01:19:21 -0700 PDT
[9387] [INF]   Expires : 1969-12-31 16:00:00 -0800 PST
[9387] [INF] Managing all jwt in exclusive directory /Users/wallyqs/repos/nats-dev/src/github.com/nats-io/kubecon2020/jwt
[9387] [INF] Starting http monitor on 0.0.0.0:8222
[9387] [INF] Listening for client connections on 0.0.0.0:4222
[9387] [INF] Server id is NCISK3EUGETHBB4OHXWQ3EPBE2P6FKX6Y7IW36GLNS3A4MSWCNDRBGWL
[9387] [INF] Server is ready

Actual result:

Without the jwt folder from the conf, it fails:

nats-server -c resolver.conf  -DV -m 8222
nats-server: resolver.conf:14:5: runtime error: invalid memory address or nil pointer dereference

created time in 7 days

issue closednats-io/nats-server

NATS v1.4 outage due to Slow consumer issue

Hi! We run cloud foundry. In the cluster we have two NATS VMs which share messages each receive so that subscribes (in our case gorouters) can connect to any instances (apps on the diego cells). It was the issue that one VM was detected as Slow Consumer and the connection between had been closed. After that we observed that it was retried to create connection (we see a couple of times Route Connection created once a second). After ~20s the NATS VM was the "Slow Consumer" again. We see again Route Connection created. This scenario had been repeated 6 times within 2 minutes (the NATS VM detected as Slow Consumer and the Route Connection created). We assume that the NATS stopped to exchange the messages (but we don't know why) as gorouters did not get the infos about routes and began to prune all of them. One minute after the last "Route Connection created" we observed many errors related to "TLS handshake: remote error: tls: bad certificate" logged by both of NATS VMs (related to the tls-communication with diego-cells). At the same time both of VMs CPUs increased and went up nearly 100%.

After restart of one VM, then the second one and then the first one again, NATS started to work without issues.

We would like to understand to root cause of the issue.

closed time in 7 days

b1tamara

issue commentnats-io/nats-server

NATS v1.4 outage due to Slow consumer issue

Closing this one since related issue from CF also closed, recommend to follow the recommendations from their issue: https://github.com/cloudfoundry/nats-release/issues/25 Thanks!

b1tamara

comment created time in 7 days

Pull request review commentnats-io/nsc

[Added] cmds to push/pull/generate to/from/for nats based resolver

 func (p *PushCmdParams) Run(ctx ActionCtx) (store.Status, error) { 	if err != nil { 		return nil, err 	}- 	r := store.NewDetailedReport(true)-	for _, v := range p.targeted {-		sub := store.NewReport(store.OK, "push %s to account server", v)-		sub.Opt = store.DetailsOnErrorOrWarning-		r.Add(sub)-		ps, err := p.pushAccount(v, ctx)-		if ps != nil {-			sub.Add(store.HoistChildren(ps)...)+	if p.sysAccUser == "" {+		for _, v := range p.targeted {+			sub := store.NewReport(store.OK, "push %s to account server", v)+			sub.Opt = store.DetailsOnErrorOrWarning+			r.Add(sub)+			ps, err := p.pushAccount(v, ctx)+			if ps != nil {+				sub.Add(store.HoistChildren(ps)...)+			}+			if err != nil {+				sub.AddError("failed to push account %q: %v", v, err)+			}+			if sub.OK() {+				sub.Label = fmt.Sprintf("pushed %q to account server", v)+			} 		}+	} else {+		sysAcc, sysAccUser, opt, err := systemAccountUser(ctx, p.sysAccUser) 		if err != nil {-			sub.AddError("failed to push account %q: %v", v, err)+			r.AddError("error obtaining system account user: %v", err)+			return r, nil 		}-		if sub.OK() {-			sub.Label = fmt.Sprintf("pushed %q to account server", v)+		nc, err := nats.Connect(p.ASU, opt, nats.Name("nsc-client"))+		if err != nil {+			r.AddError("failed to connect: %v", err)+			return r, nil+		}+		defer nc.Close()+		if len(p.targeted) != 0 {+			sub := store.NewReport(store.OK, `push to nats-server "%s" using system account "%s" user "%s"`,+				p.ASU, sysAcc, sysAccUser)+			r.Add(sub)+			for _, v := range p.targeted {+				subAcc := store.NewReport(store.OK, "push %s to nats-server with nats account resolver", v)+				sub.Add(subAcc)+				if raw, err := ctx.StoreCtx().Store.Read(store.Accounts, v, store.JwtName(v)); err != nil {+					subAcc.AddError("failed to read account %q: %v", v, err)+				} else {+					resp := multiRequest(nc, subAcc, "push account", "$SYS.REQ.CLAIMS.UPDATE", raw,

I see these subs from the system account:

[8752] 2020/10/20 03:12:24.847901 [TRC] SYSTEM - <<- [SUB $SYS.ACCOUNT.*.CLAIMS.UPDATE  32]
[8752] 2020/10/20 03:12:24.847920 [TRC] SYSTEM - <<- [SUB $SYS.REQ.ACCOUNT.*.CLAIMS.UPDATE  33]
[8752] 2020/10/20 03:12:24.847925 [TRC] SYSTEM - <<- [SUB $SYS.REQ.ACCOUNT.*.CLAIMS.LOOKUP  34]
[8752] 2020/10/20 03:12:24.847934 [TRC] SYSTEM - <<- [SUB $SYS.REQ.CLAIMS.PACK responder 35]

But do not see $SYS.REQ.CLAIMS.UPDATE, is this on a different branch from the nats-server?

matthiashanel

comment created time in 7 days

PullRequestReviewEvent

issue commentdhall-lang/dhall-lang

List of Records with optional values does not work despite type annotations

You also could use the default and only use non optional fields:

let Object = {
  , Type = {
    , name: Text
    , description: Text
  }
  , default = {
    , name = ""
    , description = ""
  }
}

let list = [
   Object::{ 
     , name = "Screwdriver"
   },
   Object::{ 
     , name = "Plate"
     , description = "drive screws"
   }
]

in list

Using the optional types is really useful though since they can be used with merge to match into a function like:

let Object = {
  , Type = {
    , name: Text
    , description: Optional Text
  }
  , default = {
    , name = ""
    , description = None Text
  }
}

let list : List Object.Type = [ 
   , Object::{ 
     name = "Screwdriver" 
   , description = Some "drive screws"
   }
   , Object::{ 
       name = "Plate"
   }
]

let testA = Object::{ name = "screwdriver", description = Some "example" }
let testB = Object::{ name = "plate", description = None Text }

in merge {
   , None = "Empty description"
   , Some = \(description : Text) -> description
} testA.description
xerus2000

comment created time in 7 days

Pull request review commentnats-io/nsc

[Added] cmds to push/pull/generate to/from/for nats based resolver

+/*+ * Copyright 2018-2020 The NATS Authors+ * Licensed under the Apache License, Version 2.0 (the "License");+ * you may not use this file except in compliance with the License.+ * You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS,+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ * See the License for the specific language governing permissions and+ * limitations under the License.+ */++package cmd++import (+	"errors"+	"fmt"++	"github.com/nats-io/jwt"+)++type NatsResolverConfigBuilder struct {+	operator       string+	operatorName   string+	sysAccountSubj string+	sysAccount     string+	sysAccountName string+}++func NewNatsResolverConfigBuilder() *NatsResolverConfigBuilder {+	cb := NatsResolverConfigBuilder{}+	return &cb+}++func (cb *NatsResolverConfigBuilder) Add(rawClaim []byte) error {+	token := string(rawClaim)+	gc, err := jwt.DecodeGeneric(token)+	if err != nil {+		return err+	}+	switch gc.Type {+	case jwt.OperatorClaim:+		if claim, err := jwt.DecodeOperatorClaims(token); err != nil {+			return err+		} else {+			cb.operator = token+			cb.operatorName = claim.Name+		}+	case jwt.AccountClaim:+		if claim, err := jwt.DecodeAccountClaims(token); err != nil {+			return err+		} else if claim.Subject == cb.sysAccountSubj {+			cb.sysAccount = token+			cb.sysAccountName = claim.Name+		}+	}+	return nil+}++func (cb *NatsResolverConfigBuilder) SetOutputDir(fp string) error {+	return errors.New("nats-resolver configurations don't support directory output")+}++func (cb *NatsResolverConfigBuilder) SetSystemAccount(id string) error {+	cb.sysAccountSubj = id+	return nil+}++const tmpl = `# Operator named %s+operator: %s+# System Account named %s+system_account: %s++# configuration of the nats based resolver+resolver {+    type: full+    # Directory in which the account jwt will be stored+    dir: './jwt'

Changing to double quotes to be consistent with interval:

    dir: "./jwt"
matthiashanel

comment created time in 7 days

PullRequestReviewEvent

Pull request review commentnats-io/nsc

[Added] cmds to push/pull/generate to/from/for nats based resolver

+/*+ * Copyright 2018-2020 The NATS Authors+ * Licensed under the Apache License, Version 2.0 (the "License");+ * you may not use this file except in compliance with the License.+ * You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS,+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ * See the License for the specific language governing permissions and+ * limitations under the License.+ */++package cmd++import (+	"errors"+	"fmt"++	"github.com/nats-io/jwt"+)++type NatsResolverConfigBuilder struct {+	operator       string+	operatorName   string+	sysAccountSubj string+	sysAccount     string+	sysAccountName string+}++func NewNatsResolverConfigBuilder() *NatsResolverConfigBuilder {+	cb := NatsResolverConfigBuilder{}+	return &cb+}++func (cb *NatsResolverConfigBuilder) Add(rawClaim []byte) error {+	token := string(rawClaim)+	gc, err := jwt.DecodeGeneric(token)+	if err != nil {+		return err+	}+	switch gc.Type {+	case jwt.OperatorClaim:+		if claim, err := jwt.DecodeOperatorClaims(token); err != nil {+			return err+		} else {+			cb.operator = token+			cb.operatorName = claim.Name+		}+	case jwt.AccountClaim:+		if claim, err := jwt.DecodeAccountClaims(token); err != nil {+			return err+		} else if claim.Subject == cb.sysAccountSubj {+			cb.sysAccount = token+			cb.sysAccountName = claim.Name+		}+	}+	return nil+}++func (cb *NatsResolverConfigBuilder) SetOutputDir(fp string) error {+	return errors.New("nats-resolver configurations don't support directory output")+}++func (cb *NatsResolverConfigBuilder) SetSystemAccount(id string) error {+	cb.sysAccountSubj = id+	return nil+}++const tmpl = `# Operator named %s+operator: %s+# System Account named %s+system_account: %s++# configuration of the nats based resolver+resolver {+    type: full+    # Directory in which the account jwt will be stored+    dir: './jwt'+    # Interval at which a nats-server with a nats based account resolver will compare+    # it's state with one random nats based account resolver in the cluster and if needed, +    # exchange jwt and converge on the same set of jwt.+    interval: "2m"+}++# Preload the nats based resolver with the system account jwt.+# This is not necessary but avoids a bootstrapping system account. +# This only applies to the system account. Therefore other account jwt are not included here.+# To populate the resolver:+# 1) make sure that your operator has Service URLs set and account server URL unset. +#        nsc edit operator --account-jwt-server-url "" --service-url <nats-url1> --service-url <nats-url...> +# 2) push your accounts using: nsc push --all+# 3) prune accounts use: nsc push --prune +# The argument to push -u is optional if your operator has Service URLs set.+# Later changes to the system account take precedence over the system account jwt listed here.+resolver_preload: {+	%s:%s,

Extra space here would be nice:

	%s: %s,
matthiashanel

comment created time in 7 days

PullRequestReviewEvent

pull request commentnats-io/nsc

[Added] cmds to push/pull/generate to/from/for nats based resolver

fyi gave it a try with the operator using service URLs but didn't seem to get picked up:

nsc describe jwt -f ./nsc/accounts/nats/KO/KO.jwt
╭──────────────────────────────────────────────────────────────────────────────────╮
│                                 Operator Details                                 │
├───────────────────────┬──────────────────────────────────────────────────────────┤
│ Name                  │ KO                                                       │
│ Operator ID           │ ODDLNCXYORNYEYVHHQKKOFPRDGIRCSUUF5XE2RDSQMHHJD3QO2LEHRYQ │
│ Issuer ID             │ ODDLNCXYORNYEYVHHQKKOFPRDGIRCSUUF5XE2RDSQMHHJD3QO2LEHRYQ │
│ Issued                │ 2020-10-20 08:19:21 UTC                                  │
│ Expires               │                                                          │
│ Operator Service URLs │ nats://127.0.0.1:4222                                    │
╰───────────────────────┴──────────────────────────────────────────────────────────╯
nsc push --all
Error: no account server url or nats-server url was provided by the operator jwt
matthiashanel

comment created time in 7 days

issue commentnats-io/nats-box

nats-box changes default URL to be NGS

sounds good to me as well

aricart

comment created time in 8 days

startedantlad/nats_asio

started time in 8 days

issue commentdhall-lang/dhall-lang

List of Records with optional values does not work despite type annotations

Maintainers will probably chime-in but I think one of the recommended ways to make this flexible is by using default and letting that fill in the record:

let Object = { 
  Type = { 
    name: Text, 
    description: Optional Text 
  },
  default = { 
    description = None Text 
  }, 
}

let list : List Object.Type = [
  Object::{
    name = "Screwdriver",
    description = Some "drives screws"
  },
  Object::{
    name = "Plate"
  },
]

in list : List Object.Type

Then you could have Screwdriver and Plate types for example which are different types but could be part of an union type etc... and either could then belong to the same list.

xerus2000

comment created time in 8 days

startedm3o/dev

started time in 10 days

starteddoc-org/doc-org

started time in 10 days

push eventnats-io/nats.go

R.I.Pienaar

commit sha 8b0826855343f00458cbd14259d18f1a3edf91e2

Allow the Subscribe functions to access JetStream consumers Signed-off-by: R.I.Pienaar <rip@devco.net>

view details

R.I.Pienaar

commit sha a08c0a0959347ac020ea891236fd65329f1bb91b

unexport various things copied from jsm Signed-off-by: R.I.Pienaar <rip@devco.net>

view details

R.I.Pienaar

commit sha 8c8e1e9644c86670426a43fa4815b2bcbb50276c

remove DeliverySubject from the public ConsumerConfig Signed-off-by: R.I.Pienaar <rip@devco.net>

view details

R.I.Pienaar

commit sha 5a33ce07522f4f491007b87406f238a524ed9ccc

Merge pull request #596 from ripienaar/js_consumer Allow the Subscribe functions to access JetStream consumers

view details

Waldemar Quevedo

commit sha a7e27ac5148aa7c9b7963d6d065af4f76d491e1d

Add separate Go modfile for tests (requires Go 1.14) Signed-off-by: Waldemar Quevedo <wally@synadia.com>

view details

push time in 11 days

issue openednats-io/nack

Update jsm.go deps

created time in 11 days

delete branch wallyqs/StreamingExamples.net

delete branch : k8s

delete time in 12 days

PR opened ColinSullivan1/StreamingExamples.net

Use 0.17.0 in ssd test

Signed-off-by: Waldemar Quevedo wally@synadia.com

+1 -1

0 comment

1 changed file

pr created time in 12 days

push eventwallyqs/StreamingExamples.net

Waldemar Quevedo

commit sha 03bdf209f60e6aadc981f2f3676d7a6ccf4ac9a4

Use 0.17.0 Signed-off-by: Waldemar Quevedo <wally@synadia.com>

view details

push time in 12 days

delete branch wallyqs/nats-server

delete branch : ws-err-typo-fix

delete time in 12 days

PR opened nats-io/nats-server

Fix typo in websocket err message

Signed-off-by: Waldemar Quevedo wally@synadia.com

+2 -2

0 comment

2 changed files

pr created time in 12 days

create barnchwallyqs/nats-server

branch : ws-err-typo-fix

created branch time in 12 days

PR opened ColinSullivan1/StreamingExamples.net

Bump disk size

Signed-off-by: Waldemar Quevedo wally@synadia.com

+1 -1

0 comment

1 changed file

pr created time in 12 days

create barnchwallyqs/StreamingExamples.net

branch : k8s

created branch time in 12 days

PullRequestReviewEvent

delete branch wallyqs/StreamingExamples.net

delete branch : k8s

delete time in 12 days

PR opened ColinSullivan1/StreamingExamples.net

Add k8s manifest

Signed-off-by: Waldemar Quevedo wally@synadia.com

+300 -0

0 comment

3 changed files

pr created time in 12 days

create barnchwallyqs/StreamingExamples.net

branch : k8s

created branch time in 12 days

push eventnats-io/nats-box

Waldemar Quevedo

commit sha 0af01c7068287284c490c723e4b11ad39d1baaff

Update README.md

view details

push time in 12 days

fork wallyqs/AksNodePublicIPController

A project that can be deployed to an Azure Kubernetes Cluster and will allow each node to obtain a Public IP address.

fork in 12 days

fork wallyqs/nats.ws

WebSocket NATS

fork in 12 days

issue openednats-io/nats-box

Include stan bench

created time in 13 days

push eventwallyqs/nats.dhall

Waldemar Quevedo

commit sha ee57e401ee5af0bb7b92876e367c63716f0d152b

Update examples.org

view details

push time in 13 days

issue commenthelm/helm

Feature Request: Disable escaping HTML in toPrettyJSON

That's fair, sounds like toPrettyRawJson would fit better

wallyqs

comment created time in 13 days

delete branch nats-io/nats.py

delete branch : sub-next-msg

delete time in 13 days

push eventwallyqs/asyncio-nats

Waldemar Quevedo

commit sha 926ef1f2043e0f653724679562aecac79fdb73ee

Merge pull request #177 from wallyqs/more-async-for Updates for async for handling

view details

Brian Shannan

commit sha c0c15bc2914f3708198441fe7612c214442f36ce

add type hints

view details

Brian Shannan

commit sha 1d527bd091feb6b7f174544d8dcdac4c65d4464b

add Msg::respond()

view details

Waldemar Quevedo

commit sha b2cab62ab556c77e4983ac46694ddfa224e4f6bb

Merge pull request #182 from brianshannan/msg-respond add Msg::respond()

view details

Brian Shannan

commit sha d00bfe4ed064dac1864a4e7aae3b592dd19574cd

use fstrings

view details

Brian Shannan

commit sha 2346f9689bb0c0525061fb1833e420af23b8659e

Merge branch 'v1-dev' of github.com:nats-io/nats.py into typing

view details

Brian Shannan

commit sha 4a4d611ccfb6f4cf31f2cc60cd3a1090593d7995

Merge pull request #181 from brianshannan/typing add type hints to publicly available apis

view details

Waldemar Quevedo

commit sha 90751e8719ac572fc5f8441a5a5b3beba687b0e0

Merge pull request #183 from brianshannan/fstrings use fstrings where possible

view details

Waldemar Quevedo

commit sha e55a72d636aec2a20475d9995ac4d4f58c510c62

Add next_msg to subscription Signed-off-by: Waldemar Quevedo <wally@synadia.com>

view details

push time in 13 days

issue closednats-io/nats.py

Add msg.respond()

Similar to the Go client.

closed time in 13 days

wallyqs

issue commentnats-io/nats.py

Add msg.respond()

Fixed via #182

wallyqs

comment created time in 13 days

issue commentnats-io/nats.py

DeprecationWarning: The loop argument is scheduled for removal in Python 3.10.

Fixed in v1-dev branch already

wallyqs

comment created time in 13 days

create barnchnats-io/nats.py

branch : sub-next-msg

created branch time in 13 days

startednginxinc/nginx-service-mesh

started time in 14 days

push eventnats-io/nats-operator

Waldemar Quevedo

commit sha a0f6ef4f5e50ac3f29a629d3c6057f7498c0fe4c

Update README.md

view details

push time in 14 days

PR opened nats-io/nats-operator

Reviewers
Release v0.7.6

Bump version

+1 -1

0 comment

1 changed file

pr created time in 14 days

create barnchnats-io/nats-operator

branch : release/v0.7.6

created branch time in 14 days

delete branch nats-io/nats-operator

delete branch : websocket-config

delete time in 14 days

push eventnats-io/nats-operator

Waldemar Quevedo

commit sha 01a2b7f776ab4648f58db126749d67d880bded6e

Add websocket support This change exposes a websocketConfig block in the NatsCluster CRD. This allows clients to connect to a cluster by using WebSockets. Note: TLS is required for WebSocket connections. Also, currently, WebSockets are only supported in NATS Server nightly images.

view details

Waldemar Quevedo

commit sha 65c21eaf680c7fdcafc67083655a95826ae891b6

Merge pull request #288 from nats-io/websocket-config Add websocket support

view details

push time in 14 days

PR merged nats-io/nats-operator

Add websocket support

This change exposes a websocketConfig block in the NatsCluster CRD. This allows clients to connect to a cluster by using WebSockets. Note: TLS is required for WebSocket connections.

Also, currently, WebSockets are only supported in NATS Server nightly images.

+252 -15

0 comment

10 changed files

variadico

pr closed time in 14 days

PullRequestReviewEvent

Pull request review commentnats-io/nats-operator

Add websocket support

 func natsPodContainer(container v1.Container, clusterName, version string, serve 		} 		ports = append(ports, port) 	}+	if websocketPort > 0 {+		port := v1.ContainerPort{+			Name:          "websocket",+			ContainerPort: int32(websocketPort),+			Protocol:      v1.ProtocolTCP,+			HostPort:      int32(websocketPort),

TODO: Make optional

variadico

comment created time in 14 days

PullRequestReviewEvent

issue commenthelm/helm

Feature Request: Disable escaping HTML in toPrettyJSON

I'd say that there are actually a lot of charts using it out there unintentionally outputting JSON with escaped html (which is a very common format for automated still readable configuration), since it is a time saver over having to implement the complete schema: https://github.com/search?l=YAML&q=toPrettyJson&type=Code

wallyqs

comment created time in 14 days

issue commentnats-io/nats-streaming-server

Question: Rust client

Yes, you can have both push based consumers and pull based consumers.

delucca

comment created time in 15 days

PullRequestReviewEvent

issue commentnats-io/nats-streaming-server

Question: Rust client

@delucca np, you can find more info about JetStream here: https://github.com/nats-io/jetstream it is the next iteration for NATS Streaming and would not need a NATS Streaming client. This snippet is a bit old but to give you an idea of how it looks when using it from Rust: https://gist.github.com/wallyqs/05516d550b756e8b453394be0e9cbf24#file-jetstream-hello-world-rs

delucca

comment created time in 15 days

push eventnats-io/k8s

Waldemar Quevedo

commit sha fe80bd817e56e533fd836bb19fb54d9304824026

Update README.md

view details

push time in 15 days

more