profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/ericsyh/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

ericsyh/apache-pulsar-grafana-dashboard 0

Apache Pulsar Grafana Dashboard

ericsyh/apisix-dashboard 0

Dashboard for Apache APISIX

ericsyh/BigData-Notes 0

大数据入门指南 :star:

ericsyh/blog 0

blog

ericsyh/chaos-mesh 0

A Chaos Engineering Platform for Kubernetes.

ericsyh/charts 0

StreamNative Helm Charts Repository: Apache Pulsar, Pulsar Operators, StreamNative Platform, Function Mesh

ericsyh/cockroach 0

CockroachDB - the open source, cloud-native SQL database.

ericsyh/cp-ansible 0

Ansible playbooks for the Confluent Platform

push eventericsyh/pulsar

Matteo Merli

commit sha da27b9eac613e65621822b18c0ec40cebcfe55b0

Avoid potentially blocking calls to metadata on critical threads (#12339) * Avoid potentially blocking calls to metadata on critical threads * Fixed log arguments order * Addressed comments * Fixed mock in PersistentSubscriptionTest * Fixed issue in mocked tests * Fixed test that was force policies modification under the hood

view details

Andrey Yegorov

commit sha 21b758ccedda40e025a951b1711a58a701c967ac

Fixed: "Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Login failed for user 'sa'.." in MS SQL integration test (#12374) * set specific tag for image, use env var for the new image * moved to an older docker image * Updated wait strategy as outlined in https://github.com/microsoft/mssql-docker/issues/625

view details

Nicolò Boschi

commit sha cc70a1f5e4ef228ca040433249d54c6e22711c20

[managedledger] NPE on OpAddEntry while ManagedLedger is closing (#12364)

view details

Rajan Dhabalia

commit sha 1ce016c5782e02ead6cddd2ea46c063f3746727f

[pulsar-broker] support split largest bundle of the namespace (#12361) * [pulsar-broker] support split largest bundle of the namespace * fix formatting violation

view details

Yan

commit sha 4498b8b742d543922d40f286b76af5622d1ae691

doc migration version-2.7.2 chapter Get Started (#12323)

view details

Yuri Mizushima

commit sha 1f8945addb18bf1f94366d2c7f5286ee6f62c675

[PIP 79][client] Add lazy-loading feature to PartitionedProducer (#10279) * feat: add lazy loading feature to PartitionedProducerImpl * feat: add partial round robin routing mode * test: add tests for lazy-loading * fix: fix producer closing code at lazy-loading * test: remove unnecessary handling, fail from test codes * feat: add enableLazyStartPartitionedProducers config * test: fix test for lazy-loading config * fix: address comments * fix: add partition-change interceptor * fix: address comments

view details

Eron Wright

commit sha 38539b351869b829c99a2b96354ce852f137015d

[PIP 95][Issue 12040][broker] Decouple advertised listeners from bind addresses (#12079)

view details

Xiangying Meng

commit sha 70c9f610078be57bee4c1581e6df64f694bbb13a

[Transaction] Transaction buffer take snapshot max read position (#12219) ### Motivation In the previous implementation of transactionBuffer, if no transaction committed or aborted, TransactionBuffer will not take a snapshot. Even if it is a timed task, it will judge whether there has been a transaction committed or aborted. If no, the timer will be skipped. **This will have a disadvantage:** * If there is no snapshot , maxReadPosition needs to be recovered from the earliest when the broker restart. ### Modifications Add a TransactionBuffer State: Unused. If no message with transaction was sent, the state of transactionBuffer is Unused. And if TransactionBuffer state is Unused, the TransactionBuffer is no need to be recovered. When the first message with transaction was sent to buffer, it will take a snapshot. And we have change the return value of takeSnapshot(), make it can be control whether to execute synchronously

view details

Eron Wright

commit sha 6382be84ae023edeae1ccc996d70391e8b01aec4

[Issue 12040][broker] Fix advertised listener confusion (#12353) * [Issue 12040] - deprecate ServiceConfigurationUtils::getAppliedAdvertisedAddress - clarify the purpose of PulsarService::advertisedAddress - introduce a helper method for webServiceAddress - stabilize the ordering of advertised listeners - add a utility class for local port-forwarding - enhance the MockedPulsarServiceBaseTest to port-forward from advertised port to listen port - fix the PulsarMultiListenersWithInternalListenerNameTest Co-authored-by: penghui <penghui@apache.org>

view details

Yunze Xu

commit sha d5e3ebba2baacb9db57ec0b9621fd7672e2be95f

[C++] Delay ClientCredentialFlow::initialize to the first authenticate call (#12372) * Delay ClientCredentialFlow::initialize to the first time authenticate() is called * Make once_flag object a field

view details

Eron Wright

commit sha d1c89324824ccd150857982755ee6d7e61321d9a

[Issue 12040] (#12387) - fix MultipleListenerValidatorTest

view details

litao

commit sha 9d10b8b3aa19647f6ff81e39866bd5d409c4133a

Fix java doc for MultipleListenerValidator (#12389) ### Motivation *Fix the java doc mentioned in [#12353](https://github.com/apache/pulsar/pull/12353/files#r729887373)* ### Modifications *Fix java doc for MultipleListenerValidator#validateAndAnalysisAdvertisedListener.*

view details

Anonymitaet

commit sha 4bf9d89e403f3f97417f28d7a49830e321f8a49a

[Doc] add scope explanations (#12380) * [Doc] add scope explanations * update

view details

congbo

commit sha 69fb80236d61e7594fdf8e84724f435d7820cc3f

[Admin] Get schema validation enforce add applied. (#12349) now, namespace get schema validation enforce don't return broker config, if namespace policy the schema validation enforce is false and we use --applied, we should return this config in broker level.

view details

fengtao1998

commit sha 0e0b3afc551a96ac230ec2f14401089c0c5e179a

[Docs] Add document label check robot (#12371) * Update PULL_REQUEST_TEMPLATE.md * Create ci-documentbot.yml * Update ci-documentbot.yml

view details

sijia-w

commit sha 6ab50353cdab2cc45ebc0e4072719a4651ef1202

[docs] [ISSUE 11526] Update the description of `topic` (#12375) * update versioned docs * add changes * correct an error

view details

Li Li

commit sha b3ac2ed66a0c26927e4b02bbad1e097e2a4bfe60

[website][upgrade]feat: home page (#12383) * feat: add tailwindcss supported Signed-off-by: LiLi <urfreespace@gmail.com> * feat: index page content Signed-off-by: LiLi <urfreespace@gmail.com> * feat: patch Signed-off-by: LiLi <urfreespace@gmail.com> * feat: add material design Signed-off-by: LiLi <urfreespace@gmail.com> * patch Signed-off-by: LiLi <urfreespace@gmail.com>

view details

ZhangJian He

commit sha 4da43109066ebea7a0b43a5a5c598b4a923e9d9b

fix windows test path probleam (#12398) ### Motivation Make unit test path compatible to windows, make these tests can run on windows ### Modifications Same to #12329 , change `"file://" + tokenFile` to `"file:///" + tokenFile.toString().replace('\\', '/')` Run in windows can't recognize format like `file://C\xxx\yyy\zzz` It should be `file:///C` and the `\` should be `/`

view details

Matteo Merli

commit sha 4e43a1dd85809f0242e354aef7a27973820e0dda

[C++] Use weak ref to ClientConnection for timeout task (#12409) ### Motivation Fixes #12408. Using a weak reference in the timeout task for `ClientConnection` to break a circular reference dependency between the connection instance and the task.

view details

Bowen Li

commit sha fd5bd6ae62bf9bada18ec017908cebbe22d004ef

[docs][Website] add docs of broker entry metadata (#12404) * add docs of broker entry metadata * Update site2/docs/developing-binary-protocol.md Co-authored-by: Anonymitaet <50226895+Anonymitaet@users.noreply.github.com> * Update site2/docs/developing-binary-protocol.md Co-authored-by: Anonymitaet <50226895+Anonymitaet@users.noreply.github.com> * Update site2/docs/developing-binary-protocol.md Co-authored-by: Anonymitaet <50226895+Anonymitaet@users.noreply.github.com> * Update site2/docs/developing-binary-protocol.md Co-authored-by: Anonymitaet <50226895+Anonymitaet@users.noreply.github.com> * Update site2/docs/developing-binary-protocol.md Co-authored-by: Anonymitaet <50226895+Anonymitaet@users.noreply.github.com> * Update site2/docs/developing-binary-protocol.md Co-authored-by: Anonymitaet <50226895+Anonymitaet@users.noreply.github.com> * Update site2/docs/developing-binary-protocol.md Co-authored-by: Anonymitaet <50226895+Anonymitaet@users.noreply.github.com> * fix comments. * Update site2/docs/developing-binary-protocol.md Co-authored-by: Anonymitaet <50226895+Anonymitaet@users.noreply.github.com> Co-authored-by: Anonymitaet <50226895+Anonymitaet@users.noreply.github.com>

view details

push time in a day

push eventericsyh/pulsar-helm-chart

Eric Shen

commit sha a4ddc78eff4e4b4524e2e5cce2612156def1974e

add limits to initContainers Signed-off-by: Eric Shen <ericshenyuhao@outlook.com>

view details

push time in 2 days

push eventericsyh/pulsar-helm-chart

Eric Shen

commit sha 99a70b78487653a27ce298d3d45d52b9355781cc

change proxy replica count Signed-off-by: Eric Shen <ericshenyuhao@outlook.com>

view details

push time in 2 days

push eventericsyh/pulsar-helm-chart

Eric Shen

commit sha c0231975bdd9695e77796451a94d889661eeb0ff

add limits for metadata Signed-off-by: Eric Shen <ericshenyuhao@outlook.com>

view details

push time in 2 days

push eventericsyh/pulsar-helm-chart

ericsyh

commit sha e397214c918192c6d534ca6063e5c92e5ef22b38

add limits Signed-off-by: ericsyh <ericshenyuhao@outlook.com>

view details

push time in 3 days

create barnchericsyh/pulsar-helm-chart

branch : htsc

created branch time in 3 days

pull request commentapache/pulsar

docs(cli):add restart command in pulsar-daemon

@Anonymitaet This change only affect master. Previously versions do not support this command.

ericsyh

comment created time in 3 days

push eventericsyh/pulsar

Eric Shen

commit sha 35338550e74fec651d4b38ec946fa70f7eaee3c6

Update site2/docs/reference-cli-tools.md Co-authored-by: Anonymitaet <50226895+Anonymitaet@users.noreply.github.com>

view details

push time in 3 days

startedrancher/local-path-provisioner

started time in 6 days

PR opened apache/pulsar

docs(cli):add restart command in pulsar-daemon

Signed-off-by: Eric Shen ericshenyuhao@outlook.com

<!--

Contribution Checklist

  • Name the pull request in the form "[Issue XYZ][component] Title of the pull request", where XYZ should be replaced by the actual issue number. Skip Issue XYZ if there is no associated github issue for this pull request. Skip component if you are unsure about which is the best component. E.g. [docs] Fix typo in produce method.

  • Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.

  • Each pull request should address only one issue, not mix up code from multiple issues.

  • Each commit in the pull request has a meaningful commit message

  • Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.

(The sections below can be removed for hotfixes of typos) -->

(If this PR fixes a github issue, please add Fixes #<xyz>.)

Related PR: https://github.com/apache/pulsar/pull/12279

Motivation

Update the pulsar-daemon command docs since a new command added.

Modifications

Added the restart command in cli tools docs.

Documentation

Check the box below and label this PR (if you have committer privilege).

Need to update docs?

  • [ ] doc-required

    (If you need help on updating docs, create a doc issue)

  • [ ] no-need-doc

    (Please explain why)

  • [x] doc

    (If this PR contains doc changes)

+6 -1

0 comment

1 changed file

pr created time in 7 days

create barnchericsyh/pulsar

branch : docs-pulsar-daemon-newcommand

created branch time in 7 days

push eventericsyh/pulsar

kaushik-develop

commit sha 7d9e9f33bf27c9b3d5beaa958cfe7945654b96c0

Support for draining workers (#12178) Co-authored-by: Kaushik Ghosh <kaushikg@splunk.com>

view details

Michael Marshall

commit sha 2ad0e5afccaaae85969d2924920a55ce95e248f6

[Java Client] Use failPendingMessages to ensure proper cleanup (#12259) * [Java Client] Use failPendingMessages to ensure proper cleanup * Update method name from code review comments * Update pulsar-broker/src/test/java/org/apache/pulsar/client/impl/ProducerCloseTest.java Co-authored-by: Matteo Merli <mmerli@apache.org> * Move setState into sync block; consolidate client.cleanupProducer call * Move cleanupProducer into sync block * Make method closeAndClearPendingMessages synchronized Co-authored-by: Matteo Merli <mmerli@apache.org>

view details

lipenghui

commit sha 65fb6c6cce3ffac221a559279eb17ce1ef9b885b

Fix incorrect returned last message ID while the `lastConfirmedEntry` with negative entry ID. (#12277) Which recover a ManagedLedger, if the ManagedLedgers does not contain any ledgers, the ManagedLedger will use the current Ledger ID and -1 to generate the `lastConfirmedEntry`. For more details to see: https://github.com/apache/pulsar/blob/4bc3c405a565b1c756b9b70ff02a63ea06c32c0d/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L477 But for compacted topic, all the data might be compacted and move to the compacted Ledger, In this case, the broker will return X:-1 as the last message ID of the topic to the consumer, the consumer will treat the negative entry ID as no data in this topic, so hasMoreMessages will return false, but there is compacted data in the topic. The fix is as #12161 does to return the last message ID from the compacted Ledger if `lastConfirmedEntry` of the ManagedLedger with negative entry ID. Improved the `testLastMessageIdForCompactedLedger` test to cover the new changes.

view details

Boyang Jerry Peng

commit sha cd9356a03063653e0097f4b85d6b3546698cc325

Allow Pulsar Functions localrun to exit on error (#12278) Co-authored-by: Jerry Peng <jerryp@splunk.com>

view details

Yunze Xu

commit sha 86b2a212d4bf38d536814517d6e91ea8d6be7a58

[PIP 96] Add message payload processor for Pulsar client (#12088)

view details

Ming

commit sha 8b55636cc3df8c4cb1d2dd3896bc2bab79c267cc

[Issue-11966][pulsar-proxy] set default http proxy request timeout (#11971)

view details

Rajan Dhabalia

commit sha bfa2b29f3d4eee5270799000dee2b4218c78fdd6

[pulsar-broker] Fix: handle failed partitions topic creation (#10374) * [pulsar-broker] Fix: handle failed partitions topic creation * fix test

view details

Addison Higham

commit sha 6b93a1688b25b937379b763b4e2cb7c5423fb41a

Fix Pulsar Proxy to re-use authentication instance (#12245) * Fix Pulsar Proxy to re-use authentication instance Currently, the Pulsar Proxy creates a new PulsarClientImpl with a new instance of the client authentication plugin. For certain client auth implementations, this can cause issues. For example, if a client plugin needs to generate a token and then cache and re-use it (which is very common with typical Pulsar client usage) this pattern breaks, because the client auth plugin is tied to the lifecycle of the connection and not the more "singleton" usage of the Pulsar client. Arguably, we should instead figure out how to re-use the entire Pulsar client, but that likely has more complexity, instead this "quick fix" will get one of the most obvious cases solved. * add test for ensuring all same auth instance * Simplify ProxyAuthTest It isn't clear why this test was doing timeouts... it doesn't really seem to be testing anything as the auth token was not being refreshed and it appears the test was expected to pass (it almost looks like they were expected to fail?) This allows us to make this test faster and more reliable, as the timeouts don't really seem to be adding anything

view details

Enrico Olivelli

commit sha b3540d28ff1e91e9766f53ad2cab0cd7915b9dbe

Release: allow Integration Tests Jar to be deployed to Maven central (#12292) - integration tests framework is needed by Pulsar Adapters project - during the release procedure, while running 'mvn deploy -DskipTests -Papache-release -DintegrationTests --settings src/settings.xml' allow to deploy integration tests jars

view details

congbo

commit sha c14d079fa992d308e5ae04f7e7d0b9ada4f28cac

[Transaction] Transaction coordinator fence mechanism. (#11357)

view details

feynmanlin

commit sha 6ee0d340028e470e3b05f2ea4f9eb15142273843

[PIP-92] Part-1: Separate Topic Policies into a separate class (#12034) ### Motivation When setting the topic policy for a geo-replicated cluster, some policies want to affect the whole geo-replicated cluster but some only want to affect the local cluster. https://github.com/apache/pulsar/wiki/PIP-92%3A-Topic-policy-across-multiple-clusters And I accept penghui‘s advice: https://lists.apache.org/thread.html/r1c4be3c3b97618625cc7b3410634b43704e0a290df21b7356a491152%40%3Cdev.pulsar.apache.org%3E ### Modifications Separate Topic Policies into a separate class

view details

Andrey Yegorov

commit sha 3545a29b34d91272f9de3af76c521acad0759eeb

Upgrading Debezium to 1.7 (#12295)

view details

Enrico Olivelli

commit sha d4ca1ff357585136d4c05068770f17874dca557c

[Proxy] set default httpProxyTimeout to 5 minutes (#12299)

view details

feynmanlin

commit sha 7b14e569f8b9265af6866ab357e98797dd9283e4

Fix NPE when removing cursor (#12297) ### Motivation Cursor may not exists, so keep it consistent with the persistent Cursor, add a new judgment whether it exists

view details

Yunze Xu

commit sha 44dcc04d037511984ec383fb3f2c75170e4cfefc

[C++] Support configuring optional scope field for OAuth2 authentication (#12305) ### Motivation It's a C++ client catchup for https://github.com/apache/pulsar/pull/11931. ### Modifications - Add a `scope_` field to `ClientCredentialFlow` and load it from `ParamMap` object whose key is `scope`. - Refactor `ClientCredentialFlow` to simplify code and make it testable: - Use only one constructor instead of two overloaded constructors that might look confused - Add a `generateJsonBody` public method for generating JSON body for post fields in `authenticate` so that it can be tested. - Add a `KeyFile` class like what Java client does to load client id and client secret from `ParamMap` or file. ### Verifying this change - [x] Make sure that the change passes the CI checks. This change added test `AuthPluginTest.testOauth2RequestBody` for the cases that scope exists or doesn't exist.

view details

congbo

commit sha e29f720a177c95d42ccdbfa76f987846b90d4d9a

[Transaction] Transaction coordinator client connect add response command. (#12301) ## Motivation now https://github.com/apache/pulsar/pull/11357 has merged, https://github.com/apache/pulsar/pull/11357 use `SUCCESS` command handle tcClientConnectRequest, This is not conducive to later expansion. So add a individual response for `CommandTcClientConnectRequest` ## implement add command ``` message CommandTcClientConnectResponse { required uint64 request_id = 1; optional ServerError error = 2; optional string message = 3; } ``` In order to ensure that the new client is compatible with the old broker, I update protocol version to 19. ``` v19 = 19; // Add CommandTcClientConnectRequest and CommandTcClientConnectResponse ``` if broker protocol version > 18 we should send TcClientConnectCommand if broker protocol version <= 18 we don't need to send TcClientConnectCommand

view details

congbo

commit sha 8b50af517255c314c773c6fe0c2530e09fafa312

[Transaction] Transaction pending ack lazy init. (#11091) ## Motivation now, in `broker.conf` `transactionCoordinatorEnabled=true` MLPendingAck will init manageLedger, some ack will not use transaction, so don't need to init manageLedger. When this sub use transaction, we can lazy init `PendingAckHandle`. ## implement When this sub use transaction, we can lazy init `PendingAckHandle`.

view details

Zixuan Liu

commit sha f9057c7b581384158aa0927b2c9a08cbafefc4a0

Improve exceptions thrown when handling the schema resource (#12155) Fixes #12010, #10694 ### Motivation When handing the schema resource, if an error occurs, there are no more error message responses, and the client only knows that the server returns 500 internal errors without the more helpful error message. You can use the `pulsarctl schemas upload test-schema-topic -f "./test.json"` command to upload this schema: ``` { "type": "PROTOBUF", "schema": "syntax = \"proto3\";\n\npackage com.mycorp.mynamespace;\n\nimport 'google/protobuf/any.proto';\n \nmessage SampleRecord { \n\n int32 my_field1 = 1; \n double my_field2 = 2; \n string my_field3 = 3; \n}", "properties": {} } ``` Before improvement: ``` code: 500 reason: Unknown error ``` After improvement: ``` code: 500 reason: --- An unexpected error occurred in the server --- Message: org.apache.pulsar.broker.service.schema.exceptions.InvalidSchemaDataException: Invalid schema definition data for PROTOBUF schema Stacktrace: java.util.concurrent.CompletionException: org.apache.pulsar.broker.service.schema.exceptions.InvalidSchemaDataException: Invalid schema definition data for PROTOBUF schema at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331) at java.base/java.util.concurrent.CompletableFuture.uniAcceptNow(CompletableFuture.java:743) at java.base/java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:731) at java.base/java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2108) at org.apache.pulsar.broker.admin.impl.SchemasResourceBase.lambda$postSchema$6(SchemasResourceBase.java:168) at java.base/java.util.concurrent.CompletableFuture.uniAcceptNow(CompletableFuture.java:753) at java.base/java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:731) at java.base/java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2108) at org.apache.pulsar.broker.admin.impl.SchemasResourceBase.postSchema(SchemasResourceBase.java:139) at org.apache.pulsar.broker.admin.v2.SchemasResource.postSchema(SchemasResource.java:199) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626) at org.apache.pulsar.broker.web.ResponseHandlerFilter.doFilter(ResponseHandlerFilter.java:67) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:179) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: org.apache.pulsar.broker.service.schema.exceptions.InvalidSchemaDataException: Invalid schema definition data for PROTOBUF schema at org.apache.pulsar.broker.service.schema.validator.StructSchemaDataValidator.throwInvalidSchemaDataException(StructSchemaDataValidator.java:75) at org.apache.pulsar.broker.service.schema.validator.StructSchemaDataValidator.validate(StructSchemaDataValidator.java:65) at org.apache.pulsar.broker.service.schema.validator.SchemaDataValidator.validateSchemaData(SchemaDataValidator.java:42) at org.apache.pulsar.broker.service.schema.validator.SchemaRegistryServiceWithSchemaDataValidator.putSchemaIfAbsent(SchemaRegistryServiceWithSchemaDataValidator.java:92) at org.apache.pulsar.broker.admin.impl.SchemasResourceBase.lambda$postSchema$6(SchemasResourceBase.java:163) ... 68 more Caused by: org.apache.avro.SchemaParseException: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'syntax': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false') at [Source: (String)"syntax = "proto3"; package com.mycorp.mynamespace; import 'google/protobuf/any.proto'; message SampleRecord { int32 my_field1 = 1; double my_field2 = 2; string my_field3 = 3; }"; line: 1, column: 7] at org.apache.avro.Schema$Parser.parse(Schema.java:1432) at org.apache.avro.Schema$Parser.parse(Schema.java:1418) at org.apache.pulsar.broker.service.schema.validator.StructSchemaDataValidator.validate(StructSchemaDataValidator.java:52) ... 71 more Caused by: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'syntax': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false') at [Source: (String)"syntax = "proto3"; package com.mycorp.mynamespace; import 'google/protobuf/any.proto'; message SampleRecord { int32 my_field1 = 1; double my_field2 = 2; string my_field3 = 3; }"; line: 1, column: 7] at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2337) at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:720) at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:2903) at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1949) at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:781) at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2902) at org.apache.avro.Schema$Parser.parse(Schema.java:1430) ... 73 more ```

view details

Aloys

commit sha aa408fff0244b4e4a4d1a8b1475b50704c422a63

fix ttl expiry does not take effect (#12266) Fixes #12265 ### Motivation fix ttl expiry does not take effect ### Modifications When `ManagedLedgerImpl.getPositionAfterN` 1. check whether the `currentLeder` contains entry when count the entries of `currentLeder` 2. check the result and `LAC` first before return ### Verifying this change add test in `ManagedLedgerTest.testGetPositionAfterN`

view details

ZhangJian He

commit sha 3de54d8e8dc7525cb2730e12b8585f953adc4fd7

presto sample doc missing the set field method (#12127)

view details

push time in 7 days

issue commentstreamnative/charts

[sn-platform]: kafka producer cannot connect to KoP in sn-platform charts

Close the issue since i already got the root cause. Will submit a new issue for enhancement.

ericsyh

comment created time in 7 days

issue closedstreamnative/charts

[sn-platform]: kafka producer cannot connect to KoP in sn-platform charts

Background

I followed the platform-quickstart to deploy the platform in Kind K8s environment with the values_cluster.yaml. Then i created a Kafka client pod to connect the KoP by the kafka-console-producer.sh but get connection error.

Kafka client command

In the Kafka client pod, i run the below command to connect KoP:

kafka-console-producer.sh \
--producer-property security.protocol=SASL_PLAINTEXT \
--producer-property sasl.mechanism=PLAIN \
--producer-property 'sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="public/default" password="token:M2ZjMTZlMmEtODVkNi02ZTQ4LTZhY2EtMTljMmE1ZWIyY2FhOjMxYTUyZGI3LTJmMWMtZjAwMC00OTY3LTM1YjE4ZDhjMDIyYw==";' \
--broker-list sn-platform-broker:9092 --topic test-kop-1

sn-platform-broker is the broker service by ClusterIP type:

$ kc get svc -n sn
NAME                                        TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
sn-platform-broker                          ClusterIP      10.96.219.184   <none>        6650/TCP,8080/TCP,9092/TCP            45h
sn-platform-broker-headless                 ClusterIP      None            <none>        6650/TCP,8080/TCP,9092/TCP            45h

Error msg

After execute the kafka-console-producer.sh command, the connection to KoP should be successful. But after running the message in the console, i will get the connection error message.

root@sn-platform-toolset-0:/kafka_2.13-2.6.0/bin# ./kafka-console-producer.sh \
> --producer-property security.protocol=SASL_PLAINTEXT \
> --producer-property sasl.mechanism=PLAIN \
> --producer-property 'sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="public/default" password="token:M2ZjMTZlMmEtODVkNi02ZTQ4LTZhY2EtMTljMmE1ZWIyY2FhOjMxYTUyZGI3LTJmMWMtZjAwMC00OTY3LTM1YjE4ZDhjMDIyYw==";' \
> --broker-list sn-platform-broker:9092 --topic test-kop-1
>
root@sn-platform-toolset-0:/kafka_2.13-2.6.0/bin# ./kafka-console-producer.sh \
> --producer-property security.protocol=SASL_PLAINTEXT \
> --producer-property sasl.mechanism=PLAIN \
> --producer-property 'sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="public/default" password="token:M2ZjMTZlMmEtODVkNi02ZTQ4LTZhY2EtMTljMmE1ZWIyY2FhOjMxYTUyZGI3LTJmMWMtZjAwMC00OTY3LTM1YjE4ZDhjMDIyYw==";' \
> --broker-list sn-platform-broker:9092 --topic test-kop-1
>test
[2021-08-25 03:05:50,391] WARN [Producer clientId=console-producer] Error connecting to node sn-platform-broker-0.sn-platform-broker-headless..svc.cluster.local:9092 (id: 2015490984 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: sn-platform-broker-0.sn-platform-broker-headless..svc.cluster.local: Name or service not known
        at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
        at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929)
        at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1519)
        at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848)
        at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1509)
        at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1368)
        at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1302)
        at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
        at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:403)
        at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:363)
        at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:151)
        at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:958)
        at org.apache.kafka.clients.NetworkClient.access$600(NetworkClient.java:74)
        at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1131)
        at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1019)
        at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:542)
        at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:325)
        at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:240)
        at java.base/java.lang.Thread.run(Thread.java:829)
[2021-08-25 03:05:50,489] WARN [Producer clientId=console-producer] Error connecting to node sn-platform-broker-0.sn-platform-broker-headless..svc.cluster.local:9092 (id: 2015490984 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: sn-platform-broker-0.sn-platform-broker-headless..svc.cluster.local
        at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:797)
        at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1509)
        at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1368)
        at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1302)
        at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
        at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:403)
        at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:363)
        at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:151)
        at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:958)
        at org.apache.kafka.clients.NetworkClient.access$600(NetworkClient.java:74)
        at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1131)
        at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1019)
        at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:542)
        at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:325)
        at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:240)
        at java.base/java.lang.Thread.run(Thread.java:829)
[2021-08-25 03:05:50,589] WARN [Producer clientId=console-producer] Error connecting to node sn-platform-broker-1.sn-platform-broker-headless..svc.cluster.local:9092 (id: 688476253 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: sn-platform-broker-1.sn-platform-broker-headless..svc.cluster.local: Name or service not known
        at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
        at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929)
        at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1519)
        at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848)
        at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1509)
        at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1368)
        at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1302)
        at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
        at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:403)

Version info

sn-platform version: streamnative/sn-platform:2.8.0.5 kafka client version: kafka_2.13-2.6.0

closed time in 7 days

ericsyh

issue openedapache/pulsar-manager

[bug](topics) Topics unsubscribe failed on pulsar-manager

Background

After the test, we found that topics unsubscribe function didn't work on pulsar-manager while pulsar-admin executes the same subscription work as expected. This should be a bug on Pulsar Manger.

截屏2021-10-14 下午2 43 06

created time in 7 days

delete branch ericsyh/pulsar

delete branch : add-restart-cmd-for-pulsar-daemon

delete time in 9 days

startedray-project/ray

started time in 9 days

startedbytebase/bytebase

started time in 11 days

PublicEvent

issue commentstreamnative/charts

[pulsar-operator] CRDs invalid and deprecated

@MisterTimn I migrated the pulsar-operator crds from v1beta1 to v1 and can be deployed on k8s v1.22. But i think this change won't be merged and release so quick, so i upload the v1 crds to my gist where you can get them.

MisterTimn

comment created time in 14 days

issue commentstreamnative/charts

[pulsar-operator] CRDs invalid and deprecated

@MisterTimn I just submitted a PR to migrate che CRDs from v1beta1 to v1 for k8s v1.22 but it is still under review. So i put these CRDs on my gist, you can take them to go on the installation os pulsar-operator.

MisterTimn

comment created time in 14 days

startedapache/skywalking

started time in 15 days

issue commentstreamnative/charts

[pulsar-operator] CRDs invalid and deprecated

kubernetes cluster v1.21.5 should be okay, apiextensions.k8s.io/v1beta1 is removed in kubernetes v1.22

MisterTimn

comment created time in 15 days

PR opened streamnative/charts

feat(crd): migrate crd from v1beta1 to v1

Signed-off-by: Eric Shen ericshenyuhao@outlook.com

Motivation

CRD v1beta1 is removed in k8s v1.22 API removals for Kubernetes v1.22, so we need to migrate the crd to v1.

Changes

Migrate the CRDs in function-mesh-operator and pulsar-operator from v1beta1 to v1.

$ kc version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.1", GitCommit:"632ed300f2c34f6d6d15ca4cef3d3c7073412212", GitTreeState:"clean", BuildDate:"2021-08-19T15:45:37Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.1", GitCommit:"632ed300f2c34f6d6d15ca4cef3d3c7073412212", GitTreeState:"clean", BuildDate:"2021-08-25T19:54:13Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"linux/amd64"}

$ kc get crds
NAME                                            CREATED AT
bookkeeperclusters.bookkeeper.streamnative.io   2021-10-06T23:09:38Z
functionmeshes.compute.functionmesh.io          2021-10-06T23:10:16Z
functions.compute.functionmesh.io               2021-10-06T23:10:28Z
pulsarbrokers.pulsar.streamnative.io            2021-10-06T23:09:46Z
pulsarproxies.pulsar.streamnative.io            2021-10-06T23:09:52Z
sinks.compute.functionmesh.io                   2021-10-06T23:10:35Z
sources.compute.functionmesh.io                 2021-10-06T23:10:40Z
zookeeperclusters.zookeeper.streamnative.io     2021-10-06T23:09:58Z
+27632 -27648

0 comment

8 changed files

pr created time in 15 days

create barnchericsyh/charts

branch : crd-v1beta1-to-v1

created branch time in 15 days

push eventericsyh/charts

Sijie Guo

commit sha a9ee87f7f12638885f338d076f15fb4d126cbb48

Fix authentication and authorization in the platform chart (#321)

view details

Guangning E

commit sha 7cc2260ee7a41c46d7b371252821b2e99f4d784d

Fixed version (#322)

view details

Marvin Cai

commit sha fc947acbd06cb0a88d7594348fee7bfe0b356b03

Remove protocol as listMapKey for container port as it'll block kubectl apply of operator crd. (#320)

view details

push time in 15 days

fork ericsyh/charts

StreamNative Helm Charts Repository: Apache Pulsar, Pulsar Operators, StreamNative Platform, Function Mesh

https://charts.streamnative.io

fork in 15 days

startedwurstmeister/kafka-docker

started time in 16 days

PR opened apache/pulsar

feat(cli): add restart command to pulsar-daemon

Signed-off-by: Eric Shen ericshenyuhao@outlook.com

<!--

Contribution Checklist

  • Name the pull request in the form "[Issue XYZ][component] Title of the pull request", where XYZ should be replaced by the actual issue number. Skip Issue XYZ if there is no associated github issue for this pull request. Skip component if you are unsure about which is the best component. E.g. [docs] Fix typo in produce method.

  • Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.

  • Each pull request should address only one issue, not mix up code from multiple issues.

  • Each commit in the pull request has a meaningful commit message

  • Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.

(The sections below can be removed for hotfixes of typos) -->

Motivation

It's inconvenient to restart the broker/bk/zk process when users change the config, they have use pulsar-daemon stop the process first then start it again.

Modifications

I added a command as restart in pulsar-daemon.

Verifying this change

  • [x] Make sure that the change passes the CI checks.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

Check the box below and label this PR (if you have committer privilege).

Need to update docs?

  • [ ] doc-required

    (If you need help on updating docs, create a doc issue)

  • [x] no-need-doc

    (Please explain why)

  • [ ] doc

    (If this PR contains doc changes)

+61 -1

0 comment

1 changed file

pr created time in 16 days