profile
viewpoint

Graylog2/graylog-project 17

Packaging/development helper for Graylog

kroepke/brew-packages 6

"Lost" brew packages

kroepke/luna 4

Lua 5.3 implementation for the JVM

kroepke/elasticsearch-head 1

A web front end for an elastic search cluster

kroepke/homebrew 1

The missing package manager for OS X.

kroepke/acme.sh 0

A pure Unix shell script implementing ACME client protocol

kroepke/auto 0

A collection of source code generators for Java.

kroepke/elasticsearch-kopf 0

web admin interface for elasticsearch

issue openedGraylog2/graylog2-server

Unhandled exception while executing event processor

Periodically, we get a few ERROR entries in the Graylog server.log file that says "Unhandled exception while executing event processor" (see log sample below). I have searched in the documentation but couldn't find an indication on how to interpret this error or how to troobleshoot it further.

Can anyone help explaining what these are or how to address them ?

2020-11-27T16:14:56.872+01:00 ERROR [EventProcessorEngine] Caught an unhandled exception while executing event processor <aggregation-v1/REDACTED/5f36423060848518575ae77f> - Make sure to modify the event processor to throw only EventProcessorExecutionException so we get more context! org.graylog2.indexer.ElasticsearchException: Unable to scroll indices. at org.graylog.storage.elasticsearch6.jest.JestUtils.execute(JestUtils.java:54) ~[?:?] at org.graylog.storage.elasticsearch6.jest.JestUtils.execute(JestUtils.java:65) ~[?:?] at org.graylog.storage.elasticsearch6.Scroll.scroll(Scroll.java:45) ~[?:?] at org.graylog.storage.elasticsearch6.Scroll.scroll(Scroll.java:41) ~[?:?] at org.graylog.storage.elasticsearch6.MoreSearchAdapterES6.scrollEvents(MoreSearchAdapterES6.java:163) ~[?:?] at org.graylog.events.search.MoreSearch.scrollQuery(MoreSearch.java:147) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.AggregationEventProcessor.filterSearch(AggregationEventProcessor.java:210) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.AggregationEventProcessor.createEvents(AggregationEventProcessor.java:121) ~[graylog.jar:?] at org.graylog.events.processor.EventProcessorEngine.execute(EventProcessorEngine.java:92) ~[graylog.jar:?] at org.graylog.events.processor.EventProcessorExecutionJob.execute(EventProcessorExecutionJob.java:115) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.executeJob(JobExecutionEngine.java:166) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.lambda$handleTrigger$2(JobExecutionEngine.java:144) ~[graylog.jar:?] at com.codahale.metrics.Timer.time(Timer.java:151) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.handleTrigger(JobExecutionEngine.java:144) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.lambda$execute$0(JobExecutionEngine.java:119) ~[graylog.jar:?] at org.graylog.scheduler.worker.JobWorkerPool.lambda$execute$0(JobWorkerPool.java:110) ~[graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275] at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275] Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_275] at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_275] at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_275] at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_275] at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) ~[graylog.jar:?] at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) ~[graylog.jar:?] at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282) ~[graylog.jar:?] at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) ~[graylog.jar:?] at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[graylog.jar:?] at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[graylog.jar:?] at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[graylog.jar:?] at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) ~[graylog.jar:?] at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[graylog.jar:?] at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[graylog.jar:?] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[graylog.jar:?] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[graylog.jar:?] at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) ~[graylog.jar:?] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[graylog.jar:?] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[graylog.jar:?] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[graylog.jar:?] at io.searchbox.client.http.JestHttpClient.executeRequest(JestHttpClient.java:151) ~[?:?] at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:77) ~[?:?] at org.graylog.storage.elasticsearch6.jest.JestUtils.execute(JestUtils.java:49) ~[?:?] ... 20 more 2020-11-27T16:14:56.874+01:00 ERROR [EventProcessorExecutionJob] Event processor <aggregation-v1/5f36423060848518575ae77f> failed to execute: Couldn't create events for: EventDefinitionDto{id=5f36423060848518575ae77f, title=REDACTED, description=, priority=3, alert=true, config=AggregationEventProcessorConfig{type=aggregation-v1, query="REDACTED", queryParameters=[], streams=[], groupBy=[], series=[], conditions=Optional[AggregationConditions{expression=Optional.empty}], searchWithinMs=120000, executeEveryMs=120000}, fieldSpec={}, keySpec=[], notificationSettings=EventNotificationSettings{gracePeriodMs=120000, backlogSize=50}, notifications=[Config{notificationId=5ee0e355eaec315f0aaa582e, notificationParameters=Optional.empty}], storage=[Config{type=persist-to-streams-v1, streams=[000000000000000000000002]}]} (retry in 5000 ms) org.graylog.events.processor.EventProcessorException: Couldn't create events for: EventDefinitionDto{id=5f36423060848518575ae77f, title=REDACTED, description=, priority=3, alert=true, config=AggregationEventProcessorConfig{type=aggregation-v1, query="High Availability status changed", queryParameters=[], streams=[], groupBy=[], series=[], conditions=Optional[AggregationConditions{expression=Optional.empty}], searchWithinMs=120000, executeEveryMs=120000}, fieldSpec={}, keySpec=[], notificationSettings=EventNotificationSettings{gracePeriodMs=120000, backlogSize=50}, notifications=[Config{notificationId=5ee0e355eaec315f0aaa582e, notificationParameters=Optional.empty}], storage=[Config{type=persist-to-streams-v1, streams=[000000000000000000000002]}]} at org.graylog.events.processor.EventProcessorEngine.execute(EventProcessorEngine.java:106) ~[graylog.jar:?] at org.graylog.events.processor.EventProcessorExecutionJob.execute(EventProcessorExecutionJob.java:115) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.executeJob(JobExecutionEngine.java:166) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.lambda$handleTrigger$2(JobExecutionEngine.java:144) ~[graylog.jar:?] at com.codahale.metrics.Timer.time(Timer.java:151) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.handleTrigger(JobExecutionEngine.java:144) ~[graylog.jar:?] at org.graylog.scheduler.JobExecutionEngine.lambda$execute$0(JobExecutionEngine.java:119) ~[graylog.jar:?] at org.graylog.scheduler.worker.JobWorkerPool.lambda$execute$0(JobWorkerPool.java:110) ~[graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275] at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275] Caused by: org.graylog2.indexer.ElasticsearchException: Unable to scroll indices. at org.graylog.storage.elasticsearch6.jest.JestUtils.execute(JestUtils.java:54) ~[?:?] at org.graylog.storage.elasticsearch6.jest.JestUtils.execute(JestUtils.java:65) ~[?:?] at org.graylog.storage.elasticsearch6.Scroll.scroll(Scroll.java:45) ~[?:?] at org.graylog.storage.elasticsearch6.Scroll.scroll(Scroll.java:41) ~[?:?] at org.graylog.storage.elasticsearch6.MoreSearchAdapterES6.scrollEvents(MoreSearchAdapterES6.java:163) ~[?:?] at org.graylog.events.search.MoreSearch.scrollQuery(MoreSearch.java:147) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.AggregationEventProcessor.filterSearch(AggregationEventProcessor.java:210) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.AggregationEventProcessor.createEvents(AggregationEventProcessor.java:121) ~[graylog.jar:?] at org.graylog.events.processor.EventProcessorEngine.execute(EventProcessorEngine.java:92) ~[graylog.jar:?] ... 12 more Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_275] at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_275] at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_275] at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_275] at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) ~[graylog.jar:?] at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) ~[graylog.jar:?] at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282) ~[graylog.jar:?] at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) ~[graylog.jar:?] at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[graylog.jar:?] at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[graylog.jar:?] at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[graylog.jar:?] at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) ~[graylog.jar:?] at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[graylog.jar:?] at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[graylog.jar:?] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[graylog.jar:?] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[graylog.jar:?] at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) ~[graylog.jar:?] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[graylog.jar:?] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[graylog.jar:?] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[graylog.jar:?] at io.searchbox.client.http.JestHttpClient.executeRequest(JestHttpClient.java:151) ~[?:?] at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:77) ~[?:?] at org.graylog.storage.elasticsearch6.jest.JestUtils.execute(JestUtils.java:49) ~[?:?] at org.graylog.storage.elasticsearch6.jest.JestUtils.execute(JestUtils.java:65) ~[?:?] at org.graylog.storage.elasticsearch6.Scroll.scroll(Scroll.java:45) ~[?:?] at org.graylog.storage.elasticsearch6.Scroll.scroll(Scroll.java:41) ~[?:?] at org.graylog.storage.elasticsearch6.MoreSearchAdapterES6.scrollEvents(MoreSearchAdapterES6.java:163) ~[?:?] at org.graylog.events.search.MoreSearch.scrollQuery(MoreSearch.java:147) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.AggregationEventProcessor.filterSearch(AggregationEventProcessor.java:210) ~[graylog.jar:?] at org.graylog.events.processor.aggregation.AggregationEventProcessor.createEvents(AggregationEventProcessor.java:121) ~[graylog.jar:?] at org.graylog.events.processor.EventProcessorEngine.execute(EventProcessorEngine.java:92) ~[graylog.jar:?] ... 12 more

Your Environment

  • Graylog Version: 4.0.1
  • Elasticsearch Version: 6
  • Operating System: Ubuntu

created time in 9 minutes

issue closedGraylog2/graylog2-server

Cluster not working after es upgrade on graylog 4.0

Hi

Everthing was running fine after graylog version upgrade from 3.3 to 4.0 with es on 6.8 but later upgraded es from 6.8 to 7.9.3 . We are started seeing the following issues:

  • search became 20%-30% slow ( not important)
  • stream permissions issues , even for admins, this happened randomly ( I deleted session collection from mongo to fix this up but couldn't test as graylog is not working)
  • We started seeing timeout errors on graylog apis like /api/system/jobs /api/system/metrics/multiple
  • And major issue that I feel we are getting repeatedly is: An I/O error has occurred while writing a response message entity to the container output stream. - {} Caused by: java.io.IOException: Broken pipe , details are down below

2020-11-27 13:15:13,611 ERROR [ServerRuntime$Responder] - An I/O error has occurred while writing a response message entity to the container output stream. - {} org.glassfish.jersey.server.internal.process.MappableException: java.io.IOException: Connection is closed at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:67) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116) ~[graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:638) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:371) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:361) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) [graylog.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [graylog.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275] Caused by: java.io.IOException: Connection is closed at org.glassfish.grizzly.nio.NIOConnection.assertOpen(NIOConnection.java:441) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:663) ~[graylog.jar:?] at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:462) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:722) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:643) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.JsonValueSerializer.serialize(JsonValueSerializer.java:181) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[graylog.jar:?] at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?] ... 20 more Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:1.8.0_275] at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:1.8.0_275] at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) ~[?:1.8.0_275] at sun.nio.ch.IOUtil.write(IOUtil.java:51) ~[?:1.8.0_275] at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470) ~[?:1.8.0_275] at org.glassfish.grizzly.nio.transport.TCPNIOUtils.flushByteBuffer(TCPNIOUtils.java:125) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOUtils.writeCompositeBuffer(TCPNIOUtils.java:64) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:105) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:82) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:236) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:145) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:47) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleWrite(TCPNIOTransportFilter.java:102) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.TransportFilter.handleWrite(TransportFilter.java:167) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:87) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88) ~[graylog.jar:?] at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:866) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:834) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.flushBuffer(OutputBuffer.java:1068) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:695) ~[graylog.jar:?] at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:457) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[graylog.jar:?] at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?] ... 20 more 2020-11-27 13:15:29,484 INFO [EventProcessorExecutionJob] - Event processor <5fb4f93e3a7f354004e965c8> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T21:56:46.325Z to=2020-11-26T22:56:46.324Z - {} 2020-11-27 13:15:34,481 INFO [EventProcessorExecutionJob] - Event processor <5fb4f93a3a7f354004e96429> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T21:56:42.731Z to=2020-11-26T22:56:42.730Z - {} 2020-11-27 13:15:36,481 INFO [EventProcessorExecutionJob] - Event processor <5fb4f93543102921ea49d2a4> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T21:46:37.170Z to=2020-11-26T22:46:37.169Z - {} 2020-11-27 13:15:56,484 INFO [EventProcessorExecutionJob] - Event processor <5fb4f93943102921ea49d43b> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T22:36:41.015Z to=2020-11-26T23:36:41.014Z - {} 2020-11-27 13:15:59,554 ERROR [ServerRuntime$Responder] - An I/O error has occurred while writing a response message entity to the container output stream. - {} org.glassfish.jersey.server.internal.process.MappableException: java.io.IOException: Connection is closed at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:67) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116) ~[graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:638) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:371) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:361) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) [graylog.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [graylog.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275] Caused by: java.io.IOException: Connection is closed at org.glassfish.grizzly.nio.NIOConnection.assertOpen(NIOConnection.java:441) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:663) ~[graylog.jar:?] at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:462) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:722) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:643) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.JsonValueSerializer.serialize(JsonValueSerializer.java:181) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[graylog.jar:?] at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?] ... 20 more Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:1.8.0_275] at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:1.8.0_275] at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) ~[?:1.8.0_275] at sun.nio.ch.IOUtil.write(IOUtil.java:51) ~[?:1.8.0_275] at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470) ~[?:1.8.0_275] at org.glassfish.grizzly.nio.transport.TCPNIOUtils.flushByteBuffer(TCPNIOUtils.java:125) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOUtils.writeCompositeBuffer(TCPNIOUtils.java:64) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:105) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:82) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:236) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:145) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:47) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleWrite(TCPNIOTransportFilter.java:102) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.TransportFilter.handleWrite(TransportFilter.java:167) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:87) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88) ~[graylog.jar:?] at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:866) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:834) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.flushBuffer(OutputBuffer.java:1068) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:695) ~[graylog.jar:?] at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:457) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[graylog.jar:?] at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?] ... 20 more 2020-11-27 13:15:59,559 ERROR [ServerRuntime$Responder] - An I/O error has occurred while writing a response message entity to the container output stream. - {} org.glassfish.jersey.server.internal.process.MappableException: java.io.IOException: Connection is closed at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:67) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116) ~[graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:638) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:371) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:361) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) [graylog.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [graylog.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275] Caused by: java.io.IOException: Connection is closed at org.glassfish.grizzly.nio.NIOConnection.assertOpen(NIOConnection.java:441) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:663) ~[graylog.jar:?] at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:462) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:722) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:643) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.JsonValueSerializer.serialize(JsonValueSerializer.java:181) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[graylog.jar:?] at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?] ... 20 more Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:1.8.0_275] at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:1.8.0_275] at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) ~[?:1.8.0_275] at sun.nio.ch.IOUtil.write(IOUtil.java:51) ~[?:1.8.0_275] at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470) ~[?:1.8.0_275] at org.glassfish.grizzly.nio.transport.TCPNIOUtils.flushByteBuffer(TCPNIOUtils.java:125) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOUtils.writeCompositeBuffer(TCPNIOUtils.java:64) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:105) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:82) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:236) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:145) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:47) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleWrite(TCPNIOTransportFilter.java:102) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.TransportFilter.handleWrite(TransportFilter.java:167) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:87) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88) ~[graylog.jar:?] at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:866) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:834) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.flushBuffer(OutputBuffer.java:1068) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:695) ~[graylog.jar:?] at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:457) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[graylog.jar:?] at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?] ... 20 more 2020-11-27 13:16:00,473 INFO [EventProcessorExecutionJob] - Event processor <5fb4f931a8e0bf4bc0dfd9f3> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T22:46:33.096Z to=2020-11-26T23:46:33.095Z - {} 2020-11-27 13:16:02,493 INFO [EventProcessorExecutionJob] - Event processor <5fb4f93743102921ea49d433> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T21:36:39.154Z to=2020-11-26T22:36:39.153Z - {} 2020-11-27 13:16:03,481 INFO [EventProcessorExecutionJob] - Event processor <5fb4f93c3a7f354004e96431> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T21:56:44.502Z to=2020-11-26T22:56:44.501Z - {} 2020-11-27 13:16:04,650 WARN [RestClient] - request [POST http://graylog-es-http.prod-graylog1.svc.cluster.local:9200/_msearch?typed_keys=true] returned 1 warnings: [299 Elasticsearch-7.9.3-c4138e51121ef06a6404866cddc601906fe5c868 "'y' year should be replaced with 'u'. Use 'y' for year-of-era. Use new java.time date format specifiers."] - {} 2020-11-27 13:16:06,492 INFO [EventProcessorExecutionJob] - Event processor <5fb380379e225b1283ce8e10> is catching up on old data. Combining 12 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T21:03:07.424Z to=2020-11-26T22:06:07.423Z - {} 2020-11-27 13:16:16,634 WARN [ProxiedResource] - Unable to call http://graylog-cluster-0.graylog-cluster.prod-graylog1.svc.cluster.local:9000/api/system/jobs on node <c2e845cb-3fe0-4f7b-8b24-fddf646a6e51>: timeout - {} 2020-11-27 13:16:16,664 WARN [ProxiedResource] - Unable to call http://graylog-cluster-0.graylog-cluster.prod-graylog1.svc.cluster.local:9000/api/system/jobs on node <c2e845cb-3fe0-4f7b-8b24-fddf646a6e51>: timeout - {} 2020-11-27 13:16:17,730 INFO [IndexRangesCleanupPeriodical] - Removing index range information for unavailable indices: [prod_sg1_service_epay-integration-critical_0, prod_sg1_java_lang_no_classdef_found_error_idx_0] - {} continuation to above, graylog keep on reporting api timeouts as stated previously

image

We run docker host in k8s env, and we see that graylog is getting stuck at the below position for long after a restart - image

And if runs after that long wait but after a few minutes, it will go down again without throwing any logs. Though on container side it throws 137 exit code but I verified that this is not an OOM issue. Also as it is taking long time, my liveliness thresholds are pretty high to not kill the container. But still to no rescue.

Right now, graylog will run for sometime and then we dead and restart. I can also see spike in the mongodb ops/s: image es load is pretty low: image pod utilization is also very low: image config is 16cpu 32gb , 3 replicas for graylog pods config is 2cpu 4gb , 2 replicas for mongo pods config is 4cpu 32gb for es data pods

  • Graylog Version:4.0.1
  • Java Version:1.8
  • Elasticsearch Version:7.9.3
  • MongoDB Version:
  • Operating System:
  • Browser version:

closed time in 14 minutes

sowhatim

issue commentGraylog2/graylog2-server

Cluster not working after es upgrade on graylog 4.0

Update - it was mongo which was getting bottlenecked on cpu. Interesting we had upscaled it from 500m cpu to 1 CPU earlier to see if that was the issue but didn't pay much attention to it because mongo has been never the problem in resource management for graylog cluster. But here it seems that due to lag or recovery that graylog might be trying to do, it was putting mongo into too much load and required high cpu.

Now after graylog is recovered(it's still ingesting pending messages), load on mongo is again very low around 200m of usage. So we will downscale it again.

One more thing that I found that docker container exit code 137 is not just for oom as even If I send kill -9 signal, it is exiting with 137. So graylog container was exiting out most probably due to mongo api timeout but sadly it was not throwing any errors before exiting out.

Few concerns that remains -

  • why graylog is not throwing any logs when it is getting stuck waiting from mongo ? (I did enable debug logs as well)
  • before graylog exits/dies out , why no message is thrown
  • why searching became slower after es 7.9.3 upgrade on graylog 4 but these are not bugcritical so

I am closing this out now with a learning that mongo also needs to be looked upon in graylog cluster : )

sowhatim

comment created time in 14 minutes

issue openedGraylog2/graylog2-server

Streams page stopped working after the update to Graylog 4.0.1

After updating to Graylog 4.0.1 the streams page does not work anymore. Search and message processing works without problems.

Expected Behavior

The page should load without errors.

Current Behavior

Error displayed in browser: Could not load streams Loading streams failed with status: Error: cannot GET https://graylog.XXX.de/api/streams/paginated?page=1&per_page=10 (500)

Context

Graylog server log (Loglevel set to TRACE):

Graylog server log:
2020-11-27T13:02:39.629+01:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
java.lang.RuntimeException: IOException encountered while reading from a byte array input stream
        at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:67) ~[graylog.jar:?]
        at com.mongodb.DBDecoderAdapter.decode(DBDecoderAdapter.java:49) ~[graylog.jar:?]
        at com.mongodb.DBDecoderAdapter.decode(DBDecoderAdapter.java:29) ~[graylog.jar:?]
        at com.mongodb.operation.CommandResultArrayCodec.decode(CommandResultArrayCodec.java:52) ~[graylog.jar:?]
        at com.mongodb.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:60) ~[graylog.jar:?]
        at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:84) ~[graylog.jar:?]
        at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:41) ~[graylog.jar:?]
        at org.bson.internal.LazyCodec.decode(LazyCodec.java:48) ~[graylog.jar:?]
        at org.bson.codecs.BsonDocumentCodec.readValue(BsonDocumentCodec.java:101) ~[graylog.jar:?]
        at com.mongodb.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:63) ~[graylog.jar:?]
        at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:84) ~[graylog.jar:?]
        at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:41) ~[graylog.jar:?]
        at com.mongodb.internal.connection.ReplyMessage.<init>(ReplyMessage.java:51) ~[graylog.jar:?]
        at com.mongodb.internal.connection.InternalStreamConnection.getCommandResult(InternalStreamConnection.java:413) ~[graylog.jar:?]
        at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:309) ~[graylog.jar:?]
        at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:259) ~[graylog.jar:?]
        at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99) ~[graylog.jar:?]
        at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:450) ~[graylog.jar:?]
        at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:72) ~[graylog.jar:?]
        at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:226) ~[graylog.jar:?]
        at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:269) ~[graylog.jar:?]
        at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:131) ~[graylog.jar:?]
        at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:123) ~[graylog.jar:?]
        at com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:343) ~[graylog.jar:?]
        at com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:334) ~[graylog.jar:?]
        at com.mongodb.operation.CommandOperationHelper.executeCommandWithConnection(CommandOperationHelper.java:220) ~[graylog.jar:?]
        at com.mongodb.operation.FindOperation$1.call(FindOperation.java:731) ~[graylog.jar:?]
        at com.mongodb.operation.FindOperation$1.call(FindOperation.java:725) ~[graylog.jar:?]
        at com.mongodb.operation.OperationHelper.withReadConnectionSource(OperationHelper.java:463) ~[graylog.jar:?]
        at com.mongodb.operation.FindOperation.execute(FindOperation.java:725) ~[graylog.jar:?]
        at com.mongodb.operation.FindOperation.execute(FindOperation.java:89) ~[graylog.jar:?]
        at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:196) ~[graylog.jar:?]
        at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:177) ~[graylog.jar:?]
        at com.mongodb.DBCursor.initializeCursor(DBCursor.java:989) ~[graylog.jar:?]
        at com.mongodb.DBCursor.hasNext(DBCursor.java:172) ~[graylog.jar:?]
        at org.mongojack.DBCursor.hasNext(DBCursor.java:330) ~[graylog.jar:?]
        at java.util.Iterator.forEachRemaining(Iterator.java:115) ~[?:1.8.0_272]
        at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) ~[?:1.8.0_272]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_272]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_272]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_272]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_272]
        at java.util.stream.LongPipeline.reduce(LongPipeline.java:461) ~[?:1.8.0_272]
        at java.util.stream.LongPipeline.sum(LongPipeline.java:419) ~[?:1.8.0_272]
        at java.util.stream.ReferencePipeline.count(ReferencePipeline.java:593) ~[?:1.8.0_272]
        at org.graylog2.database.PaginatedDbService.findPaginatedWithQueryFilterAndSort(PaginatedDbService.java:159) ~[graylog.jar:?]
        at org.graylog2.streams.PaginatedStreamService.findPaginated(PaginatedStreamService.java:48) ~[graylog.jar:?]
        at org.graylog2.rest.resources.streams.StreamResource.getPage(StreamResource.java:210) ~[graylog.jar:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_272]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_272]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_272]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_272]
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) ~[graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) [graylog.jar:?]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [graylog.jar:?]
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?]
        at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_272]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_272]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_272]
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.graylog2.rest.models.alarmcallbacks.requests.AlertReceivers`, problem: Null users
 at [Source: de.undercouch.bson4jackson.io.LittleEndianInputStream@3f07f13e; pos: 157] (through reference chain: org.graylog2.streams.StreamDTO["alert_receivers"])
        at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1608) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException(StdValueInstantiator.java:484) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem(StdValueInstantiator.java:503) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:285) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromObjectWith(ValueInstantiator.java:229) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:195) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:488) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1287) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:530) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:528) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:417) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1287) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3985) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2343) ~[graylog.jar:?]
        at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:80) ~[graylog.jar:?]
        at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:64) ~[graylog.jar:?]
        ... 74 more
Caused by: java.lang.NullPointerException: Null users
        at org.graylog2.rest.models.alarmcallbacks.requests.$AutoValue_AlertReceivers.<init>($AutoValue_AlertReceivers.java:22) ~[graylog.jar:?]
        at org.graylog2.rest.models.alarmcallbacks.requests.AutoValue_AlertReceivers.<init>(AutoValue_AlertReceivers.java:9) ~[graylog.jar:?]
        at org.graylog2.rest.models.alarmcallbacks.requests.AlertReceivers.create(AlertReceivers.java:40) ~[graylog.jar:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_272]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_272]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_272]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_272]
        at com.fasterxml.jackson.databind.introspect.AnnotatedMethod.call(AnnotatedMethod.java:109) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:283) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromObjectWith(ValueInstantiator.java:229) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:195) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:488) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1287) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:530) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:528) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:417) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1287) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3985) ~[graylog.jar:?]
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2343) ~[graylog.jar:?]
        at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:80) ~[graylog.jar:?]
        at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:64) ~[graylog.jar:?]
        ... 74 more
2020-11-27T13:02:39.630+01:00 DEBUG [accesslog] 10.1.2.69 57f373d6dd20fa5ac503b194 [-] "GET api/streams/paginated?page=1&per_page=10" Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36 500 -1

Your Environment

  • Graylog Version: 4.0.1
  • Java Version: 1.8.0.272.b10-1.el7_9
  • Elasticsearch Version: 6.8.4
  • MongoDB Version: 4.0.21-1.el7
  • Operating System: Centos 7
  • Browser version: Chrome 86

created time in 3 hours

issue openedGraylog2/graylog2-server

Cluster not working after es upgrade on graylog 4.0

Hi

Everthing was running fine after graylog version upgrade from 3.3 to 4.0 with es on 6.8 but later upgraded es from 6.8 to 7.9.3 . We are started seeing the following issues:

  • search became 20%-30% slow ( not important)
  • stream permissions issues , even for admins, this happened randomly ( I deleted session collection from mongo to fix this up but couldn't test as graylog is not working)
  • We started seeing timeout errors on graylog apis like /api/system/jobs /api/system/metrics/multiple
  • And major issue that I feel we are getting repeatedly is: An I/O error has occurred while writing a response message entity to the container output stream. - {} Caused by: java.io.IOException: Broken pipe , details are down below

2020-11-27 13:15:13,611 ERROR [ServerRuntime$Responder] - An I/O error has occurred while writing a response message entity to the container output stream. - {} org.glassfish.jersey.server.internal.process.MappableException: java.io.IOException: Connection is closed at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:67) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116) ~[graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:638) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:371) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:361) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) [graylog.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [graylog.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275] Caused by: java.io.IOException: Connection is closed at org.glassfish.grizzly.nio.NIOConnection.assertOpen(NIOConnection.java:441) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:663) ~[graylog.jar:?] at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:462) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:722) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:643) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.JsonValueSerializer.serialize(JsonValueSerializer.java:181) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[graylog.jar:?] at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?] ... 20 more Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:1.8.0_275] at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:1.8.0_275] at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) ~[?:1.8.0_275] at sun.nio.ch.IOUtil.write(IOUtil.java:51) ~[?:1.8.0_275] at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470) ~[?:1.8.0_275] at org.glassfish.grizzly.nio.transport.TCPNIOUtils.flushByteBuffer(TCPNIOUtils.java:125) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOUtils.writeCompositeBuffer(TCPNIOUtils.java:64) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:105) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:82) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:236) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:145) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:47) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleWrite(TCPNIOTransportFilter.java:102) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.TransportFilter.handleWrite(TransportFilter.java:167) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:87) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88) ~[graylog.jar:?] at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:866) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:834) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.flushBuffer(OutputBuffer.java:1068) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:695) ~[graylog.jar:?] at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:457) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[graylog.jar:?] at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?] ... 20 more 2020-11-27 13:15:29,484 INFO [EventProcessorExecutionJob] - Event processor <5fb4f93e3a7f354004e965c8> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T21:56:46.325Z to=2020-11-26T22:56:46.324Z - {} 2020-11-27 13:15:34,481 INFO [EventProcessorExecutionJob] - Event processor <5fb4f93a3a7f354004e96429> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T21:56:42.731Z to=2020-11-26T22:56:42.730Z - {} 2020-11-27 13:15:36,481 INFO [EventProcessorExecutionJob] - Event processor <5fb4f93543102921ea49d2a4> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T21:46:37.170Z to=2020-11-26T22:46:37.169Z - {} 2020-11-27 13:15:56,484 INFO [EventProcessorExecutionJob] - Event processor <5fb4f93943102921ea49d43b> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T22:36:41.015Z to=2020-11-26T23:36:41.014Z - {} 2020-11-27 13:15:59,554 ERROR [ServerRuntime$Responder] - An I/O error has occurred while writing a response message entity to the container output stream. - {} org.glassfish.jersey.server.internal.process.MappableException: java.io.IOException: Connection is closed at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:67) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116) ~[graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:638) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:371) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:361) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) [graylog.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [graylog.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275] Caused by: java.io.IOException: Connection is closed at org.glassfish.grizzly.nio.NIOConnection.assertOpen(NIOConnection.java:441) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:663) ~[graylog.jar:?] at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:462) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:722) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:643) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.JsonValueSerializer.serialize(JsonValueSerializer.java:181) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[graylog.jar:?] at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?] ... 20 more Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:1.8.0_275] at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:1.8.0_275] at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) ~[?:1.8.0_275] at sun.nio.ch.IOUtil.write(IOUtil.java:51) ~[?:1.8.0_275] at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470) ~[?:1.8.0_275] at org.glassfish.grizzly.nio.transport.TCPNIOUtils.flushByteBuffer(TCPNIOUtils.java:125) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOUtils.writeCompositeBuffer(TCPNIOUtils.java:64) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:105) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:82) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:236) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:145) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:47) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleWrite(TCPNIOTransportFilter.java:102) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.TransportFilter.handleWrite(TransportFilter.java:167) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:87) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88) ~[graylog.jar:?] at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:866) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:834) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.flushBuffer(OutputBuffer.java:1068) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:695) ~[graylog.jar:?] at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:457) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[graylog.jar:?] at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?] ... 20 more 2020-11-27 13:15:59,559 ERROR [ServerRuntime$Responder] - An I/O error has occurred while writing a response message entity to the container output stream. - {} org.glassfish.jersey.server.internal.process.MappableException: java.io.IOException: Connection is closed at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:67) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116) ~[graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:638) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:371) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:361) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) [graylog.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [graylog.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275] Caused by: java.io.IOException: Connection is closed at org.glassfish.grizzly.nio.NIOConnection.assertOpen(NIOConnection.java:441) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:663) ~[graylog.jar:?] at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:462) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:722) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:643) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.JsonValueSerializer.serialize(JsonValueSerializer.java:181) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[graylog.jar:?] at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?] ... 20 more Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:1.8.0_275] at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) ~[?:1.8.0_275] at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) ~[?:1.8.0_275] at sun.nio.ch.IOUtil.write(IOUtil.java:51) ~[?:1.8.0_275] at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470) ~[?:1.8.0_275] at org.glassfish.grizzly.nio.transport.TCPNIOUtils.flushByteBuffer(TCPNIOUtils.java:125) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOUtils.writeCompositeBuffer(TCPNIOUtils.java:64) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:105) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:82) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:236) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:145) ~[graylog.jar:?] at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:47) ~[graylog.jar:?] at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleWrite(TCPNIOTransportFilter.java:102) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.TransportFilter.handleWrite(TransportFilter.java:167) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:87) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88) ~[graylog.jar:?] at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:866) ~[graylog.jar:?] at org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:834) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.flushBuffer(OutputBuffer.java:1068) ~[graylog.jar:?] at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:695) ~[graylog.jar:?] at org.glassfish.grizzly.http.server.NIOOutputStreamImpl.write(NIOOutputStreamImpl.java:59) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) ~[graylog.jar:?] at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:457) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) ~[graylog.jar:?] at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) ~[graylog.jar:?] at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:625) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85) ~[graylog.jar:?] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139) ~[graylog.jar:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61) ~[graylog.jar:?] ... 20 more 2020-11-27 13:16:00,473 INFO [EventProcessorExecutionJob] - Event processor <5fb4f931a8e0bf4bc0dfd9f3> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T22:46:33.096Z to=2020-11-26T23:46:33.095Z - {} 2020-11-27 13:16:02,493 INFO [EventProcessorExecutionJob] - Event processor <5fb4f93743102921ea49d433> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T21:36:39.154Z to=2020-11-26T22:36:39.153Z - {} 2020-11-27 13:16:03,481 INFO [EventProcessorExecutionJob] - Event processor <5fb4f93c3a7f354004e96431> is catching up on old data. Combining 6 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T21:56:44.502Z to=2020-11-26T22:56:44.501Z - {} 2020-11-27 13:16:04,650 WARN [RestClient] - request [POST http://graylog-es-http.prod-graylog1.svc.cluster.local:9200/_msearch?typed_keys=true] returned 1 warnings: [299 Elasticsearch-7.9.3-c4138e51121ef06a6404866cddc601906fe5c868 "'y' year should be replaced with 'u'. Use 'y' for year-of-era. Use new java.time date format specifiers."] - {} 2020-11-27 13:16:06,492 INFO [EventProcessorExecutionJob] - Event processor <5fb380379e225b1283ce8e10> is catching up on old data. Combining 12 search windows with catchUpWindowSize=3600000ms: from=2020-11-26T21:03:07.424Z to=2020-11-26T22:06:07.423Z - {} 2020-11-27 13:16:16,634 WARN [ProxiedResource] - Unable to call http://graylog-cluster-0.graylog-cluster.prod-graylog1.svc.cluster.local:9000/api/system/jobs on node <c2e845cb-3fe0-4f7b-8b24-fddf646a6e51>: timeout - {} 2020-11-27 13:16:16,664 WARN [ProxiedResource] - Unable to call http://graylog-cluster-0.graylog-cluster.prod-graylog1.svc.cluster.local:9000/api/system/jobs on node <c2e845cb-3fe0-4f7b-8b24-fddf646a6e51>: timeout - {} 2020-11-27 13:16:17,730 INFO [IndexRangesCleanupPeriodical] - Removing index range information for unavailable indices: [prod_sg1_service_epay-integration-critical_0, prod_sg1_java_lang_no_classdef_found_error_idx_0] - {} continuation to above, containers keep on throwing timeout

image

We run docker host in k8s env, and we see that graylog is getting stuck at the below position for long after a restart - image

And if runs after that long wait but after a few minutes, it will go down again without throwing any logs. Though on container side it throws 137 exit code but I verified that this is not an OOM issue. Also as it is taking long time, my liveliness thresholds are pretty high to not kill the container. But still to no rescue.

Right now, graylog will run for sometime and then we dead and restart. I can also see spike in the mongodb ops/s: image es load is pretty low: image pod utilization is also very low: image config is 16cpu 32gb , 3 replicas for graylog pods config is 2cpu 4gb , 2 replicas for mongo pods config is 4cpu 32gb for es pods

  • Graylog Version:4.0.1
  • Java Version:1.8
  • Elasticsearch Version:7.9.3
  • MongoDB Version:
  • Operating System:
  • Browser version:

created time in 3 hours

issue commentGraylog2/graylog2-server

processing pipeline - function `parse_string_as_json`

+1 !! It definitely shoould be possible to extract json in a pipeline rule without manually handling the structure.

jalogisch

comment created time in 8 hours

push eventGraylog2/graylog2-server

Bernd Ahlers

commit sha 049079f25cdbb6881f5385d82519f6d253eb0565

Use empty EventSearchParameters in EventsResource by default (#9635) Avoids an NPE when executing a search without payload.

view details

push time in 18 hours

delete branch Graylog2/graylog2-server

delete branch : fix/events-resource-npe

delete time in 18 hours

PR merged Graylog2/graylog2-server

Use empty EventSearchParameters in EventsResource by default ready-for-review

Avoids an NPE when executing a search without payload.

+7 -1

0 comment

2 changed files

bernd

pr closed time in 18 hours

issue commentGraylog2/graylog2-server

Keep AD group mapping in GL 4 OS

So why not thinking about making this LDAP / SAML as a Module which has a "good" price for the community version of graylog. You can still keep it bundled with the enterprise version, if there is a nice price around this featured as a module i think many people who are using this will be able and willing to afford this?

With this in mind, you might even be able to package more feature in a module an offer them the enterprise as well as the community.

What do you think about this?

I had exactly the same thought, there is only one pricing scheme, and we simply can't afford it. So we are looking now for another tool. Look Bacula systems for example, you can pay for volume, or for functionalities.

aim4r

comment created time in 20 hours

PR opened Graylog2/graylog2-server

Reviewers
Use empty EventSearchParameters in EventsResource by default ready-for-review

Avoids an NPE when executing a search without payload.

+7 -1

0 comment

2 changed files

pr created time in a day

create barnchGraylog2/graylog2-server

branch : fix/events-resource-npe

created branch time in a day

PR opened Graylog2/graylog2-server

Reviewers
Implement Highlighting Rule Form search ux web

Description

Make it possible to create a Highlighting rule via a creation form.

Motivation and Context

To create more complex Highlighting rules we need a form for creating these rules.

Screenshots (if appropriate):

Graylog (21)

Types of changes

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Refactoring (non-breaking change)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

Todo

Form validation and using the form for edit

+219 -28

0 comment

8 changed files

pr created time in a day

create barnchGraylog2/graylog2-server

branch : highlight-form

created branch time in a day

issue commentGraylog2/graylog2-server

Sources that haven't sent any logs in X amount of time

actually this is a job that can be done via the correlation engine in Graylog but it is not possible without.

swelcher

comment created time in a day

issue openedGraylog2/graylog2-server

processing pipeline - function `parse_string_as_json`

Expected Behavior

It should be possible to parse a string in the processing pipeline like the extractors do.

Current Behavior

It is always needed to know the json structure and use parse_json and json_path to create fields out of a KNOWN json structure.

Possible Solution

Make it possible to have the same functionality the current extractor for json parsing has. Parse a string and make all new fields with values. Plus have the option to flatten.

Context

Multiple issues show that JSON parsing is an issue. We should make it as easy as possible to consume json logs.

Your Environment

  • Graylog Version: 4 / 3

[Z#844347]

created time in a day

push eventGraylog2/documentation

dependabot-preview[bot]

commit sha 783b7f0fd22e1af6708aab9e90e8fc92b3ca12f7

Bump watchdog from 0.10.3 to 0.10.4 (#1007) Bumps [watchdog](https://github.com/gorakhargosh/watchdog) from 0.10.3 to 0.10.4. - [Release notes](https://github.com/gorakhargosh/watchdog/releases) - [Changelog](https://github.com/gorakhargosh/watchdog/blob/master/changelog.rst) - [Commits](https://github.com/gorakhargosh/watchdog/compare/v0.10.3...v0.10.4) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

view details

push time in a day

delete branch Graylog2/documentation

delete branch : dependabot/pip/watchdog-0.10.4

delete time in a day

PR merged Graylog2/documentation

Bump watchdog from 0.10.3 to 0.10.4 dependencies

Bumps watchdog from 0.10.3 to 0.10.4. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/gorakhargosh/watchdog/releases">watchdog's releases</a>.</em></p> <blockquote> <h2>0.10.4</h2> <ul> <li>Add <code>logger</code> parameter for the <code>LoggingEventHandler</code> (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/issues/676">#676</a>)</li> <li>Replace mutable default arguments with <code>if None</code> implementation in <code>RegexMatchingEventHandler</code> (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/issues/677">#677</a>)</li> <li>Expand tests to Python 2.7 and 3.5-3.10 for GNU/Linux, macOS and Windows</li> <li>[mac] Performance improvements for the <code>fsevents</code> module (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/issues/680">#680</a>)</li> <li>[mac] Prevent compilation of <code>watchdog_fsevents.c</code> on non-macOS machines (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/issues/687">#687</a>)</li> <li>[watchmedo] Handle shutdown events from <code>SIGTERM</code> and <code>SIGINT</code> more reliably (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/issues/693">#693</a>)</li> <li>Thanks to our beloved contributors: <a href="https://github.com/Sraw">@Sraw</a>, <a href="https://github.com/CCP-Aporia">@CCP-Aporia</a>, <a href="https://github.com/BoboTiG">@BoboTiG</a>, <a href="https://github.com/maybe-sybr">@maybe-sybr</a></li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/gorakhargosh/watchdog/blob/master/changelog.rst">watchdog's changelog</a>.</em></p> <blockquote> <h1>0.10.4</h1> <p>2020-11-21 • <a href="https://github.com/gorakhargosh/watchdog/compare/v0.10.3...v0.10.4">full history</a></p> <ul> <li>Add <code>logger</code> parameter for the <code>LoggingEventHandler</code> (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/pull/676">#676</a>)</li> <li>Replace mutable default arguments with <code>if None</code> implementation (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/pull/677">#677</a>)</li> <li>Expand tests to Python 2.7 and 3.5-3.10 for GNU/Linux, macOS and Windows</li> <li>[mac] Performance improvements for the <code>fsevents</code> module (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/pull/680">#680</a>)</li> <li>[mac] Prevent compilation of <code>watchdog_fsevents.c</code> on non-macOS machines (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/pull/687">#687</a>)</li> <li>[watchmedo] Handle shutdown events from <code>SIGTERM</code> and <code>SIGINT</code> more reliably (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/pull/693">#693</a>)</li> <li>Thanks to our beloved contributors: <a href="https://github.com/Sraw">@Sraw</a>, <a href="https://github.com/CCP-Aporia">@CCP-Aporia</a>, <a href="https://github.com/BoboTiG">@BoboTiG</a>, <a href="https://github.com/maybe-sybr">@maybe-sybr</a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/gorakhargosh/watchdog/commit/186a39c4367debb3a84cff86653be4f6de493a83"><code>186a39c</code></a> Release 0.10.4</li> <li><a href="https://github.com/gorakhargosh/watchdog/commit/14a94060ca6f0a4266898d206e7f5275cfed4d3e"><code>14a9406</code></a> [mac] Prevent compilation of watchdog_fsevents.c on non-macOS machines</li> <li><a href="https://github.com/gorakhargosh/watchdog/commit/d35ef6be8deedd65206db991175a1ca8fdf4606d"><code>d35ef6b</code></a> watchdog is working on Python 3.9</li> <li><a href="https://github.com/gorakhargosh/watchdog/commit/7ddd5d4a7e407e94724bee756e6f6b8c4d6a55fa"><code>7ddd5d4</code></a> doc: Use <code>finally</code> for teardown in doco examples</li> <li><a href="https://github.com/gorakhargosh/watchdog/commit/b803b3e4ea1c76c51c5f79a0a06a1121b3dda376"><code>b803b3e</code></a> fix: Handle repeated signals better in watchmedo</li> <li><a href="https://github.com/gorakhargosh/watchdog/commit/60c57aee2d39cfa8c5280cc92e3b3d278633b793"><code>60c57ae</code></a> Expand tests to Python 2.7 and 3.5-3.10 for GNU/Linux, macOS and Windows</li> <li><a href="https://github.com/gorakhargosh/watchdog/commit/3b9904c74ad0eae628471d22d584b4582344e761"><code>3b9904c</code></a> [Mac] Performance improvements for the fsevents module (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/issues/680">#680</a>)</li> <li><a href="https://github.com/gorakhargosh/watchdog/commit/d95692b5ff8fe0f459b58f758f5cb6a7dce112df"><code>d95692b</code></a> Add watchdog.tricks to the API documentation</li> <li><a href="https://github.com/gorakhargosh/watchdog/commit/1c6f52a8e1fda0a5e9cb20a42628d12675c88e55"><code>1c6f52a</code></a> [Mac] Remove unused initwatchdog_fsevents() prototype (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/issues/681">#681</a>)</li> <li><a href="https://github.com/gorakhargosh/watchdog/commit/c36a7df255ad80bc5812f864b1d571cfd07ff37c"><code>c36a7df</code></a> Rename docs/dependencies.txt to docs/requirements.txt (<a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/issues/678">#678</a>, fixes <a href="https://github-redirect.dependabot.com/gorakhargosh/watchdog/issues/650">#650</a>, fixe...</li> <li>Additional commits viewable in <a href="https://github.com/gorakhargosh/watchdog/compare/v0.10.3...v0.10.4">compare view</a></li> </ul> </details> <br />

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


<details> <summary>Dependabot commands and options</summary> <br />

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
  • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
  • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
  • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language
  • @dependabot badge me will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot dashboard:

  • Update frequency (including time of day and day of week)
  • Pull request limits (per update run and/or open at any time)
  • Automerge options (never/patch/minor, and dev/runtime dependencies)
  • Out-of-range updates (receive only lockfile updates, if desired)
  • Security updates (receive only security updates, if desired)

</details>

+1 -1

1 comment

1 changed file

dependabot-preview[bot]

pr closed time in a day

delete branch Graylog2/documentation

delete branch : dependabot/pip/certifi-2020.11.8

delete time in a day

delete branch Graylog2/documentation

delete branch : dependabot/pip/sphinx-3.3.1

delete time in a day

PR closed Graylog2/documentation

Bump certifi from 2020.6.20 to 2020.11.8 dependencies

Bumps certifi from 2020.6.20 to 2020.11.8. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/certifi/python-certifi/commit/015cba9d2492a4cddaf5efe40666c18a2b259c93"><code>015cba9</code></a> 2020.11.08</li> <li><a href="https://github.com/certifi/python-certifi/commit/9da350977a877ac18a44fcf267fb77e937e40f5c"><code>9da3509</code></a> Declare support for Python 3.9 (<a href="https://github-redirect.dependabot.com/certifi/python-certifi/issues/138">#138</a>)</li> <li>See full diff in <a href="https://github.com/certifi/python-certifi/compare/2020.06.20...2020.11.08">compare view</a></li> </ul> </details> <br />

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


<details> <summary>Dependabot commands and options</summary> <br />

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
  • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
  • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
  • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language
  • @dependabot badge me will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot dashboard:

  • Update frequency (including time of day and day of week)
  • Pull request limits (per update run and/or open at any time)
  • Automerge options (never/patch/minor, and dev/runtime dependencies)
  • Out-of-range updates (receive only lockfile updates, if desired)
  • Security updates (receive only security updates, if desired)

</details>

+1 -1

1 comment

1 changed file

dependabot-preview[bot]

pr closed time in a day

PR closed Graylog2/documentation

Bump sphinx from 3.2.1 to 3.3.1 dependencies

Bumps sphinx from 3.2.1 to 3.3.1. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/sphinx-doc/sphinx/blob/3.x/CHANGES">sphinx's changelog</a>.</em></p> <blockquote> <h1>Release 3.3.1 (released Nov 12, 2020)</h1> <h2>Bugs fixed</h2> <ul> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8372">#8372</a>: autodoc: autoclass directive became slower than Sphinx-3.2</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/7727">#7727</a>: autosummary: raise PycodeError when documenting python package without <strong>init</strong>.py</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8350">#8350</a>: autosummary: autosummary_mock_imports causes slow down builds</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8364">#8364</a>: C, properly initialize attributes in empty symbols.</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8399">#8399</a>: i18n: Put system locale path after the paths specified by configuration</li> </ul> <h1>Release 3.3.0 (released Nov 02, 2020)</h1> <h2>Deprecated</h2> <ul> <li><code>sphinx.builders.latex.LaTeXBuilder.usepackages</code></li> <li><code>sphinx.builders.latex.LaTeXBuilder.usepackages_afger_hyperref</code></li> <li><code>sphinx.ext.autodoc.SingledispatchFunctionDocumenter</code></li> <li><code>sphinx.ext.autodoc.SingledispatchMethodDocumenter</code></li> </ul> <h2>Features added</h2> <ul> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8100">#8100</a>: html: Show a better error message for failures on copying html_static_files</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8141">#8141</a>: C: added a <code>maxdepth</code> option to :rst:dir:<code>c:alias</code> to insert nested declarations.</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8081">#8081</a>: LaTeX: Allow to add LaTeX package via <code>app.add_latex_package()</code> until just before writing .tex file</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/7996">#7996</a>: manpage: Add :confval:<code>man_make_section_directory</code> to make a section directory on build man page</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8289">#8289</a>: epub: Allow to suppress "duplicated ToC entry found" warnings from epub builder using :confval:<code>suppress_warnings</code>.</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8298">#8298</a>: sphinx-quickstart: Add :option:<code>sphinx-quickstart --no-sep</code> option</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8304">#8304</a>: sphinx.testing: Register public markers in sphinx.testing.fixtures</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8051">#8051</a>: napoleon: use the obj role for all See Also items</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8050">#8050</a>: napoleon: Apply :confval:<code>napoleon_preprocess_types</code> to every field</li> <li>C and C++, show line numbers for previous declarations when duplicates are detected.</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8183">#8183</a>: Remove substitution_reference nodes from doctree only on LaTeX builds</li> </ul> <h2>Bugs fixed</h2> <ul> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8085">#8085</a>: i18n: Add support for having single text domain</li> <li><a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/6640">#6640</a>: i18n: Failed to override system message translation</li> </ul> <!-- raw HTML omitted --> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/sphinx-doc/sphinx/commit/c09795c19421a01aa03e465aaf26834941d5b245"><code>c09795c</code></a> Bump to 3.3.1 final</li> <li><a href="https://github.com/sphinx-doc/sphinx/commit/5cabdecb9e47c50fd7de32f02acef1ddfde3b1cf"><code>5cabdec</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8412">#8412</a> from tk0miya/8164_autosummary_mock_imports_causes_sl...</li> <li><a href="https://github.com/sphinx-doc/sphinx/commit/953a5ec89801edb7047a779bc443b40acf066a27"><code>953a5ec</code></a> Fix <a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8350">#8350</a>: autosummary_mock_imports causes slow down builds</li> <li><a href="https://github.com/sphinx-doc/sphinx/commit/de2d2cc6ac7e381a4d4dacac146d09cb8e4e0c18"><code>de2d2cc</code></a> Update CHANGES for PR <a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8399">#8399</a></li> <li><a href="https://github.com/sphinx-doc/sphinx/commit/52432c7e3e675fa9763c3f5519dfb520cb7b7095"><code>52432c7</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8399">#8399</a> from mitya57/fix-system-translation-override</li> <li><a href="https://github.com/sphinx-doc/sphinx/commit/f92989ca6fe4d51f606cfc97610d6e1087b81969"><code>f92989c</code></a> Put system locale path after the paths specified by configuration</li> <li><a href="https://github.com/sphinx-doc/sphinx/commit/90f6fd90e0e32e6b2e80fa1783dc5f045a2d5de1"><code>90f6fd9</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8390">#8390</a> from sphinx-doc/7727_autosummary_PycodeError</li> <li><a href="https://github.com/sphinx-doc/sphinx/commit/ed5af2461f77a5bd7d1cc371877a0ab961224f79"><code>ed5af24</code></a> Merge branch '3.3.x' into 7727_autosummary_PycodeError</li> <li><a href="https://github.com/sphinx-doc/sphinx/commit/0be0cd606cdc240480ee5e57089562a9007da6bf"><code>0be0cd6</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/8387">#8387</a> from tk0miya/8372_slow_autoclass</li> <li><a href="https://github.com/sphinx-doc/sphinx/commit/787444ff182c4a3c490e26647f1cd99872a575ba"><code>787444f</code></a> Fix <a href="https://github-redirect.dependabot.com/sphinx-doc/sphinx/issues/7727">#7727</a>: autosummary raises PycodeError for namespace python package</li> <li>Additional commits viewable in <a href="https://github.com/sphinx-doc/sphinx/compare/v3.2.1...v3.3.1">compare view</a></li> </ul> </details> <br />

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


<details> <summary>Dependabot commands and options</summary> <br />

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
  • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
  • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
  • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language
  • @dependabot badge me will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot dashboard:

  • Update frequency (including time of day and day of week)
  • Pull request limits (per update run and/or open at any time)
  • Automerge options (never/patch/minor, and dev/runtime dependencies)
  • Out-of-range updates (receive only lockfile updates, if desired)
  • Security updates (receive only security updates, if desired)

</details>

+1 -1

1 comment

1 changed file

dependabot-preview[bot]

pr closed time in a day

pull request commentGraylog2/documentation

Bump certifi from 2020.6.20 to 2020.11.8

Looks like certifi is up-to-date now, so this is no longer needed.

dependabot-preview[bot]

comment created time in a day

pull request commentGraylog2/documentation

Bump sphinx from 3.2.1 to 3.3.1

Looks like sphinx is up-to-date now, so this is no longer needed.

dependabot-preview[bot]

comment created time in a day

delete branch Graylog2/documentation

delete branch : dependabot/pip/babel-2.9.0

delete time in a day

push eventGraylog2/documentation

Jan Doberstein

commit sha 989bede88cc0ec71651b1db40db03a4bcb8751ac

Update manual_setup.rst (#895) * Update manual_setup.rst adjust the logger configuration to the latest * fixed code block * final fix

view details

Jan Doberstein

commit sha 24bed456b6db2e7a596e19f0004750030429e85c

Update all Python Deps (#908) Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

view details

Sascha Henke

commit sha 9297db4039d14812916f6827fcb9df158de1067c

Changed Elasticsearch output to current version (#907)

view details

Bernd Ahlers

commit sha ad1bcfc4ec3e4254fd2f277e67c0c63a211cef58

Add changelog for 3.3.2 (#899)

view details

David Greenwald

commit sha d0c6ffd28b6db91fe7e0d113eaafc2f0558edef1

Update dashboards.rst (#903) Fix two typos.

view details

Jan Doberstein

commit sha 0e55d2fbaf6447ac58b98cf93ce3963f43663b29

updated numbers and little missing pieces (#909) * add recommendation for docker-compose * overview more verbose to enable the ability to search with the browser based search in the headlines I made the index overview page more verbose (1 level) * adjusted latest version * updated elasticsearch versions in docs * removed blank line * adjusted AWS link * added enterprise plugins links * adjusted Graylog version

view details

Jan Doberstein

commit sha b5bd5344ddd9547467fb57dfb8cb9cca944e9aa4

tested and updated centos install (#910) to have the installation tested on centOS 8 (and other distributions based on that) I have done the verification and updated the documentation. FIX: https://github.com/Graylog2/graylog2-server/issues/8241 Co-authored-by: Konrad Merz <konrad@graylog.com>

view details

Jan Doberstein

commit sha aa4da96256f8a4751d7841dee57fefc94f02b4d7

fixed wording (#911) * fixed wording After thinking about https://github.com/Graylog2/documentation/issues/865 and the knowledge that we not only have the journal that you want to be persistent. * fixed missing change Co-authored-by: Konrad Merz <konrad@graylog.com>

view details

Jan Doberstein

commit sha 12c2bb92da91122ab55a251f1ece8b2245676bec

change to copy&paste action (#912) I just realised that we try to have as many copy&paste options as possible. With this change we have removed the reference to vim to edit a file and just added the option to copy&paste from the docs. Hopefully this will lower the amount of people forgetting the setting. Co-authored-by: Konrad Merz <konrad@graylog.com>

view details

Jan Doberstein

commit sha 05386dc6d314c2e8f693f77d3a2636c0df858254

wording

view details

Jan Doberstein

commit sha 6bd0feac84c7e0252700219589d914d084386135

Rework `sending log messages` section of the docs (#917) This is a complete rework of the sending messages to graylog part. This fixes the following issues: FIX:https://github.com/Graylog2/documentation/issues/630 FIX:https://github.com/Graylog2/documentation/issues/147 Co-authored-by: Donald Morton <donald.morton@graylog.com>

view details

Marco Pfatschbacher

commit sha e443051bcb1f1f520c867c1a359647f77ed4ffa2

update ldap screenshot after changes (#919) "Allow self-signed Certificates" -> "Allow untrusted certificates"

view details

Jan Doberstein

commit sha 2d38c9fe91ba2cdc984636cb80a9fe9d00e45847

updated docs (#913) Co-authored-by: Donald Morton <donald.morton@graylog.com>

view details

dependabot-preview[bot]

commit sha 2ce01359aaa4972897c8227d507ab623596f0d51

Bump urllib3 from 1.25.9 to 1.25.10 (#916)

view details

Priya

commit sha ec9268558fa64da0259d1308132e4af7d9242127

Documentation for O365 Plugin (#918) Co-authored-by: Rob Curtis <rob.curtis@graylog.com> Co-authored-by: Jan Doberstein <jd@jalogis.ch>

view details

Rob Curtis

commit sha 45b1c24d05dc9251491a7ba8ec4e007aa56fbfdf

Add documentation for Enterprise Output Framework (#920) Co-authored-by: Jan Doberstein <jd@jalogis.ch>

view details

Jan Doberstein

commit sha 6c66eb8db0884b9501addced5310c68d22620b30

Add changelog and upgrade notes for 3.3.3 (#921)

view details

Nick Carstensen

commit sha f0096e91ab80268629e35af7fa60f142b868943c

Update ubuntu.rst Added in line about working on 20.04 LTS as well

view details

Abe

commit sha 28a06b0fc07c3bb94fddf7a0d1646eb4813ee50e

Update parameters.rst (#933) Co-authored-by: Jan Doberstein <jd@jalogis.ch>

view details

Jan Doberstein

commit sha 35940f27d4ec478de20cd5f527a1f062e4ed1320

added note about missing boolean (#914)

view details

push time in a day

PR closed Graylog2/documentation

Bump babel from 2.8.0 to 2.9.0 dependencies

Bumps babel from 2.8.0 to 2.9.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/python-babel/babel/releases">babel's releases</a>.</em></p> <blockquote> <h2>Version 2.8.1</h2> <p>This patch version only differs from 2.8.0 in that it backports in <a href="https://github-redirect.dependabot.com/python-babel/babel/issues/752">#752</a>.</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/python-babel/babel/blob/master/CHANGES">babel's changelog</a>.</em></p> <blockquote> <h2>Version 2.9.0</h2> <p>Upcoming version support changes</p> <pre><code>

  • This version, Babel 2.9, is the last version of Babel to support Python 2.7, Python 3.4, and Python 3.5. <p>Improvements</p> <pre><code>
  • CLDR: Use CLDR 37 – Aarni Koskela (#734)
  • Dates: Handle ZoneInfo objects in get_timezone_location, get_timezone_name - Alessio Bogon (#741)
  • Numbers: Add group_separator feature in number formatting - Abdullah Javed Nesar (#726)

Bugfixes


* Dates: Correct default Format().timedelta format to 'long' to mute deprecation warnings – Aarni Koskela
* Import: Simplify iteration code in &amp;quot;import_cldr.py&amp;quot; – Felix Schwarz
* Import: Stop using deprecated ElementTree methods &amp;quot;getchildren()&amp;quot; and &amp;quot;getiterator()&amp;quot; – Felix Schwarz
* Messages: Fix unicode printing error on Python 2 without TTY. – Niklas Hambüchen
* Messages: Introduce invariant that _invalid_pofile() takes unicode line. – Niklas Hambüchen
* Tests: fix tests when using Python 3.9 – Felix Schwarz
* Tests: Remove deprecated 'sudo: false' from Travis configuration – Jon Dufresne
* Tests: Support Py.test 6.x – Aarni Koskela
* Utilities: LazyProxy: Handle AttributeError in specified func – Nikiforov Konstantin ([#724](https://github.com/python-babel/babel/issues/724))
* Utilities: Replace usage of parser.suite with ast.parse – Miro Hrončok

Documentation
</code></pre>
<ul>
<li>Update parse_number comments – Brad Martin (<a href="https://github-redirect.dependabot.com/python-babel/babel/issues/708">#708</a>)</li>
<li>Add <strong>iter</strong> to Catalog documentation – <a href="https://github.com/CyanNani123">@CyanNani123</a></li>
</ul>
<h2>Version 2.8.1</h2>
<p>This is solely a patch release to make running tests on Py.test 6+ possible.</p>
<p>Bugfixes</p>
<pre><code>
* Support Py.test 6 - Aarni Koskela ([#747](https://github.com/python-babel/babel/issues/747), [#750](https://github.com/python-babel/babel/issues/750), [#752](https://github.com/python-babel/babel/issues/752))
&lt;/code&gt;&lt;/pre&gt;
&lt;/blockquote&gt;
&lt;/details&gt;
&lt;details&gt;
&lt;summary&gt;Commits&lt;/summary&gt;

&lt;ul&gt;
&lt;li&gt;See full diff in &lt;a href=&quot;https://github.com/python-babel/babel/commits&quot;&gt;compare view&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/details&gt;

&lt;br /&gt;
</code></pre>


[![Dependabot compatibility score](https://api.dependabot.com/badges/compatibility_score?dependency-name=babel&package-manager=pip&previous-version=2.8.0&new-version=2.9.0)](https://dependabot.com/compatibility-score/?dependency-name=babel&package-manager=pip&previous-version=2.8.0&new-version=2.9.0)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Pull request limits (per update run and/or open at any time)
- Automerge options (never/patch/minor, and dev/runtime dependencies)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)



</details>
+1 -1

2 comments

1 changed file

dependabot-preview[bot]

pr closed time in a day

delete branch Graylog2/documentation

delete branch : dependabot/pip/requests-2.25.0

delete time in a day

delete branch Graylog2/documentation

delete branch : dependabot/pip/urllib3-1.26.2

delete time in a day

more