profile
viewpoint
Junya Hayashi ledmonster Groove X Tokyo

ledmonster/ddd-python-inject 75

DDD, CQRS and Hexagonal Architecture example using inject package

groove-x/mqtt_bridge 61

mqtt_bridge provides a functionality to bridge between ROS and MQTT in bidirectional

ledmonster/alloy-doc-ja 19

Alloy Framework のドキュメントの日本語訳です。

ledmonster/japanese-law 7

Sphinx repository for Japanese law

groove-x/gxredis 3

Simple redis-py wrapper library

ledmonster/Hatena-Textbook 2

はてな研修用教科書

ledmonster/djblets 1

A collection of useful extensions for Django.

ledmonster/hornetq 1

HornetQ is an open source project to build a multi-protocol, embeddable, very high performance, clustered, asynchronous messaging system.

groove-x/step-bloom-generate 0

generate debian package for ROS package

ledmonster/adv-r 0

Advanced R programming: a book

issue commentVictoriaMetrics/VictoriaMetrics

series count API returns old value after calling delete series API

I see. Let me explain my background. Since I implements dynamic logic to ingest metrics from IoT devices to VM, I write unit tests for ingestion program with VM and confirm my logic before deploying it to production.

  • start VM container locally
  • run tests
  • stop container

To make each test independent, I delete time series on tear down. And I noticed that count API doesn't be reset with series deletion. For tests, I can achieve my purpose without count API, so current behavior doesn't matter for me. Just for your information. Thanks.

ledmonster

comment created time in 9 days

issue commentVictoriaMetrics/VictoriaMetrics

series count API returns old value after calling delete series API

@dxtrzhang thanks, now I tried that API, but count API still returns old value.

$ curl http://localhost:8428/internal/resetRollupResultCache
$ curl -G 'http://localhost:8428/api/v1/series' --data-urlencode 'match[]={__name__=~".*"}' --data 'start=0'
{"status":"success","data":[]}
$ curl http://localhost:8428/api/v1/labels/count
{"status":"success","data":{}}
$ curl http://localhost:8428/api/v1/series/count
{"status":"success","data":[780]}
ledmonster

comment created time in 10 days

issue openedVictoriaMetrics/VictoriaMetrics

series count API returns old value after calling delete series API

Describe the bug I got old return value from /api/v1/series/count after calling /api/v1/admin/tsdb/delete_series.

To Reproduce

After ingesting multiple time series,

$ curl http://localhost:8428/api/v1/series/count
{"status":"success","data":[150]}

then, delete series from VM.

$ curl -G 'http://localhost:8428/api/v1/admin/tsdb/delete_series' --data-urlencode 'match[]={__name__=~".*"}'

Now, series API returns empty result.

$ curl -G 'http://localhost:8428/api/v1/series' --data-urlencode 'match[]={__name__=~".*"}' --data 'start=0'
{"status":"success","data":[]}

But count API still returns old stats.

$ curl http://localhost:8428/api/v1/series/count
{"status":"success","data":[150]}

Expected behavior

I want to get a correct value from series count API after series deletion.

Version

  • v1.41.0

Used command-line flags

  • nothing

created time in 10 days

issue closedVictoriaMetrics/VictoriaMetrics

Support timestamp option on prometheus import API

Is your feature request related to a problem? Please describe.

When I push metrics to /api/v1/import/prometheus, I want to specify a timestamp.

Describe the solution you'd like

/api/v1/import/prometheus?timestamp=1594370496905

Describe alternatives you've considered

add timestamps on prom file one by one

Additional context

our architecture:

IoT devices --> gateway --> pubsub --> worker --> VictoriaMetrics

IoT devices post prom formatted metrics (without timestamp) to gateway, and we push it to VictoriaMetrics through pubsub. Since there exist some latency on this path, I want to specify metrics timestamp by myself, instead of using current timestamp in VM import API layer.

closed time in 12 days

ledmonster

issue commentVictoriaMetrics/VictoriaMetrics

Support timestamp option on prometheus import API

Thank you, it works as expected.

$ curl -v  --data-binary @node-exporter.prom  'http://localhost:8428/api/v1/import/prometheus?extra_label=job=curl&extra_label=instance=pc&timestamp=1599894041764'
$ curl -G 'http://localhost:8428/api/v1/query' --data-urlencode 'query=go_gc_duration_seconds[5m]' | jq . 
{
  "status": "success",
  "data": {
    "resultType": "matrix",
    "result": [
      {
        "metric": {
          "__name__": "go_gc_duration_seconds",
          "job": "curl",
          "instance": "pc",
          "quantile": "1"
        },
        "values": [
          [
            1599894041.764,
            "0.007358708"
          ]
        ]
      },
      {
        "metric": {
          "__name__": "go_gc_duration_seconds",
          "job": "curl",
          "instance": "pc",
          "quantile": "0.25"
        },
        "values": [
          [
            1599894041.764,
            "0.000048134"
          ]
        ]
      },
      {
        "metric": {
          "__name__": "go_gc_duration_seconds",
          "job": "curl",
          "instance": "pc",
          "quantile": "0.5"
        },
        "values": [
          [
            1599894041.764,
            "0.000071184"
          ]
        ]
      },
      {
        "metric": {
          "__name__": "go_gc_duration_seconds",
          "job": "curl",
          "instance": "pc",
          "quantile": "0"
        },
        "values": [
          [
            1599894041.764,
            "0.000009038"
          ]
        ]
      },
      {
        "metric": {
          "__name__": "go_gc_duration_seconds",
          "job": "curl",
          "instance": "pc",
          "quantile": "0.75"
        },
        "values": [
          [
            1599894041.764,
            "0.001780718"
          ]
        ]
      }
    ]
  }
}
ledmonster

comment created time in 14 days

issue commentVictoriaMetrics/VictoriaMetrics

Support timestamp option on prometheus import API

Thank you for quick implementation. I'll check the behavior hopefully on this weekend.

ledmonster

comment created time in 14 days

PR opened prometheus/docs

fix typo: Nan -> NaN

@juliusv @brian-brazil I assume this should be "NaN", thanks.

+1 -1

0 comment

1 changed file

pr created time in 16 days

create barnchledmonster/docs

branch : feature/typo-nan

created branch time in 16 days

fork ledmonster/docs

Prometheus documentation: content and static site generator

https://prometheus.io

fork in 16 days

issue openedVictoriaMetrics/VictoriaMetrics

Support timestamp option on prometheus import API

Is your feature request related to a problem? Please describe.

When I push metrics to /api/v1/import/prometheus, I want to specify a timestamp.

Describe the solution you'd like

/api/v1/import/prometheus?timestamp=1594370496905

Describe alternatives you've considered

add timestamps on prom file one by one

Additional context

our architecture:

IoT devices --> gateway --> pubsub --> worker --> VictoriaMetrics

IoT devices post prom formatted metrics (without timestamp) to gateway, and we push it to VictoriaMetrics through pubsub. Since there exist some latency on this path, I want to specify metrics timestamp by myself, instead of using current timestamp in VM import API layer.

created time in 18 days

issue commentVictoriaMetrics/VictoriaMetrics

Support additional labels on prometheus import API

@valyala I'll try it later, thank you!!

ledmonster

comment created time in 23 days

issue commentVictoriaMetrics/VictoriaMetrics

Support additional labels on prometheus import API

I think it would be great adding labels={<LABEL_NAME>:<LABEL_VALUE>,...} query arg to all the supported data ingestion protocols that work over http.

That's great!!

I'm also using reliable feature, thank you.

ledmonster

comment created time in 24 days

PullRequestReviewEvent

issue commentVictoriaMetrics/helm-charts

Can I mount prometheus.yml on victoria-metrics-single?

@tenmozes I see. Now, I copied prometheus.yml to storage directly, and it works.

ledmonster

comment created time in 25 days

issue openedVictoriaMetrics/helm-charts

Can I mount prometheus.yml on victoria-metrics-single?

I found extraConfigmapMounts parameter is used in server-deployment.yaml, but volumes are not configured there. Should I modify chart to use prometheus.yml with victoria-metrics-single?

created time in a month

issue openedVictoriaMetrics/VictoriaMetrics

Support additional labels on prometheus import API

Is your feature request related to a problem? Please describe.

When I try to substitute VictoriaMetrics for Prometheus+Pushgateway, lack of additional labels feature on prometheus import API (/api/v1/import/prometheus) makes work harder. Pushgateway does support additional labels on API, and I want to use it in the import API, too.

Describe the solution you'd like

One idea is to support same API as Pushgateway.

/api/v1/import/prometheus{/<LABEL_NAME>/<LABEL_VALUE>}

or some query strings (e.g. labels) may also work.

/api/v1/import/prometheus?labels={<LABEL_NAME>%3D<LABEL_VALUE>,}

Describe alternatives you've considered

Now I'm using remote_write API instead. I parse Prometheus exposition format to WriteRequest and post it to the API with following code.

  • https://gist.github.com/ledmonster/7d0de61a788b164a510c0359465a41f4

created time in a month

issue commentVictoriaMetrics/VictoriaMetrics

Is it possible for VictoriaMetrics to substitue Prometheus+Pushgateway?

For me, remote_write based direct access to VM is working well, now. I had to fill timestamps to data after parsing prometheus text-based format, though.(Now, I want a kind of VM client library. 😄 )

Anyway, I'll create a feature request to support additional labels on api/v1/import/prometheus. Since pushgateway is supporting additional labels, user can easily switch to VM with this feature.

So, this issue can be closed, isn't it?

ozn0417

comment created time in a month

issue commentVictoriaMetrics/VictoriaMetrics

Is it possible for VictoriaMetrics to substitue Prometheus+Pushgateway?

@hagen1778 Hi!! I was not aware that pushgateway protocol is defined as prometheus text-based format.

In my case, my services can reach VM directly, so I adopted direct access approach. I succeeded to convert text protocol to remote_write one. Also, I added additional labels ("instance" and "job"), because I want to make data pushgateway compatible. It would be nice if /api/v1/import/prometheus supports additional labels. Thank you!!

ozn0417

comment created time in a month

issue openedgoogleapis/api-common-protos

do you have a plan to include actions type?

Here is some date range related proto definitions. Do you have a plan to port these protos to this repository?

  • https://github.com/googleapis/googleapis/tree/master/google/actions/type

created time in a month

issue commentVictoriaMetrics/VictoriaMetrics

Is it possible for VictoriaMetrics to substitue Prometheus+Pushgateway?

I found scrapework in lib. I'll try following parser logic.

  • https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/lib/promscrape/scrapework_test.go#L314
ozn0417

comment created time in a month

issue commentVictoriaMetrics/VictoriaMetrics

Is it possible for VictoriaMetrics to substitue Prometheus+Pushgateway?

@valyala Thank you for providing an exciting module. Does vmagent support Pushgateway protocol now? Or do you have a plan to support it in near future?

If vmagent doesn't support pushgateway protocol, I'm considering that convert pushgateway protocol to Prometheus remote_write protocol and push it to vmagent. Do you think does it feasible? Are there any library to do so?

Sorry for asking many questions, thanks.

ozn0417

comment created time in a month

startedVictoriaMetrics/operator

started time in a month

pull request commentgolang/mock

mockgen: handle more cases of "duplicate" imports

Could you please release this PR?

stevendanna

comment created time in 2 months

startedthanos-io/thanos

started time in 2 months

startedVictoriaMetrics/VictoriaMetrics

started time in 2 months

startedgo-oauth2/oauth2

started time in 3 months

startedkubernetes-sigs/krew

started time in 3 months

startedsubmariner-io/submariner

started time in 3 months

more