profile
viewpoint
Jim Pick jimpick Backblend Systems Inc. Vancouver, BC, Canada http://jimpick.com/

ipfs-inactive/dynamic-data-and-capabilities 54

[ARCHIVED] Dynamic Data and Capabilities in IPFS Working Group

ipfs-shipyard/integration-mini-projects 15

Ideas and tracking for small one week "mini projects" that integrate across IPFS

Digital-MOB-Filecoin/QABot 14

Quality Bot: Periodically makes storage and retrieval deals with each of the top 100 miners, tracking state & success/fail.

filecoin-project/sentinel 8

A Filecoin Network Monitoring and Analysis System

ipfs-shipyard/camp-course-c-demo 7

IPFS Camp 2019 Course C Materials

filecoin-shipyard/filecoin-big-head 6

Displays Filecoin chain height in big letters in terminal

backblend/peer-pad-nano 5

📝 Simplified version of peerpad.net for learning

automerge/pixelpusherd 4

An archiver daemon for pixelpusher

filecoin-shipyard/filecoin-pickaxe 3

Tool for storing/retrieving data in Filecoin

issue commentipfs/notes

SpaceX's Starlink

Yeah, I'd see this mostly being useful for very large, very popular downloads. For example, game of thrones on the release day, live TV, etc.

The benefit of building this into IPFS is that you'd be able to fallback on bitswap/graphsync if no satellite link is available, you'd be able to find available satellites through content addressing instead of needing to trust some centralized service, and you'd be able to verify the data. You could always build a custom protocol and/or proprietary system to do the same thing, but integrating into an existing ecosystem is probably simpler in the long run.

RubenKelevra

comment created time in an hour

starteddenysdovhan/spaceship-prompt

started time in an hour

Pull request review commentfilecoin-project/sentinel

docs: guide to getting it all working locally. update telegraf defaults for dev

 Follow the [Lotus installation instructions](https://lotu.sh/en+getting-started) 6. `make run-docker` to start Docker services 7. (In separate windows) `make run-drone` and `make run-visor`. -### Configure Grafana+## Getting Started Step-by-Step++Clone the repo and fetch the submodules. `make` will help you out:++```console+$ git clone git@github.com:filecoin-project/sentinel.git+$ cd sentinel+$ make deps+git submodule update --init --recursive+...+```++Now we need to get Lotus built and syncing the chain.++### Lotus++Sentinel requires a running lotus daemon that has completed synching the chain locally. A full sync of the [Mainnet](https://network.filecoin.io/#mainnet) takes days. You can test against the [Nerpa network](https://network.filecoin.io/#nerpa) to try Sentinel out.++Follow the [Lotus install and setup instructions](https://docs.filecoin.io/get-started/lotus/installation), to install the dependencies, check out the branch for the network you want to join, (e.g. `ntwrk-nerpa`), and build a local copy of lotus. If you need to test against mainnet, see [Testing against Mainnet](#testing-against-mainnet).++Run Lotus to start syncing.+```console+$ lotus daemon+```++In a sperate shell, ask lotus to tell us when it has finished syncing.+```console+$ lotus sync wait+lotus sync wait+Worker 0: Target Height: 9099	Target: [bafy2bzacecqrt46shkioecxkt6mrdvi5xd73wh2gonrp3bhxfut6szc2labj6]	State: complete	Height: 9099+Done!+```++Now let's set up the Database.++### TimescaleDB++A [`docker-compose`](./docker-compose.yml) file is provided that will spin up a TimescaleDB and a Grafana instance to query it. Check that you have `docker` and `docker-compose` installed. [Docker Desktop](https://docs.docker.com/desktop/) is relatively painless. ++```console+$ make run-docker+``` -1. Visit [http://localhost:3000](http://localhost:3000) to open Grafana.-2. Login with username and password as `admin`. Change the admin password.+Note that TimescaleDB is packaged as an Postgres extension, so some other components will refer to it as Postgres. -The datasource and dashboards are provisioned by the config in +Now we need to generate some data with Visor and Drone -- `grafana/provisioning/dashboards/dashboards.yml`-- `grafana/provisioning/datasources/timescaledb.yml`+### Visor++In a new shell run Visor. [By default](https://github.com/filecoin-project/sentinel/blob/d1623b47649113c12aa53c943b4ddb60401f4632/Makefile#L93-L94) it will read Lotus data from `$(HOME)/.lotus` and writes to the local TimescaleDB container we just started at `localhost:5432`.

Should probably note this will also be using the API lens.

olizilla

comment created time in an hour

Pull request review commentfilecoin-project/sentinel

docs: guide to getting it all working locally. update telegraf defaults for dev

 Follow the [Lotus installation instructions](https://lotu.sh/en+getting-started) 6. `make run-docker` to start Docker services 7. (In separate windows) `make run-drone` and `make run-visor`. -### Configure Grafana+## Getting Started Step-by-Step++Clone the repo and fetch the submodules. `make` will help you out:++```console+$ git clone git@github.com:filecoin-project/sentinel.git+$ cd sentinel+$ make deps+git submodule update --init --recursive+...+```++Now we need to get Lotus built and syncing the chain.++### Lotus++Sentinel requires a running lotus daemon that has completed synching the chain locally. A full sync of the [Mainnet](https://network.filecoin.io/#mainnet) takes days. You can test against the [Nerpa network](https://network.filecoin.io/#nerpa) to try Sentinel out.++Follow the [Lotus install and setup instructions](https://docs.filecoin.io/get-started/lotus/installation), to install the dependencies, check out the branch for the network you want to join, (e.g. `ntwrk-nerpa`), and build a local copy of lotus. If you need to test against mainnet, see [Testing against Mainnet](#testing-against-mainnet).

Perhaps it would be better to point to the test network instead of nerpa net? Unless there is a strong case for using nerpa.

olizilla

comment created time in an hour

issue commentfilecoin-project/lotus

Creating "tracked" addresses and tooling for tracking miner account activities.

So the idea is to have lotus track add <address> and all changes in value from/to that address (even sub-transactions and burns, everything) would get saved and could be exported instantly for audit purposes. The system would work only after you enabled tracking for a given address.

jennijuju

comment created time in 2 hours

push eventfilecoin-project/specs-actors

zgfzgf

commit sha 51482ab9f7aeb57bc778ad607046b04f41792cfa

Use StateReadonly in ProveCommitSector (#1312)

view details

push time in 2 hours

issue closedfilecoin-project/specs-actors

optimize miner.ProveCommitSector use StateReadonly

rt.StateTransaction should be: rt.StateReadonly https://github.com/filecoin-project/specs-actors/blob/585b71e2b89642f2b01157c94e72a4ca9a8e869a/actors/builtin/miner/miner_actor.go#L733-L741

closed time in 2 hours

zgfzgf

issue commentipfs/notes

SpaceX's Starlink

I guess that is basically what they do for TV, where a large number of clients request the exact same bits at the exact same time. Another use-case would be firmware update of the antennas themselves. Other than that, I don't see why many clients would request the exact same bits at the exact same time. And they don't need IPFS for that.

I think sattelite ISPs can be useful to have a one-hop connection to a ground node near you (direct connection as seen from IPFS of course). But Sat ISP don't need to run sky IPFS nodes for that.

Having them run smart IPFS local DHT listing only nodes under a sattelite would help, though, as people would be able to quickly find nodes with the requested data one sat hop away from them. But it require a very large IPFS adoption before Starlink even consider maintaining such large DHT on every sat (with the additional problem that you have to constantly pass the info to the next sattelite, as it is a geographically bounded DHT).

RubenKelevra

comment created time in 2 hours

pull request commentfilecoin-project/sentinel

docs: guide to getting it all working locally. update telegraf defaults for dev

I resolved conflicts and removed the errors. @frrist Want to give it another look before I merge?

olizilla

comment created time in 2 hours

push eventfilecoin-project/sentinel

Mike Greenberg

commit sha ab0ede9e4c0ea01a02fc3bb2493acdbb86944025

fix: Makefile bug; Disambiguate doc header

view details

push time in 2 hours

startedNerdyPepper/taizen

started time in 2 hours

issue commentfilecoin-project/replication-game

Issue with building

It can be fixed by running cargo update -p pear, which updates your Cargo.lock to use the latest version of Pear (which includes a bugfix for the regression).

From link: rust-lang/rust#73293

Don't forget to remove Cargo.lock after the update.

jmanchuck

comment created time in 2 hours

startedd3/d3-geo

started time in 3 hours

issue openedfilecoin-project/lotus

Creating "tracked" addresses and tooling for tracking miner account activities.

Miners brought up that it could be nice if there is a way to track where all the money goes. burned/minerfees/gas vs collateral, and eventually earnings for tax purpose. This is to create an address so that related operations can be easily tracked

created time in 3 hours

issue commentipfs/notes

SpaceX's Starlink

I doubt storing data in space will be an attractive option any time soon. The benefit of LEO satellites is low latency communication in remote places. It's much cheeper to just drop a bunch of large datacenters around the world and then transfer data on demand. A 2x bandwidth cost for not storing anything on the satellite is worth it.

However, one could use satellites to broadcast popular data then use content addressing to find nearby satellites/frequencies providing the data. But this would require new protocols.

RubenKelevra

comment created time in 3 hours

startedchrisveness/geodesy

started time in 3 hours

push eventfilecoin-project/sentinel

Ian Davis

commit sha f86c3fd687de72a2028b9c7d7b6ef0fd5c0ada8a

feat: add jaeger and redis to docker compose

view details

Ian Davis

commit sha 30ff5b7a1a9102de9384e80d1dbb6249f59930c5

Specify service dependencies

view details

Ian Davis

commit sha 1920ddf68227d13223b723ee316ec52bbca87f3d

Merge pull request #140 from filecoin-project/iand/docker-compose-additional feat: add jaeger and redis to docker compose

view details

Mike Greenberg

commit sha d1623b47649113c12aa53c943b4ddb60401f4632

Merge v0.3.2 (#158) * feat(telefraf): change the prometheus->telegraf metrics mapping (#113) This updates the metric_version of the prometheus input to version 2, which brings the telegraf mapping more inline with how prometheus natively outputs metrics, especially histograms and summaries. See https://github.com/influxdata/telegraf/issues/4415 for more information. * feat(dashboard): Update dashboards for Grafana 7.1 (#135) * feat(dashboard): Update dashboards for Grafana 7.1 * fix(docker): Set specific version of grafana for docker-compose * feat: Rename Telegraf to Sentinel Drone (#131) * feat: Add visor to Makefile; CI build (#149) * feat: add visor to makefile * Update Makefile Co-authored-by: Oli Evans <oli@tableflip.io> * chore: Remove sentinel- prefix in CLI usage * chore: Add rust deps to CI build env * chore: Add latest visor Co-authored-by: Oli Evans <oli@tableflip.io> Co-authored-by: Mike Greenberg <mg+github@protocol.ai> * dep: Update to lotus@v0.8.0 (#155) * feat(grafana): Update dashboards for latest Visor schema (#156) * chore: Fixup Makefile and docker-compose; Update submodules (#157) * chore: Remove redis docker-compose svcs; Create jaeger config * chore: Update docker-compose timescale svc to pg12 * chore: visor-indexer and -processor make directives use new subcommand * deps: Update to drone@v0.3.0 and visor@v0.2.0 * deps: Update to lotus@v0.9.1 * chore: Remove chainwatch services; Fix visor services * fix(docs): Correct documentation for visor and naming * Apply suggestions: Service desc edits; log output adj in service Co-authored-by: Adrian Lanzafame <adrianlanzafame92@gmail.com> * Update dashboards chain_power -> chain_powers Co-authored-by: Adrian Lanzafame <adrianlanzafame92@gmail.com> Co-authored-by: Adrian Lanzafame <adrianlanzafame92@gmail.com> Co-authored-by: Ian Davis <nospam@iandavis.com> Co-authored-by: Oli Evans <oli@tableflip.io>

view details

Mike Greenberg

commit sha 921c2e7d89e4553256607384a07d8477b7908958

Merge branch 'master' into document-getting-started * master: Merge v0.3.2 (#158) Specify service dependencies feat: add jaeger and redis to docker compose

view details

Mike Greenberg

commit sha e0cc4387f89fa0a732517a824d82f56afc47530f

fix: Default connection string for drone

view details

push time in 3 hours

issue openedfilecoin-project/lotus

After restarting miner one worker id is 00000000-0000-0000-0000-000000000000

After restarting the miner, all worker reconnects automatically except one who is reporting a 00000000-0000-0000-0000-000000000000 worker ID and is in status ret-wait

Describe the problem

Today, I restart my miner to compact the chain. all the steps:

one of my worker (lamia) was processing a C2 job. I stopped the miner, I stopped the daemon, (I didn't stopped the workers) I compact the chain following the official procedure : https://docs.filecoin.io/get-started/lotus/chain/#compacting-the-chain-data After restarting properly the daemon and the miner, all my workers reconnect properly. Except lamia.

Lamia appears in the status ret-wait Lamia doesn't appear in the worker list Lamia appears in the job list with a worker id is 00000000-0000-0000-0000-000000000000

 lotus@keats:~$ miner sealing workers
Worker 0dd98dae-f8f9-46ce-a1a8-f5a185710e07, host kassad
	CPU:  [                                                                ] 0/32 core(s) in use
	RAM:  [||                                                              ] 3% 4.698 GiB/125.7 GiB
	VMEM: [||                                                              ] 3% 4.698 GiB/125.7 GiB
	GPU: GeForce RTX 3080, not used
Worker 41d3ab72-6a5b-46ca-b36d-c065497036cf, host keats
	CPU:  [||||||||||||||||||||||||||||||||||||||||||||||||||||||||||      ] 29/32 core(s) in use
	RAM:  [||||||                                                          ] 9% 37.28 GiB/377.6 GiB
	VMEM: [||||||||||||||||||||||||||                                      ] 41% 157.3 GiB/377.6 GiB
	GPU: GeForce GTX 1080 Ti, used
Worker 46d1d366-037e-419d-917a-c18e44128fe2, host masteen
	CPU:  [                                                                ] 0/16 core(s) in use
	RAM:  [                                                                ] 0% 4.74 GiB/503.6 GiB
	VMEM: [                                                                ] 0% 4.74 GiB/511.6 GiB
Worker 4737e0b9-8dd8-4805-abc3-73b47e4800e5, host hoyt
	CPU:  [                                                                ] 0/32 core(s) in use
	RAM:  [|||                                                             ] 5% 10.32 GiB/188.6 GiB
	VMEM: [|||                                                             ] 5% 10.32 GiB/188.6 GiB
Worker 83a4c4a9-88bf-44cb-8f1e-919c3610a72c, host silenus
	CPU:  [                                                                ] 0/16 core(s) in use
	RAM:  [                                                                ] 0% 4.9 GiB/503.6 GiB
	VMEM: [                                                                ] 0% 4.9 GiB/503.6 GiB
L lotus@keats:~$ miner sealing jobs
ID        Sector  Worker    Hostname  Task  State        Time
483f238c  1473    00000000  lamia     C2    ret-wait     21m34.5s
ea252d6b  1475    41d3ab72  keats     C2    running      9m20.5s
00000000  1472    41d3ab72  keats     C2    assigned(0)  8m20.2s
{
  "jsonrpc": "2.0",
  "result": {
    "00000000-0000-0000-0000-000000000000": [
      {
        "ID": {
          "Sector": {
            "Miner": 10479,
            "Number": 1473
          },
          "ID": "483f238c-8a42-4e70-b656-581289ca4ef8"
        },
        "Sector": {
          "Miner": 10479,
          "Number": 1473
        },
        "Task": "seal/v0/commit/2",
        "RunWait": -1,
        "Start": "2020-11-30T22:57:04+01:00",
        "Hostname": "lamia"
      }
    ],
    "41d3ab72-6a5b-46ca-b36d-c065497036cf": [
      {
        "ID": {
          "Sector": {
            "Miner": 10479,
            "Number": 1475
          },
          "ID": "ea252d6b-bcb7-460b-9b3c-b1319dad7bd2"
        },
        "Sector": {
          "Miner": 10479,
          "Number": 1475
        },
        "Task": "seal/v0/commit/2",
        "RunWait": 0,
        "Start": "2020-11-30T23:09:17.945788076+01:00"
      },
      {
        "ID": {
          "Sector": {
            "Miner": 0,
            "Number": 0
          },
          "ID": "00000000-0000-0000-0000-000000000000"
        },
        "Sector": {
          "Miner": 10479,
          "Number": 1472
        },
        "Task": "seal/v0/commit/2",
        "RunWait": 1,
        "Start": "2020-11-30T23:10:18.311687171+01:00"
      }
    ]
  },
  "id": 3
}

Worker (lamia) logs :

[...]
2020-11-30T23:08:59.179+0100	WARN	stores	stores/local.go:324	error reporting storage health for 036b462c-caa7-4524-b0e7-26df70b296ae ({Stat:{Capacity:982899539968 Available:776974663680 Reserved:0} Err:}): RPC client error: sendRequest failed: Post "http://192.168.43.1:2345/rpc/v0": dial tcp 192.168.43.1:2345: connect: connection refused
2020-11-30T23:09:01.122+0100	ERROR	main	lotus-seal-worker/main.go:490	heartbeat: checking remote session failed: RPC client error: sendRequest failed: Post "http://192.168.43.1:2345/rpc/v0": dial tcp 192.168.43.1:2345: connect: connection refused
2020-11-30T23:09:09.180+0100	WARN	stores	stores/local.go:324	error reporting storage health for 036b462c-caa7-4524-b0e7-26df70b296ae ({Stat:{Capacity:982899539968 Available:776974663680 Reserved:0} Err:}): RPC client error: sendRequest failed: Post "http://192.168.43.1:2345/rpc/v0": dial tcp 192.168.43.1:2345: connect: connection refused
2020-11-30T23:09:11.123+0100	ERROR	main	lotus-seal-worker/main.go:490	heartbeat: checking remote session failed: RPC client error: sendRequest failed: Post "http://192.168.43.1:2345/rpc/v0": dial tcp 192.168.43.1:2345: connect: connection refused
2020-11-30T23:09:19.181+0100	WARN	stores	stores/local.go:324	error reporting storage health for 036b462c-caa7-4524-b0e7-26df70b296ae ({Stat:{Capacity:982899539968 Available:776974663680 Reserved:0} Err:}): health report for unknown storage: 036b462c-caa7-4524-b0e7-26df70b296ae
2020-11-30T23:09:21.123+0100	ERROR	main	lotus-seal-worker/main.go:515	LOTUS-MINER CONNECTION LOST
2020-11-30T23:09:21.123+0100	INFO	main	lotus-seal-worker/main.go:467	Redeclaring local storage
2020-11-30T23:09:21.123+0100	INFO	main	lotus-seal-worker/main.go:483	Making sure no local tasks are running
2020-11-30T23:11:25.867 INFO bellperson::gpu::locks > GPU is available for Multiexp!
2020-11-30T23:11:26.000 INFO bellperson::gpu::multiexp > Multiexp: 1 working device(s) selected. (CPU utilization: 0)
2020-11-30T23:11:26.000 INFO bellperson::gpu::multiexp > Multiexp: Device 0: GeForce RTX 2080 Ti (Chunk-size: 7488999)
2020-11-30T23:11:26.000 INFO bellperson::multiexp > GPU Multiexp kernel instantiated!

** Code modifications **

none

Version

Daemon: 1.2.1+git.df66f48f6+api1.0.0 Local: lotus version 1.2.1+git.df66f48f6

created time in 3 hours

pull request commentipfs/pinning-services-api-spec

docs: improve Pin.origins example

one thing that I would mention is that users make sure they're providing the multi address with their public key (as often many addresses are listed when you do ipfs id. This is a small thing though.

lidel

comment created time in 3 hours

PR opened ipfs/pinning-services-api-spec

docs: improve Pin.origins example

This updates example in docs to show that multiple multiaddrs of different types can be added:

2020-11-30--23-23-36

Thanks @obo20 for noticing this.

+2 -1

0 comment

2 changed files

pr created time in 4 hours

create barnchipfs/pinning-services-api-spec

branch : docs/origins-multiaddrs

created branch time in 4 hours

fork triptych/SuperTinyIcons

Under 1KB each! Super Tiny Icons are miniscule SVG versions of your favourite website and app logos

fork in 4 hours

startededent/SuperTinyIcons

started time in 4 hours

push eventfilecoin-project/lotus

Łukasz Magiera

commit sha 6dea0a6281232b41047e8ed5e01370dc40015a58

worker: fix math-related issue

view details

push time in 4 hours

push eventfilecoin-project/lotus

Łukasz Magiera

commit sha a4f3758f4ca172eb2771ff59130e10db05d16376

worker api: better grouping

view details

push time in 4 hours

push eventfilecoin-project/lotus

Łukasz Magiera

commit sha b18bc90dbc8c38a5caaf0cb0dc7ab06d9fc4cd49

client list-asks --by-ping

view details

Łukasz Magiera

commit sha 3e143cac4bd430aa40deb29d17be693bd6d63f98

Merge pull request #5060 from filecoin-project/feat/listasks-ping client list-asks --by-ping

view details

push time in 4 hours

delete branch filecoin-project/lotus

delete branch : feat/listasks-ping

delete time in 4 hours

startedsindresorhus/awesome

started time in 4 hours

PR merged filecoin-project/lotus

Reviewers
client list-asks --by-ping
+40 -11

0 comment

1 changed file

magik6k

pr closed time in 4 hours

more