profile
viewpoint
Dmitry Shemetov dshemetov Math Department, UCDavis https://dshemetov.github.io Code happens!

dshemetov/GraphletLift 2

Python code to compare GraphletLift to a few other algorithms.

cmu-delphi/covidcast-modeling 0

Public-facing data analyses surrounding Delphi's COVIDcast effort.

dshemetov/Advent-of-Code-2015 0

I can't believe it's been 3 years already!

dshemetov/Advent-of-Code-2018 0

Puzzles + Christmas = yess.

dshemetov/cockeyedgaming 0

Repository for streaming tools used on twitch.tv/cockeyedgaming

dshemetov/delphi-epidata 0

An open API for Epidemiological Data, from the Delphi research group.

dshemetov/factorio-init 0

Factorio init script

pull request commentcmu-delphi/covidcast-indicators

Refactor claims_hosp to use smoothing util

Just to make sure I understand, the savgol smoother is not meant to be equivalent to the left gauss filter previously used. The results don't line up, but do match up to 0.1 tolerance. I ran using Smoother.left_gauss_linear_smoother from the smoother utility, and those results line up, so we're changing the smoothing method (and not just moving the function out).

Sorry for the delay, the PR looks good. @krivard are we going to announce this change? The documentation as well?

My understanding is that they should be very close with some slight differences in parameters, but I haven't spent a whole lot of time digging into details. See this slack post.

chinandrew

comment created time in 29 minutes

issue commentcmu-delphi/covidcast-indicators

Release SafeGraph_Patterns

Update: we decided to hold off on pushing to prod until Monday but this is currently in flight.

jingjtang

comment created time in an hour

issue commentcmu-delphi/covidcast-indicators

Release Google Symptoms

Update: we decided to hold off on pushing to prod until Monday, but this is currently in progress. Marking automation as done.

krivard

comment created time in an hour

delete branch cmu-delphi/covidcast-indicators

delete branch : safegraph-export

delete time in an hour

push eventcmu-delphi/covidcast-indicators

andrew

commit sha 0348db672a7d3724feee8ca04561e9a9a94a8aad

Refactor safegraph to use export util

view details

andrew

commit sha 715b5a0d65f2ba05b828d3216eabbc4e0bbe7fb1

Update imports

view details

Katie Mazaitis

commit sha a9eb30d5359087e941d4ab850a770b0c91081267

Merge pull request #574 from cmu-delphi/safegraph-export Refactor safegraph to use export util

view details

push time in an hour

PR merged cmu-delphi/covidcast-indicators

Refactor safegraph to use export util

Description

Migrate safegraph to use the common export function rather than an indicator-specific one. Tested by storing the test output from original code and diff'ing against the new code to make sure nothing changed.

Changelog

Itemize code/test/documentation changes and files added/removed.

  • process_window() now calls delphi_utils.export.create_export_csv() instead of to_csv()
  • intermediate timestamp column is added so the util function can be used.

Fixes

  • Partially addresses #497
+8 -6

0 comment

1 changed file

chinandrew

pr closed time in an hour

delete branch cmu-delphi/covidcast-indicators

delete branch : safegraph-hrr-msa

delete time in an hour

push eventcmu-delphi/covidcast-indicators

andrew

commit sha ea320565ae17334c7f11443c52c850f2fc112d1f

Add HRR and MSA

view details

andrew

commit sha f6d8fc2cde0905f2f9fc8debc6bc3e196e7fb7f0

Update tests

view details

Katie Mazaitis

commit sha a627e1c023593f6c61ddbac08b465398d66464a1

Merge pull request #573 from cmu-delphi/safegraph-hrr-msa Add HRR and MSA to safegraph

view details

push time in an hour

PR merged cmu-delphi/covidcast-indicators

Add HRR and MSA to safegraph

Description

Adding additional georesolutions to safegraph signal. See #114 for context.

Changelog

Itemize code/test/documentation changes and files added/removed.

  • Add HRR and MSA to constants file
  • Add conditional statements to map to HRR and MSA
  • Add HRR and MSA tests, and add a dimension check to existing tests since I had a typo where I was mapping to MSA instead of HRR and the HRR tests still passed with the original checks.

Fixes

  • Fixes #114
+48 -6

0 comment

3 changed files

chinandrew

pr closed time in an hour

issue closedcmu-delphi/covidcast-indicators

Add MSA and HRR levels to SafeGraph data

Though we do not ourselves use MSA-level data for forecasting, it may be of general interest to journalists and researchers interested in studying outbreaks in specific cities. Currently the SafeGraph pipeline does not do any aggregation to MSAs, only to states.

We should also add HRRs for completeness, since anyone using the HRR-level data from other sources may want to compare against HRR-level data from SafeGraph.

@jsharpna, maybe this is something that would be easy for you to do as you refactor the Python pipelines to use common geographic aggregation functions?

closed time in an hour

capnrefsmmat

delete branch cmu-delphi/covidcast-indicators

delete branch : large_spikes_validator

delete time in an hour

push eventcmu-delphi/covidcast-indicators

Ananya Ashish Joshi

commit sha 14d85be834a520a168521e2b488f493bc095e66d

finished outliers and tests

view details

Ananya Ashish Joshi

commit sha 7ac7964b817e7d238e194714d14800edd972d1f4

finished rebase and updating tests

view details

Ananya Ashish Joshi

commit sha 5acde059cc4b3df0e8c5271c2a019197f12e5313

finished outliers and tests

view details

Ananya Ashish Joshi

commit sha f19cb97324d51998c9312cabfd1a0a1ab782b41b

finished rebase and updating tests

view details

Ananya Ashish Joshi

commit sha 8cd7f8932fb94ba8480ca22b91ad9b764adb57b0

changes from Nat's code review

view details

Ananya Ashish Joshi

commit sha 08dc6a5138e9a4c99f4c462e366b568d40889eb4

rebased code

view details

Ananya Ashish Joshi

commit sha 5ca4406e0c307b4938e9b5450d06195c75a1f030

added more comments

view details

Ananya-Joshi

commit sha e509ab210267b774860c84774291980ded61396c

Update validator/delphi_validator/validate.py Co-authored-by: nmdefries <42820733+nmdefries@users.noreply.github.com>

view details

Ananya-Joshi

commit sha ce7db0a1c00c13a79f6cdb2d0f5dc94c1aa2460f

Update validator/delphi_validator/validate.py Co-authored-by: nmdefries <42820733+nmdefries@users.noreply.github.com>

view details

Ananya-Joshi

commit sha e9b9537f233f97faebb851bf1166759826ec1852

Update validator/delphi_validator/validate.py Co-authored-by: nmdefries <42820733+nmdefries@users.noreply.github.com>

view details

Ananya-Joshi

commit sha ef1f28217c29ede7e41ec15b7acbff23aea13384

Update validator/delphi_validator/validate.py Co-authored-by: nmdefries <42820733+nmdefries@users.noreply.github.com>

view details

Ananya-Joshi

commit sha f3931b4c0377487b7215ad0bb207a54d489027bb

Update validator/delphi_validator/validate.py Co-authored-by: nmdefries <42820733+nmdefries@users.noreply.github.com>

view details

Ananya-Joshi

commit sha 7cd087213f24541cf37874611a685bcc18ffdbe3

Update validator/delphi_validator/validate.py Co-authored-by: nmdefries <42820733+nmdefries@users.noreply.github.com>

view details

Ananya-Joshi

commit sha e259fb8c0e8c4ce46260244409c8db33900d21a8

Update validator/tests/test_checks.py Co-authored-by: nmdefries <42820733+nmdefries@users.noreply.github.com>

view details

Ananya Ashish Joshi

commit sha b1f00b0dcad4456cc462dbf1cac8b53e79e8ba99

changes from the PR

view details

Ananya Ashish Joshi

commit sha 730535f1c3751237d5cfaa56de9348784fea1067

changed the names of the tests correctly

view details

Ananya Ashish Joshi

commit sha 74c5a26541952b5175e0bb11e147cb7afd0dc747

added a new test when the source and api frames overlap

view details

Ananya Ashish Joshi

commit sha bf8cf181b6bfee25f4b7d1a160f899332db8df92

pylint on tests cleanup

view details

Ananya Ashish Joshi

commit sha 38bbe4497a084946bc6048ea13099a3f9f625fb4

formatting change

view details

Ananya Ashish Joshi

commit sha 7cd0306b33256ac31e3f074a9ef96dba1582ecd5

removed remaining print statement

view details

push time in an hour

PR merged cmu-delphi/covidcast-indicators

Large spikes validator

Modified R Code from Dan to be added to the Validator

+369 -13

1 comment

4 changed files

Ananya-Joshi

pr closed time in an hour

push eventcmu-delphi/covidcast-indicators

Nat DeFries

commit sha a6d3b0108d980eaa2bbb3778a7cee4e8b0def25a

everything working except write_to_api feature. add TODOs to address

view details

push time in 19 hours

delete branch cmu-delphi/covidcast-indicators

delete branch : nchs

delete time in 2 days

create barnchcmu-delphi/covidcast-indicators

branch : nchs

created branch time in 2 days

push eventcmu-delphi/covidcast-modeling

Addison Hu

commit sha bf40c8df094574ea48b59d6c510b9b9e939e8f93

update

view details

Addison Hu

commit sha 47009102dde3eca1430ec9bdd6d0be74a3389c07

add sanity check plots

view details

push time in 5 days

push eventcmu-delphi/covidcast-indicators

Nat DeFries

commit sha 05e3b7ca6cf045668e67f502562ee9201a7f9975

debugging with example data. move aggregation specifications to code

view details

push time in 5 days

issue openedcmu-delphi/covidcast-modeling

DAP: Leading Indicators

  • [ ] Stakeholder: @INSERT_GITHUB_NAME
  • [ ] Advisor(s): @INSERT_GITHUB_NAME
  • [ ] Analyst: @INSERT_GITHUB_NAME <!-- Add as Assignees on this issue) -->

DAP Deliverable: <!-- (text or link to doc) --> An explicit description of what deliverables a successful DAP for this topic must contain; to be written by the stakeholder

Timeline: <!-- (text or link to doc) --> The timeline to adhere to in terms of completing the DAP, ideally with one or more "milestones" as intermediate check-ins with the stakeholder.

created time in 5 days

Pull request review commentcmu-delphi/covidcast-indicators

Check for duplicate rows

 def test_invalid_geo_id_national(self):         assert "US" not in validator.raised_errors[0].expression         assert "SP" not in validator.raised_errors[0].expression +class TestDuplicatedRows:+    params = {}

Copied params from another test, see if that resolves it. If it's possible to set reasonable defaults for any of these values, we should add that to our TODOs.

JedGrabman

comment created time in 5 days

Pull request review commentcmu-delphi/covidcast-indicators

Check for duplicate rows

  ### Starter/small issues -* Check for duplicate rows

Done

JedGrabman

comment created time in 5 days

issue commentcmu-delphi/covidcast-indicators

Develop Change Healthcare data source (counts signal)

Not sure what the last bullet point above means, but is this done with #479? (or if further development is desired, should we make separate specific tickets?)

krivard

comment created time in 5 days

issue closedcmu-delphi/covidcast-indicators

JHU MSA has inf values in some _prop signals

In receiving, for many msa level wip files with prop, there are inf values. The cause is that the aggregation (which should be happening on 'population') yields some 0 populations, when run through GeoMapper. Here is an example of some rows...

10180,inf,,,1012.7696318559967,,
10380,inf,,,,,
10420,inf,,,556.8447266067241,,
10500,inf,,,2591.1270959864937,,
10540,inf,,,187.28468042142902,,

closed time in 5 days

jsharpna

issue commentcmu-delphi/covidcast-indicators

JHU MSA has inf values in some _prop signals

@krivard I tested for any inf msa values in the output of my local jhu from the deploy_jhu branch and found none, so this seems fixed.

closing given this comment.

jsharpna

comment created time in 5 days

issue commentcmu-delphi/covidcast-indicators

Geo refactor for Quidel covidtests

Is this resolved?

krivard

comment created time in 5 days

issue closedcmu-delphi/covidcast-indicators

Resolve Jenkins linter regression problem in JHU

#272 failed to check due to a regression in astroid, a component of pylint. This appears to only be a problem for certain version combinations of python, pylint, and pandas. We have temporarily disabled linting in the JHU indicator (#274), but that's not a good long-term solution.

Figure out what's up, decide how to fix it, and repair the Jenkins job for JHU.

closed time in 5 days

krivard

issue commentcmu-delphi/covidcast-indicators

Resolve Jenkins linter regression problem in JHU

This was fixed in #333

krivard

comment created time in 5 days

issue closedcmu-delphi/covidcast-indicators

Clean up docstrings to pass pydocstyle linter (PEP257)

Both in anticipation of running a docstring linter and also just to standardize everything.

indicator status
cdc_covidnet #374
~cds~[1] ~NA~
claims_hosp #379
combo_cases_and_deaths #375
doctor_visits blocked - waiting on merge to main
~emr_hosp~ ~NA~
facebook[3] ~NA~
~google_health~[4] ~NA~
jhu not touching yet since lots of stuff in progress
nchs_mortality #376
quidel #372
quidel_covidtest #371
safegraph #373
safegraph_patterns #378
usafacts #377

closed time in 5 days

chinandrew

issue commentcmu-delphi/covidcast-indicators

Clean up docstrings to pass pydocstyle linter (PEP257)

All PRs are completed and this is now tracked in #452

chinandrew

comment created time in 5 days

issue commentcmu-delphi/covidcast-indicators

Remove deprecated google health indicator directory

Now that google symptoms is released in #577, is this unblocked? Are there any additional steps that need to be taken to remove from production?

@krivard

chinandrew

comment created time in 5 days

more