profile
viewpoint
Jake Mensch jmensch1 Los Angeles

jmensch1/pm2-git-hook 3

automated webhooks for pm2

jmensch1/atom-pong 1

pong in the atom text editor

jmensch1/email-endpoint 1

Instantly create a live endpoint that sends emails to a specified address.

jmensch1/react-mapbox 1

basic demo of mapbox with react

gvost/civilianagency-staic 0

Static site for Civilian Agency

jmensch1/311-data 0

Empowering Neighborhood Associations to improve the analysis of their initiatives using 311 data

jmensch1/react-navigation 0

Routing and navigation for your React Native apps

push eventhackforla/ballotnav

aNullValue

commit sha 52c06329fc8bd889716002e5657e432199cd98a8

Corrects deficiencies, provides logical json views

view details

aNullValue

commit sha 6c597e36620bdfd8f4af6a4ef7a84b6781724f96

Adds sample data and manual loading instructions

view details

Jake Mensch

commit sha fae9b2138fbe8d832f8c23e3234327c494a105bc

Merge pull request #141 from hackforla/db-migrations-202009261438 database improvements

view details

push time in 8 hours

PR merged hackforla/ballotnav

Reviewers
database improvements
  • corrects some models/tables to split date/time into separate columns because we cannot necessarily know UTC offset for times
  • adds pg-level validation of text columns that must contain a "time" value (i.e., "23:59", "11:59 pm", "3:04 am", etc)
  • provides default now() to all updated_at and created_at columns that sequelize creates
  • adds triggers to ensure fipsCode and fipsNumber columns are always in sync
  • creates views that return precompiled json responses: jurisdiction_json, state_json, state_with_jurisdictions_json, states_json
  • adds sample data with manual loading instructions
+700 -16

0 comment

13 changed files

aNullValue

pr closed time in 8 hours

PullRequestReviewEvent

push eventhackforla/ballotnav

jafow

commit sha 688ee025527581a0a37e8ee340b55f9e932f7670

install logging package

view details

jafow

commit sha 7f94907968fba0b8c0d437277bad7ac414557bba

setup logging

view details

jafow

commit sha a7fdb51717f9d98176905e67fc35d2541da304ae

add logging on register route

view details

jafow

commit sha 9c9135c1d7feced4d92271035c7e9226cdaf7f59

fmt

view details

Jake Mensch

commit sha 710b76fffd2c4f6e6d5a235522aa5793e6b10e2b

Merge pull request #140 from hackforla/backend/logging setup backend logging

view details

push time in 18 hours

PR merged hackforla/ballotnav

Reviewers
setup backend logging

setup winston and logging on user routes

+239 -3

0 comment

4 changed files

jafow

pr closed time in 18 hours

PullRequestReviewEvent

PR opened hackforla/ballotnav

Reviewers
using umzug for migrations when server starts (instead of shell script)

Uses umzug (JS library) to migrate when server starts instead of the entrypoint script. Migration doesn't run if that env var is set.

This seems to have fixed the problem @aNullValue discovered where the server wouldn't start the first time you run docker because it wasn't waiting for the DB. I was able to recreate that issue using:

docker-compose down --volumes 
docker-compose up

But after this change it wasn't happening.

+31 -3

0 comment

5 changed files

pr created time in a day

create barnchhackforla/ballotnav

branch : JavascriptMigration

created branch time in a day

push eventhackforla/ballotnav

aNullValue

commit sha 4b3cddf2dd1300a0f6b57b23324987ebc2083d52

Provides initial database migration

view details

aNullValue

commit sha 90b2ad3d92a060242ed20ad96f5ee979c0edc722

Formatting

view details

Jake Mensch

commit sha cc5096480595ba8719f3a4de7c249b352715943e

Merge pull request #138 from hackforla/db-migrations-init Provides initial database migration

view details

push time in a day

PR merged hackforla/ballotnav

Reviewers
Provides initial database migration
+1045 -16

0 comment

5 changed files

aNullValue

pr closed time in a day

PullRequestReviewEvent

Pull request review commenthackforla/ballotnav

Provides initial database migration

+'use strict'++const fs = require("fs");+const init_up_sql = fs.readFileSync("./migrations/sql/20200925163300-init-up.sql", {encoding: "utf-8",});+const init_down_sql = fs.readFileSync("./migrations/sql/20200925163300-init-down.sql", {encoding: "utf-8",});

can you just change these vars to camelCase and then run npm run format so we get consistent code styling? (It only effects JS files). Otherwise this PR looks great.

aNullValue

comment created time in a day

PullRequestReviewEvent

issue commenthackforla/311-data

Update NC Boundaries per City

Accounts have been mentioned but it’s always been pretty vague. I don’t think we’ve ever had a clear picture of what kind of user-specific functionality we want to provide.

On Fri, Sep 25, 2020 at 1:35 PM Regan notifications@github.com wrote:

I agree.

If we don’t have user logins then there would have to be a way to unsubscribe.

Has there been discussion of supporting user accounts?

On Sep 25, 2020, at 10:48 AM, Jake Mensch notifications@github.com wrote:

Ohh nice, that’s a great idea

On Fri, Sep 25, 2020 at 10:13 AM Matt Webster notifications@github.com wrote:

Building on @jmensch1 https://github.com/jmensch1 here, what I'd really

like to do as a user is set up an immediate area around me and "watch" it

for new 311 calls. For instance, to get an alert whenever there's a new graffiti report on my block.

For me, and many people I suspect, the NC is still too big of an area. I

live in DTLA but that doesn't mean I care about what's going on around the

Staples Center which is 12 blocks from me. The NC is fine as an organizational unit and way to get the attention of the city, but I think

in order to really engage people we're going to need to have some way of

supporting smaller geographic units.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/hackforla/311-data/issues/807#issuecomment-699048164>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/ACFYMD6XCCO3N6IKKPFMCTDSHTFR3ANCNFSM4RL6NTWQ

.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or unsubscribe.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hackforla/311-data/issues/807#issuecomment-699142414, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACFYMD74ZIICT3Z6U7HGIKDSHT5ILANCNFSM4RL6NTWQ .

johnr54321

comment created time in a day

issue commenthackforla/311-data

Update NC Boundaries per City

Ohh nice, that’s a great idea

On Fri, Sep 25, 2020 at 10:13 AM Matt Webster notifications@github.com wrote:

Building on @jmensch1 https://github.com/jmensch1 here, what I'd really like to do as a user is set up an immediate area around me and "watch" it for new 311 calls. For instance, to get an alert whenever there's a new graffiti report on my block.

For me, and many people I suspect, the NC is still too big of an area. I live in DTLA but that doesn't mean I care about what's going on around the Staples Center which is 12 blocks from me. The NC is fine as an organizational unit and way to get the attention of the city, but I think in order to really engage people we're going to need to have some way of supporting smaller geographic units.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hackforla/311-data/issues/807#issuecomment-699048164, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACFYMD6XCCO3N6IKKPFMCTDSHTFR3ANCNFSM4RL6NTWQ .

johnr54321

comment created time in 2 days

push eventhackforla/ballotnav

jafow

commit sha 365bbdc44cce4dbbb84ebe62cf682d962db0ba23

add configuration options to support cloud deployment; we don't want to use the config.json file for prod db connection params and instead take from the env.

view details

jafow

commit sha b8929d92776f67abd9a27baa4da4ff861442fd0d

add dynamic config for prod db setup

view details

jafow

commit sha 4539246fd92d9a72c943d0771624099e14f9b6f1

update module alias

view details

jafow

commit sha 89512c564db79fb94a9be78e0acfa49aef0c75da

use config.js specifically

view details

jafow

commit sha a353c3a0f2d725725654bca682a8b5f00e51e855

db: allow connection override with env vars

view details

jafow

commit sha 3b5c814ba35864b691c84cca5013c8dab63020fe

move sequelize meta table to config.js; we'll use a js file instead of json so we can do dynamic lookup from env var

view details

jafow

commit sha 50a619759fa9ce4ec45fc4ebd42c5add710608f8

fix method call on toLowerCase

view details

jafow

commit sha 5bcdc8b2bd91a0c7194cfa61a4cd8e8dab1b9de8

fix the config module alias

view details

jafow

commit sha b8231c5c795a6705074851eb04a847d99a4266c1

format

view details

Jake Mensch

commit sha f990963bdd582f65d2e80d2bc1442d67eccc2d58

Merge pull request #137 from hackforla/db-config update db config options for deployment;

view details

push time in 2 days

PR merged hackforla/ballotnav

Reviewers
update db config options for deployment;

we allow passing secrets via environment vars so we dont have to code it into the json file. yay!

+48 -11

0 comment

10 changed files

jafow

pr closed time in 2 days

PullRequestReviewEvent

Pull request review commenthackforla/ballotnav

update db config options for deployment;

     "prettier": "2.1.2"   },   "_moduleAliases": {-    "@config": "./config/config.json",+    "@config": "./config/config.js",

like in app.js

jafow

comment created time in 2 days

PullRequestReviewEvent

Pull request review commenthackforla/ballotnav

update db config options for deployment;

     "prettier": "2.1.2"   },   "_moduleAliases": {-    "@config": "./config/config.json",+    "@config": "./config/config.js",

hmm, not sure. just fyi require('module-alias/register') has to be called at some point in the execution path for the aliases to work

jafow

comment created time in 2 days

PullRequestReviewEvent

Pull request review commenthackforla/ballotnav

update db config options for deployment;

 require('module-alias/register') const fs = require('fs') const Sequelize = require('sequelize') const env = process.env.NODE_ENV || 'development'-const config = require('@config')[env]+const config = require('../config/config.js')[env]++if (env.toLowerCase === 'production') {

oh I just meant the () on the function

jafow

comment created time in 2 days

PullRequestReviewEvent

Pull request review commenthackforla/ballotnav

update db config options for deployment;

 require('module-alias/register') const fs = require('fs') const Sequelize = require('sequelize') const env = process.env.NODE_ENV || 'development'-const config = require('@config')[env]+const config = require('../config/config.js')[env]++if (env.toLowerCase === 'production') {

does this need to be env.toLowerCase()?

jafow

comment created time in 2 days

PullRequestReviewEvent

Pull request review commenthackforla/ballotnav

update db config options for deployment;

+const path = require('path');++module.exports = {+  'config': path.resolve('config', 'config.js')+}

nice! this will make the config more flexible

jafow

comment created time in 2 days

PullRequestReviewEvent

push eventhackforla/ballotnav

aNullValue

commit sha 020fc0438a09f10de14cc7b6392c9763980e51d1

Amends models

view details

aNullValue

commit sha 4691bebb3bb32578c09f48aeed90c446b40790ab

Normalizes migration storage table name

view details

aNullValue

commit sha 92a0b8fb3d62bcb73b13405252dea5b27cec2e5e

Removes broken seeders

view details

Jake Mensch

commit sha 1cfe34a088da65c4d36cb6191ab8a99a340d7df1

Merge pull request #135 from hackforla/model-work-202009242200 Database work

view details

push time in 2 days

PullRequestReviewEvent

push eventhackforla/ballotnav

aNullValue

commit sha 1e7142e090db461df43d458107a42aa01d9198f5

Fixed docker-compose per Jared

view details

Jake Mensch

commit sha 0b6f529f0de4a4603a70d34e67a3499ce26d681f

Merge pull request #136 from hackforla/docker-update Fixed docker-compose per Jared

view details

push time in 2 days

PR merged hackforla/ballotnav

Reviewers
Fixed docker-compose per Jared
+2 -1

0 comment

1 changed file

aNullValue

pr closed time in 2 days

PullRequestReviewEvent

issue commenthackforla/311-data

Update NC Boundaries per City

So the only specific functionality I had in mind was a location-based search in regards to "version 2". Like finding all requests within a certain radius of the user's location. I imagine there's other cool stuff we could do with it, seems like cool tech I just haven't explored it very much.

On Thu, Sep 24, 2020 at 7:15 PM Regan notifications@github.com wrote:

@jmensch1 https://github.com/jmensch1 What functionality are you thinking of opening up with PostGIS? It seems like Mapbox is opening up a lot of new options from what was in the demo you shared. Ex. Calculating the total of each category within a radius of the address entered. Are there things that mapbox won't support or are there interfaces where we wouldn't want to use mapbox?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hackforla/311-data/issues/807#issuecomment-698681710, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACFYMD2YF66BOQ72GAODZ3LSHP4L7ANCNFSM4RL6NTWQ .

johnr54321

comment created time in 2 days

issue commenthackforla/311-data

Update NC Boundaries per City

We haven't discussed GIS extensions for Postgres but I think it's a great idea and would probably open a bunch of doors. There's some code in ballotnav that might be helpful. https://github.com/hackforla/ballotnav/blob/master/backend/db/create-postgis-extension.sql

johnr54321

comment created time in 2 days

push eventhackforla/ballotnav

aNullValue

commit sha 2a58ebd842109dca72532c07dab99c7d9ce6d222

Creates new models (draft)

view details

aNullValue

commit sha 93e4d6309078fe221fcf2788eabb71eac895a6b5

Corrects typo and formats

view details

aNullValue

commit sha 9be5ecd641391d193d55965d2247d815cbfecfb8

File case change part 1

view details

aNullValue

commit sha 95cb65ecf4b8e1526a4bb17dc0d48b315c0616a5

File case change part 2

view details

Jake Mensch

commit sha 30664f4f9551fb7fa8fd1424202c37da0051a862

Merge pull request #134 from hackforla/new-models Creates new models (draft)

view details

push time in 2 days

PR merged hackforla/ballotnav

Reviewers
Creates new models (draft)
+2832 -90

0 comment

32 changed files

aNullValue

pr closed time in 2 days

PullRequestReviewEvent

PR closed hackforla/ballotnav

Reviewers
Creates new models (draft)
+2597 -105

0 comment

31 changed files

aNullValue

pr closed time in 2 days

push eventhackforla/ballotnav

Jake Mensch

commit sha 8ac1cc660460bfdc46e4df4224e9c15e9e1e52a3

controllers fix

view details

push time in 3 days

PR opened hackforla/ballotnav

Reviewers
role-based auth

Very simple role-based auth, based on slack discussion. Here's how it works:

  1. every user is either 'volunteer' or 'admin'. The role is stored in an enum column the users table.

  2. There's no distinction between roles and capabilities. Every volunteer can do all volunteer stuff, every admin can do all admin stuff.

  3. on registration, a user is made a volunteer. If we want to upgrade them we do it manually in the DB.

  4. on registration or login, the user's role is added to their auth token. When they hit a route that requires auth, we send a 401 if they don't have a valid token, or a 403 if they have a token but not one of the allowed roles for that route.

  5. The auth requirements are specified in the router, which looks like this:

const router = require('express').Router()
const control = require('@controllers/jurisdictions')
const auth = require('@middleware/auth')

router.get('/', control.list)
router.post('/', auth(['volunteer', 'admin']), control.create)
router.delete('/', auth(['admin']), control.delete)

This router says that (1) anyone can list jurisdictions (no auth required), (2) volunteers and admins can create jurisdictions, (3) only admins can delete jurisdictions.

+63 -25

0 comment

11 changed files

pr created time in 3 days

create barnchhackforla/ballotnav

branch : SuperSimpleAuthRoles

created branch time in 3 days

PR opened hackforla/ballotnav

Reviewers
added aNullValue as codeowner
+1 -2

0 comment

1 changed file

pr created time in 3 days

create barnchhackforla/ballotnav

branch : CodeOwners

created branch time in 3 days

push eventjmensch1/ballotnav

Jake Mensch

commit sha 03586d5478a61d2f655b62b4ba62eeadb3b56fba

fixed production api url

view details

push time in 3 days

push eventhackforla/ballotnav

jafow

commit sha 77a7337aa529014e2093bd5e8b3736433ba46c0d

updates to dockerfile for better startup; we no longer need to run the migrations in entrypoint; remove npm from the dockerfile command;

view details

jafow

commit sha f61bbb62fe471932017c613a7713e93d3d3c65b4

Revert "updates to dockerfile for better startup;" This reverts commit 77a7337aa529014e2093bd5e8b3736433ba46c0d.

view details

jafow

commit sha b7b3ff751ad92b6b9a4bb63550b5b4a15c8809ee

adds back entrypoint, we do need this for migrations

view details

jafow

commit sha a0fc98636961683076ab340035c444dee86e9881

use postgres image with gis extension; we can use this for local dev to test & write the geocode lookups

view details

Jake Mensch

commit sha c306d14e8f1f397a664ab29fff2dc64b7551af08

Merge pull request #129 from hackforla/dockerfixup some docker fixups that @aNullValue discovered

view details

push time in 4 days

PR merged hackforla/ballotnav

Reviewers
some docker fixups that @aNullValue discovered
+17 -3

3 comments

3 changed files

jafow

pr closed time in 4 days

PullRequestReviewEvent

Pull request review commenthackforla/ballotnav

some docker fixups that @aNullValue discovered

 services:     command: npm run dev    db:-    image: postgres:11+    image: jred/postgres:11-postgis

nice!

jafow

comment created time in 4 days

PullRequestReviewEvent

Pull request review commenthackforla/ballotnav

admin auth: server and client

+const bcrypt = require('bcrypt')+const jwt = require('jsonwebtoken')++//// CONFIG ////++const TOKEN_SECRET = process.env.TOKEN_SECRET+const TOKEN_EXPIRY = Number(process.env.TOKEN_EXPIRY)+const SALT_ROUNDS = 10++//// HELPERS ////++async function hashPassword(password) {+  return await bcrypt.hash(password, SALT_ROUNDS)+}++async function checkPassword(password, passwordHash) {+  return await bcrypt.compare(password, passwordHash)+}++async function createToken(claims) {+  return await jwt.sign(claims, TOKEN_SECRET, { expiresIn: TOKEN_EXPIRY })+}++async function decodeToken(token) {+  const claims = await jwt.verify(token, TOKEN_SECRET)+  delete claims.iat+  return claims+}++//// EXPORTS ////++exports.getUser = async (req, res) => res.json(req.user)++exports.register = async (req, res, next) => {+  const { firstName, lastName, email, password } = req.body++  const existingUser = await req.db.User.findOne({ where: { email } })+  if (existingUser) return res.json({ duplicateEmail: true })++  try {+    const passwordHash = await hashPassword(password)+    const user = await req.db.User.create({+      firstName,+      lastName,+      email,+      passwordHash,+    })+    delete user.passwordHash+    const token = await createToken({ user })+    return res.json({ isSuccess: true, token, user })+  } catch (e) {+    return res.status(400).send(e)+  }+}++exports.login = async (req, res, next) => {+  const { email, password } = req.body++  const user = await req.db.User.findOne({ where: { email } })+  if (!user) return res.json({ emailNotFound: true })

The '/login' route returns a 200 if email is not found. But we return a 401 if the user ever attempts to access a route requiring authentication, and their request doesn't include a valid, unexpired token (see @middleware/authenticate).

My thinking is that we should return a 200 in the email-not-found scenario because because the /login route itself doesn't require a token, and it would confuse things a bit to return a 401 for a route not requiring a token.

jmensch1

comment created time in 4 days

PullRequestReviewEvent

pull request commenthackforla/ballotnav

some docker fixups that @aNullValue discovered

Ok, yeah I was thinking that the sync would be temporary. Once we get some real data I think we're gonna need to switch to the migrations.

jafow

comment created time in 4 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commenthackforla/ballotnav

some docs is better than no docs

+# ballotnav backend+source code and scripts for the ballotnav application++# prerequisites+for local development with a full webserver and postgres instance within docker, ensure the following are installed:+- docker at v19.03+ [Install instructions by platform](https://docs.docker.com/get-docker/)++## getting started+`cd` to the `backend` folder and in a terminal do:++```bash+docker-compose up --build+```++:eyes: The `--build` option is necessary for the first time you boot the app, as it+builds the application and its dependencies (`node_modules`) but you+can drop it on subsequent start ups if you've not added new packages. ++**example docker-compose up**+```bash+$ docker-compose up+Starting backend_db_1 ... done+Recreating backend_ballotnav_1 ... done+Attaching to backend_db_1, backend_ballotnav_1+ballotnav_1  | Checking status of pending migrations:+db_1         |+db_1         | PostgreSQL Database directory appears to contain a database; Skipping initialization+db_1         |+db_1         | 2020-09-23 02:46:16.469 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432+db_1         | 2020-09-23 02:46:16.469 UTC [1] LOG:  listening on IPv6 address "::", port 5432+db_1         | 2020-09-23 02:46:16.493 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"+db_1         | 2020-09-23 02:46:16.636 UTC [25] LOG:  database system was shut down at 2020-09-20 19:56:47 UTC+db_1         | 2020-09-23 02:46:16.688 UTC [1] LOG:  database system is ready to accept connections+ballotnav_1  |+ballotnav_1  | Sequelize CLI [Node: 12.18.4, CLI: 6.2.0, ORM: 6.3.5]+ballotnav_1  |+ballotnav_1  | Loaded configuration file "config/config.json".+ballotnav_1  | Using environment "development".+ballotnav_1  | up 20200919055731-state-and-county-tables.js+ballotnav_1  | up 20200919061608-create-state.js+ballotnav_1  | up 20200920042755-initial-sync.js+ballotnav_1  | up 20200920054750-seed-states.js+ballotnav_1  | up 20200920061547-seed-jurisdictions.js+ballotnav_1  | up 20200920064108-seed-locations.js+ballotnav_1  | Running sequelize migrations...+ballotnav_1  |+ballotnav_1  | Sequelize CLI [Node: 12.18.4, CLI: 6.2.0, ORM: 6.3.5]+ballotnav_1  |+ballotnav_1  | Loaded configuration file "config/config.json".+ballotnav_1  | Using environment "development".+ballotnav_1  | No migrations were executed, database schema was already up to date.+ballotnav_1  | Success applying migrations+ballotnav_1  |+ballotnav_1  | > ballot-nav-backend@1.0.0 dev /ballotnav+ballotnav_1  | > npx nodemon -L app.js+ballotnav_1  |+ballotnav_1  | [nodemon] 2.0.4+ballotnav_1  | [nodemon] to restart at any time, enter `rs`+ballotnav_1  | [nodemon] watching path(s): *.*+ballotnav_1  | [nodemon] watching extensions: js,mjs,json+ballotnav_1  | [nodemon] starting `node app.js`+ballotnav_1  | App listening on port 8080+```++### development+The `docker-compose` should launch a database instance running on 127.0.0.1:5432+and an express webserver at 127.0.0.1:8080.

Should we mention that the host port is 5434?

jafow

comment created time in 4 days

PullRequestReviewEvent

issue openedhackforla/ballotnav

deep-linking search results

Overview

The URL on the search results page needs to completely determine the results so that we can support bookmarking and linking directly to the page from other sites. Since there are multiple ways to arrive at that page, we'll need to support several different formats for that URL.

1. /search?jurisdiction_id=1234&lng=X&lat=Y

Arrival path: The user would arrive at this URL if they start on the home page and enter an address or use "Current Location".

Behavior: When the user uses the search bar or "Current Location" on the home page, we get their jurisdiction id and lng/lat from the backend and then send them to the search results page at the URL above. Once on the search results page, we'd send the jurisdiction id to the backend to get the complete list of locations in the jurisdiction. And then on the frontend we'd sort those results according to distance from the lng/lat in the URL. (The lng/lat would not be included in the API call because then the calls wouldn't be cacheable).

2. /search?state_abbr=CA&county=orange

Arrival path: Two ways to get here: (1) the user selects their state/county from dropdowns instead of using the search bar, (2) linking from other sites.

Behavior: In this case we'd send the state abbreviation and county to the backend instead of the jurisdiction id. If we can convert them to a jurisdiction id, we return the complete list of locations in the jurisdiction. (They would not be ordered by distance since we don't have the user's lng/lat.) If we can't convert their query to a jurisdiction id then we'd send them to the error page.

3. /search?fips_code=56378

Arrival path: This would only be used for linking from other sides.

Behavior: Same as # 2 except we send the fips code to the backend instead of the state/county.

Resources/Instructions

For ordering the results in #1 above we can use the haversine (i.e. great circle) distance. Here's a javascript implementation. https://stackoverflow.com/questions/27928/calculate-distance-between-two-latitude-longitude-points-haversine-formula

created time in 4 days

Pull request review commenthackforla/ballotnav

admin auth: server and client

+const db = require('@models')++function database(req, res, next) {

yeah exactly, I saw that pattern in food oasis and thought it was smart

jmensch1

comment created time in 4 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commenthackforla/ballotnav

admin auth: server and client

+'use strict'+module.exports = {+  up: async (queryInterface, Sequelize) => {+    await queryInterface.createTable('Users', {+      id: {+        allowNull: false,+        autoIncrement: true,+        primaryKey: true,+        type: Sequelize.INTEGER,+      },+      email: {+        type: Sequelize.STRING,+      },+      password_hash: {

so the files in _misc aren't being used right now, I'm just saving them for when I make the migrations real. I'll def fix that when I do

jmensch1

comment created time in 4 days

Pull request review commenthackforla/ballotnav

admin auth: server and client

+REACT_APP_API_URL=https://api.ballotnav.com

ok thanks

jmensch1

comment created time in 4 days

PullRequestReviewEvent

delete branch jmensch1/ballotnav

delete branch : AdminAuthentication

delete time in 4 days

delete branch jmensch1/ballotnav

delete branch : AdminAuth

delete time in 4 days

PR opened hackforla/ballotnav

Reviewers
admin auth: server and client
+18080 -158

0 comment

62 changed files

pr created time in 4 days

create barnchjmensch1/ballotnav

branch : AdminAuthStuff

created branch time in 4 days

create barnchjmensch1/ballotnav

branch : AdminAuth

created branch time in 4 days

create barnchjmensch1/ballotnav

branch : AdminAuthentication

created branch time in 4 days

Pull request review commenthackforla/ballotnav

Backend: core models

+'use strict'+module.exports = {+  up: async (queryInterface, Sequelize) => {+    await queryInterface.createTable('Locations', {+      id: {+        allowNull: false,+        autoIncrement: true,+        primaryKey: true,+        type: Sequelize.INTEGER,+      },+      jurisdiction_id: {+        allowNull: false,+        type: Sequelize.INTEGER,+      },+      location_name: {+        type: Sequelize.STRING,+      },+      location_info: {+        type: Sequelize.STRING,+      },+      address_1: {+        type: Sequelize.STRING,+      },+      address_2: {+        type: Sequelize.STRING,+      },+      address_3: {+        type: Sequelize.STRING,+      },+      contact_name: {+        type: Sequelize.STRING,+      },+      contact_email: {+        type: Sequelize.STRING,+      },+      contact_fax: {+        type: Sequelize.STRING,+      },+      contact_phone: {+        type: Sequelize.STRING,+      },+      internal_note: {+        type: Sequelize.STRING,+      },+      latitude: {+        type: Sequelize.FLOAT,+      },+      longitude: {+        type: Sequelize.FLOAT,+      },+      is_early_dropoff_location: {+        type: Sequelize.BOOLEAN,+      },+      is_early_voting_location: {+        type: Sequelize.BOOLEAN,+      },+      is_elections_office: {+        type: Sequelize.BOOLEAN,+      },+      is_polling_location: {+        type: Sequelize.BOOLEAN,+      },+      scheduletype: {

I'd go with boolean/null, unless there's a chance we end up with more than three states.

jmensch1

comment created time in 4 days

PullRequestReviewEvent
PullRequestReviewEvent

PR opened hackforla/ballotnav

Reviewers
Backend: core models

This sets up the core models we (@jafow @theswerd @aNullValue) discussed last night, and sets up some basic code patterns that will be useful going forward.

Models

We've got models for State, Jurisdiction, and Location. The fields are based on the gist from @aNullValue (https://gist.github.com/aNullValue/b97e5d7645772a69c3a2238d0991f067). The relationships between them are: (1) State has many Jurisdictions and (2) Jurisdiction has many Locations.

Migrations/dummy data

The migrations are set up to run when the docker container starts (thanks @jared). They create three tables, one for each model, and populate each table with dummy data from faker.js.

I'm currently using a forced sync for the initial migration because the models will likely change a lot in the next few days and it's annoying and error-prone to have to keep two sets of files in sync for every little change (i.e. the model file and the migration file). I can convert these to proper migrations when things settle down a little. In the meantime there may be a bit of deleting/recreating the database while we get things in order.

Routing

We've got two routes for each model:

GET /states POST /states

GET /jurisdictions POST /jurisdictions

GET /locations POST /locations

The GET routes return all of the entries for the given model, and the POST routes create a single entry for each model (with validation). I can add some bulk-create routes if it's necessary. And note that I changed /dropoffs to /locations just to be consistent with the model names.

Other stuff

The app.js file is a lot cleaner:


require('module-alias/register')

const express = require('express')
const cors = require('cors')
const routes = require('@routes')
const errorHandler = require('@middleware/errorHandler')

const PORT = 8080

const app = express()
app.use(express.json())
app.use(cors())
app.use(routes)
app.use(errorHandler)

app.listen(PORT, () => console.log(`App listening on port ${PORT}`))

The details of the routes are now in the /routes folder.

Also I added some useful commands to the "scripts" section of the package.json, and added prettier.js for linting our code (I'd like to get that included in the CI if possible). And I'm using some module aliases (see package.json) cuz it's easier to write const db = require('@models') than const db = require('../../../models'), and also makes it easier to move things around if we need to.

+782 -244

0 comment

29 changed files

pr created time in 7 days

create barnchjmensch1/ballotnav

branch : BACK-CoreModels

created branch time in 7 days

issue commenthackforla/ballotnav

we need points!!

looks good! I can jump on this later today or maybe tomorrow

jafow

comment created time in 8 days

PullRequestReviewEvent

Pull request review commenthackforla/ballotnav

/seed, /dropoff and /dropoffs working

 const csv = require('csvtojson'); const fetch = require("node-fetch");  const app = express();+app.use(express.json()); const router = express.Router(); const port = 8080; const Dropoffs = require("./models/Dropoffs.js"); const db = require('./models/index');+const loadDropoffs = require('./seeders/loadDropoffs');  app.get("/status", (req, res) => {   res.send(`ok at ${new Date()}`) }) +app.get("/seed", (req, res, next) => {

good call on the migration script, I'll look into it

jmensch1

comment created time in 8 days

PR opened hackforla/ballotnav

/seed, /dropoff and /dropoffs working

This adds three endpoints:

GET /seed

Creates the Dropoffs table and adds 500 rows of dummy data. The data is from faker.js. Figure we can replace it with a csv file when we have some real data.

POST /dropoff

Adds a single dropoff to the table. Validates the input and returns the entry that was created.

POST /dropoffs

Adds an array of dropoffs to the table. Validates the input and returns an array of new entries.

+77 -9

0 comment

6 changed files

pr created time in 8 days

create barnchjmensch1/ballotnav

branch : BACK-SeedingAndDropoffEndpoints

created branch time in 8 days

PR opened hackforla/311-data

807 front nc boundary update

This replaces the old NC boundaries with the new ones that @JRHutson generated from the city's file.

@adamkendis @johnr54321 You can see the mismatch between the new boundaries and way socrata codes requests if you do a search for Noho. The requests are being displayed north of the boundary line because the boundary got moved a couple blocks south, but socrata doesn't know about that yet.

Screen Shot 2020-09-18 at 8 33 30 AM

  • [x] Up to date with dev branch
  • [x] Branch name follows guidelines
  • [ ] All PR Status checks are successful
  • [ ] Peer reviewed and approved

Any questions? See the getting started guide

+108 -1

0 comment

2 changed files

pr created time in 9 days

push eventhackforla/311-data

Jake Mensch

commit sha b8789a5e1c0b80ccd7b78b660fb2647395a0da42

deleted geojson with bad coords

view details

push time in 9 days

push eventhackforla/311-data

Jake Mensch

commit sha be7195905149ec348dc938ba7148ceba3af6f432

update geojson import in map component

view details

push time in 9 days

issue commenthackforla/311-data

Update NC Boundaries per City

Ok, yeah I see what you’re saying. So right now the backend doesn’t assign NC ids to individual requests — the city assigns the ids to the requests (in socrata) and we just assume that the city is correct. So i guess we’re talking about adding a step to the nightly update that assigns NC ids based on the new boundaries. Plus I guess we’ll need to recode the existing requests in the DB to reflect the new boundaries.

I wonder if the city is planning to use the new boundaries in socrata. And whether they plan to update the older requests in their own databases. On Thu, Sep 17, 2020 at 11:43 AM Adam Kendis notifications@github.com wrote:

@jmensch1 https://github.com/jmensch1 Might not be able to make tonight's meeting - unsure yet. I think we'll need to update some NC ids or the NC boundaries displayed on the map will be at odds with the data. Here's an example:

North Hollywood Northeast and NoHo NCs Our site: southern boundary is Vanowen. Updated boundary data: southern boundary is Victory (part of NoHo NC on our map).

See images below. It seems we'd need to change the NoHo requests falling between Victory and Vanowen to North Hollywood NE otherwise NoHo will display some requests that actually fall in North Hollywood NE. Updated North Hollywood NE boundary

[image: Screen Shot 2020-09-17 at 11 36 37 AM] https://user-images.githubusercontent.com/40484278/93513201-5f151e80-f8da-11ea-9191-8bc3447ceb3c.png Requests for North Hollywood NE

[image: Screen Shot 2020-09-17 at 11 31 47 AM] https://user-images.githubusercontent.com/40484278/93513189-5cb2c480-f8da-11ea-8662-a9d74af93e8e.png Requests for NoHo

[image: Screen Shot 2020-09-17 at 11 32 03 AM] https://user-images.githubusercontent.com/40484278/93513192-5d4b5b00-f8da-11ea-93d3-328020d2e79f.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hackforla/311-data/issues/807#issuecomment-694426374, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACFYMD5D3DGU3FRSQRATM3DSGJKEZANCNFSM4RL6NTWQ .

johnr54321

comment created time in 10 days

issue commenthackforla/311-data

Update NC Boundaries per City

@adamkendis Hey, I'm not sure yet what kind of reconciliation will be necessary. If there aren't any changes to the NC id's we may not need to do anything on the backend. Let's discuss tonight.

johnr54321

comment created time in 10 days

push eventhackforla/311-data

Jake Mensch

commit sha 1f0df0fe6ea8001041415635804dfe44727d2f24

logging 500 errors for all positive postman tests

view details

push time in 14 days

push eventhackforla/311-data

Jake Mensch

commit sha a066a0c88380554eb62c53b5b814ea55e00bded1

linting

view details

push time in 14 days

push eventhackforla/311-data

Jake Mensch

commit sha a66ea539d64d3aab7b847ecbd47810ef66b3a5ca

commented build_cache

view details

push time in 14 days

push eventhackforla/311-data

Jake Mensch

commit sha a9c970a0df0b686e099e6ad0fb9e23141d995d35

more postman logging

view details

push time in 14 days

push eventhackforla/311-data

Jake Mensch

commit sha 155350fa63a8698278672e01b29b93506f9ca1c2

running CI on push

view details

push time in 14 days

push eventhackforla/311-data

Jake Mensch

commit sha 193b5ca91603ce16c18cad582389d9f4ce084231

postman logging

view details

push time in 14 days

push eventhackforla/311-data

Jake Mensch

commit sha e44b669374b86d5c1022fe9e57bd51623e3e9ef8

logging response text on postman test failure

view details

Jake Mensch

commit sha d603132de4b6c0040351cfac5d5d6d786a72009c

Merge branch 'add-alembic' of https://github.com/hackforla/311-data into add-alembic

view details

push time in 14 days

issue commenthackforla/311-data

Identify v1 app issues (devs)

Maybe instead of thinking of the app as two discrete categories -- the map and the visualizations -- we can think of it instead as a set of views on the underlying data. Some of them may involve a map, others might not. The views don't necessarily have to share a set of identical filters -- they can each have their own controls. We could just pick some of the coolest data-based UI's we can find and build them without worrying about whether they fit together. Here are some examples:

time sliders w/ map http://bl.ocks.org/cmdoptesc/fc0e318ce7992bed7ca8 http://io.morphocode.com/urban-layers/ (hit "get started" after the loading indicator finished)

3-dimensional map https://bl.ocks.org/Pessimistress/raw/1a4f3f5eb3b882ab4dd29f8ac122a7be/

steam graph for request types (instead of line/area charts) https://www.d3-graph-gallery.com/streamgraph.html

adamkendis

comment created time in 16 days

issue commenthackforla/311-data

Discussion: data quality / data handling issues

Hey @mattyweb, thanks for starting this. It's an important discussion, and we should definitely address a lot of these issues in version 2. Adding other devs: @adamkendis @hannahlivnat @tan-nate @JRHutson

Your description of the process above is mostly complete. There's some minor data cleaning that happens in the staging table before we copy to the requests table. Also, the API calls are based on the constants file (client/components/common/CONSTANTS.js), rather than the geojson file, which I believe is currently only used for inside the Map component. Ideally we would merge the constants with the geojson and move everything to the back end.

The constants file also has some additional notes about data issues, which I'm copying here:

  NC regions and names are from here:
    https://empowerla.org/councils-by-service-region/

  Note that Central Avenue Historic is listed there (Region 9 - South LA 2),
  but it isn't anywhere in the DB, so we don't know the id number. It also
  isn't in the nc-boundary json. It's in this list with an id of -1, which
  won't return any results but also won't hurt anything.

  Also note the 4 councils that are commented at the end of the list.
    Historic Filipinotown NC -- #122
    Old Northridge CC -- -- #2
    Brentwood CC -- -- #65
    Pacific Palisades CC --  -- #117
  These are NOT listed on empowerla's website, and they aren't in the
  nc-boundary json. But there are requests associated with them in the DB.

In addition, there are at least two other issues with the geojson. It gives ''NORTH WESTWOOD NC' an id of 0, even though there are requests in the database that give it an id of 127 (in addition to some requests that give it an id of 0). And it gives 'HISTORIC CULTURAL NORTH NC' an id of 0, even though the database that give it an id of 128 (plus some requests that give it an id of 0).

I don't think we've ever updated the geojson (at least not since I've been here) so we don't really have a process for that. And I'm not sure the city does either. We recently got a note from a user saying that the boundaries were off.

mattyweb

comment created time in 21 days

PullRequestReviewEvent

push eventhackforla/311-data

Jake Mensch

commit sha 37e2282021c4aaa4f7b97de8fb5e4c81242c043b

hotfix for compression error

view details

push time in 23 days

push eventhackforla/311-data

Jake Mensch

commit sha 5b916f8c81adb936ac28d55b58c9e575e48a50d2

comparison set validation

view details

push time in 23 days

push eventhackforla/311-data

Jake Mensch

commit sha 34c6c80f5486e16d15e7a8bab2b389bde864cdbd

comparison set validation

view details

push time in 23 days

push eventhackforla/311-data

Jake Mensch

commit sha a4cb368081136e89dd3bff60e8d2b0c5e1ad9cb0

postman tests accept either 400 or 422 for bad input

view details

push time in 24 days

more