Some utility commands you wish were in Python's pip
Make everything black
Sourcegraph blog, feature announcements, and website (about.sourcegraph.com)
Ajenti - the web admin panel everyone wants.
Reference application for AngularJS
Reliable, componentized data form controls for AngularJS
Seed project for angular apps.
push eventsourcegraph/sourcegraph
commit sha 7ba275d7126569d832ba50d9dd36a9f01f585e60
doc: codeintel: link to lsif quickstart from how-to page
push time in 2 days
push eventsourcegraph/sourcegraph
commit sha 94671fb19895e4b3b6272f511788f65e144aaebf
campaigns: migrate pre-spec campaigns to have specs (#13001) * Basic migrations. * wip * Fix capitalisation on the JSON tag for ExternalID. * Implement the rest of the migration. * Fix test breakage from the database migration. * Apply generate changes. * Update internal/campaigns/types.go Co-authored-by: Erik Seliger <erikseliger@me.com> * Continue for changesets that never had campaigns. * Remove weird duplicate log15. * Handle the partial changeset migration case in the down migration. * Add ON DELETE SET NULL constraints. * More migration updates. * Add delete cascade integration test. * Fix generated schema. Co-authored-by: Erik Seliger <erikseliger@me.com>
commit sha 380718a2624da12c6ae898bec2303b061cc2e9bd
doc/dev: Add code intelligence development documentation (#12949)
commit sha 49d206bed71386cc11a47a0786c15361d9e43094
codeintel: Add fast-path edge unmarshalling (#12878)
commit sha e0ae49d12196e18f345d877601610937dbec6ed8
campaigns: migrate to allow changeset spec user IDs to be NULL (#13020)
commit sha 15230795276fc0c3f11763f3fe297d9f5027c470
Don't break campaign resolver when a related user is deleted (#13006)
commit sha dcbd0181d398ac2e75335a075f70ed5e68baa526
e2e test: search visibility test refs wrong repo (#13021)
commit sha a9946a99a666be13562ed9934a923be047a52439
Revert "Update comby/comby Docker tag to v0.18.1 (#13003)" (#13027) This reverts commit 94dc08eb65896f04824f142092929086675da64a.
commit sha 05170f3dc459fa2e9440c3ce5aed5314b846d16a
campaigns: update a migration with an UPDATE (#13030) The boat burning migration that was merged in #13001 didn't account for the possibility that a campaign may have been created in the narrow time window between the campaign_spec_id and last_updated_at fields being added. Since the sum total of people who might be affected is likely three, and they all work here, we'll deal with this by updating the existing migration.
commit sha 41ccf166a444adf5928939a47324933b203ddea8
Cleanup campaign connection resolver (#13029) * Split up campaigns resolver in connection and node resolvers This follows the pattern we've been using with the other API entities as well. Also, I've added a separate test for the connection resolver to test pagination and moved the existing test for namespace resolvers from resolvers_test, which is way too large anyways. * Add cursor based pagination for campaigns This uses the exact same pattern we use for the changeset spec connection.
commit sha d428d27ed7b4e9c8992f5057a63b4f2ce1cb8170
Add test for reconciler state variables (#13031) Especially important for error, as that path wasn't tested at all.
commit sha b4d4ea7767ddccf35854711441f427483530b53d
Remove unused files (#13033)
commit sha b36a0e4bbc9fb90468b14b8417ce1a0f0df9e5ee
Add method to access faster pre-computed diffstat (#13034) .. rather than parsing all diffs for the list view.
commit sha a625db1e49f4fa6f314a5be8f5210d9050e23d7e
search: Monaco query editor should not syntax highlight links or unknown filters (#12992) Fixes #12961 with two changes to the Monaco query input: - URLs are no longer clickable - Input that looks like a filter (eg. 'https:' or C++ namespaces) are no longer treated as unknown filters or syntax highlighted Also fixes #10056  
commit sha 068b3ca0004aba97ebd2defcdfd0d2b8d892d3ac
changelog entries for search operators (#13040)
commit sha bde4f4a64bce185513c11539371077ac2be0775c
Unclutter the bitbucket server types (#13038)
commit sha 6b66c6def8881590975d12366777cc49b013556d
fix names of programming languages (#13042)
commit sha 505a9aadadcdd296a8b91fa2be08d3b0d7277190
Improving code intelligence docs and LSIF indexer installation instructions (#13041) * Making code intelligence documents more use friendly and direct * renamed lsif.md to feature name: precise_code_intelligence * created a features page Co-authored-by: Eric Fritz <eric@sourcegraph.com> * combined javascript/typescript into one line * replaced "built-in" with "basic" for consistency
commit sha d76e96439c98a238216de86eaf08f3d4f1978f01
auth: enforce length requirement when reset password (#12971)
commit sha 84bf8188a6a1c2efda244be83e90bdb90faf52a7
Changelog: add entries for repogroup pages and search homepage (#11921)
commit sha a76ad2c9fe821bd68ac6966859e450d8a4f7f46d
clean up PrivateCodeCta (#13044) - Use <a> not <Link> for external links (if there were no `target="_blank"` I believe this would not have worked; it would have treated the `to` prop value as a relative path) - To style a link as a button, use the Bootstrap classes directly, instead of embedding a button in a link (which is unnecessary and could have unintended consequences if this element is inside a form element) - Use consistent language: self-hosted not private, capitalize/spell Git/Perforce/Subversion in the standard way
push time in 2 days
PR closed sourcegraph/sourcegraph
First commit is what was pushed in #13782, second commit fixes #14039.
Empirically, this improves time-to-user-readiness for enterprise/dev/start.sh from 40-45s to 32-36s.
pr closed time in 2 days
Pull request review commentsourcegraph/sourcegraph
Don't regenerate if input files are older than output file (take 2)
const build = gulp.series(generate, webWebpack) /** * Watches everything and rebuilds on file changes. */-const dev = gulp.parallel(watchGenerate, webWebpackDevServer)+const dev = gulp.series(
We want to wait for the initial generate steps to complete before running the webpack dev server for 2 reasons:
- The generated files may be missing when the the webpack dev server expects them to exist, causing errors to be printed.
- The generate steps will immediately update the generated files, causing webpack dev server to immediately recompile before its first compilation is complete, wasting CPU cycles for no observable improvement in time-to-ready
comment created time in 4 days
pull request commentsourcegraph/sourcegraph
Don't regenerate if input files are older than output file (take 2)
@felixfbecker PTAL
comment created time in 4 days
push eventsourcegraph/sourcegraph
commit sha 18b93a120bf428a35292790a90c882fc7ea45bf5
web:build: don't regenerate files if generated files are newer than source
push time in 4 days
push eventsourcegraph/sourcegraph
commit sha ebb9650b1e1cf08feb167d10035181701c5ad46f
Update sha for dev/private (#14077)
commit sha a2adb9f9ea68f29d41930b259b27aa1aad889262
prometheus: only override Slack username if it is empty (#14079)
commit sha 8383a5ddf0e7354baac66b0b3b71ac0f0b4bcee7
add tracing to internal/db/repos/repos.Count (#14020)
commit sha cc74e8e33f5b61403517899a7c5c9b47a8b38325
codenotify: add bobheadxi subscriptions, fix entry (#14080)
commit sha 1976511aa1ae525a4775c0126efb1be84859eca7
update release version on docs.sourcegraph.com (#14084)
commit sha 20e2597bfb17f1a62abbd3162d8852a9bd279272
Implement a CodeSnippet component that encapsules the design (#14062)
commit sha 2008a8bfb9d6e8c60519af415b17f4040503ba6c
Drop sources column (#14094) * Drop sources column * Regenerate schema
commit sha ccafaea0b167a60ee1ff4f3c04859d3d740f2974
Update dependency typescript to ^4.0.3 (#14040) Co-authored-by: Renovate Bot <bot@renovateapp.com>
commit sha 180bbec981e36039bfa00f4da9c68bde419e5019
codenotify: Configure efritz's subscriptions (#14060)
commit sha 4ac0214132549835b99569d50395c434cc7c8ba9
codeintel: Refactor command runner in indexer (#14102)
commit sha d1e0f498d443c9dc3ab9342b0c70d0ebf396c54b
codeintel: Lower indexer output to debug level (#14103)
commit sha a9ebb0fbc4136b79471185d623733ac41ff5e476
indent and highlight JSON campaign specs (#14087) A campaign spec is a YAML document. YAML is a superset of JSON. If a campaign spec is pure-JSON, then it's nice to indent it (in case it's all on one line, for example) and highlight it using the JSON highlighter. The button is still "Download YAML" and the filename is still "whatever.yaml", but that's OK because YAML is a superset of JSON, so those are correct.
commit sha d233e758e35673446b8aae817c2cf2bc81760003
tracking-issue: Fix strikethrough on closed (unmerged) PRs (#14107)
commit sha 9263b31038b03e29bf2c358b92daa4a1cf2709aa
codeintel: Refactor index command construction (#14105)
commit sha 947f7da87983d36bdf763621045c114d948ef85f
codeintel: Move away from hardcoding image names (#14114)
commit sha d2f2b5f079f8f29716298a5a68ab4c525c581fd3
Pin cssnano version that treats :focus-visible correctly (#14086)
commit sha 1d56631d51f086e09da0f7294b18727fa8d2245f
Properly truncate path breadcrumbs (#14097) * Properly truncate path breadcrumbs Fixes #13535 * Add changelog entry * Prettier * add whitespace nowrap to path part * apply whitespace nowrap to all fp breadcrumbs Co-authored-by: TJ Kandala <kandalatj@gmail.com>
commit sha 91d59f56fad75723103c07e94d020fab28420de1
codeintel: Refactor construction of copyfiles flags (#14116)
commit sha 91ce92dd9d446011bf3e7f569d16a69bb9910151
codeintel: Add orderedKeys to indexer (#14117)
commit sha bc1864ec4dddd6362970902881a1da0fe8a14052
show command palette immediately, don't wait for fade-in (#14119) This improves UI responsiveness and is consistent with other similar menus in the UI (that immediately appear without a fade-in).
push time in 4 days
startedptitSeb/box86
started time in 5 days
startedcode-mancers/invoker
started time in 5 days
pull request commentbeyondgrep/website
Add Sourcegraph to list of indexing tools
I was going through my outstanding PRs and closing ones that hadn't had any activity on them in awhile. Sorry if this was premature!
comment created time in 5 days
push eventsourcegraph/sourcegraph
commit sha 4e59a515ee4f10281e75060d73620f45dafffce2
oauth2: better debugging and validation * debug logging toggled by INSECURE_OAUTH2_LOG_TRACES * github client secret validator * troubleshooting docs Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com>
push time in 6 days
PR merged sourcegraph/sourcegraph
Logs will look like this, when INSECURE_OAUTH2_LOG_TRACES=true in the frontend environment:
18:06:37 frontend | >>>>> HTTP Request: POST https://github.com/login/oauth/access_token
18:06:37 frontend | Header: map[Authorization:[Basic xxxxxxxxxxxxxxxxxxxxxx] Content-Type:[application/x-www-form-urlencoded]]
18:06:37 frontend | Body: code=xxxxxxxxxxx&grant_type=authorization_code
18:06:37 frontend | >>>>> HTTP Request: POST https://github.com/login/oauth/access_token
18:06:37 frontend | Header: map[Content-Type:[application/x-www-form-urlencoded]]
18:06:37 frontend | Body: client_id=xxxxxxxxxxxxx&client_secret=xxxxxxxxxxxxxxxxxxxxxxx&code=xxxxxxxxxxxxxxxxxxxxxxx&grant_type=authorization_code
Also added validation for the GitHub client ID and secret to ensure no hidden or non-alphanumeric characters are used.
pr closed time in 6 days
PR closed beyondgrep/website
Would it be appropriate to add Sourcegraph (https://github.com/sourcegraph/sourcegraph) to the list of indexing tools on the web page? It looks like the other tools listed are open source, while Sourcegraph is open core (main application is open-source, but there's an enterprise version that contains proprietary code for things like SSO). Unsure if that rules it out of this list, but a lot of our customers were using one of the indexing tools listed here (e.g., OpenGrok, Hound, cscope) before they adopted Sourcegraph, so thought it might be a worthy addition . If not, feel free to close and sorry about the noise!
P.S. thanks for creating this useful resource for people looking for better code search utilities. I've used ack a lot over the years, its speed and usability are fantastic, and from my vantage point (I used ack before anything else), it kicked off the explosion of better grep alternatives listed here!
pr closed time in 6 days
push eventsourcegraph/sourcegraph
commit sha 3761e8c3f593b51f2001bf20d5ed424cc5dbbb17
fix test
push time in 6 days
PR closed cool-repositories/sessions
Adds a GitHub action that generates LSIF for all Go repositories (selected by the query file:go.mod$ fork:yes on this Sourcegraph instance.
pr closed time in 6 days
PR closed cool-repositories/csrf
Adds a GitHub action that generates LSIF for all Go repositories (selected by the query file:go.mod$ fork:yes on this Sourcegraph instance.
pr closed time in 6 days
PR closed cool-repositories/schema
Adds a GitHub action that generates LSIF for all Go repositories (selected by the query file:go.mod$ fork:yes on this Sourcegraph instance.
pr closed time in 6 days
PR closed cool-repositories/websocket
Adds a GitHub action that generates LSIF for all Go repositories (selected by the query file:go.mod$ fork:yes on this Sourcegraph instance.
pr closed time in 6 days
PR closed cool-repositories/handlers
Adds a GitHub action that generates LSIF for all Go repositories (selected by the query file:go.mod$ fork:yes on this Sourcegraph instance.
pr closed time in 6 days
push eventsourcegraph/sourcegraph
commit sha 40da5f1228d4167d40e1f7b699aba3e4b25afce6
repo-updater: Remove NewChangesetSource (#14585) We now use the repos.Sourcer to create a repos.Source and then cast it to a ChangesetSource.
commit sha af4b27f5c0ffb4822316f6a33d09b5e6e52f0ea6
licensing: add new plans and feature tags (#14236)
commit sha 1e9624670fd101f0777e0b800f066b7da971eaaa
gosum: commit gosum values (#14615) Someone didn't run there code after updating go.mod :)
commit sha ed134c2c1cd25e18e83cbdb6d9bfad54c78da75e
graphqlbackend: Markdown resolver is string (#14616) Our Markdown resolver can just be `type Markdown string`. This will avoid some allocations and simplify interactions with the code. Code in graphqlbackend pkg updated with comby: comby -in-place '*markdownResolver' 'Markdown' 'go' comby -in-place '&markdownResolver{text: :[a]}' 'Markdown(:[a])' 'go'
commit sha 72cb6d7c9116a1df55a5b8ba82fa20528dd1c9c3
graphqlbackend: remove deprecated updateAllMirrorRepositories and noRepositoriesEnabled (#14400)
commit sha 392fbc71f5b934b305626ebe4662a3ca0a6cf4bd
Auto-retry integration tests as a bandaid (#14620)
commit sha 460c7ec5da4e265d8d2d0b88b3669807dfac1795
Split up campaigns docs into getting started, how-tos, tutorials (#14612) * Split up campaigns docs into getting started, how-tos, tutorials * Add link to campaing spec yaml * Change fonts of campaigns docs * Change getting started * Change wording * Change styling of Getting started links * Split out the explanations * Fix broken links * Rename 'Managing access' to 'Permissions in campaigns' * Rename outdated link * Add a file-icon to links on campaigns docs index * Remove images * Reorder screenshots * Pull out styling to global CSS * Add recommendation for latest version to docs * Update doc/user/campaigns/how-tos/creating_a_campaign.md Co-authored-by: Erik Seliger <erikseliger@me.com> Co-authored-by: Erik Seliger <erikseliger@me.com>
commit sha 0ab2b83b580f661fdc5269174f5ac6b1b3edb25c
release: unify command naming, remove tracking issue announcement (#14621) Documented in https://github.com/sourcegraph/about/pull/1749
commit sha 886d3f120b960c25bdc48c882d2a394b15473e83
release: add open issues to status, only post if issue created
commit sha abd1397705ae4e81a938ab3deedbfaefb8ba50d6
Remove code intel latencies from pings (#14554)
commit sha e86f5a3854dfac63a96a1307df5c0045b6d9bf86
release: align issues query with tracking issue (#14627)
commit sha b81b3b4071dffb0c5c75e8ba4a3965b1c1375475
remove stages from pings (#14551) * Remove stages from pings * Update changelog and docs pages. * Fix types
commit sha e08eb08cb4c9fe92404a7336dd193d18280a7c7d
admin docs: describes experimental validation command in the src-cli (#12001) * admin docs: describes experimental validation command in the src-cli * link to it * Update doc/admin/validation.md Co-authored-by: Robert Lin <robert@bobheadxi.dev> * Update doc/admin/validation.md Co-authored-by: Robert Lin <robert@bobheadxi.dev> * Update doc/admin/validation.md Co-authored-by: Robert Lin <robert@bobheadxi.dev> * Update doc/admin/validation.md Co-authored-by: Robert Lin <robert@bobheadxi.dev> Co-authored-by: Robert Lin <robert@bobheadxi.dev>
commit sha 17dc10bcfa091f46a9ac6a8d7c0088a8939efa95
web: import dialog css (#14631)
commit sha 16576ab2d7f86a6252daa4feb85f17407acac62d
admin doc: fix code block in validation page (#14634)
commit sha b01d3043e5595b6bb6cae4447b9f9c0d4dac5695
handle gzipped graphql requests (#14391) * handle gzipped graphql requests * cleanup * close gzip reader * revert unwanted change
commit sha ecfc49a1a2241ea7fbc09ebf2e373ea6625c9e56
search: fix version context dropdown styles (#14542)
commit sha 9f34563c12ae27e45c5da154ae92f4915ce040ca
Update dependency @sourcegraph/eslint-config to ^0.20.9 (#14602) Co-authored-by: Renovate Bot <bot@renovateapp.com> Co-authored-by: Felix Becker <felix.b@outlook.com>
commit sha 15b8df087682cbbeb3703ef6a2f480aa6d81ccdb
codeintel: Determine nearest upload efficiently in query path (#14581)
commit sha 3dabc0a78a8a6eeed48e3ab1bd88c04162107de1
Rewrite browser extension options menu with UX improvements (RFC 221) (#14345) Co-authored-by: Marek <marekz@gmail.com> Co-authored-by: Felix Becker <felix.b@outlook.com>
push time in 6 days
push eventsourcegraph/sourcegraph
commit sha 8ec977ccc6b4af26859003eb35ac7b573c029f88
use lazyregexp
commit sha 62aa053c3c3feae8a285ecdf9d2d027d95bf23c3
refactor
push time in 6 days
push eventsourcegraph/sourcegraph
commit sha 5c5db966d377d1d3324842798e00f498b8f3b834
authz: github: GitHub PAT may not have write access (#14700) When the GitHub PAT does not have write access, don't sync an empty user list for repo permissions; otherwise the repository will be rendered inaccessible to non-admin users.
push time in 7 days
PR merged sourcegraph/sourcegraph
When the GitHub PAT does not have write access, don't sync an empty user list for repo permissions; otherwise the repository will be rendered inaccessible to non-admin users.
pr closed time in 7 days
push eventsourcegraph/sourcegraph
commit sha 1e1abb4c00bf9c370a4a63dd402a7c365ee693f6
e2e: temporarily skip 2 tests to get 3.20 rc build (#13831)
commit sha e1873b6eded648d95f87015f9d5bd85123a65a37
Fix interactive search e2e test failures (#13866)
commit sha e984c51eb79a6011bd33a6a76daa07d94df657a0
e2e: keep up with search input UI changes (#13867)
commit sha 0be50ddf11c47ba47ee59e7be5bab9193628b12b
better selector and skip two tests that influence later tests (#13881)
commit sha dfa2d7915a0e166da62fbb768231a69f144727d6
search: fix NOT erasing rest of query (#13925)
commit sha 82e9bdfbe694608b08ca8dac367bf9323c82c8f3
Update latest release to 3.20.0 (#13962)
commit sha 4024d8feabe3802d11272ad987fb3b4ae90d2a73
gomod: rollback go-diff to v0.5.3 (#13973) Causing a panic in production. We need to fix the issue in v0.6.0 and then we can update again.
commit sha f085e1031a64beed9c2eccd396f59f29f8f1768c
Fix query state getting erased on location changes (#13954) As a result, every time the `location` was updated, for example when clicking on tokens in a code view, any query state that wasn't persisted to the URL (a partially edited query, or the scoped query added automatically when navigating to a repo/file) would be erased. I've simplified the useEffect() block in <GlobalNavbar/> (which could end up calling onNavbarQueryChange() multiple times, had some comments implying that it wouldn't update the query in interactive mode but still did it further down...), fixing the issue as a result, by restricting the cases in which we update the query state.
commit sha b673842d93c910a5832c03edbd88ea3affe34ebc
update docker deployment docs and handler to point to 3.20.1 (#14053) * update docker deployment docs and handler to point to 3.20.1 * correct src update version
commit sha 71a10292a91e464c397f01ca12d703230d78b920
authz: github: GitHub PAT may not have write access When the GitHub PAT does not have write access, don't sync an empty user list for repo permissions; otherwise the repository will be rendered inaccessible to non-admin users.
push time in 7 days
PR opened sourcegraph/sourcegraph
When the GitHub PAT does not have write access, don't sync an empty user list for repo permissions; otherwise the repository will be rendered inaccessible to non-admin users.
pr created time in 7 days
pull request commentsourcegraph/about
Great values, thanks for making this change!
comment created time in 8 days
push eventsourcegraph/sourcegraph
commit sha 4f5a2c55d4dfb0f3d4f14f1c7d754b5bfcd19b5b
go.mod: update golang.org/x/oauth2 (#14607)
push time in 8 days
PR merged sourcegraph/sourcegraph
Update to latest master of our fork (have a pending PR to merge to upstream: https://github.com/golang/oauth2/pull/442).
pr closed time in 8 days
Pull request review commentsourcegraph/sourcegraph
replace github.com/russross/blackfriday => github.com/russross/blackfriday v1.5. replace github.com/dghubble/gologin => github.com/sourcegraph/gologin v1.0.2-0.20181110030308-c6f1b62954d8 -replace golang.org/x/oauth2 => github.com/sourcegraph/oauth2 v1.0.0+replace golang.org/x/oauth2 => github.com/sourcegraph/oauth2 v0.0.0-20201011192344-605770292164
go mod edit -replace golang.org/x/oauth2=github.com/sourcegraph/oauth2@605770292164aba6b415b7b243a20006ac97b90d
comment created time in 8 days
push eventsourcegraph/about
commit sha 8cbfd7cbd934965ec1f7e9006501be72ae39d600
podcast: publishe ep 12
push time in 8 days
push eventsourcegraph/about
commit sha 3d6b2f77185ef5b189c24cfd1dca58a3dbe7490a
podcast fix typo
push time in 8 days
push eventsourcegraph/about
commit sha 34a18b7aa2863f11092fe576b69649929e23c68e
Revert "podcast: publish ep 12" This reverts commit c8baf27d7a58f68f6149db0a1f92004baaf17ce6.
push time in 9 days
push eventsourcegraph/about
commit sha c8baf27d7a58f68f6149db0a1f92004baaf17ce6
podcast: publish ep 12
push time in 9 days
push eventsourcegraph/about
commit sha ceb61fdb53613efc332c18cf44093d9dd6730a04
podcast: clean up ep 12 transcript
push time in 9 days
push eventsourcegraph/sourcegraph
push time in 9 days
Pull request review commentsourcegraph/sourcegraph
oauth2: debug logging toggled by INSECURE_OAUTH2_LOG_TRACES
func newOAuthFlowHandler(serviceType string) http.Handler { // golang.org/x/oauth2 package will use our http client which is configured // with proxy and TLS settings/etc. func withOAuthExternalHTTPClient(r *http.Request) *http.Request {- ctx := context.WithValue(r.Context(), oauth2.HTTPClient, httpcli.ExternalHTTPClient())+ client := httpcli.ExternalHTTPClient()+ if traceLogEnabled {
This wraps the global external HTTP client in a logging roundtripper that is used by the underlying oauth2 package. Code feels a little icky to me due to the type checks; if reviewer(s) have any suggestions for doing this in a cleaner way, please comment!
comment created time in 9 days
push eventsourcegraph/sourcegraph
commit sha 1ba61addd33e46c7dbac74053736b563818012c0
github client secret validator
commit sha 415dd82b00219baa7490a22d8f60233f714e46ed
troubleshooting docs
push time in 9 days
PR opened sourcegraph/sourcegraph
Logs will look like this, when INSECURE_OAUTH2_LOG_TRACES=true in the frontend environment:
18:06:37 frontend | >>>>> HTTP Request: POST https://github.com/login/oauth/access_token
18:06:37 frontend | Header: map[Authorization:[Basic xxxxxxxxxxxxxxxxxxxxxx] Content-Type:[application/x-www-form-urlencoded]]
18:06:37 frontend | Body: code=xxxxxxxxxxx&grant_type=authorization_code
18:06:37 frontend | >>>>> HTTP Request: POST https://github.com/login/oauth/access_token
18:06:37 frontend | Header: map[Content-Type:[application/x-www-form-urlencoded]]
18:06:37 frontend | Body: client_id=xxxxxxxxxxxxx&client_secret=xxxxxxxxxxxxxxxxxxxxxxx&code=xxxxxxxxxxxxxxxxxxxxxxx&grant_type=authorization_code
pr created time in 9 days
PR opened sourcegraph/sourcegraph
Update to latest master of our fork (have a pending PR to merge to upstream: https://github.com/golang/oauth2/pull/442).
pr created time in 10 days
push eventsourcegraph/oauth2
commit sha 605770292164aba6b415b7b243a20006ac97b90d
oauth2: return error, error_description, and error_uri when error field is present in token response
push time in 10 days
PR opened golang/oauth2
Return an error containing the error, error_description, and error_uri when the error field is present in the access token response, even if the HTTP response code is 2xx.
Fixes #441
pr created time in 10 days
issue openedgolang/oauth2
oauth2 swallows token response error if HTTP code is 200
What version of Go are you using (go version)? go version go1.15.2 linux/amd64 What operating system and processor architecture are you using? Ubuntu, x86
The OAuth2 spec states the authorization server should respond with a HTTP 400 if there is an error in the access token response. The error is conveyed in the error, error_description, and error_uri fields in the response.
GitHub OAuth does not exactly obey this spec and can return an error response (examples) with HTTP code 200. In this case, the error from GitHub is swallowed and instead we return a generic error. This makes it difficult to debug exactly what went wrong.
Reproduction steps:
- Configure GitHub as OAuth2 authorization provider for a web service that uses this package.
- Change the GitHub client secret so that it no longer matches.
- Attempt login via GitHub.
- Note that the error returned by this package is the generic
oauth2: server response missing access_token, rather than the actual error returned by GitHubincorrect_client_credentials
Proposed fix: Parse the error field in the access token response. If it is non-empty, then treat the response as an error and return an error containing the error, error_description, and error_uri fields in the access token response. An implementation of this fix is here: https://github.com/sourcegraph/oauth2/commit/605770292164aba6b415b7b243a20006ac97b90d.
created time in 10 days
push eventsourcegraph/sourcegraph
commit sha e323726bf7bce6ef02de1cf3029f8ef843512d2a
SOURCEGRAPH_AUTH_STYLE_PARAMS
push time in 14 days
push eventsourcegraph/sourcegraph
commit sha e323726bf7bce6ef02de1cf3029f8ef843512d2a
SOURCEGRAPH_AUTH_STYLE_PARAMS
push time in 14 days
push eventsourcegraph/sourcegraph
commit sha ca8413249c7c90108c3b8c5ca4abb719622cdcc5
Properly align diff-stat in changeset nodes like all other states (#13650)
commit sha 1360b5314daac2dd9db3e5c5698bdc7408dfb61b
monitoring: add panels for permissions (#13685)
commit sha 1b7c80b11d1a8e6dcbf09f3aa8f089a4e999db85
clean up NavLinks for unauthed users (#13680) When you're not signed in, the NavLinks has unnecessary items: - The external link icon is not necessary for About/Help. It just adds noise. - `Help` -> `Docs` - Remove `Extensions`. It's not important enough to put there. (It's not there for signed-in users even.)
commit sha 3697cc1e1a48352cc8b0eabeb2a20ff8e90000ae
codeintel: Load docker images into vm via docker save/load (#13701)
commit sha f5dbf0902da0ea19c092169957a49bab6664d493
Add doctype (#13658)
commit sha a8f065d4ecb16f09feff830f36a20aec9814da78
workerutil: Add num_failures column to limit RetryAfter functionality (#13623)
commit sha 62f73fa98a321bb0a78134dd837c93293692a068
add "LSIF" to title of LSIF quickstart page (#13660) Avoids confusion about whether this is quickstart for all of Sourcegraph vs. just for LSIF.
commit sha 09965489e0a8fec0035f8f8352bf7ad0f213f0d0
make GlobalNavbar a React.FunctionComponent (#13679)
commit sha affb013297c3c2e49ef67215eaf96ddca6c85ed7
Refactor nav and search components (#13697)
commit sha 3c3c1ece954fbd69811036b3fcfc042059a306ad
chore: fix event state check (#13707)
commit sha 8313d235af89e53eb9b26de4bb9a795c1617f7ff
ci: build candidate images sooner (#13689) Building the server and frontend images are the slowest steps on master CI runs. This change ensures we run those steps first (and some other slow images earlier as well). The earlier a step runs, the less time it potentially waits (vs other steps) for a free agent.
commit sha 88972485b6781e44c8b9ecac06056727a15b15a9
chore: adjust renovate action again (#13709)
commit sha a249d4b524906715107b526ee158cea55097be32
chore: move PR reference to prFooter, remove log job (#13710)
commit sha 9b6e31ae0beb4d0b7a667486e1d54f68b1f255a7
chore: do not allow codecov bot to trigger renovate (#13711) * chore: do not allow codecov bot to trigger renovate * hack a way to conditionally render pull request link for sourcegraph docker image updates
commit sha d00157ae47393b4f7fc25e93d314b70b953cf84d
Sync single external service (#13483) * repo-updater: Sync single external service at a time using workerutil Tests are still failing * repoupdater: Fix server_test * testSyncerSyncWithErrors uses SyncExternalService Also removes a test case that tests a sync of muktiple external services as we no longer support that. * Add EnqueueJobs tests * Removed old references to syncer.Sync Code now compiles but a lot of tests still fail. Added a cleanup function that needs to be called when we stop a worker to unregister prometheus metrics so that it doesn't panic when we start another worker in a subsequent test. * Fix repo-updater tests * Delete orphaned repos * Avoid constraints by changing upsert order * Fix minSyncInterval * WIP test sync * Add a test that syncs multiple external services in sequence * WIP test for orphaned repos * Only delete orphaned repos * repo-updater: Remove stale jobs on startup If a job is processing when repo-updater dies that job would never complete. We would then not requeue the associated external service for syncing. To be safe we delete non locked processing rows on startup. * Typo * db: Mark next_synced_at on service save This will cause a sync to be triggered ASAP for the saved external service. On save, we trigger a call to repo-updater which causes us to enqueue any pending sync jobs. As we've just upated next_sync_at the job for the newly saved repo will be queued. * Rename trigger We no longer trigger a full sync but instead trigger enqueueing pending sync jobs. * Remove orphaned repo code We now have a trigger that does the same thing * Improve orphaned repo test * Test external service deletion * Remove orphaned repo code from syncSubset * Remove leftover debug line * Fix deleting of repos when called via syncSubset * Improve EnqueueSyncJobs docstring Co-authored-by: ᴜɴᴋɴᴡᴏɴ <joe@sourcegraph.com> * Improve error message Co-authored-by: ᴜɴᴋɴᴡᴏɴ <joe@sourcegraph.com> * Check rows.Err() after iteration * Change log line to debug Co-authored-by: ᴜɴᴋɴᴡᴏɴ <joe@sourcegraph.com> * Remove leftover debug line * Add stalled jobs resetter * Pass prometheus register to the SyncWorker * Create ResetterMetrics inline * Remove unused field * Fix docstring * Typo * Missing period * Typo * Add failing test * Resolve name conflicts deterministically * Cleanup name conflicts resolution * Resolve name conflicts during renames * Don't wrap nil error * Don't queue jobs for Phabricator services * Handle large name lists in StoreListReposArgs.Names * Trigger rebuild * Remove stale job cleanup code We use the workerutil resetter now instead * Add failing test * Fix predicate on sourcegraphcom mode * Don't sync deleted external services * When in cloud mode DON'T sync admin added external services Co-authored-by: Asdine El Hrychy <asdine.elhrychy@gmail.com> Co-authored-by: ᴜɴᴋɴᴡᴏɴ <joe@sourcegraph.com>
commit sha bd9b03c23199d6ecbe1d0b73c090e163dfea1b6d
workerutil: demote log15.Info to Debug (#13717)
commit sha abc47c6afe03b0054f82ed7f3a23848664a6dc8e
codeowners: specify top-level catches earlier (#13719) There were a bunch of rules put in before this old top-level rule. For example internal/workerutil was assigned as owned by Stephen, even though we had a rule for it to be owned by Eric. This was because that rule was before this catch all rule.
commit sha b7599f3bf84d85ded03a81360c91e6240f03cb8c
Revert "Sync single external service (#13483)" (#13720) This reverts commit d00157ae47393b4f7fc25e93d314b70b953cf84d.
commit sha 5f42de54c1504c58277fb23e6b2cb19bd7d6b6e5
migrations: add back repo name index to speed up search (#13496)
commit sha 1f5c3f03f8ea1c6e55f11814943a8d1c7510fde3
campaigns: allow user to set git commit author (#13574) * add campaigns git commit author data
push time in 14 days
push eventsourcegraph/oauth2
commit sha c21c69abeaf41371d387b42c7b8fe91ababfcfeb
SOURCEGRAPH_OAUTH_DEBUG debug logging
push time in 14 days
push eventbeyang/hello-world
commit sha 07580ed46acb8dd8296239f67017d7858f65325f
move into subdirectories
push time in 15 days
push eventbeyang/hello-world
commit sha 5811300c96a392bf180d9955920769a534598393
hello worlds
push time in 16 days
startedrizsotto/Bear
started time in 16 days
push eventsourcegraph/lsif-clang
commit sha be00d0d82dd9e2196a233581fdbdb5343ccbf497
README: fix link
push time in 16 days
Pull request review commentsourcegraph/about
Code search code checks blog post
+---+title: Code search turned static code checker+author: Rijnard van Tonder+authorUrl: https://twitter.com/rvtond+publishDate: 2020-09-30T14:00-07:00+tags: [blog]+slug: code-search-turned-code-checker+heroImage: /blog/XXX.png+published: true+---++<style>+ .gatsby-highlight {+ max-width: 100%;+ width: 40rem;+ margin-left: auto;+ margin-right: auto;+ }+ table {+ width: 40rem;+ border: none;+ }+ table th {+ border: none;+ }+ table td {+ border: none;+ }+ table td:nth-child(2n) {+ text-align: right;+ }+ table th:nth-child(2n) {+ text-align: right;+ }+ table tr:nth-child(2n) {+ background-color: transparent;+ }+</style>++I find static code checkers most valuable when they help teach me better ways to+code in a language or framework. For example, the Go+[staticcheck](https://staticcheck.io/docs/checks#SA6005) tool finds expensive+string comparisons like:++```go+if strings.ToLower(string1) == strings.ToLower(string2) {+ ...+}+```++and suggests instead:++```go+if strings.EqualFold(string1, string2) {+ ...+}+```++I find these short-and-sweet replacements are a great way to learn framework+idioms or library functions, like `strings.EqualFold` in Go.<sup>1</sup> As a+codebase grows, these small inefficiences, inconsistencies, and missed+opportunities compound. Code patterns creep in that affect readability and+performance—[and it matters](https://www.digitalocean.com/blog/how-to-efficiently-compare-strings-in-go/?).++## Productivity workflows: Code checks vs. code search++Code checkers like `staticcheck` typically integrate with continuous integration+(CI) pipelines, or maybe a pre-commit hook in your development environment.+Others, like lint checks, typically integrate with editors. These workflows need+some upfront configuration, the code checks are fixed in place, and productivity+ensues (🤞).++A code search can also find code snippets to replace with functions like+`EqualFold`. In practice though, search engines generally treat code as+plaintext, and can't offer the fidelity of dedicated code checkers. Code+checkers do more work to gather static information of programs, e.g., parsing+syntax into trees and using build outputs like types and dependency graphs. It+takes time to gather this info, it takes knowledge to write checks that use this+info, and it takes time to hook it into your workflow. These constraints take away the+flexibility and speed of search workflows when you're simply looking for changes+to your favorite+`BananaNutChocolateCake Provider`[↗](https://sourcegraph.com/search?q=BananaNutChocolateCake&patternType=literal).++And yet, I can't shake the idea that there's a middle ground between crude+plaintext search and dedicated code checkers. What if the `EqualFold` check I+learned about could be reduced to a simpler and comparably effective _search query_?+Could we find, and maybe eradicate, all the code that should be calling+`EqualFold` instead? And without needing to clone the repos or set up a+fully fledged code checker? And so sprung the idea to explore code checking with+the ease of a flexible, push-button search workflow.++## Pushing code search toward static code checking++Earlier this year Sourcegraph introduced [structural search](blog/going-beyond-regular-expressions-with-structural-code-search/)+to search over code syntax. Structural search uses [comby](https://comby.dev) to implement+a basic building block in traditional code checkers: it interprets programs as+concrete syntax trees, not just plaintext. Sourcegraph search queries now also support patterns with+`or` clauses to search for multiple patterns. Add some file filters, and+it's becomes possible to write configurable code checks as self-contained search queries.+Let's explore this idea!++Here's a search query inspired by+[a check](https://staticcheck.io/docs/checks#S1003) where `strings.Index`+comparisons can be replaced with `strings.Contains`:++```python+language:go+-file:test+-file:vendor++strings.Index(..., ...) > -1++or++strings.Index(..., ...) >= 0++or++strings.Index(..., ...) != -1+```++This query matches all `.go` files, excluding file paths that contain+`test` or `vendor`. It's sensible to exclude `test` and `vendor` paths if we+want to actually propose changes to a project (more on that later). The+patterns `strings.Index(..., ....)` match syntax of `strings.Index` calls,+and the `...` ellipses are special placeholders that match at least two+arguments.<sup>2</sup> The `or` keywords separate the patterns into separate+expressions.++We created a set of the top 100 Go repositories on GitHub (by stars) on+Sourcegraph.com. We can search those by adding `repogroup:go-gh-100` to+the query. Have a look at some of the results:++[🔘 Find ways to improve code in popular Go projects ↗](https://sourcegraph.com/search/console?q=repogroup:go-gh-100%0Alanguage:go%0A-file:test%0A-file:vendor%0A-file:Godeps%0A%0Astrings.Index(...,%20...)%20%3E%20-1%0A%0Aor%0A%0Astrings.Index(...,%20...)%20%3E=%200%0A%0Aor%0A%0Astrings.Index(...,%20...)%20!=%20-1)+<br />+<sup>Side note: our multiline query editor is in a proof-of-concept phase.</sup>++The query finds matches in some of the most popular Go projects in a+couple of seconds. An exhaustive search shows that there are at least 14 matches+at the time of writing. For this flavor of syntactic change, I have a good sense+that these are real hits of code that can be improved (we'll explore+proposing changes to projects later in this blog.)++## Adding more code checks++Because structural search only looks at syntax, it can't yet operate at the+level of a tool like `staticcheck`, which knows more about static properties like type+information and variable scope to implement checks. At the same time,+`staticcheck` isn't a search tool, it's an entire toolset that includes a suite+of pre-written, high-precision checks that's very effective in certain+workflows, like CI. The question is not necessarily whether a search tool can+achieve parity with a tool like `staticcheck`. But given the overlap with+now-expressible search queries, it does prompt: how far can we push structural+code search to find similarly _actionable_ code checks? I.e., checks that match+real cases of code that we can improve.++So, taking inspiration from `staticcheck`, I wanted to see how many of its checks+translate to search queries that I could have high confidence in (i.e., all+patterns find legitimate issues; zero or very-near-zero false positives). I+chose to look at `staticcheck` checks for its clear documentation, which made it+easy to start developing checks. So I attempted to write search patterns+for each [simple static check](https://staticcheck.io/docs/checks).<sup>3</sup>++### Test files as reference patterns++I ran my search queries against `staticcheck`'s own test files to check that they+don't match unintended patterns (false positives) and don't miss real patterns+(false negatives). Each check may have more than one syntactic _variant_, so I+tried to implement patterns for as many variants as I could find in tests. It's+a neat exercise to develop patterns against the reference tests and discover+which variants to cover, all in a self-contained search webapp. Here's an example+where the query matches all the true hits in the test file, annotated with+`// want strings.Contains ...`:+++<img src="https://storage.googleapis.com/sourcegraph-assets/about.sourcegraph.com/blog/2020/multiline-query-editor.png">++[🔘 Example query to match known patterns in test files↗](https://sourcegraph.com/search/console?q=repo:%5Egithub%5C.com/dominikh/go-tools$%20%0Alang:go%0Afile:test%0A%0Astrings.IndexRune(...,%20...)%20%3E%20-1%0A%0Aor%0A%0Astrings.IndexRune(...,%20...)%20%3E=%200%0A%0Aor%0A%0Astrings.IndexRune(...,%20...)%20!=%20-1%0A%0Aor%0A%0Astrings.IndexRune(...,%20...)%20==%20-1%0A%0Aor%0A%0Astrings.IndexRune(...,%20...)%20%3C%200%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20%3E%20-1%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20%3E=%200%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20!=%20-1%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20==%20-1%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20%3C%200%0A%0Aor%0A%0Astrings.Index(...,%20...)%20%3E%20-1%0A%0Aor%0A%0Astrings.Index(...,%20...)%20%3E=%200%0A%0Aor%0A%0Astrings.Index(...,%20...)%20!=%20-1%0A%0Aor%0A%0Astrings.Index(...,%20...)%20==%20-1%0A%0Aor%0A%0Astrings.Index(...,%20...)%20%3C%200%0A)++Using the test files isn't a guarantee that I've implemented all the checks or+that it's entirely precise, but it adds a lot more confidence than inferring+patterns only from documentation.++### Results++Without making any claims about completeness, I was able to implement+at least one variant for 19 out of 34 checks checks that I feel confident+about. I relied only on patterns in `staticcheck`'s test data to discover syntactic+variants for checks, so I don't know if I covered all the patterns that+`staticcheck` implements in its code.++The majority of checks that I couldn't write required type information to+implement correctly (13 of 34). Other checks I couldn't write required more+complex syntax matching rules (8 of 34). This table roughly quantifies the+expressive needs for implementing checks:++| Description | # of checks |+|----------------------------------|-------------|+| Total | 34 |+| Works (all variants) | 11 |+| Works (at least one variant) | 19 |+| Needs additional syntax matching | 8 |+| Needs type info | 13 |++<sup>Note that some checks require type info _and_ additional syntax matching.+Also, one working variant for a check may not require type info, but another+variant for the same check may. I.e., the values overlap and do not sum to+Total.</sup>++Extending search queries to access static properties like type information is a+natural extension for writing better code checks, and an area of [code intelligence](https://docs.sourcegraph.com/user/code_intelligence)+work that we're exploring.++It was tempting to compare more directly with `staticcheck` output by+downloading the 100 Go repositories to disk, and running `staticcheck` on them+to see how the same patterns are found. For `staticcheck` to be effective, the+project typically needs to be built first (my experience was that running+`staticcheck` on individual files can be hit-and-miss, and understandably so). I+didn't like the idea of doing all that work, so I punted. It does nicely motivate+why less precise, but quicker code check workflows appeal.<sup>5</sup>++### Takeaway++Code search can be much more than finding your favorite provider called+`BananaNut.*Cake`. It can be a lightweight workflow for revealing+short-and-sweet ways to make your code better. Perhaps the most powerful idea is+that a universal code search can wholesale find and eradicate the code slips+we're always bound to make.++Starting small, the solution could look like a search query that finds issues in+active and popular Go projects at just the push of a button. Why not start+there? So while we're still working on ways to edit, comment, and annotate+queries, I get giddy about the idea that we can already run a check for+everything at once:++[🔘 Run all the code checks on popular Go projects ↗](https://sourcegraph.com/search/console?q=repogroup:go-gh-100%0Alang:go%0A-file:test%0A-file:vendor%0A-file:Godeps%0Apatterntype:structural%0A%0Astrings.ToLower(...)%20==%20strings.ToLower(...)%20or%0Astrings.IndexRune(...,%20...)%20%3E%20-1%20or%0Astrings.IndexRune(...,%20...)%20%3E=%200%20or%0Astrings.IndexRune(...,%20...)%20!=%20-1%20or%0Astrings.IndexRune(...,%20...)%20==%20-1%20or%0Astrings.IndexRune(...,%20...)%20%3C%200%20or%0Astrings.IndexAny(...,%20...)%20%3E%20-1%20or%0Astrings.IndexAny(...,%20...)%20%3E=%200%20or%0Astrings.IndexAny(...,%20...)%20!=%20-1%20or%0Astrings.IndexAny(...,%20...)%20==%20-1%20or%0Astrings.IndexAny(...,%20...)%20%3C%200%20or%0Astrings.Index(...,%20...)%20%3E%20-1%20or%0Astrings.Index(...,%20...)%20%3E=%200%20or%0Astrings.Index(...,%20...)%20!=%20-1%20or%0Astrings.Index(...,%20...)%20==%20-1%20or%0Astrings.Index(...,%20...)%20%3C%200%20or%0A%0Abytes.Compare(...,%20...)%20==%200%20or%0Abytes.Compare(...,%20...)%20!=%200%20or%0A%0Afor%20:%5B~_%5D,%20:%5B~_%5D%20=%20range%20or%0A%0Afor%20true%20%7B...%7D%20or%0A%0A:%5Bs.%5D%5B:len(:%5Bs%5D)%5D%20or%0A%0Atime.Now().Sub(...)%20or%0A%0Aif%20strings.HasPrefix(:%5Bstr.%5D,%20:%5Bprefix.%5D)%20%7B%0A%20%20:%5Bstr.%5D%20=%20:%5Bstr.%5D%5Blen(:%5Bprefix%5D):%5D%0A%7D%0A%0Aor%0A%0Afor%20:%5Bi.%5D%20:=%200;%20:%5Bi.%5D%20%3C%20:%5Bn.%5D;%20:%5Bi.%5D%20%20%20%7B%0A%20%20:%5Bbs.%5D%5B:%5Bi%5D%5D%20=%20:%5Bbs.%5D%5B:%5Boffset.%5D%20:%5Bi.%5D%5D%0A%7D%0A%0Aor%0A%0Amake(...,%20:%5Bx%5D,%20:%5Bx%5D)%20or%20%0Amake(map%5B:%5B%5Bw%5D%5D%5D:%5B%5Bw%5D%5D,%200)%20or%0Amake(chan%20int,%200)%0A%0Aor%0A%0Afunc%20:%5Bfn.%5D(...)%20%7B%0A%20%20...return%0A%7D%20%0A%0Aor%20%0A%0Afunc()%20%7B%0A%20%20...return%0A%7D%0A%0A.Sub(time.Now())%0A%0Aor%0A%0Afmt.Println(%22%25s%22,%20%22...%22)%0A%0Aor%0A%0Aerrors.New(fmt.Sprintf(...))%0A%0Aor%0A%0Afor%20:%5B~_%5D,%20:%5B_.%5D%20:=%20range%20%5B%5Drune(...)%0A%0Aor%0A%0Asort.Sort(sort.IntSlice(...))%20or%0Asort.Sort(sort.StringSlice(...))%20or%0Asort.Sort(sort.StringSlice(...))%20%0A%0Aor%0A%0Aif%20:%5B~_%5D,%20ok%20:=%20:%5Bm.%5D%5B:%5Bk%5D%5D;%20ok%20%7B%0A%09:%5Bm.%5D%5B:%5Bk%5D%5D%20=%20append(:%5Bm.%5D%5B:%5Bk%5D%5D,%20%22:%5Bv1%5D%22,%20%22:%5Bv2%5D%22)%0A%7D%20else%20%7B%0A%09:%5Bm.%5D%5B:%5Bk%5D%5D%20=%20%5B%5Dstring%7B%22:%5Bv1%5D%22,%20%22:%5Bv2%5D%22%7D%0A%7D%0A%0Aor%0A%0Aif%20:%5B~_%5D,%20ok%20:=%20:%5Bm.%5D%5B:%5Bk%5D%5D;%20ok%20%7B%0A%09:%5Bm.%5D%5B:%5Bk%5D%5D%20=%20append(:%5Bm.%5D%5B:%5Bk%5D%5D,%20%22:%5Bv1%5D%22)%0A%7D%20else%20%7B%0A%09:%5Bm.%5D%5B:%5Bk%5D%5D%20=%20%5B%5Dstring%7B%22:%5Bv1%5D%22%7D%0A%7D%0A%0Aor%0A%0Aselect%20%7B%0A%09case%20%3C-time.After(0):%0A%7D%0A%0Aor%0A%0Afmt.Print(fmt.Sprintf(%22...%22,%20...))%20or%0Afmt.Println(fmt.Sprintf(%22...%22,%20...))%20or%0Afmt.Fprint(nil,%20fmt.Sprintf(%22...%22,%20...))%20or%0Afmt.Fprintln(nil,%20fmt.Sprintf(%22...%22,%20...))%20or%0Afmt.Sprint(fmt.Sprintf(%22...%22,%20...))%20or%0Afmt.Sprintln(fmt.Sprintf(%22...%22,%20...))%0A%0Aor%20%0A%0Afmt.Sprintf(%22%25s%22,%20%22...%22))+<br />+<sup>Side note: patterns run in order and results return early after finding "enough" matches.</sup>++The great thing about code checks as queries is that it's easy to simply+delete patterns that we may not find as valuable. When I explored some+individual queries, it was reassuring to discover that _no_ patterns occur+in any of the Go repositories. For example,+[S1035](https://staticcheck.io/docs/checks#S1035) checks that there are unneeded+`http.CanonicalHeaderKey` calls on the first argument of certain functions:++```python+headers.Add(http.CanonicalHeaderKey(...), ...) or+headers.Del(http.CanonicalHeaderKey(...)) or+headers.Get(http.CanonicalHeaderKey(...)) or+headers.Set(http.CanonicalHeaderKey(...), ...)+```++There are [no matches ↗](https://sourcegraph.com/search/console?q=repogroup:go-gh-100%0A-file:test%0A-file:vendor%0Alang:go%0A%0Aheaders.Add(http.CanonicalHeaderKey(...),%20...)%20or%0Aheaders.Del(http.CanonicalHeaderKey(...))%20or%0Aheaders.Get(http.CanonicalHeaderKey(...))%20or%0Aheaders.Set(http.CanonicalHeaderKey(...),%20...)) for this pattern in the Go repositories, but there are [some matches ↗](https://sourcegraph.com/search/console?q=repogroup:go-gh-100%0A-file:test%0Alang:go%0A%0Aheaders.Add(http.CanonicalHeaderKey(...),%20...)%20or%0Aheaders.Del(http.CanonicalHeaderKey(...))%20or%0Aheaders.Get(http.CanonicalHeaderKey(...))%20or%0Aheaders.Set(http.CanonicalHeaderKey(...),%20...)) in vendored files, when we we remove the `-file:vendor` field.+++### Proposing changes++The most exciting part of finding real hits with any code checking tool is that+we discover an _actionable_ way to improve the code! I've narrowed down the+search query so that it (hopefully) finds only real and uncontroversial patterns+to improve and contribute to active Go projects.++Part of an effective contribution means that we need to avoid matches in files+that are tests, vendored, or external dependencies. While anyone can use the+results to make contributions, it's important to be mindful of contributor+guidelines, clearly communicate and motivate proposed changes in pull requests,+and validate that the change passes a project's tests or CI checks. And, while the+query does exclude common test and vendored files, it's best to check that+matches occur in files that really are part of the project.++If you're interested in potentially turning the results of [the query](https://sourcegraph.com/search/console?q=repogroup:go-gh-100%0Alang:go%0A-file:test%0A-file:vendor%0A-file:Godeps%0Apatterntype:structural%0A%0Astrings.ToLower(...)%20==%20strings.ToLower(...)%20or%0Astrings.IndexRune(...,%20...)%20%3E%20-1%20or%0Astrings.IndexRune(...,%20...)%20%3E=%200%20or%0Astrings.IndexRune(...,%20...)%20!=%20-1%20or%0Astrings.IndexRune(...,%20...)%20==%20-1%20or%0Astrings.IndexRune(...,%20...)%20%3C%200%20or%0Astrings.IndexAny(...,%20...)%20%3E%20-1%20or%0Astrings.IndexAny(...,%20...)%20%3E=%200%20or%0Astrings.IndexAny(...,%20...)%20!=%20-1%20or%0Astrings.IndexAny(...,%20...)%20==%20-1%20or%0Astrings.IndexAny(...,%20...)%20%3C%200%20or%0Astrings.Index(...,%20...)%20%3E%20-1%20or%0Astrings.Index(...,%20...)%20%3E=%200%20or%0Astrings.Index(...,%20...)%20!=%20-1%20or%0Astrings.Index(...,%20...)%20==%20-1%20or%0Astrings.Index(...,%20...)%20%3C%200%20or%0A%0Abytes.Compare(...,%20...)%20==%200%20or%0Abytes.Compare(...,%20...)%20!=%200%20or%0A%0Afor%20:%5B~_%5D,%20:%5B~_%5D%20=%20range%20or%0A%0Afor%20true%20%7B...%7D%20or%0A%0A:%5Bs.%5D%5B:len(:%5Bs%5D)%5D%20or%0A%0Atime.Now().Sub(...)%20or%0A%0Aif%20strings.HasPrefix(:%5Bstr.%5D,%20:%5Bprefix.%5D)%20%7B%0A%20%20:%5Bstr.%5D%20=%20:%5Bstr.%5D%5Blen(:%5Bprefix%5D):%5D%0A%7D%0A%0Aor%0A%0Afor%20:%5Bi.%5D%20:=%200;%20:%5Bi.%5D%20%3C%20:%5Bn.%5D;%20:%5Bi.%5D%20%20%20%7B%0A%20%20:%5Bbs.%5D%5B:%5Bi%5D%5D%20=%20:%5Bbs.%5D%5B:%5Boffset.%5D%20:%5Bi.%5D%5D%0A%7D%0A%0Aor%0A%0Amake(...,%20:%5Bx%5D,%20:%5Bx%5D)%20or%20%0Amake(map%5B:%5B%5Bw%5D%5D%5D:%5B%5Bw%5D%5D,%200)%20or%0Amake(chan%20int,%200)%0A%0Aor%0A%0Afunc%20:%5Bfn.%5D(...)%20%7B%0A%20%20...return%0A%7D%20%0A%0Aor%20%0A%0Afunc()%20%7B%0A%20%20...return%0A%7D%0A%0A.Sub(time.Now())%0A%0Aor%0A%0Afmt.Println(%22%25s%22,%20%22...%22)%0A%0Aor%0A%0Aerrors.New(fmt.Sprintf(...))%0A%0Aor%0A%0Afor%20:%5B~_%5D,%20:%5B_.%5D%20:=%20range%20%5B%5Drune(...)%0A%0Aor%0A%0Asort.Sort(sort.IntSlice(...))%20or%0Asort.Sort(sort.StringSlice(...))%20or%0Asort.Sort(sort.StringSlice(...))%20%0A%0Aor%0A%0Aif%20:%5B~_%5D,%20ok%20:=%20:%5Bm.%5D%5B:%5Bk%5D%5D;%20ok%20%7B%0A%09:%5Bm.%5D%5B:%5Bk%5D%5D%20=%20append(:%5Bm.%5D%5B:%5Bk%5D%5D,%20%22:%5Bv1%5D%22,%20%22:%5Bv2%5D%22)%0A%7D%20else%20%7B%0A%09:%5Bm.%5D%5B:%5Bk%5D%5D%20=%20%5B%5Dstring%7B%22:%5Bv1%5D%22,%20%22:%5Bv2%5D%22%7D%0A%7D%0A%0Aor%0A%0Aif%20:%5B~_%5D,%20ok%20:=%20:%5Bm.%5D%5B:%5Bk%5D%5D;%20ok%20%7B%0A%09:%5Bm.%5D%5B:%5Bk%5D%5D%20=%20append(:%5Bm.%5D%5B:%5Bk%5D%5D,%20%22:%5Bv1%5D%22)%0A%7D%20else%20%7B%0A%09:%5Bm.%5D%5B:%5Bk%5D%5D%20=%20%5B%5Dstring%7B%22:%5Bv1%5D%22%7D%0A%7D%0A%0Aor%0A%0Aselect%20%7B%0A%09case%20%3C-time.After(0):%0A%7D%0A%0Aor%0A%0Afmt.Print(fmt.Sprintf(%22...%22,%20...))%20or%0Afmt.Println(fmt.Sprintf(%22...%22,%20...))%20or%0Afmt.Fprint(nil,%20fmt.Sprintf(%22...%22,%20...))%20or%0Afmt.Fprintln(nil,%20fmt.Sprintf(%22...%22,%20...))%20or%0Afmt.Sprint(fmt.Sprintf(%22...%22,%20...))%20or%0Afmt.Sprintln(fmt.Sprintf(%22...%22,%20...))%0A%0Aor%20%0A%0Afmt.Sprintf(%22%25s%22,%20%22...%22)) into open source+contributions, e-mail me at <rijnard@sourcegraph.com>, I can help.
I am in favor of this idea. It would be a great way to engage new users. I do not know if there is a way to enable a private Slack like ours have a public channel that anyone can join. Creating a separate community Slack channel poses the question of whether or not this would be visible enough to members of the engineering team.
What would be ideal is if we could enable anyone to join a community channel without approval, that channel is visible to members of the team, and we can restrict visibility of more sensitive channels to only Sourcegraph teammates.
comment created time in 16 days
Pull request review commentsourcegraph/about
Code search code checks blog post
+---+title: Code search turned static code checker+author: Rijnard van Tonder+authorUrl: https://twitter.com/rvtond+publishDate: 2020-09-30T14:00-07:00+tags: [blog]+slug: code-search-turned-code-checker+heroImage: /blog/XXX.png+published: true+---++<style>+ .gatsby-highlight {+ max-width: 100%;+ width: 40rem;+ margin-left: auto;+ margin-right: auto;+ }+ table {+ width: 40rem;+ border: none;+ }+ table th {+ border: none;+ }+ table td {+ border: none;+ }+ table td:nth-child(2n) {+ text-align: right;+ }+ table th:nth-child(2n) {+ text-align: right;+ }+ table tr:nth-child(2n) {+ background-color: transparent;+ }+</style>++I find static code checkers most valuable when they help teach me better ways to+code in a language or framework. For example, the Go+[staticcheck](https://staticcheck.io/docs/checks#SA6005) tool finds expensive+string comparisons like:++```go+if strings.ToLower(string1) == strings.ToLower(string2) {+ ...+}+```++and suggests instead:++```go+if strings.EqualFold(string1, string2) {+ ...+}+```++I find these short-and-sweet replacements are a great way to learn framework+idioms or library functions, like `strings.EqualFold` in Go.<sup>1</sup> As a+codebase grows, these small inefficiences, inconsistencies, and missed+opportunities compound. Code patterns creep in that affect readability and+performance—[and it matters](https://www.digitalocean.com/blog/how-to-efficiently-compare-strings-in-go/?).++## Productivity workflows: Code checks vs. code search++Code checkers like `staticcheck` typically integrate with continuous integration+(CI) pipelines, or maybe a pre-commit hook in your development environment.+Others, like lint checks, typically integrate with editors. These workflows need+some upfront configuration, the code checks are fixed in place, and productivity+ensues (🤞).++A code search can also find code snippets to replace with functions like+`EqualFold`. In practice though, search engines generally treat code as+plaintext, and can't offer the fidelity of dedicated code checkers. Code+checkers do more work to gather static information of programs, e.g., parsing+syntax into trees and using build outputs like types and dependency graphs. It+takes time to gather this info, it takes knowledge to write checks that use this+info, and it takes time to hook it into your workflow. These constraints take away the+flexibility and speed of search workflows when you're simply looking for changes+to your favorite+`BananaNutChocolateCake Provider`[↗](https://sourcegraph.com/search?q=BananaNutChocolateCake&patternType=literal).++And yet, I can't shake the idea that there's a middle ground between crude+plaintext search and dedicated code checkers. What if the `EqualFold` check I+learned about could be reduced to a simpler and comparably effective _search query_?+Could we find, and maybe eradicate, all the code that should be calling+`EqualFold` instead? And without needing to clone the repos or set up a+fully fledged code checker? And so sprung the idea to explore code checking with+the ease of a flexible, push-button search workflow.++## Pushing code search toward static code checking++Earlier this year Sourcegraph introduced [structural search](blog/going-beyond-regular-expressions-with-structural-code-search/)+to search over code syntax. Structural search uses [comby](https://comby.dev) to implement+a basic building block in traditional code checkers: it interprets programs as+concrete syntax trees, not just plaintext. Sourcegraph search queries now also support patterns with+`or` clauses to search for multiple patterns. Add some file filters, and+it's becomes possible to write configurable code checks as self-contained search queries.+Let's explore this idea!++Here's a search query inspired by+[a check](https://staticcheck.io/docs/checks#S1003) where `strings.Index`+comparisons can be replaced with `strings.Contains`:++```python+language:go+-file:test+-file:vendor++strings.Index(..., ...) > -1++or++strings.Index(..., ...) >= 0++or++strings.Index(..., ...) != -1+```++This query matches all `.go` files, excluding file paths that contain+`test` or `vendor`. It's sensible to exclude `test` and `vendor` paths if we+want to actually propose changes to a project (more on that later). The+patterns `strings.Index(..., ....)` match syntax of `strings.Index` calls,+and the `...` ellipses are special placeholders that match at least two+arguments.<sup>2</sup> The `or` keywords separate the patterns into separate+expressions.++We created a set of the top 100 Go repositories on GitHub (by stars) on+Sourcegraph.com. We can search those by adding `repogroup:go-gh-100` to+the query. Have a look at some of the results:++[🔘 Find ways to improve code in popular Go projects ↗](https://sourcegraph.com/search/console?q=repogroup:go-gh-100%0Alanguage:go%0A-file:test%0A-file:vendor%0A-file:Godeps%0A%0Astrings.Index(...,%20...)%20%3E%20-1%0A%0Aor%0A%0Astrings.Index(...,%20...)%20%3E=%200%0A%0Aor%0A%0Astrings.Index(...,%20...)%20!=%20-1)+<br />+<sup>Side note: our multiline query editor is in a proof-of-concept phase.</sup>++The query finds matches in some of the most popular Go projects in a+couple of seconds. An exhaustive search shows that there are at least 14 matches+at the time of writing. For this flavor of syntactic change, I have a good sense+that these are real hits of code that can be improved (we'll explore+proposing changes to projects later in this blog.)++## Adding more code checks++Because structural search only looks at syntax, it can't yet operate at the+level of a tool like `staticcheck`, which knows more about static properties like type+information and variable scope to implement checks. At the same time,+`staticcheck` isn't a search tool, it's an entire toolset that includes a suite+of pre-written, high-precision checks that's very effective in certain+workflows, like CI. The question is not necessarily whether a search tool can+achieve parity with a tool like `staticcheck`. But given the overlap with+now-expressible search queries, it does prompt: how far can we push structural+code search to find similarly _actionable_ code checks? I.e., checks that match+real cases of code that we can improve.++So, taking inspiration from `staticcheck`, I wanted to see how many of its checks+translate to search queries that I could have high confidence in (i.e., all+patterns find legitimate issues; zero or very-near-zero false positives). I+chose to look at `staticcheck` checks for its clear documentation, which made it+easy to start developing checks. So I attempted to write search patterns+for each [simple static check](https://staticcheck.io/docs/checks).<sup>3</sup>++### Test files as reference patterns++I ran my search queries against `staticcheck`'s own test files to check that they+don't match unintended patterns (false positives) and don't miss real patterns+(false negatives). Each check may have more than one syntactic _variant_, so I+tried to implement patterns for as many variants as I could find in tests. It's+a neat exercise to develop patterns against the reference tests and discover+which variants to cover, all in a self-contained search webapp. Here's an example+where the query matches all the true hits in the test file, annotated with+`// want strings.Contains ...`:+++<img src="https://storage.googleapis.com/sourcegraph-assets/about.sourcegraph.com/blog/2020/multiline-query-editor.png">++[🔘 Example query to match known patterns in test files↗](https://sourcegraph.com/search/console?q=repo:%5Egithub%5C.com/dominikh/go-tools$%20%0Alang:go%0Afile:test%0A%0Astrings.IndexRune(...,%20...)%20%3E%20-1%0A%0Aor%0A%0Astrings.IndexRune(...,%20...)%20%3E=%200%0A%0Aor%0A%0Astrings.IndexRune(...,%20...)%20!=%20-1%0A%0Aor%0A%0Astrings.IndexRune(...,%20...)%20==%20-1%0A%0Aor%0A%0Astrings.IndexRune(...,%20...)%20%3C%200%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20%3E%20-1%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20%3E=%200%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20!=%20-1%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20==%20-1%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20%3C%200%0A%0Aor%0A%0Astrings.Index(...,%20...)%20%3E%20-1%0A%0Aor%0A%0Astrings.Index(...,%20...)%20%3E=%200%0A%0Aor%0A%0Astrings.Index(...,%20...)%20!=%20-1%0A%0Aor%0A%0Astrings.Index(...,%20...)%20==%20-1%0A%0Aor%0A%0Astrings.Index(...,%20...)%20%3C%200%0A)++Using the test files isn't a guarantee that I've implemented all the checks or+that it's entirely precise, but it adds a lot more confidence than inferring+patterns only from documentation.++### Results
I think it would be interesting to show a small table comparing the definitions of these checks in staticcheck (presumably long/complex) vs. comby.
comment created time in 17 days
Pull request review commentsourcegraph/about
Code search code checks blog post
+---+title: Code search turned static code checker+author: Rijnard van Tonder+authorUrl: https://twitter.com/rvtond+publishDate: 2020-09-30T14:00-07:00+tags: [blog]+slug: code-search-turned-code-checker+heroImage: /blog/XXX.png+published: true+---++<style>+ .gatsby-highlight {+ max-width: 100%;+ width: 40rem;+ margin-left: auto;+ margin-right: auto;+ }+ table {+ width: 40rem;+ border: none;+ }+ table th {+ border: none;+ }+ table td {+ border: none;+ }+ table td:nth-child(2n) {+ text-align: right;+ }+ table th:nth-child(2n) {+ text-align: right;+ }+ table tr:nth-child(2n) {+ background-color: transparent;+ }+</style>++I find static code checkers most valuable when they help teach me better ways to+code in a language or framework. For example, the Go+[staticcheck](https://staticcheck.io/docs/checks#SA6005) tool finds expensive+string comparisons like:++```go+if strings.ToLower(string1) == strings.ToLower(string2) {+ ...+}+```++and suggests instead:++```go+if strings.EqualFold(string1, string2) {+ ...+}+```++I find these short-and-sweet replacements are a great way to learn framework+idioms or library functions, like `strings.EqualFold` in Go.<sup>1</sup> As a+codebase grows, these small inefficiences, inconsistencies, and missed+opportunities compound. Code patterns creep in that affect readability and+performance—[and it matters](https://www.digitalocean.com/blog/how-to-efficiently-compare-strings-in-go/?).++## Productivity workflows: Code checks vs. code search++Code checkers like `staticcheck` typically integrate with continuous integration+(CI) pipelines, or maybe a pre-commit hook in your development environment.+Others, like lint checks, typically integrate with editors. These workflows need+some upfront configuration, the code checks are fixed in place, and productivity+ensues (🤞).++A code search can also find code snippets to replace with functions like+`EqualFold`. In practice though, search engines generally treat code as+plaintext, and can't offer the fidelity of dedicated code checkers. Code+checkers do more work to gather static information of programs, e.g., parsing+syntax into trees and using build outputs like types and dependency graphs. It+takes time to gather this info, it takes knowledge to write checks that use this+info, and it takes time to hook it into your workflow. These constraints take away the+flexibility and speed of search workflows when you're simply looking for changes+to your favorite+`BananaNutChocolateCake Provider`[↗](https://sourcegraph.com/search?q=BananaNutChocolateCake&patternType=literal).++And yet, I can't shake the idea that there's a middle ground between crude+plaintext search and dedicated code checkers. What if the `EqualFold` check I+learned about could be reduced to a simpler and comparably effective _search query_?+Could we find, and maybe eradicate, all the code that should be calling+`EqualFold` instead? And without needing to clone the repos or set up a+fully fledged code checker? And so sprung the idea to explore code checking with+the ease of a flexible, push-button search workflow.++## Pushing code search toward static code checking++Earlier this year Sourcegraph introduced [structural search](blog/going-beyond-regular-expressions-with-structural-code-search/)+to search over code syntax. Structural search uses [comby](https://comby.dev) to implement+a basic building block in traditional code checkers: it interprets programs as+concrete syntax trees, not just plaintext. Sourcegraph search queries now also support patterns with+`or` clauses to search for multiple patterns. Add some file filters, and+it's becomes possible to write configurable code checks as self-contained search queries.+Let's explore this idea!++Here's a search query inspired by+[a check](https://staticcheck.io/docs/checks#S1003) where `strings.Index`+comparisons can be replaced with `strings.Contains`:++```python+language:go+-file:test+-file:vendor++strings.Index(..., ...) > -1++or++strings.Index(..., ...) >= 0++or++strings.Index(..., ...) != -1+```++This query matches all `.go` files, excluding file paths that contain+`test` or `vendor`. It's sensible to exclude `test` and `vendor` paths if we+want to actually propose changes to a project (more on that later). The+patterns `strings.Index(..., ....)` match syntax of `strings.Index` calls,+and the `...` ellipses are special placeholders that match at least two+arguments.<sup>2</sup> The `or` keywords separate the patterns into separate+expressions.++We created a set of the top 100 Go repositories on GitHub (by stars) on+Sourcegraph.com. We can search those by adding `repogroup:go-gh-100` to+the query. Have a look at some of the results:++[🔘 Find ways to improve code in popular Go projects ↗](https://sourcegraph.com/search/console?q=repogroup:go-gh-100%0Alanguage:go%0A-file:test%0A-file:vendor%0A-file:Godeps%0A%0Astrings.Index(...,%20...)%20%3E%20-1%0A%0Aor%0A%0Astrings.Index(...,%20...)%20%3E=%200%0A%0Aor%0A%0Astrings.Index(...,%20...)%20!=%20-1)+<br />+<sup>Side note: our multiline query editor is in a proof-of-concept phase.</sup>++The query finds matches in some of the most popular Go projects in a+couple of seconds. An exhaustive search shows that there are at least 14 matches+at the time of writing. For this flavor of syntactic change, I have a good sense+that these are real hits of code that can be improved (we'll explore+proposing changes to projects later in this blog.)++## Adding more code checks++Because structural search only looks at syntax, it can't yet operate at the+level of a tool like `staticcheck`, which knows more about static properties like type+information and variable scope to implement checks. At the same time,+`staticcheck` isn't a search tool, it's an entire toolset that includes a suite+of pre-written, high-precision checks that's very effective in certain+workflows, like CI. The question is not necessarily whether a search tool can+achieve parity with a tool like `staticcheck`. But given the overlap with+now-expressible search queries, it does prompt: how far can we push structural+code search to find similarly _actionable_ code checks? I.e., checks that match+real cases of code that we can improve.++So, taking inspiration from `staticcheck`, I wanted to see how many of its checks+translate to search queries that I could have high confidence in (i.e., all+patterns find legitimate issues; zero or very-near-zero false positives). I+chose to look at `staticcheck` checks for its clear documentation, which made it+easy to start developing checks. So I attempted to write search patterns+for each [simple static check](https://staticcheck.io/docs/checks).<sup>3</sup>++### Test files as reference patterns++I ran my search queries against `staticcheck`'s own test files to check that they+don't match unintended patterns (false positives) and don't miss real patterns+(false negatives). Each check may have more than one syntactic _variant_, so I+tried to implement patterns for as many variants as I could find in tests. It's+a neat exercise to develop patterns against the reference tests and discover+which variants to cover, all in a self-contained search webapp. Here's an example+where the query matches all the true hits in the test file, annotated with+`// want strings.Contains ...`:+++<img src="https://storage.googleapis.com/sourcegraph-assets/about.sourcegraph.com/blog/2020/multiline-query-editor.png">++[🔘 Example query to match known patterns in test files↗](https://sourcegraph.com/search/console?q=repo:%5Egithub%5C.com/dominikh/go-tools$%20%0Alang:go%0Afile:test%0A%0Astrings.IndexRune(...,%20...)%20%3E%20-1%0A%0Aor%0A%0Astrings.IndexRune(...,%20...)%20%3E=%200%0A%0Aor%0A%0Astrings.IndexRune(...,%20...)%20!=%20-1%0A%0Aor%0A%0Astrings.IndexRune(...,%20...)%20==%20-1%0A%0Aor%0A%0Astrings.IndexRune(...,%20...)%20%3C%200%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20%3E%20-1%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20%3E=%200%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20!=%20-1%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20==%20-1%0A%0Aor%0A%0Astrings.IndexAny(...,%20...)%20%3C%200%0A%0Aor%0A%0Astrings.Index(...,%20...)%20%3E%20-1%0A%0Aor%0A%0Astrings.Index(...,%20...)%20%3E=%200%0A%0Aor%0A%0Astrings.Index(...,%20...)%20!=%20-1%0A%0Aor%0A%0Astrings.Index(...,%20...)%20==%20-1%0A%0Aor%0A%0Astrings.Index(...,%20...)%20%3C%200%0A)++Using the test files isn't a guarantee that I've implemented all the checks or+that it's entirely precise, but it adds a lot more confidence than inferring+patterns only from documentation.++### Results++Without making any claims about completeness, I was able to implement+at least one variant for 19 out of 34 checks checks that I feel confident+about. I relied only on patterns in `staticcheck`'s test data to discover syntactic+variants for checks, so I don't know if I covered all the patterns that+`staticcheck` implements in its code.++The majority of checks that I couldn't write required type information to+implement correctly (13 of 34). Other checks I couldn't write required more+complex syntax matching rules (8 of 34). This table roughly quantifies the+expressive needs for implementing checks:++| Description | # of checks |+|----------------------------------|-------------|+| Total | 34 |+| Works (all variants) | 11 |+| Works (at least one variant) | 19 |+| Needs additional syntax matching | 8 |+| Needs type info | 13 |++<sup>Note that some checks require type info _and_ additional syntax matching.+Also, one working variant for a check may not require type info, but another+variant for the same check may. I.e., the values overlap and do not sum to+Total.</sup>++Extending search queries to access static properties like type information is a+natural extension for writing better code checks, and an area of [code intelligence](https://docs.sourcegraph.com/user/code_intelligence)+work that we're exploring.++It was tempting to compare more directly with `staticcheck` output by+downloading the 100 Go repositories to disk, and running `staticcheck` on them+to see how the same patterns are found. For `staticcheck` to be effective, the+project typically needs to be built first (my experience was that running+`staticcheck` on individual files can be hit-and-miss, and understandably so). I+didn't like the idea of doing all that work, so I punted. It does nicely motivate+why less precise, but quicker code check workflows appeal.<sup>5</sup>++### Takeaway++Code search can be much more than finding your favorite provider called
Maybe a table showing relative advantages/disadvantages of staticcheck vs seach+comby would be evocative here.
Potential rows:
- Accuracy (syntax vs semantics)
- Scalability
- Ease of definition
- Ease of execution
comment created time in 17 days
Pull request review commentsourcegraph/about
Code search code checks blog post
+---+title: Code search turned static code checker+author: Rijnard van Tonder+authorUrl: https://twitter.com/rvtond+publishDate: 2020-09-30T14:00-07:00+tags: [blog]+slug: code-search-turned-code-checker+heroImage: /blog/XXX.png+published: true+---++<style>+ .gatsby-highlight {+ max-width: 100%;+ width: 40rem;+ margin-left: auto;+ margin-right: auto;+ }+ table {+ width: 40rem;+ border: none;+ }+ table th {+ border: none;+ }+ table td {+ border: none;+ }+ table td:nth-child(2n) {+ text-align: right;+ }+ table th:nth-child(2n) {+ text-align: right;+ }+ table tr:nth-child(2n) {+ background-color: transparent;+ }+</style>++I find static code checkers most valuable when they help teach me better ways to+code in a language or framework. For example, the Go+[staticcheck](https://staticcheck.io/docs/checks#SA6005) tool finds expensive+string comparisons like:++```go+if strings.ToLower(string1) == strings.ToLower(string2) {+ ...+}+```++and suggests instead:++```go+if strings.EqualFold(string1, string2) {+ ...+}+```++I find these short-and-sweet replacements are a great way to learn framework+idioms or library functions, like `strings.EqualFold` in Go.<sup>1</sup> As a+codebase grows, these small inefficiences, inconsistencies, and missed+opportunities compound. Code patterns creep in that affect readability and+performance—[and it matters](https://www.digitalocean.com/blog/how-to-efficiently-compare-strings-in-go/?).++## Productivity workflows: Code checks vs. code search++Code checkers like `staticcheck` typically integrate with continuous integration+(CI) pipelines, or maybe a pre-commit hook in your development environment.+Others, like lint checks, typically integrate with editors. These workflows need+some upfront configuration, the code checks are fixed in place, and productivity+ensues (🤞).++A code search can also find code snippets to replace with functions like+`EqualFold`. In practice though, search engines generally treat code as+plaintext, and can't offer the fidelity of dedicated code checkers. Code+checkers do more work to gather static information of programs, e.g., parsing+syntax into trees and using build outputs like types and dependency graphs. It+takes time to gather this info, it takes knowledge to write checks that use this+info, and it takes time to hook it into your workflow. These constraints take away the+flexibility and speed of search workflows when you're simply looking for changes+to your favorite+`BananaNutChocolateCake Provider`[↗](https://sourcegraph.com/search?q=BananaNutChocolateCake&patternType=literal).++And yet, I can't shake the idea that there's a middle ground between crude
To me, this intro leaves out a huge advantage of search over staticcheck, which is scalability. With search+comby, you can execute your find-replace over the entire universe of code, whereas staticcheck requires manual setup/effort for each repository you want to cover. This difference in scale/automaticness means if I'm someone who added a new API, I would totally reach for search+comby to update existing usages of that API to take advantage of my new function. Not as easy with staticcheck, because other repository maintainers have to set it up first. Big difference in friction.
comment created time in 17 days
Pull request review commentsourcegraph/about
Code search code checks blog post
+---+title: Code search turned static code checker+author: Rijnard van Tonder+authorUrl: https://twitter.com/rvtond+publishDate: 2020-09-30T14:00-07:00+tags: [blog]+slug: code-search-turned-code-checker+heroImage: /blog/XXX.png+published: true+---++<style>+ .gatsby-highlight {+ max-width: 100%;+ width: 40rem;+ margin-left: auto;+ margin-right: auto;+ }+ table {+ width: 40rem;+ border: none;+ }+ table th {+ border: none;+ }+ table td {+ border: none;+ }+ table td:nth-child(2n) {+ text-align: right;+ }+ table th:nth-child(2n) {+ text-align: right;+ }+ table tr:nth-child(2n) {+ background-color: transparent;+ }+</style>++I find static code checkers most valuable when they help teach me better ways to
I find static code checkers most valuable when they teach me better ways to
comment created time in 18 days
push eventsourcegraph/sourcegraph
commit sha f86335240f77a006f6887a7c057073c299d69428
CODENOTIFY: beyang subscriptions (#14396)
push time in 17 days
PR merged sourcegraph/sourcegraph
<!-- Reminder: Have you updated the changelog and relevant docs (user docs, architecture diagram, etc) ? -->
pr closed time in 17 days
PR opened sourcegraph/sourcegraph
<!-- Reminder: Have you updated the changelog and relevant docs (user docs, architecture diagram, etc) ? -->
pr created time in 18 days
Pull request review commentsourcegraph/about
Blog: A different way to think about code ownership
+---+title: "A different way to think about code ownership"+author: Nick Snyder+authorUrl: https://twitter.com/nickdsnyder+publishDate: 2020-10-05+tags: [blog]+slug: a-different-way-to-think-about-code-ownership+heroImage: https://sourcegraphstatic.com/blog/codenotify-survey-results.png+published: true+---++We’re [experimenting](https://github.com/sourcegraph/sourcegraph/pull/13838) with a new way to think about code ownership, and have built a new tool called [Codenotify](https://github.com/sourcegraph/codenotify). Codenotify enables developers to easily subscribe to file changes in a Git repository without creating the expectation that changes to those files are blocked on the subscriber’s review.++## Defining the problem++We host all of [our code on GitHub](https://github.com/sourcegraph/), so up until now we have been using a GitHub [CODEOWNERS](https://docs.github.com/en/free-pro-team@latest/github/creating-cloning-and-archiving-repositories/about-code-owners) file in each repository. When someone opens a pull request, GitHub will look at the CODEOWNERS file to determine which reviewers to automatically add.++As our team has grown over the past year, I started to observe patterns that made me think that the way we assigned code reviewers was flawed.++- Pull requests that touched multiple files of different kinds and in multiple places would sometimes end up with a large list of automatically added reviews.+- Reviewers wouldn’t review all the changes they were notified about because it wasn’t clear if their personal review was relevant (for example: they may have gotten notified because they are on the team that owns the code, but aren’t the best reviewer for that particular change).+- Authors (especially newer teammates) were getting slowed down because it was not clear whether they needed to wait for all automatically added reviewers or not. This was exacerbated by the fact that some teammates used CODEOWNERS as a way to be notified of changes they cared about, even if they didn’t have the intention or ability to promptly review all those changes.++My diagnosis was that there were two problems:++1. The low signal to noise ratio of automatically assigned reviews was slowing the team down.+1. The design of CODEOWNERS (its name, its UI treatment, it automatically adding code reviewers) created a gatekeeper culture to code review that discouraged a [high agency](https://about.sourcegraph.com/company/values#high-agency) mindset.++## Validation++I had a hypothesis—that we needed something other than CODEOWNERS—but it needed validation. Was the team feeling the same pains and observing the same patterns as I was? Was continuing to use CODEOWNERS a viable solution moving forward?++To get the sentiment of the team, [I opened a pull request that proposed deleting CODEOWNERS with my reasons](https://github.com/sourcegraph/sourcegraph/pull/11718) and asked the entire team to submit anonymous feedback. Here were the results:++<div class="text-center">+ <img src="https://sourcegraphstatic.com/blog/codenotify-survey-results.png" alt="50% yes, 31% maybe, 19% no">+</div>++In the free-form comments, responders provided extra context about their support, concerns, and suggestions for the proposal. Here are a few representative examples:++- "I've honestly not gotten any value out of CODEOWNERS files. I never look into them, I never trust the auto-add-as-reviewer feature in GitHub PRs."+- "Would this result in people who wrote more LOC being enlisted into more reviews?"+- "I use CODEOWNERS to get notified of PRs"+- "Replace with [OWNERS](https://chromium.googlesource.com/chromium/src/+/master/docs/code_reviews.md) (something people will actually update)"+- "I think removing CODEOWNERS makes it more difficult for new developers without historical context to determine who the right reviewers are."++It was clear we needed _something_, but CODEOWNERS didn’t do exactly what we needed and we didn’t have the ability to change its behavior.++## Exploring alternatives++There was enough evidence that a problem existed to justify exploring alternatives.++I found [Fullstory’s blog post about how they ran into similar problems with CODEOWNERS and ended up creating a bot](https://bionic.fullstory.com/taming-github-codeowners-with-bots/). I would have loved to experiment with using their bot, but unfortunately it was not open source.++A few of our teammates, including myself, worked at Google so we were familiar with the [OWNERS](https://chromium.googlesource.com/chromium/src/+/master/docs/code_reviews.md) format used internally and in the Chromium project. This would have been an improvement over CODEOWNERS because multiple files are easier to maintain than a single large file and [Chromium’s implementation explicitly attempts to minimize the number of reviewers](https://chromium.googlesource.com/chromium/tools/depot_tools/+/master/owners.py#607), but I was worried this would continue to create a gatekeeper culture around modifying code.++I wanted a solution that would:++1. Allow developers to subscribe to the code changes that they cared about without creating an expectation that they would review all those changes.+1. Allow authors to use their judgement and agency to intentionally select reviewers whose feedback they think would be valuable (for example: Who wrote the code being edited? Who reviewed the code being edited? Who provides valuable feedback in a timely manner?).+1. Be easy to understand and maintain in a large and constantly evolving codebase.++Neither CODEOWNERS nor OWNERs were designed with all these requirements in mind, so I started to think about what a new tool might look like, and I ended up with [Codenotify](https://github.com/sourcegraph/codenotify).++## Designing Codenotify++As the name implies, Codenotify is designed around the concept of notifications, not about establishing "ownership" of code or mandating "reviewers". Codenotify works on the command line and is codehost agnostic at its core, but it is also packaged as a GitHub Action so it is easy for repositories on GitHub, like ours, to adopt. The GitHub Action sends notifications to subscribers by mentioning them in a comment instead of adding them to the official "reviewers" or "assignees" lists on GitHub.++Notification rules are stored in [CODENOTIFY files](https://sourcegraph.com/github.com/sourcegraph/codenotify/-/blob/README.md#codenotify-files) in any directory. This makes rules more maintainable and less brittle to moving code because the CODENOTIFY files are adjacent to the files they have rules for.++Codenotify rules and files are familiar in syntax to CODEOWNERS, but simpler to understand because each rule is additive, not hierarchical. This means that you can understand how a single rule behaves without needing to consider any other rules (unlike OWNERS or CODEOWNERS).++## Starting our experiment
Maybe add a small section here with a CTA to readers to try it out. Can link to https://github.com/sourcegraph/codenotify.
comment created time in 18 days
startedsourcegraph/codenotify
started time in 18 days
push eventbeyang/google-keys
commit sha 59f3f2f21298ed164db5cb8cfd40751c8222fa5c
spacing
commit sha 6308ecb7a13f5a8911e7b18e235d602227a0bf93
fix google keys
push time in 19 days
push eventsourcegraph/about
commit sha 216d7b4dca6c14cb07b8132f24d32996a54c3bbd
fix formatting
push time in 19 days
push eventsourcegraph/about
commit sha c7721c94bedd9cee15294459803115c320c4dabf
podcast transcript cleanup
push time in 19 days
push eventsourcegraph/about
commit sha c9b172262a2e850b3847a5a593a622c8e0fea706
podcast: ep12
push time in 19 days
push eventsourcegraph/ghdump
commit sha a5484a8c6894ac6f63a54b163f2d4c8a4c3c8d60
add back all 100k repos
push time in 20 days
push eventbeyang/google-keys
commit sha 75cc03c2f90e16a74e4a73f823517c0bf3e34609
fix google keys
push time in 22 days
startedbellard/quickjs
started time in 22 days
startedwspl/go-quickjs
started time in 22 days
startedelsaland/elsa
started time in 22 days