profile
viewpoint
Ryan Slade ryanslade Sourcegraph Cape Town Developer at @sourcegraph

ryanslade/goautotest 26

Automatically run Go unit tests when code changes

ryanslade/nsq-ocaml 15

Ocaml client library for the NSQ messaging platform

arussellsaw/e2e 1

a package for testing your infrastructure end to end, with an interface identical to testing.T

ryanslade/apns 1

Go APNS

create barnchsourcegraph/sourcegraph

branch : repo_sync_worker

created branch time in a day

issue commentsourcegraph/sourcegraph

Cloud: 3.19 Tracking issue

Team update for Goal 1:

We finished up RFC 211 which is now approved.

We broke out the engineering tasks required to implement the RFC, #12699 . Our goal was to try and break it down to tasks that are all smaller than one day of effort.

Currently:

  • @asdine is working on migrating source information from the sources column on the external_services table to a new external_service_repos table as part of the "Database changes" section of the tracking issue
  • @unknwon is working through the "Fetch external services section" of the tracking issue
  • @ryanslade is working though the "Repo syncing section" of the tracking issue

We also synced with @daxmc99 and @chayim to ensure that we're not stepping on each other's toes regarding DB changes and have a clear path forward where the two sub teams can work independently.

tsenart

comment created time in a day

push eventsourcegraph/sourcegraph

Ryan Slade

commit sha 8436ad194ddb8624a2b7b3d5acfe54320ca10c02

usagestats: Use context for Redis connections (#12787)

view details

push time in 2 days

delete branch sourcegraph/sourcegraph

delete branch : redis_context

delete time in 2 days

pull request commentsourcegraph/sourcegraph

usagestats: Use context for Redis connections

I noticed this in some lint checks on another PR

ryanslade

comment created time in 2 days

PR opened sourcegraph/sourcegraph

Reviewers
usagestats: Use context for Redis connections
+25 -13

0 comment

4 changed files

pr created time in 2 days

create barnchsourcegraph/sourcegraph

branch : redis_context

created branch time in 2 days

Pull request review commentsourcegraph/sourcegraph

RFC211: list external services allow filtering by user

 func (*schemaResolver) DeleteExternalService(ctx context.Context, args *struct { 	return &EmptyResponse{}, nil } -func (r *schemaResolver) ExternalServices(ctx context.Context, args *struct {+type ExternalServicesArgs struct {+	Namespace *graphql.ID 	graphqlutil.ConnectionArgs-}) (*externalServiceConnectionResolver, error) {-	// 🚨 SECURITY: Only site admins may read external services (they have secrets).-	if err := backend.CheckCurrentUserIsSiteAdmin(ctx); err != nil {-		return nil, err+}++func (r *schemaResolver) ExternalServices(ctx context.Context, args *ExternalServicesArgs) (*externalServiceConnectionResolver, error) {+	var namespaceUserID int32+	if args.Namespace != nil {+		var err error+		switch relay.UnmarshalKind(*args.Namespace) {+		case "User":+			err = relay.UnmarshalSpec(*args.Namespace, &namespaceUserID)+		default:+			err = errors.Errorf("invalid namespace %q", *args.Namespace)+		}++		if err != nil {+			return nil, err+		}+	}++	// 🚨 SECURITY: Only site admins may read all or a user's external services.+	// Otherwise, the authenticated user can only read external services under the same namespace.+	if backend.CheckSiteAdminOrSameUser(ctx, namespaceUserID) != nil {+		// NOTE: We do not directly return the err here because it contains the desired username,+		// which then allows attacker to brute force over our database ID and get corresponding+		// username.+		return nil, backend.ErrMustBeSiteAdmin

I agree the check is correct, but the error is too specific as it is must be site admin, it should be something like must be site admin or owner of external service

unknwon

comment created time in 3 days

issue commentsourcegraph/sourcegraph

Update ExternalServicesStore.List to allow filtering by user

Are you going to add pagination as part of this?

unknwon

comment created time in 3 days

issue closedsourcegraph/sourcegraph

Add sync time columns to the external_services table

We need to record last_synced_at and next_sync_at timestamps per external service to support the move to having a syncer per external service.

Part of: https://github.com/sourcegraph/sourcegraph/issues/12699

closed time in 3 days

ryanslade

Pull request review commentsourcegraph/sourcegraph

RFC211: list external services allow filtering by user

 func (*schemaResolver) DeleteExternalService(ctx context.Context, args *struct { 	return &EmptyResponse{}, nil } -func (r *schemaResolver) ExternalServices(ctx context.Context, args *struct {+type ExternalServicesArgs struct {+	Namespace *graphql.ID 	graphqlutil.ConnectionArgs-}) (*externalServiceConnectionResolver, error) {-	// 🚨 SECURITY: Only site admins may read external services (they have secrets).-	if err := backend.CheckCurrentUserIsSiteAdmin(ctx); err != nil {-		return nil, err+}++func (r *schemaResolver) ExternalServices(ctx context.Context, args *ExternalServicesArgs) (*externalServiceConnectionResolver, error) {+	var namespaceUserID int32+	if args.Namespace != nil {+		var err error+		switch relay.UnmarshalKind(*args.Namespace) {+		case "User":+			err = relay.UnmarshalSpec(*args.Namespace, &namespaceUserID)+		default:+			err = errors.Errorf("invalid namespace %q", *args.Namespace)+		}++		if err != nil {+			return nil, err+		}+	}++	// 🚨 SECURITY: Only site admins may read all or a user's external services.+	// Otherwise, the authenticated user can only read external services under the same namespace.+	if backend.CheckSiteAdminOrSameUser(ctx, namespaceUserID) != nil {+		// NOTE: We do not directly return the err here because it contains the desired username,+		// which then allows attacker to brute force over our database ID and get corresponding+		// username.+		return nil, backend.ErrMustBeSiteAdmin

This will also fail if the current user doesn't match namespaceUserID so we should update the error

unknwon

comment created time in 3 days

PR opened sourcegraph/about

Reviewers
Add a section on referrals
+4 -0

0 comment

1 changed file

pr created time in 3 days

create barnchsourcegraph/about

branch : ryanslade-patch-1

created branch time in 3 days

push eventsourcegraph/sourcegraph

Ryan Slade

commit sha 55116953b64e2542fa70438858bc30320e25a88c

db: Add last_sync_at, next_sync_at and namespace_user_id (#12713) * Add last_sync_at and next_sync_at to external services * Update repo store * Add new columns to ExternalService in api and types * Add namespace_user_id to external_services * Populate NamespaceUserID on external service types * Regenerate db schema

view details

push time in 4 days

delete branch sourcegraph/sourcegraph

delete branch : add_external_service_columns

delete time in 4 days

PR merged sourcegraph/sourcegraph

Reviewers
db: Add last_sync_at, next_sync_at and namespace_user_id

Add last_sync_at, next_sync_at and namespace_user_id columns to the external_services table.

This change only adds the columns and updates the SQL level functions that update and scan columns so that we can unblock works on RFC 211.

One additional change is that we now populate the DeletedAt field on types.ExternalService which seems to have been missed before.

Part of: https://github.com/sourcegraph/sourcegraph/issues/12701

+178 -52

1 comment

11 changed files

ryanslade

pr closed time in 4 days

push eventsourcegraph/sourcegraph

Ryan Slade

commit sha 44c5cc216bf09fd022780bb9d81fabe84fb37cc4

Regenerate db schema

view details

push time in 4 days

PR opened sourcegraph/sourcegraph

db: Add last_sync_at, next_sync_at and namespace_user_id

Add last_sync_at, next_sync_at and namespace_user_id columns to the external_services table.

This change only adds the columns and updates the SQL level functions that update and scan columns so that we can unblock works on RFC 211.

+165 -43

0 comment

11 changed files

pr created time in 4 days

create barnchsourcegraph/sourcegraph

branch : add_external_service_columns

created branch time in 4 days

delete branch sourcegraph/sourcegraph

delete branch : 3.16

delete time in 4 days

delete branch sourcegraph/sourcegraph

delete branch : testutil-deepcompare

delete time in 4 days

delete branch sourcegraph/sourcegraph

delete branch : source-rate-limit

delete time in 4 days

delete branch sourcegraph/sourcegraph

delete branch : external_service_sync_columns

delete time in 4 days

create barnchsourcegraph/sourcegraph

branch : external_service_sync_columns

created branch time in 4 days

issue openedsourcegraph/sourcegraph

Add sync time columns to the external_services table

We need to record last_synced_at and next_sync_at timestamps per external service to support the move to having a syncer per external service.

Part of: https://github.com/sourcegraph/sourcegraph/issues/12699

created time in 4 days

issue openedsourcegraph/sourcegraph

RFC 211 Tracking Issue

Database changes

  • [ ] Create the external_service_repos table and migrate external_service sources column data to that table. Update stores to read sources data from that new table.
  • [ ] Add namespace_user_id column to the external_services table and update stores to load and store data from that column.
  • [ ] Update repo-updater to only sync private repos that were added by at least one external service owned by a site-admin
  • [ ] Fetching external services
  • [ ] Update ExternalServicesStore.List to allow filtering by user, and site admins are able to list all.
  • [ ] Update call sites of ExternalServicesStore.List to avoid application-level aggregation over all external services, instead accomplish via SQL query.
  • [ ] Update ExternalServicesStore.List to support pagination, and update corresponding GraphQL queries.
  • [ ] Update repo-updater SyncRateLimiter to support pagination.
  • [ ] Update API clients of external services endpoints to support pagination.

Repo syncing

  • [ ] Add last_synced_at and next_sync to external services table
  • [ ] Create worker function that syncs single external service and updates next_sync
  • [ ] Note that it was recommended to ignore repositoryPathPattern
  • [ ] Use internal/workerutil to schedule concurrent syncing of external services based on next_sync time. Worker concurrency should be configurable
  • [ ] Create repo cleanup process to soft delete repos that are no longer linked to any external services

Search

  • [ ] Amend search logic to fetch a list of repos configured by a user and include them by default when no repo filter is provided.

Handling user deletion

  • [ ] When a user is deleted, update their external services deleted_at column.
  • [ ] When a user is nuked, delete any external service they own.
  • [ ] Ensure any entry in external_service_repos is deleted in cascade
  • [ ] When an external service is (soft or hard?) deleted, determine which repos are orphaned by counting the number of references in the external_service_repos table and marked them as deleted.
  • [ ] This must be done within the same transaction as the one deleting the external service

Progressive rollout

  • [ ] Add the external_service_user_mode feature flag as part of the configuration. By default, it will be disabled but if it contains public it will be considered enabled for public code. Anything other than public will disable the flag. Use the flag to protect the external service config page and related GraphQL api to users.
  • [ ] Add external_service_user_allow_list configuration option. It will contain a list of regex patterns that must be used to limit the users who will be able to create external services if the external_service_user_mode flag is enabled.
  • [ ] Add external_service_user_allow_percentage configuration option. It will contain a value from 0 to 100. If the external_service_user_mode flag is enabled, use that value to limit the number of users that can create external services, by hashing their user id and applying a modulo 100 on it. If the result is under that value, the user has access to that feature.
  • [ ] If a user is already allowed by external_service_user_allow_list, ignore this check

Metrics

  • [ ] Track number of code host connections added by users
  • [ ] Track total number of repositories added by users
  • [ ] Average repos per user can be derived from the above
  • [ ] Track whether a user has added any external service
  • [ ] Given the above, we can derive the visits per user with their own external services vs without
  • [ ] Track when feature flag is enabled

Abuse mitigation

  • [ ] Update repo-updater to support user-added repo listing limits per user.
  • [ ] Update repo-updater to support user-added repo listing limits in total.

UI Changes

  • [ ] Replicate pages for “manage repositories”.
  • [ ] Backend changes to allow CRUD of external services by users.

created time in 4 days

issue commentsourcegraph/sourcegraph

Cloud: 3.19 Tracking issue

Goal 1 Progress

Did a number of pairing sessions with @asdine on RFC 211 as we iterated based on feedback to the point where we think it's ready for implementation. We aimed to reduce the scope of the RFC to just what is required to support public code on public code hosts while keeping in mind that private code and organisation support is next. We wanted to focus on getting the groundwork right.

Looked into some memory and CPU profiles on Sourcegraph.com to see if there was any low hanging fruit there. Fixed one issue which was leading to a large number of pending timers (https://github.com/sourcegraph/sourcegraph/pull/12553)

This week

Extract engineering tasks from RFC 211 and work with @asdine and @joe on implementation.

tsenart

comment created time in 5 days

delete branch sourcegraph/sourcegraph

delete branch : cache-mutex

delete time in 8 days

PR closed sourcegraph/sourcegraph

Reviewers
db: Use mutex around cache

This ensures that when the cache has expired, only one goroutine will refresh it while others will block until the it has refreshed.

+14 -8

3 comments

1 changed file

ryanslade

pr closed time in 8 days

pull request commentsourcegraph/sourcegraph

db: Use mutex around cache

Ok, fair enough.

Another option I considered was keeping atomic.Value and returning stale cache data and kicking off a background process to update the cache.. but that's even more complicated 😄

ryanslade

comment created time in 8 days

push eventsourcegraph/sourcegraph

Ryan Slade

commit sha 5709b5a2384c9f6bfcd40fd422f85ab094c211c6

Update comment again

view details

push time in 9 days

pull request commentsourcegraph/sourcegraph

db: Use mutex around cache

@keegancsmith hah, I think using a mutex may actually be better 😄

ryanslade

comment created time in 9 days

push eventsourcegraph/sourcegraph

Ryan Slade

commit sha 0ae8dcc5fe68676252914e4a829abf0189c53cd8

Update comment

view details

push time in 9 days

PR opened sourcegraph/sourcegraph

db: Use mutex around cache

This ensures that when the cache has expired, only one goroutine will refresh it while others will block until the it has refreshed.

+14 -8

0 comment

1 changed file

pr created time in 9 days

create barnchsourcegraph/sourcegraph

branch : cache-mutex

created branch time in 9 days

push eventsourcegraph/sourcegraph

Ryan Slade

commit sha 4d254b2f7e5c6e9c451a74f8eb7112a7128143dd

conf: More efficient deadlock check (#12555) Perform a non blocking check early so that we have a fast path when we know config is already initialsed.

view details

push time in 9 days

delete branch sourcegraph/sourcegraph

delete branch : improve-deadlock-check

delete time in 9 days

PR merged sourcegraph/sourcegraph

Reviewers
conf: More efficient deadlock check

Perform a non blocking check early so that we have a fast path when we know config is already initialised.

+37 -27

0 comment

1 changed file

ryanslade

pr closed time in 9 days

PR opened sourcegraph/sourcegraph

Reviewers
conf: More efficient deadlock check

Perform a non blocking check early so that we have a fast path when we know config is already initialised.

+37 -27

0 comment

1 changed file

pr created time in 9 days

create barnchsourcegraph/sourcegraph

branch : improve-deadlock-check

created branch time in 9 days

push eventsourcegraph/sourcegraph

Ryan Slade

commit sha a4d9c17aa0785b19f829fcf162fea2b19a350967

conf: Cancel timer to free goroutine (#12553) Previously all calls to conf.Get, via WaitUntilInitialized, would spawn a timer that would leave a pending goroutine running, even if config had already been initialised. The default timeout period is 5 minutes so this can lead to a large number of pending routines.

view details

push time in 9 days

delete branch sourcegraph/sourcegraph

delete branch : cancel-timer

delete time in 9 days

PR merged sourcegraph/sourcegraph

Reviewers
conf: Cancel timer to free goroutine

Previously all calls to WaitUntilInitialized via conf.Get, would spawn a timer that would leave a pending goroutine running, even if config had already been initialised.

The default timeout period is 5 minutes so this can lead to a large number of pending routines.

<!-- Reminder: Have you updated the changelog and relevant docs (user docs, architecture diagram, etc) ? -->

+4 -1

1 comment

1 changed file

ryanslade

pr closed time in 9 days

Pull request review commentsourcegraph/sourcegraph

search: more trace logging

 func (m *meteredSearcher) Search(ctx context.Context, q query.Q, opts *zoekt.Sea 		code = "error" 	} -	requestDuration.WithLabelValues(m.hostname, cat, code).Observe(time.Since(start).Seconds())+	d := time.Since(start)+	requestDuration.WithLabelValues(m.hostname, cat, code).Observe(d.Seconds())  	tr.SetError(err) 	if zsr != nil { 		tr.LogFields( 			log.Int("filematches", len(zsr.Files)),-			log.Object("stats", &zsr.Stats),+			log.Int64("rpc.latencyms", (d-zsr.Stats.Duration-zsr.Stats.Wait).Milliseconds()),

nit: You capitalised MS below

			log.Int64("rpc.latencyMS", (d-zsr.Stats.Duration-zsr.Stats.Wait).Milliseconds()),
keegancsmith

comment created time in 9 days

pull request commentsourcegraph/sourcegraph

conf: Cancel timer to free goroutine

I noticed this when looking at an inuse_space profile:

Screenshot 2020-07-30 at 11 54 59

ryanslade

comment created time in 9 days

PR opened sourcegraph/sourcegraph

conf: Cancel timer to free goroutine

Previously all calls to conf.Get, via WaitUntilInitialized, would spawn a timer that would leave a pending goroutine running, even if config had already been initialised.

The default timeout period is 5 minutes so this can lead to a large number of pending routines.

<!-- Reminder: Have you updated the changelog and relevant docs (user docs, architecture diagram, etc) ? -->

+4 -1

0 comment

1 changed file

pr created time in 9 days

create barnchsourcegraph/sourcegraph

branch : cancel-timer

created branch time in 9 days

PR closed sourcegraph/graphql-go

Reuse byte slices from relay handler

When using the relay handler the previous code would:

Create a empty byte buffer Use it to build up the response (as raw json) Return the raw bytes, converted to a json.RawMessage Marshal the RawMessage which simply returns the bytes unhanged Write the bytes to the a ResponseWriter Throw away the bytes

The new code instead passes the final bytes back into a buffer pool so that they can be reused when handling subsequent requests.

+38 -10

2 comments

3 changed files

ryanslade

pr closed time in 12 days

pull request commentsourcegraph/graphql-go

Reuse byte slices from relay handler

I did some testing locally and although this did reduce the number of allocations I don't think it's worth the hack. We still end up with a lot of byte allocations due to json encoding in many places and I think a better long term option is actually going to be to investigate some other graphql libraries.

ryanslade

comment created time in 12 days

delete branch sourcegraph/sourcegraph

delete branch : graphql-dep

delete time in 12 days

push eventsourcegraph/sourcegraph

Ryan Slade

commit sha 65d2eae2857724b745ad953f6b719a7bd1ee746c

go: Update out graphql-go dependency (#12459)

view details

push time in 12 days

PR merged sourcegraph/sourcegraph

go: Update our graphql-go dependency

This reverses a change we made to pool some buffers as it didn't help in practice and the code was more complicated

+3 -1

2 comments

2 changed files

ryanslade

pr closed time in 12 days

issue commentsourcegraph/sourcegraph

Cloud: 3.19 Tracking issue

Last Week

Worked with @unknwon and @asdine on the product document

Looked into memory leaks on the frontend. Haven't found the culprit yet but I've been increasing my understanding of where we're currently producing the most garbage. Currently it's mostly coming from the our graphql layer due to the amount of json encoding and reflection involved there.

This Week

I'll work with @asdine on preparing an RFC that proposes how we want to allow any user to create external services with the constraint that we only allow public code for now.

I'll continue investigating memory leaks as a second priority.

tsenart

comment created time in 12 days

Pull request review commentsourcegraph/graphql-go

Reuse byte slices from relay handler

 func (r *Request) execSelections(ctx context.Context, sels []selected.Selection, 	var fields []*fieldToExec 	collectFieldsToResolve(sels, s, resolver, &fields, make(map[string]*fieldToExec)) +	defer func() {+		for _, f := range fields {+			if f.out != nil {+				BytePool.Put(f.out.Bytes())+			}+		}+	}()+ 	if async { 		var wg sync.WaitGroup 		wg.Add(len(fields)) 		for _, f := range fields { 			go func(f *fieldToExec) {

Our change only limited the number of list items we handle concurrently, not fields: https://github.com/sourcegraph/graphql-go/blob/e67ac72893d245ce10092afba20bd2289a2c31c1/internal/exec/exec.go#L341

ryanslade

comment created time in 15 days

push eventsourcegraph/graphql-go

Ryan Slade

commit sha e67ac72893d245ce10092afba20bd2289a2c31c1

Rename out to buf

view details

push time in 15 days

push eventsourcegraph/graphql-go

Ryan Slade

commit sha 2589d57ce7e8136fd59e33099cb6e08912fd7a98

Revert "Use a buffer pool for field buffers" This reverts commit 7ee5d02fd6c4defbffcee26cc976d8232ac7e4c5.

view details

Ryan Slade

commit sha e542e89564849f50365081e6c76396b8c9eee490

Merge pull request #3 from sourcegraph/revert-pool Revert "Use a buffer pool for field buffers"

view details

Ryan Slade

commit sha c12f85f33c63c290c72b9c054ede2b86cec7565a

Merge branch 'master' into byte-pool

view details

push time in 15 days

pull request commentsourcegraph/graphql-go

Reuse byte slices from relay handler

This feels like a hack because I needed to expose the byte pool, but this accounts for a huge amount of allocations in frontend so I think it might be worth a try.

ryanslade

comment created time in 15 days

PR opened sourcegraph/graphql-go

Reuse byte slices from relay handler

When using the relay handler the previous code would:

Create a empty byte buffer Use it to build up the response (as raw json) Return the raw bytes, converted to a json.RawMessage Marshal the RawMessage which simply returns the bytes unhanged Write the bytes to the a ResponseWriter Throw away the bytes

The new code instead passes the final bytes back into a buffer pool so that they can be reused when handling subsequent requests.

+31 -18

0 comment

3 changed files

pr created time in 15 days

create barnchsourcegraph/graphql-go

branch : byte-pool

created branch time in 15 days

pull request commentsourcegraph/sourcegraph

go: Update our graphql-go dependency

I'm going to hold off on merging this until I've investigated another approach

ryanslade

comment created time in 15 days

push eventsourcegraph/sourcegraph

Ryan Slade

commit sha c70637d0ce3093ea8d1ad9d2ae71d93d1ecc183f

basestore: Fix doc comment (#12429)

view details

push time in 15 days

delete branch sourcegraph/sourcegraph

delete branch : store-comment

delete time in 15 days

PR merged sourcegraph/sourcegraph

Reviewers
basestore: Fix doc comment

<!-- Reminder: Have you updated the changelog and relevant docs (user docs, architecture diagram, etc) ? -->

+2 -2

0 comment

1 changed file

ryanslade

pr closed time in 15 days

pull request commentsourcegraph/sourcegraph

go: Update out graphql-go dependency

This reverses a change we made to pool some buffers as it didn't help in practice and the code was more complicated.

ryanslade

comment created time in 16 days

create barnchsourcegraph/sourcegraph

branch : graphql-dep

created branch time in 16 days

push eventsourcegraph/graphql-go

Ryan Slade

commit sha 2589d57ce7e8136fd59e33099cb6e08912fd7a98

Revert "Use a buffer pool for field buffers" This reverts commit 7ee5d02fd6c4defbffcee26cc976d8232ac7e4c5.

view details

Ryan Slade

commit sha e542e89564849f50365081e6c76396b8c9eee490

Merge pull request #3 from sourcegraph/revert-pool Revert "Use a buffer pool for field buffers"

view details

push time in 16 days

PR merged sourcegraph/graphql-go

Revert "Use a buffer pool for field buffers"

This reverts commit 7ee5d02fd6c4defbffcee26cc976d8232ac7e4c5.

+2 -18

1 comment

1 changed file

ryanslade

pr closed time in 16 days

push eventsourcegraph/zoekt

Ryan Slade

commit sha 4e4dd94c5d1ee9b10eb1be5bd25088d4a77ca07f

Fix spelling (#51)

view details

push time in 16 days

delete branch sourcegraph/zoekt

delete branch : fix-typo

delete time in 16 days

PR merged sourcegraph/zoekt

Fix spelling
+2 -2

0 comment

1 changed file

ryanslade

pr closed time in 16 days

pull request commentsourcegraph/graphql-go

Revert "Use a buffer pool for field buffers"

In practice this doesn't seem to help for out use case and I'd rather keep the previous code as it's simpler.

ryanslade

comment created time in 16 days

PR opened sourcegraph/graphql-go

Revert "Use a buffer pool for field buffers"

This reverts commit 7ee5d02fd6c4defbffcee26cc976d8232ac7e4c5.

+2 -18

0 comment

1 changed file

pr created time in 16 days

create barnchsourcegraph/graphql-go

branch : revert-pool

created branch time in 16 days

PR opened sourcegraph/zoekt

Fix spelling
+2 -2

0 comment

1 changed file

pr created time in 16 days

create barnchsourcegraph/zoekt

branch : fix-typo

created branch time in 16 days

delete branch sourcegraph/sourcegraph

delete branch : graphql-dep

delete time in 16 days

push eventsourcegraph/sourcegraph

Ryan Slade

commit sha f085ae76798b6c7f8351875fb4587d3a564a0923

go: Update our graphql-go dependency (#12430)

view details

push time in 16 days

PR merged sourcegraph/sourcegraph

go: Update our graphql-go dependency

To include this commit: https://github.com/sourcegraph/graphql-go/commit/8462b0b708c44d9a44eade8973be857b1a05029a

+3 -1

1 comment

2 changed files

ryanslade

pr closed time in 16 days

create barnchsourcegraph/sourcegraph

branch : graphql-dep

created branch time in 17 days

push eventsourcegraph/graphql-go

Ryan Slade

commit sha 7ee5d02fd6c4defbffcee26cc976d8232ac7e4c5

Use a buffer pool for field buffers Previsouly we created a large number of short lived buffers. Instead we can place them in a pool to reduce allocations and reduce pressure on the garbage collector.

view details

Ryan Slade

commit sha 8462b0b708c44d9a44eade8973be857b1a05029a

Merge pull request #2 from sourcegraph/buffer-pool Use a buffer pool for field buffers

view details

push time in 17 days

PR merged sourcegraph/graphql-go

Use a buffer pool for field buffers

Previously we created a large number of short lived buffers. Instead we can place them in a pool to reduce allocations and reduce pressure on the garbage collector.

+18 -2

1 comment

1 changed file

ryanslade

pr closed time in 17 days

PR opened sourcegraph/sourcegraph

Reviewers
basestore: Fix doc comment

<!-- Reminder: Have you updated the changelog and relevant docs (user docs, architecture diagram, etc) ? -->

+2 -2

0 comment

1 changed file

pr created time in 17 days

pull request commentsourcegraph/sourcegraph

codeintel: Add Lock to store

LGTM but worth pointing out that after looking into advisory locks for some work to scale repo-updater we decided against using them.

The main issue is that if a task is holding the lock and loses its connection to the db the lock is released. If another worker then attempts to grab the lock it'll succeed and you'll end up with two workers who think they hold the lock.

Instead we decided to use Redis via https://github.com/sourcegraph/sourcegraph/blob/5cf616b29a9c53a6e6ab7784aaebe7596cc8a3a5/internal/leader/leader.go#L27

efritz

comment created time in 17 days

create barnchsourcegraph/sourcegraph

branch : store-comment

created branch time in 17 days

pull request commentsourcegraph/graphql-go

Use a buffer pool for field buffers

A memory profile of our frontend showed this as a large contributor to overall allocations

ryanslade

comment created time in 17 days

PR opened sourcegraph/graphql-go

Use a buffer pool for field buffers

Previsouly we created a large number of short lived buffers. Instead we can place them in a pool to reduce allocations and reduce pressure on the garbage collector.

+18 -2

0 comment

1 changed file

pr created time in 17 days

create barnchsourcegraph/graphql-go

branch : buffer-pool

created branch time in 17 days

Pull request review commentsourcegraph/sourcegraph

authz: remove on-demand permissions fetching

 All notable changes to Sourcegraph are documented in this file. - Configuration for `observability.alerts` has changed and notifications are now provided by Prometheus Alertmanager. [#11832](https://github.com/sourcegraph/sourcegraph/pull/11832)   - Removed: `observability.alerts.id`.   - Removed: Slack notifiers no longer accept `mentionUsers`, `mentionGroups`, `mentionChannel`, and `token` options.+- [Background permissions syncing](https://docs.sourcegraph.com/admin/repo/permissions#background-permissions-syncing) (`permissions.backgroundSync`) has become the only option for mirroring repository permissions from code hosts. All relevant site configrations are deprecated.
- [Background permissions syncing](https://docs.sourcegraph.com/admin/repo/permissions#background-permissions-syncing) (`permissions.backgroundSync`) has become the only option for mirroring repository permissions from code hosts. All relevant site configurations are deprecated.
unknwon

comment created time in 18 days

issue commentsourcegraph/sourcegraph

[WIP] Cloud: 3.19 Tracking issue

In my sync with @christinaforney today, we had the idea of using the new Product Document template to capture the context around Goal 1. The exact format and prompts aren't fixed in stone (and you should change them as you see fit) but I think they provide very good structure, aligned with what we were thinking of writing. What do you think @unknwon, @ryanslade, @asdine? Check it out here: https://docs.google.com/document/d/1MBZxnRlDG69Fyvzpai5rBqxizvX5zVeZiUe6z7VZrjk/edit#heading=h.g0gjwch98szj

I think that makes sense after we've done the prototype. Initially I was thinking of a simpler document that would describe the feature as user documentation as if it had already been implemented.

tsenart

comment created time in 19 days

issue commentsourcegraph/sourcegraph

Cloud: 3.18 Tracking issue

Last Week

  • Finished work on RFC 191 which is now ready to be implemented
  • Added a leader election helper package

This week

  • Planning for 3.19
  • Looking into a slow memory leak on frontend: https://github.com/sourcegraph/sourcegraph/issues/11790
  • Adding tests for our raw endpoint: https://github.com/sourcegraph/sourcegraph/issues/11259
tsenart

comment created time in 19 days

push eventsourcegraph/sourcegraph

Ryan Slade

commit sha 9da26cb86fbf866625cd5133c23ee21c16dd124c

repo-updater: Sleep on error during syncClone (#12299) * repo-updater: Sleep on error during syncClone Instead of immediatley looping around and potentially putting more pressure on gitserver or the repo store. * Changes from review * Move err check * Refactor to remove sleep / continue pattern

view details

push time in 19 days

delete branch sourcegraph/sourcegraph

delete branch : error-backoff

delete time in 19 days

PR merged sourcegraph/sourcegraph

repo-updater: Sleep on error during syncClone

Instead of immediately looping around and potentially putting more pressure on gitserver or the repo store.

+6 -4

0 comment

1 changed file

ryanslade

pr closed time in 19 days

push eventsourcegraph/sourcegraph

Chayim

commit sha e4a647a4e2c043828519cc712be00ef65e7c796e

Secure Token Schema (#12207) Creating the migrations for the token table, database schema change to accommodate key-value pairs.

view details

Felix Becker

commit sha 24ee7987c4e671d0df8ae3609652dc5b082b7cf9

Increase contrast of placeholders to meet WCAG (#12213)

view details

Felix Becker

commit sha d81fc27fa3d444becf126bf8fc4b7da60c1c7631

Add integration tests for spaces in paths (#12297) * Add integration tests for spaces in paths * Use test-* classes

view details

Eric Fritz

commit sha 80c27d5e64abb974b1ae857c9b0757c3cdfe9627

Add docs for rolling updates to migration readme. (#12298)

view details

ᴜɴᴋɴᴡᴏɴ

commit sha 5ce8ae587d734252af7d8ab439510a8fddb4f421

gqltest: add docs to run and add tests (#12285) Co-authored-by: Rijnard van Tonder <rvantonder@gmail.com>

view details

Keegan Carruthers-Smith

commit sha 3cb30b5ff3f15ba152a9153873ea4b1ad3929519

honey: sample 1 in 16 events (#12317) Band-aid solution until we can investigate further. Again hitting very high usage. Likely due to recent scaling up of the indexed search cluster. Will require more investigation, but we should probably segment user request path.

view details

Ryan Slade

commit sha 74a64327bb69f14399d25b6293d464d0adefa13f

Merge branch 'master' into error-backoff

view details

push time in 20 days

push eventsourcegraph/sourcegraph

Ryan Slade

commit sha f1fc5598dffa6db7e46514caf6257b980d78a7ff

Move err check

view details

Ryan Slade

commit sha 939d11c6ab92bf3b6dbe69753e7abe81cc3a7d62

Refactor to remove sleep / continue pattern

view details

push time in 20 days

more