profile
viewpoint
Adam Harvey LawnGnome Sourcegraph Vancouver, British Columbia http://about.me/LawnGnome

LawnGnome/dubnium 5

Cross-platform GUI debugger for PHP/XDebug (and anything else that talks DBGp).

LawnGnome/broadcom-wl 1

Broadcom Linux hybrid wireless driver (64-bit)

LawnGnome/c25k-pebble 1

C25K watchapp for Pebble

LawnGnome/cinejs 1

Video and image processing in Javascript.

LawnGnome/fastcgi-react-adapter 1

An adapter that allows ebernhardson/fastcgi to be used within a React event loop.

LawnGnome/homebrew 1

The missing package manager for OS X.

LawnGnome/about 0

Sourcegraph blog, feature announcements, and website (about.sourcegraph.com)

LawnGnome/actiontec-insights 0

Actiontec V2200H → New Relic Insights

LawnGnome/animportantdate 0

Our open source wedding website built on top of Django.

issue commentsourcegraph/sourcegraph

Design a GraphQL API that would allow us to show either the "Update campaign" or "Create campaign" button on the apply page

You both know I'm Australian, right? Some people would argue I don't speak proper English.

I'd go with applies, though, for the same reasons @eseliger said.

mrnugget

comment created time in a day

pull request commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

Yeah I still have it in my database, and for some spurious reason, there actually is one event in the DB:

Thanks!

It's interesting. So there was a pipeline, which correctly created a pending event, and then it got cancelled. I'll investigate why that didn't turn into a more appropriate status on the next sync, if not on a webhook that wasn't recorded.

That said, I don't think this is on the critical path for this PR. I've filed #12667 to fix that issue. Beyond that, if there's no screaming, I'm going to merge this tomorrow morning. (For today is a holiday. I'm not here. This isn't happening.)

LawnGnome

comment created time in a day

issue openedsourcegraph/sourcegraph

campaigns: cancelled GitLab pipelines don't take check state out of pending

Originally posted by @eseliger in https://github.com/sourcegraph/sourcegraph/pull/12139#issuecomment-667984395

I won't reproduce the full comment here, but basically: a cancelled pipeline was synced and didn't change the check state from pending to whatever is appropriate (probably unknown). I need to investigate the offending pipeline and merge request in the API to ensure we're doing the right thing.

created time in a day

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

+package webhooks

It felt logically separate to me: while there's overlap between GitLab's REST API and webhooks (the types have most, but not all, of their fields in common, and the webhooks package reuses those types where appropriate), the webhook API also has plenty of its own quirks, and I preferred to isolate the behaviour we needed around typing and parsing webhook objects outside of the general gitlab package to avoid clutter and make it easier to reason about and test.

LawnGnome

comment created time in a day

issue commentsourcegraph/sourcegraph

Campaigns 3.19 Tracking issue

Weekly update: 2020-07-27 to 2020-07-31

Last week, I was optimistic that I would be done GitLab webhooks and could move on with my other big 3.19 tasks: most notably, CLI support. I was, sadly, wrong.

However, we made good progress, if slower than I would have liked. Erik did a terrific job testing the GitLab webhook PR, and found a couple of bugs I'm very glad we didn't ship. I also continued to bang my head against the inconsistencies between GitLab's webhooks and their other APIs. It's very quirky.

Beyond that, I'd been having trouble with BitBucket Server and personal repos, so I fixed that. I also put a fair amount of time into reviewing ongoing work. It feels like we're at a crucial point right now in shipping campaigns, between Thorsten's work to implement the new reconciler state machine and Erik's ongoing work on our frontend and GraphQL API, and I want to make sure that we're giving this important work the focus it deserves.

This week, after a public holiday Monday, it's full steam ahead on CLI design and implementation. Well, you know, just as soon as I merge that pesky GitLab webhook PR... :man_facepalming:

mrnugget

comment created time in a day

Pull request review commentsourcegraph/sourcegraph

Implement ApplyCampaign and changeset reconciler

 type ChangesetSpec struct {  	RawSpec string 	// TODO(mrnugget): should we rename the "spec" column to "description"?+	// TODO(mrnugget): should we make this a pointer? Otherwise we're passing

Yes, IMO.

mrnugget

comment created time in 4 days

Pull request review commentsourcegraph/sourcegraph

Implement ApplyCampaign and changeset reconciler

 type Changeset struct { 	ExternalState       ChangesetExternalState 	ExternalReviewState ChangesetReviewState 	ExternalCheckState  ChangesetCheckState-	CreatedByCampaign   bool-	AddedToCampaign     bool 	DiffStatAdded       *int32 	DiffStatChanged     *int32 	DiffStatDeleted     *int32 	SyncState           ChangesetSyncState++	// The campaign that "owns" this changeset: it can create/close it on code host.+	OwnedByCampaignID int64+	// Whether this changeset was created by a campaign on a code host.+	CreatedByCampaign bool

Is this field useful, given the presence of OwnedByCampaignID?

mrnugget

comment created time in 4 days

Pull request review commentsourcegraph/sourcegraph

Implement ApplyCampaign and changeset reconciler

 func (c *Campaign) GenChangesetBody(externalURL string) string { 	return description } -// ChangesetState defines the possible states of a Changeset.-type ChangesetState string+// ChangesetPublicationState defines the possible publication states of a Changeset.+type ChangesetPublicationState string  // ChangesetState constants. const (-	ChangesetStateUnpublished ChangesetState = "UNPUBLISHED"-	ChangesetStatePublishing  ChangesetState = "PUBLISHING"-	ChangesetStateErrored     ChangesetState = "ERRORED"-	ChangesetStateSynced      ChangesetState = "SYNCED"+	ChangesetPublicationStateUnpublished ChangesetPublicationState = "UNPUBLISHED"+	ChangesetPublicationStatePublished   ChangesetPublicationState = "PUBLISHED"+)++// Valid returns true if the given ChangesetPublicationState is valid.+func (s ChangesetPublicationState) Valid() bool {+	switch s {+	case ChangesetPublicationStateUnpublished, ChangesetPublicationStatePublished:+		return true+	default:+		return false+	}+}

This is exhibit A in my whitepaper on why Go's type system is insufficient. In this thesis, I wil… writer was eaten by a bear

mrnugget

comment created time in 4 days

Pull request review commentsourcegraph/sourcegraph

Implement ApplyCampaign and changeset reconciler

 import ( 	"github.com/inconshreveable/log15" 	"github.com/pkg/errors" 	"github.com/sourcegraph/go-diff/diff"+	"github.com/sourcegraph/sourcegraph/cmd/frontend/types"

This import feels a little icky (internal → cmd). Would CheckRepoSupported fit better in enterprise/internal/campaigns/service.go?

(Which has the same problem, admittedly, but it's more of a frontend concern.)

mrnugget

comment created time in 4 days

Pull request review commentsourcegraph/sourcegraph

Implement ApplyCampaign and changeset reconciler

 func testChangeset(repoID api.RepoID, campaign int64, changesetJob int64, extSta  	return changeset }++func createCampaign(t *testing.T, ctx context.Context, store *Store, name string, userID int32, spec int64) *campaigns.Campaign {+	t.Helper()++	c := &campaigns.Campaign{+		AuthorID:        userID,+		NamespaceUserID: userID,+		CampaignSpecID:  spec,+		Name:            name,+		Description:     "campaign description",+	}++	if err := store.CreateCampaign(ctx, c); err != nil {+		t.Fatal(err)+	}++	return c+}++func createCampaignSpec(t *testing.T, ctx context.Context, store *Store, name string, userID int32) *campaigns.CampaignSpec {+	t.Helper()++	s := &campaigns.CampaignSpec{+		UserID:          userID,+		NamespaceUserID: userID,+		Spec: campaigns.CampaignSpecFields{+			Name:        name,+			Description: "the description",+			ChangesetTemplate: campaigns.ChangesetTemplate{+				Branch: "branch-name",+			},+		},+	}++	if err := store.CreateCampaignSpec(ctx, s); err != nil {+		t.Fatal(err)+	}++	return s+}++type testSpecOpts struct {+	user         int32+	repo         api.RepoID+	campaignSpec int64++	// If this is non-blank, the changesetSpec will be an import/track spec for+	// the changeset with the given externalID in the given repo.+	externalID string++	// If this is set, the changesetSpec will be a "create commit on this+	// branch" changeset.+	headRef string+}++func createChangesetSpec(+	t *testing.T,+	ctx context.Context,+	store *Store,+	opts testSpecOpts,+) *campaigns.ChangesetSpec {+	t.Helper()++	spec := &campaigns.ChangesetSpec{+		UserID:         opts.user,+		RepoID:         opts.repo,+		CampaignSpecID: opts.campaignSpec,+		Spec: campaigns.ChangesetSpecDescription{+			BaseRepository: graphqlbackend.MarshalRepositoryID(opts.repo),+		},+	}++	spec.Spec.ExternalID = opts.externalID+	spec.Spec.HeadRef = opts.headRef++	if err := store.CreateChangesetSpec(ctx, spec); err != nil {+		t.Fatal(err)+	}++	return spec+}++func createTestRepos(t *testing.T, ctx context.Context, db *sql.DB) []*repos.Repo {+	t.Helper()++	rstore := repos.NewDBStore(db, sql.TxOptions{})++	ext := &repos.ExternalService{+		Kind:        extsvc.TypeGitHub,+		DisplayName: "GitHub",+		Config: marshalJSON(t, &schema.GitHubConnection{+			Url:   "https://github.com",+			Token: "SECRETTOKEN",+		}),+	}+	if err := rstore.UpsertExternalServices(ctx, ext); err != nil {+		t.Fatal(err)+	}++	var rs []*repos.Repo+	for i := 0; i < 4; i++ {+		r := testRepo(i, extsvc.TypeGitHub)+		r.Sources = map[string]*repos.SourceInfo{ext.URN(): {ID: ext.URN()}}++		rs = append(rs, r)+	}++	err := rstore.UpsertRepos(ctx, rs...)+	if err != nil {+		t.Fatal(err)+	}++	return rs+}++func createChangesetWithSpec(+	repoID api.RepoID,+	campaign int64,+	externalID string,+	spec int64,+	extState campaigns.ChangesetExternalState,+) *campaigns.Changeset {++	changeset := &campaigns.Changeset{+		RepoID:              repoID,+		ExternalServiceType: extsvc.TypeGitHub,+		ExternalID:          externalID,+		CurrentSpecID:       spec,+	}++	if campaign != 0 {+		changeset.CampaignIDs = []int64{campaign}+	}++	return changeset+}++type changesetAssertions struct {+	repo             api.RepoID+	currentSpec      int64+	previousSpec     int64+	ownedByCampaign  int64+	reconcilerState  campaigns.ReconcilerState+	publicationState campaigns.ChangesetPublicationState+	externalID       string+	externalBranch   string+}++func assertChangeset(t *testing.T, c *campaigns.Changeset, a changesetAssertions) {+	t.Helper()

Do you think we (I) should be more consistent on using this in our assertion and setup helpers?

mrnugget

comment created time in 4 days

Pull request review commentsourcegraph/sourcegraph

Implement ApplyCampaign and changeset reconciler

 func TestService(t *testing.T) { 		}) 	}) -	t.Run("MoveCampaign", func(t *testing.T) {-		svc := NewServiceWithClock(store, cf, clock)--		createCampaign := func(t *testing.T, name string, authorID, userID, orgID int32) *campaigns.Campaign {-			t.Helper()--			c := &campaigns.Campaign{-				AuthorID:        authorID,-				NamespaceUserID: userID,-				NamespaceOrgID:  orgID,-				Name:            name,+	// These tests focus on changesetSpecs and wiring them up with changesets.+	// The applying/re-applying of a campaignSpec to an existing campaign is+	// covered in the tests above.+	t.Run("campaignSpec with changesetSpecs", func(t *testing.T) {+		// We need to mock SyncChangesets because ApplyCampaign syncs+		// changesets. Once that moves to the background, we can remove this+		// mock.

:rocket:

mrnugget

comment created time in 4 days

Pull request review commentsourcegraph/sourcegraph

Implement ApplyCampaign and changeset reconciler

 func TestService(t *testing.T) { 		}) 	}) -	t.Run("MoveCampaign", func(t *testing.T) {-		svc := NewServiceWithClock(store, cf, clock)--		createCampaign := func(t *testing.T, name string, authorID, userID, orgID int32) *campaigns.Campaign {-			t.Helper()--			c := &campaigns.Campaign{-				AuthorID:        authorID,-				NamespaceUserID: userID,-				NamespaceOrgID:  orgID,-				Name:            name,+	// These tests focus on changesetSpecs and wiring them up with changesets.+	// The applying/re-applying of a campaignSpec to an existing campaign is+	// covered in the tests above.+	t.Run("campaignSpec with changesetSpecs", func(t *testing.T) {+		// We need to mock SyncChangesets because ApplyCampaign syncs+		// changesets. Once that moves to the background, we can remove this+		// mock.+		syncedBranchName := "synced-branch-name"+		MockSyncChangesets = func(_ context.Context, _ RepoStore, tx SyncStore, _ *httpcli.Factory, cs ...*campaigns.Changeset) error {+			for _, c := range cs {+				c.ExternalBranch = syncedBranchName+				if err := tx.UpdateChangesets(ctx, c); err != nil {+					return err+				} 			}+			return nil+		}+		t.Cleanup(func() { MockSyncChangesets = nil }) -			if err := store.CreateCampaign(ctx, c); err != nil {-				t.Fatal(err)-			}+		t.Run("new campaign", func(t *testing.T) {+			campaignSpec := createCampaignSpec(t, ctx, store, "campaign3", admin.ID) -			return c-		}+			spec1 := createChangesetSpec(t, ctx, store, testSpecOpts{+				user:         admin.ID,+				repo:         repos[0].ID,+				campaignSpec: campaignSpec.ID,+				externalID:   "1234",+			}) -		t.Run("new name", func(t *testing.T) {-			campaign := createCampaign(t, "old-name", admin.ID, admin.ID, 0)+			spec2 := createChangesetSpec(t, ctx, store, testSpecOpts{+				user:         admin.ID,+				repo:         repos[1].ID,+				campaignSpec: campaignSpec.ID,+				headRef:      "refs/heads/my-branch",+			}) -			opts := MoveCampaignOpts{CampaignID: campaign.ID, NewName: "new-name"}-			moved, err := svc.MoveCampaign(ctx, opts)-			if err != nil {-				t.Fatal(err)-			}+			campaign, cs := applyAndListChangesets(ctx, t, svc, campaignSpec.RandID, 2) -			if have, want := moved.Name, opts.NewName; have != want {-				t.Fatalf("wrong name. want=%q, have=%q", want, have)+			if have, want := campaign.Name, "campaign3"; have != want {+				t.Fatalf("wrong campaign name. want=%s, have=%s", want, have) 			}++			c1 := cs.Find(campaigns.WithExternalID(spec1.Spec.ExternalID))+			assertChangeset(t, c1, changesetAssertions{+				repo:             spec1.RepoID,+				externalBranch:   syncedBranchName,+				externalID:       "1234",+				reconcilerState:  campaigns.ReconcilerStateCompleted,+				publicationState: campaigns.ChangesetPublicationStatePublished,+			})++			c2 := cs.Find(campaigns.WithCurrentSpecID(spec2.ID))+			assertChangeset(t, c2, changesetAssertions{+				repo:             spec2.RepoID,+				currentSpec:      spec2.ID,+				ownedByCampaign:  campaign.ID,+				reconcilerState:  campaigns.ReconcilerStateQueued,+				publicationState: campaigns.ChangesetPublicationStateUnpublished,+			}) 		}) -		t.Run("new user namespace", func(t *testing.T) {-			campaign := createCampaign(t, "old-name", admin.ID, admin.ID, 0)+		t.Run("campaign with changesets", func(t *testing.T) {+			// First we create a campaignSpec and apply it, so that we have+			// changesets and changesetSpecs in the database, wired up+			// correctly.+			campaignSpec1 := createCampaignSpec(t, ctx, store, "campaign4", admin.ID)++			createChangesetSpec(t, ctx, store, testSpecOpts{+				user:         admin.ID,+				repo:         repos[0].ID,+				campaignSpec: campaignSpec1.ID,+				externalID:   "1234",+			}) -			user2 := createTestUser(ctx, t)+			createChangesetSpec(t, ctx, store, testSpecOpts{+				user:         admin.ID,+				repo:         repos[0].ID,+				campaignSpec: campaignSpec1.ID,+				externalID:   "5678",+			}) -			opts := MoveCampaignOpts{CampaignID: campaign.ID, NewNamespaceUserID: user2.ID}-			moved, err := svc.MoveCampaign(ctx, opts)-			if err != nil {-				t.Fatal(err)-			}+			oldSpec3 := createChangesetSpec(t, ctx, store, testSpecOpts{+				user:         admin.ID,+				repo:         repos[1].ID,+				campaignSpec: campaignSpec1.ID,+				headRef:      "refs/heads/repo-1-branch-1",+			}) -			if have, want := moved.NamespaceUserID, opts.NewNamespaceUserID; have != want {-				t.Fatalf("wrong NamespaceUserID. want=%d, have=%d", want, have)-			}+			oldSpec4 := createChangesetSpec(t, ctx, store, testSpecOpts{+				user:         admin.ID,+				repo:         repos[2].ID,+				campaignSpec: campaignSpec1.ID,+				headRef:      "refs/heads/repo-2-branch-1",+			}) -			if have, want := moved.NamespaceOrgID, opts.NewNamespaceOrgID; have != want {-				t.Fatalf("wrong NamespaceOrgID. want=%d, have=%d", want, have)-			}-		})+			// Apply and expect 4 changesets+			_, oldChangesets := applyAndListChangesets(ctx, t, svc, campaignSpec1.RandID, 4) -		t.Run("new user namespace but current user is not admin", func(t *testing.T) {-			campaign := createCampaign(t, "old-name", user.ID, user.ID, 0)+			// Now we create another campaign spec with the same campaign name+			// and namespace.+			campaignSpec2 := createCampaignSpec(t, ctx, store, "campaign4", admin.ID) -			user2 := createTestUser(ctx, t)+			// Same+			spec1 := createChangesetSpec(t, ctx, store, testSpecOpts{+				user:         admin.ID,+				repo:         repos[0].ID,+				campaignSpec: campaignSpec2.ID,+				externalID:   "1234",+			}) -			opts := MoveCampaignOpts{CampaignID: campaign.ID, NewNamespaceUserID: user2.ID}+			// DIFFERENT: Track #9999 in repo[0]+			spec2 := createChangesetSpec(t, ctx, store, testSpecOpts{+				user:         admin.ID,+				repo:         repos[0].ID,+				campaignSpec: campaignSpec2.ID,+				externalID:   "5678",+			}) -			userCtx := actor.WithActor(context.Background(), actor.FromUser(user.ID))-			_, err := svc.MoveCampaign(userCtx, opts)-			if !errcode.IsUnauthorized(err) {-				t.Fatalf("expected unauthorized error but got %s", err)-			}+			// Same+			spec3 := createChangesetSpec(t, ctx, store, testSpecOpts{+				user:         admin.ID,+				repo:         repos[1].ID,+				campaignSpec: campaignSpec2.ID,+				headRef:      "refs/heads/repo-1-branch-1",+			})++			// DIFFERENT: branch changed in repo[2]+			spec4 := createChangesetSpec(t, ctx, store, testSpecOpts{+				user:         admin.ID,+				repo:         repos[2].ID,+				campaignSpec: campaignSpec2.ID,+				headRef:      "refs/heads/repo-2-branch-2",+			})++			// NEW: repo[3]+			spec5 := createChangesetSpec(t, ctx, store, testSpecOpts{+				user:         admin.ID,+				repo:         repos[3].ID,+				campaignSpec: campaignSpec2.ID,+				headRef:      "refs/heads/repo-3-branch-1",+			})++			// Before we apply the new campaign spec, we set up the assertion+			// for changesets to be closed.+			wantClosed := oldChangesets.Find(campaigns.WithCurrentSpecID(oldSpec4.ID))+			// We need to make it look "published", otherwise it won't be closed.+			setChangesetPublished(t, ctx, store, wantClosed, oldSpec4.Spec.HeadRef, "98765")++			verifyClosed := assertChangesetsClose(t, wantClosed)++			// Apply and expect 5 changesets+			campaign, cs := applyAndListChangesets(ctx, t, svc, campaignSpec2.RandID, 5)++			verifyClosed()++			c1 := cs.Find(campaigns.WithExternalID(spec1.Spec.ExternalID))+			assertChangeset(t, c1, changesetAssertions{+				repo:             repos[0].ID,+				currentSpec:      0,+				previousSpec:     0,+				externalBranch:   syncedBranchName,+				externalID:       "1234",+				reconcilerState:  campaigns.ReconcilerStateCompleted,+				publicationState: campaigns.ChangesetPublicationStatePublished,+			})++			c2 := cs.Find(campaigns.WithExternalID(spec2.Spec.ExternalID))+			assertChangeset(t, c2, changesetAssertions{+				repo:             repos[0].ID,+				currentSpec:      0,+				previousSpec:     0,+				externalBranch:   syncedBranchName,+				externalID:       "5678",+				reconcilerState:  campaigns.ReconcilerStateCompleted,+				publicationState: campaigns.ChangesetPublicationStatePublished,+			})++			c3 := cs.Find(campaigns.WithCurrentSpecID(spec3.ID))+			assertChangeset(t, c3, changesetAssertions{+				repo:             repos[1].ID,+				currentSpec:      spec3.ID,+				previousSpec:     oldSpec3.ID,+				ownedByCampaign:  campaign.ID,+				reconcilerState:  campaigns.ReconcilerStateQueued,+				publicationState: campaigns.ChangesetPublicationStateUnpublished,+			})++			c4 := cs.Find(campaigns.WithCurrentSpecID(spec4.ID))+			assertChangeset(t, c4, changesetAssertions{+				repo:             repos[2].ID,+				currentSpec:      spec4.ID,+				ownedByCampaign:  campaign.ID,+				reconcilerState:  campaigns.ReconcilerStateQueued,+				publicationState: campaigns.ChangesetPublicationStateUnpublished,+			})++			c5 := cs.Find(campaigns.WithCurrentSpecID(spec5.ID))+			assertChangeset(t, c5, changesetAssertions{+				repo:             repos[3].ID,+				currentSpec:      spec5.ID,+				ownedByCampaign:  campaign.ID,+				reconcilerState:  campaigns.ReconcilerStateQueued,+				publicationState: campaigns.ChangesetPublicationStateUnpublished,+			}) 		}) -		t.Run("new org namespace", func(t *testing.T) {-			campaign := createCampaign(t, "old-name", admin.ID, admin.ID, 0)+		t.Run("campaign tracking changesets owned by another campaign", func(t *testing.T) {

I'm really glad you added this specific case. It's one of the ones I'm most worried about. Thanks!

mrnugget

comment created time in 4 days

Pull request review commentsourcegraph/sourcegraph

Implement ApplyCampaign and changeset reconciler

 func (s *Service) ApplyCampaign(ctx context.Context, opts ApplyCampaignOpts) (ca 	campaign.NamespaceOrgID = campaignSpec.NamespaceOrgID 	campaign.NamespaceUserID = campaignSpec.NamespaceUserID 	campaign.Name = campaignSpec.Spec.Name-	campaign.Description = campaignSpec.Spec.Description +	campaign.Description = campaignSpec.Spec.Description 	// TODO(mrnugget): This doesn't need to be populated, since the branch is 	// now ChangesetSpec.Spec.HeadRef. 	campaign.Branch = campaignSpec.Spec.ChangesetTemplate.Branch  	if campaign.ID == 0 {-		return campaign, tx.CreateCampaign(ctx, campaign)+		err := tx.CreateCampaign(ctx, campaign)+		if err != nil {+			return nil, err+		}+	}++	// Now we need to wire up the ChangesetSpecs of the new CampaignSpec+	// correctly with the Changesets so that the reconciler can create/update+	// them.++	// Load all of the new ChangesetSpecs+	newChangesetSpecs, _, err := tx.ListChangesetSpecs(ctx, ListChangesetSpecsOpts{+		Limit:          -1,+		CampaignSpecID: campaign.CampaignSpecID,+	})+	if err != nil {+		return nil, err+	}++	// Load all Changesets attached to this Campaign.+	changesets, _, err := tx.ListChangesets(ctx, ListChangesetsOpts{CampaignID: campaign.ID})+	if err != nil {+		return nil, err+	}++	// We load all the repositories involved, checking for repository permissions+	// under the hood.+	repoIDs := make([]api.RepoID, 0, len(newChangesetSpecs)+len(changesets))+	for _, spec := range newChangesetSpecs {+		repoIDs = append(repoIDs, spec.RepoID)+	}+	for _, changeset := range changesets {+		repoIDs = append(repoIDs, changeset.RepoID)+	}+	accessibleReposByID, err := accessibleRepos(ctx, repoIDs)+	if err != nil {+		return nil, err+	}++	// Now we have two lists:++	// ┌───────────────────────────────────────┐   ┌───────────────────────────────┐+	// │Changeset 1 | Repo A | #111 | run-gofmt│   │  Spec 1 | Repo A | run-gofmt  │+	// └───────────────────────────────────────┘   └───────────────────────────────┘+	// ┌───────────────────────────────────────┐   ┌───────────────────────────────┐+	// │Changeset 2 | Repo B |      | run-gofmt│   │  Spec 2 | Repo B | run-gofmt  │+	// └───────────────────────────────────────┘   └───────────────────────────────┘+	// ┌───────────────────────────────────────┐   ┌───────────────────────────────────┐+	// │Changeset 3 | Repo C | #222 | run-gofmt│   │  Spec 3 | Repo C | run-goimports  │+	// └───────────────────────────────────────┘   └───────────────────────────────────┘+	// ┌───────────────────────────────────────┐   ┌───────────────────────────────┐+	// │Changeset 4 | Repo C | #333 | older-pr │   │    Spec 4 | Repo C | #333     │+	// └───────────────────────────────────────┘   └───────────────────────────────┘++	// We need to:+	// 1. Find out whether our new specs should _update_ an existing+	//    changeset, or whether we need to create a new one.+	// 2. Since we can have multiple changesets per repository, we need to match+	//    based on repo and external ID.+	// 3. But if a changeset wasn't published yet, it doesn't have an external ID.+	//    In that case, we need to check whether the branch on which we _might_+	//    push the commit (because the changeset might not be published+	//    yet) is the same.++	// What we want:+	//+	// ┌───────────────────────────────────────┐    ┌───────────────────────────────┐+	// │Changeset 1 | Repo A | #111 | run-gofmt│───▶│  Spec 1 | Repo A | run-gofmt  │+	// └───────────────────────────────────────┘    └───────────────────────────────┘+	// ┌───────────────────────────────────────┐    ┌───────────────────────────────┐+	// │Changeset 2 | Repo B |      | run-gofmt│───▶│  Spec 2 | Repo B | run-gofmt  │+	// └───────────────────────────────────────┘    └───────────────────────────────┘+	// ┌───────────────────────────────────────┐+	// │Changeset 3 | Repo C | #222 | run-gofmt│+	// └───────────────────────────────────────┘+	// ┌───────────────────────────────────────┐    ┌───────────────────────────────┐+	// │Changeset 4 | Repo C | #333 | older-pr │───▶│    Spec 4 | Repo C | #333     │+	// └───────────────────────────────────────┘    └───────────────────────────────┘+	// ┌───────────────────────────────────────┐    ┌───────────────────────────────────┐+	// │Changeset 5 | Repo C | | run-goimports │───▶│  Spec 3 | Repo C | run-goimports  │+	// └───────────────────────────────────────┘    └───────────────────────────────────┘+	//+	// Spec 1 should be attached to Changeset 1 and (possibly) update its title/body/diff.+	// Spec 2 should be attached to Changeset 2 and publish it on the code host.+	// Spec 3 should get a new Changeset, since its branch doesn't match Changeset 3's branch.+	// Spec 4 should be attached to Changeset 4, since it tracks PR #333 in Repo C.+	// Changeset 3 doesn't have a matching spec and should be detached from the campaign (and closed).++	type repoHeadRef struct {+		repo    api.RepoID+		headRef string+	}+	changesetsByRepoHeadRef := map[repoHeadRef]*campaigns.Changeset{}++	type repoExternalID struct {+		repo       api.RepoID+		externalID string+	}+	changesetsByRepoExternalID := map[repoExternalID]*campaigns.Changeset{}++	currentSpecsByChangeset := map[int64]*campaigns.ChangesetSpec{}++	for _, c := range changesets {+		// This is an n+1+		s, err := tx.GetChangesetSpecByID(ctx, c.CurrentSpecID)+		if err != nil {+			return nil, err+		}+		currentSpecsByChangeset[c.ID] = s++		if c.ExternalID != "" {+			k := repoExternalID{repo: c.RepoID, externalID: c.ExternalID}+			changesetsByRepoExternalID[k] = c++			// If it has an externalID but no CurrentSpecID, it is a tracked+			// changeset, and we're done and don't need to match it by HeadRef+			if c.CurrentSpecID == 0 {+				continue+			}+		}++		k := repoHeadRef{repo: c.RepoID}+		if c.ExternalBranch != "" {+			k.headRef = git.EnsureRefPrefix(c.ExternalBranch)+			changesetsByRepoHeadRef[k] = c+			continue+		}++		// If we don't have an ExternalBranch, the changeset hasn't been+		// published yet (or hasn't been synced yet).+		if c.CurrentSpecID != 0 {+			// If we're here, the changeset doesn't have an external branch+			//+			// So we load the spec to get the branch where we _would_ push+			// the commit.++			k.headRef = git.EnsureRefPrefix(s.Spec.HeadRef)+			changesetsByRepoHeadRef[k] = c+		}+	}++	attachedChangesets := map[int64]bool{}+	for _, spec := range newChangesetSpecs {+		// If we don't have access to a repository, we return an error. Why not+		// simply skip the repository? If we skip it, the user can't reapply+		// the same campaign spec, since it's already applied and re-applying+		// would require a new spec.+		repo, ok := accessibleReposByID[spec.RepoID]+		if !ok {+			return nil, &db.RepoNotFoundErr{ID: spec.RepoID}+		}++		if err := campaigns.CheckRepoSupported(repo); err != nil {+			return nil, err+		}++		// If we need to track a changeset, we need to find it.+		if spec.Spec.IsExisting() {

This confused me a bit in conjunction with the existing variable below, which refers to something else: this Existing means "we're tracking this PR that already exists", whereas the later existing means "this changeset already exists in the Sourcegraph database".

I don't have a concrete suggestion right now because English is the worst, but the first existing feels... off to me.

mrnugget

comment created time in 4 days

Pull request review commentsourcegraph/sourcegraph

Implement ApplyCampaign and changeset reconciler

 type CampaignConnection {     pageInfo: PageInfo! } -# The internal state of a changeset on Sourcegraph.-enum ChangesetState {-    # The changeset has not yet been created on the code host and is not scheduled to be.+# The publication state of a changeset on Sourcegraph+enum ChangesetPublicationState {+    # The changeset has not yet been created on the code host.     UNPUBLISHED-    # The changeset is currently being created or updated on the code host.-    PUBLISHING-    # An error occurred while publishing or syncing this changeset.+    # The changeset has been created on the code host.+    PUBLISHED+}++# The reconciler state of a changeset on Sourcegraph+enum ChangesetReconcilerState {+    # The changeset is enqueued for the reconciler to process it.+    QUEUED+    # The changeset reconciler is currently computing the delta between the

    # The changeset reconciler is currently computing the delta between the

For consistency, there should be a bonus newline here.

mrnugget

comment created time in 4 days

Pull request review commentsourcegraph/sourcegraph

Implement ApplyCampaign and changeset reconciler

+package main++import (+	"bytes"+	"context"+	"encoding/json"+	"fmt"+	"log"+	"net/http"+	"os"+	"strings"++	"github.com/sourcegraph/sourcegraph/internal/db/dbutil"+)++// This is a test script that I use to manually integration-test the new+// changeset reconciler.+//+// It's pretty self-explanatory, I guess, and I want to keep it around for a bit.++const (+	// It's local access token, don't worry.+	authHeader = `token 1b13a0a1217377aa9a43d7cc46782f24b648ab0c`++	graphqlEndpoint = `http://localhost:3082/.api/graphql`+)++func main() {+	// deleteEverything()++	err := applySpecs(applyOpts{+		namespace:    "VXNlcjoxCg==", // User:1+		campaignSpec: newCampaignSpec("thorstens-campaign", "Updated description of my campaign"),+		changesetSpecs: []string{+			// "UmVwb3NpdG9yeToy" is "Repository:1"+			//+			//+			// `{"baseRepository":"UmVwb3NpdG9yeToy","externalID":"1"}`,+			// `{"baseRepository":"UmVwb3NpdG9yeToy","externalID":"311"}`,+			// `{"baseRepository":"UmVwb3NpdG9yeToy","externalID":"309"}`,+			`{+			     "baseRepository": "UmVwb3NpdG9yeToy",+			     "baseRev": "e4435274b43033cf0c212f61a2c16f7f2210cf56",+			     "baseRef":"refs/heads/master",++			     "headRepository": "UmVwb3NpdG9yeToy",+			     "headRef":"refs/heads/reconciler-testing-update-5555",++			     "title": "UPDATED This is the title of the PR",+			     "body": "UPDATED And this is the body. What is **up**?",++			     "published": true,++			     "commits": [{+			    "message": "Pretty cool new commit message",+				"diff":` + fmt.Sprintf("%q", automationTestingDiff) + `}]+			}`,+		},+	})+	if err != nil {+		log.Fatalf("failed to apply specs: %s", err)+	}++	err = applySpecs(applyOpts{+		namespace:    "VXNlcjoxCg==",+		campaignSpec: newCampaignSpec("thorstens-2nd-campaign", "This is the second campaign"),+		changesetSpecs: []string{+			// `{"baseRepository":"UmVwb3NpdG9yeToy","externalID":"1"}`,+			`{"baseRepository":"UmVwb3NpdG9yeToy","externalID":"311"}`,+			`{"baseRepository":"UmVwb3NpdG9yeToy","externalID":"309"}`,+			`{+			     "baseRepository": "UmVwb3NpdG9yeToy",+			     "baseRev": "e4435274b43033cf0c212f61a2c16f7f2210cf56",+			     "baseRef":"refs/heads/master",++			     "headRepository": "UmVwb3NpdG9yeToy",+			     "headRef":"refs/heads/thorstens-2nd-campaign",++			     "title": "PR in second campaign",+			     "body": "PR body in second campaign",++			     "published": true,++			     "commits": [{+			    "message": "Pretty commit message",+				"diff":` + fmt.Sprintf("%q", automationTestingDiff2) + `}]+			}`,+		},+	})+	if err != nil {+		log.Fatalf("failed to apply specs: %s", err)+	}+}++type applyOpts struct {+	namespace      string+	changesetSpecs []string+	campaignSpec   string+}++func applySpecs(opts applyOpts) error {+	var changesetSpecIDs []string++	for i, spec := range opts.changesetSpecs {+		fmt.Printf("Creating changesetSpec %d... ", i)++		q := fmt.Sprintf(createChangesetSpecTmpl, spec)+		res, err := sendRequest(q)+		if err != nil {+			return err+		}+		id := res.Data.CreateChangesetSpec.ID+		changesetSpecIDs = append(changesetSpecIDs, id)++		fmt.Printf("Done. ID: %s\n", id)+	}++	fmt.Printf("Creating campaignSpec... ")+	q := fmt.Sprintf(createCampaignSpecTmpl,+		opts.namespace,+		opts.campaignSpec,+		graphqlIDList(changesetSpecIDs...),+	)++	res, err := sendRequest(q)+	if err != nil {+		return fmt.Errorf("failed to create campaign spec: %s\n", err)+	}+	campaignSpecID := res.Data.CreateCampaignSpec.ID+	fmt.Printf("Done. ID: %s\n", campaignSpecID)++	fmt.Printf("Applying campaignSpec... ")+	q = fmt.Sprintf(applyCampaignTmpl, campaignSpecID)+	res, err = sendRequest(q)+	if err != nil {+		return fmt.Errorf("failed to apply campaign: %s\n", err)+	}+	campaignID := res.Data.ApplyCampaign.ID+	fmt.Printf("Done. Campaign ID: %s\n", campaignID)+	return nil+}++const applyCampaignTmpl = `+mutation ApplyCampaign { applyCampaign(campaignSpec: %q) { id } }+`++const createChangesetSpecTmpl = `+mutation CreateChangesetSpec {+  createChangesetSpec(changesetSpec: %q) {+    ... on HiddenChangesetSpec { id }+    ... on VisibleChangesetSpec { id }+  }+}+`++const createCampaignSpecTmpl = `+mutation CreateCampaignSpec {+  createCampaignSpec(namespace: %q, campaignSpec: %q, changesetSpecs: %s) {+    id+  }+}+`++type graphqlPayload struct {+	Query string+}++func graphqlIDList(ids ...string) string {+	var quoted []string+	for _, id := range ids {+		quoted = append(quoted, fmt.Sprintf("%q", id))+	}+	return fmt.Sprintf("[%s]", strings.Join(quoted, ", "))+}++type graphqlResponse struct {+	Data struct {+		CreateChangesetSpec struct {+			ID string+		}+		CreateCampaignSpec struct {+			ID string+		}+		ApplyCampaign struct {+			ID string+		}+	}+	Errors []struct {+		Message string+	}+}++func sendRequest(query string) (graphqlResponse, error) {+	var res graphqlResponse+	b := new(bytes.Buffer)+	json.NewEncoder(b).Encode(graphqlPayload{Query: query})++	req, err := http.NewRequest("POST", graphqlEndpoint, b)+	if err != nil {+		return res, err+	}++	req.Header.Add("Authorization", authHeader)+	resp, err := http.DefaultClient.Do(req)+	if err != nil {+		return res, err+	}++	if err := json.NewDecoder(resp.Body).Decode(&res); err != nil {+		return res, err+	}++	if len(res.Errors) != 0 {+		var messages []string+		for _, e := range res.Errors {+			messages = append(messages, e.Message)+		}+		list := strings.Join(messages, "\n- ")+		return res, fmt.Errorf("graphql errors:\n\t- %s\n", list)+	}++	return res, nil+}++func newCampaignSpec(name, description string) string {+	return fmt.Sprintf(`{+  "name": %q,+  "description": %q,+  "on": [+    {"repositoriesMatchingQuery": "lang:go func main"},+    {"repository": "github.com/sourcegraph/src-cli"}+  ],+  "steps": [+    {+      "run": "echo 'foobar'",+      "container": "alpine",+      "env": {+        "PATH": "/work/foobar:$PATH"+      }+    }+  ],+  "changesetTemplate": {+    "title": "Hello World",+    "body": "My first campaign!",+    "branch": "hello-world",+    "commit": {+      "message": "Append Hello World to all README.md files"+    },+    "published": false+  }+}`, name, description)+}++const automationTestingDiff = `diff --git test.md test.md+index 52ada66..0aaaf37 100644+--- test.md++++ test.md+@@ -1 +1,3 @@+-# This is a test++# This is a test.++++And this is another line+`++const automationTestingDiff2 = `diff --git test.md test.md+index 52ada66..0aaaf37 100644+--- test.md++++ test.md+@@ -1 +1,3 @@+-# This is a test++# What an amazing test!++++And this is another line+`++func deleteEverything() {+	ctx := context.Background()++	dsn := dbutil.PostgresDSN("sourcegraph", os.Getenv)+	db, err := dbutil.NewDB(dsn, "campaigns-reconciler")+	if err != nil {+		log.Fatalf("failed to initialize db store: %v", err)+	}++	if _, err := db.ExecContext(ctx, "DELETE FROM campaigns;"); err != nil {+		log.Fatal(err)+	}+	if _, err := db.ExecContext(ctx, "DELETE FROM changesets;"); err != nil {+		log.Fatal(err)+	}+	if _, err := db.ExecContext(ctx, "DELETE FROM changeset_specs;"); err != nil {+		log.Fatal(err)+	}+	if _, err := db.ExecContext(ctx, "DELETE FROM campaign_specs;"); err != nil {+		log.Fatal(err)+	}+}++//+//            ____  __  ______________   _________    ____  ______+//           / __ \/ / / / ____/_  __/  /_  __/   |  / __ \/ ____/+//          / / / / / / / /     / /      / / / /| | / /_/ / __/+//         / /_/ / /_/ / /___  / /      / / / ___ |/ ____/ /___+//        /_____/\____/\____/ /_/      /_/ /_/  |_/_/   /_____/+//+//+//+//+//+//+//                                 .,/(##%##(((##+//          ,#((#(((((((((((((((((((((((((((((((((((#+//       (/////(((((//(((((((((((((////////////////////.+//     (///////(((/(//////////////////*//***/***********/+//    /**//*///(/((///(/*****************,,**,,**,**,**,,*/+//   ********/*///(//////*.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,*+//  ***********///(////**** ................................,+// .*,**********//////****** ..........  ....................,+// *,,,,*****/*,*/////*******        .   .....................*+// ,,,,,,*,/.....,(#/******,,,  .. . .....................,.,.,,+// ,,,,,,,(....,,,,,,,***,*,,,, ................,..,,,,,,,,,,,,*+// ,,,,,,,..&*%.(%&@.,,,***,,** ...........,,,,,,,,,,,,,,,******/+// .,,,,,/..,%,.,/.(,,,,,**,****.,,,,,,,,,,,,,*,****************/+// ..,,,.*.  .... *#(,**,,******,*********************///////////(+// ..,,.,* .#.#.,,,,*,**/,******,,*******////////////////////////(+// ,...,,* *#&  /,,,,,***.,****,**////////////////////////////////+// ,..,,., /* % #.,,,&,***,**,,,**////////////////////////////////+//  ..,..., .,@....,,,,***,,,,,*,,//////////////////////////////*/.+//  ......,,    .#...,&,**,,,,***,///////////////*****************.+//   ......* *    %...,,,,,,,,,,,,********************************+//   ,......, #    &....,.,,,,,,,,************************////////+//    ....,,,* /   .. ...,,,,,,,,,***/////////////////////////////+//    ,.,,,.,,,, (   ...,,,,,,,,,/**//////////////////////////////+//     *,,,,,,,,,,,,,,,,,,,,,,,,,/,*******////////////////////////,+//      *,,,,,,,,,,,,,*,,,,,,,*,./#********/*/*////////////////////+//       ,,,,,,,,,,,,,*,*,*,,,,,/((,***************/**/**/**/******+//         ,,,,,,,,,,,,***,**,,*(((#*********************////////**/+//          *,,,,,,,,,,,,,**,**(((((.***********/*******************+//            .,,,,,,,,,,**,./((((###********************************+//               *,,,,,,*,,((###%&@&/********************************.+//                        ............********************************+//                        ............/******************************,*+//                          ...........******************************,,+//                            .........****,***,*******,,,,,,,,,,,,,,,,,+//                               .......*,,,,,*,,,,,,.,,,,,,,,,,,.,,,,,**/+//                                  .....,,,..............,,,,***////////////(.+//                                      ... .....,,,***//*/***********//////*+//                                         /*//****************//,+//                                            ./****//,

365a4da46b04b1ec6d97da771b9e9a07

mrnugget

comment created time in 4 days

pull request commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

Unknown events are still logged as errors - but they can happen, think milestone assignment events or so, they're covered in merge request events but not supported by us. I think they should be marked less harmful

I fixed that one up in 8d1e6df. I agree it was too noisy.

Label events are still not handled - we do that for GitHub though. Doesn’t need to take the color into account yet, but I think feature parity for webhooks would be great, so it’s less documentation work why certain things aren’t real time while others are. Not sure I understood your comment regarding those events correctly, but I think you only referred to color support right?

Sadly, GitLab does not give us "label" events per se: we get a merge request update event, along with a changes field that is unspecified in nature but might include some labels if the labels change. Rather than trying to pull that apart, I've made update events do the same thing as approval/unapproval: ask repo-updater to do a sync ASAP, which also means we'll have faster updates for other changes.

That's db3a23e for the unmarshalling, and fc6f1a9 for the event handling.

https://gitlab.sgdev.org/sourcegraph/bitbucket-server-plugin/merge_requests/22 has no pipeline, also no devops one but check state is PENDING

I can't reproduce this one, nor can I figure out how the code would have gotten to pending: looking at the return value from the REST API, there's no pipeline (as you'd expect), so the relevant code should short circuit out to UNKNOWN, which is (as far as I know) correct.

Do you have the database floating around for that changeset? I'd love to see the changeset record and any events associated with it.

LawnGnome

comment created time in 4 days

push eventsourcegraph/sourcegraph

Rijnard van Tonder

commit sha ff14a9340feb96f587ccee57a108f8773339cd0c

search: dedup results in union operation (#12531)

view details

Keegan Carruthers-Smith

commit sha 9997438a8ba2fdc54920f1f6ad22dd08d4a37215

zoekt: binary encoder for repo branches (#12546) This updates zoekt to include a custom binary marshaller for repo branches. When CPU profiling two of the major contributors is Gob encoding and garbage collection. This should significantly improve time spent gob encoding (~4x faster). It also will only allocate one large buffer vs 2 per key, value in gob. This should help reduce pressure on frontend. For example on Sourcegraph.com we have 100k repos indexed with 12 search replicas. This means when marshalling the repobranches for default we will go from allocating 100k * 2 * 12 = 2.4 million objects to 12. This commit requires zoekt and sourcegraph-frontend to have the same version of query.RepoBranches. For customers this will require a small amount of downtime when upgrading. For sourcegraph.com I will ensure they roll out at the same time. Changes: - c908c13 shards: optimize RepoBranches when all branches are the same - 809c853 query: binary encoder for repo branches - 2a821ee query: exactly match on repobranch branches

view details

Ryan Slade

commit sha a4d9c17aa0785b19f829fcf162fea2b19a350967

conf: Cancel timer to free goroutine (#12553) Previously all calls to conf.Get, via WaitUntilInitialized, would spawn a timer that would leave a pending goroutine running, even if config had already been initialised. The default timeout period is 5 minutes so this can lead to a large number of pending routines.

view details

Ryan Slade

commit sha 4d254b2f7e5c6e9c451a74f8eb7112a7128143dd

conf: More efficient deadlock check (#12555) Perform a non blocking check early so that we have a fast path when we know config is already initialsed.

view details

Keegan Carruthers-Smith

commit sha 5b0a6bc46fb008703672f20028e1b46107a9817f

search: more trace logging (#12554) I am investigating traces on Sourcegraph.com. Resolving repositories is half the time, so added more detailed trace logging for it. Additionally I improved how we log for Zoekt to make it more pleasant (and efficient) to read those traces.

view details

Stefan Hengl

commit sha 370226e109b04888578884b2e475a3c9eb65818f

search: reactivate dead code path in alertForOverRepoLimit (#12556)

view details

Keegan Carruthers-Smith

commit sha 05db081805ef89db3f58c53bac3d4f0307e4d00d

db: in-memory cache of default repos (#12557) This is a sourcegraph.com specific optimization. We add an in-memory cache of the list of repos search on Sourcegraph.com when no repo filters are specified. Currently retrieving this list from the database takes 350ms. It is a very rarely changing value and is shared across users/requests. Co-authored-by: Ryan Slade <ryanslade@gmail.com>

view details

Loïc Guychard

commit sha 70b6f64e2a16bcdb1d3c85336778f22da05c2c89

Code host integrations: always add utm_source (#12534) Rel. #11447 Ensures we always add `?utm_source=platform_name` to links pointing back to the Sourcegraph instance from code host integrations, where the platform name is one of: `firefox-extension`, `chrome-extension`, `phabricator-integration`, `bitbucket-integration`, `gitlab-integration`

view details

Felix Becker

commit sha fa4ea5469043872715574aafaf107baef7e1ebe9

Badge design (#12390)

view details

Felix Becker

commit sha 3cac2fd7a4ddebda6585cb0e6bda0c55a10c28f4

Adopt graphql-code-generator (#12559) Co-authored-by: Simon <twop.sk@gmail.com>

view details

Adam Harvey

commit sha 0a74343cb22d08981ecdacab05f3faf4d77b659b

doc: make webhook configuration consistent (#12529)

view details

TJ Kandala

commit sha 4ffdde482aa64d112dcf5f582f0e2f1b6201db7b

Fix browser extension broken links when branch is specified (#12511) Fix #11057. The Sourcegraph browser extension link used to break on the root directory of a repo when a branch was specified. This happened because we would throw an error when the path structure wasn't exactly: <empty>/<user>/<repo>/(blob | tree)/<commitID>/<path/to/file> The href for the a.js-permalink-shortcut element on the root directory looks like this: "/sourcegraph/sourcegraph/tree/cff20a845c2263deacdcbf18e3b50e2b3c4af94c" (no path). I added a check for page type in getFilePath() in order to allow tree pages to have an empty path. This resulted in a complication in resolveFileInfo() because it prepended all paths with a slash, so I made an exception for falsy strings.

view details

Erik Seliger

commit sha a8a890b684f33696c01644a3873a3192444b4944

Fix filtering by external state and nil panic when state is given in args (#12568)

view details

Robert Lin

commit sha 67cd6a56b98de6b46c965df15f868bad1e7de27e

monitoring: let opsgenie apiurl be optional (#12571)

view details

Erik Seliger

commit sha 004bb0b48d89476a2116dd74ea919733dcc798c2

Fix URL for open on sourcegraph button (#12570)

view details

Robert Lin

commit sha a306f4124906b1103fdb262cdde2e25935813891

monitoring: link alerts back to grafana service dashboard (#12550) Previously, alerts would only provide links to the alert solutions page. This change adds links to the appropriate service dashboard, and also updates more notifiers to leverage built-in capabilities for adding context instead of crowding the message body with links. Other changes: * add owner to notification body * frontend page load latency may not exist * increase Alertmanager retention from 120h to 168h (7d)

view details

Stefan Hengl

commit sha 202ff3793ba887ad9513f6413d5aef5fb14c01e0

search: add annotation to VisitField (#12558)

view details

Stefan Hengl

commit sha 9ce4152e8384264d1bceb20d8ff3a5df2ec78c9d

search: fix count for queries with trailing spaces (#12579)

view details

renovate[bot]

commit sha e92279d4d0b0a8545e3f9b2db083aefce7e3b4e0

Update dependency rxjs to ^6.6.2 (#12582) Co-authored-by: Renovate Bot <bot@renovateapp.com> Co-authored-by: Felix Becker <felix.b@outlook.com>

view details

Erik Seliger

commit sha c5836e3cff6281998444022c2710937152929d3d

Migrate campaigns components to use the new graphql types (#12562)

view details

push time in 4 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha c2d67458122af0ab1b832b53da4841ddefe71807

Update enterprise/internal/campaigns/webhooks_gitlab.go Co-authored-by: Erik Seliger <erikseliger@me.com>

view details

push time in 4 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha 4b8f6776b96730403dd4c7469fa5a4e1f15a9c0d

Remove some unnecessary logging.

view details

Adam Harvey

commit sha b2867257784e9b28ca321d9f5662741c27a7b204

Add changelog.

view details

Adam Harvey

commit sha afce87d3933b814d9a361797a67274d3e35b915f

Inline respond() so we don't error.

view details

push time in 4 days

Pull request review commentsourcegraph/sourcegraph

Migrate campaigns components to use the new graphql types

  .changeset-node {     &__content {-        &--no-collapse {

There's another use of this class in HiddenExternalChangesetNode, so I'm not sure we want to get rid of this. (Or we want to get rid of that class in the component.)

eseliger

comment created time in 5 days

Pull request review commentsourcegraph/sourcegraph

Migrate campaigns components to use the new graphql types

 export const CampaignChangesets: React.FunctionComponent<Props> = ({     telemetryService,     queryChangesets = _queryChangesets, }) => {-    const [state, setState] = useState<GQL.ChangesetState | undefined>()+    const [externalState, setExternalState] = useState<GQL.ChangesetExternalState | undefined>()

:+1:

eseliger

comment created time in 5 days

pull request commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

It took a bit to verify it, but I believe b39e54ed32b5674f4956815cfe86650e0e8180a5 fixed the remaining issues @eseliger found as well.

Which leaves only this question from @mrnugget:

  • Do we have a bug in syncChangesetsWithSources and we need to first insert the new changeset events, then load all events again, and then set the derived state?

  • Did you know all of this and had this in mind? smile

In summary, yes. I do think we have some ordering issues, and I think fixing #12565 will fix those, but I've tried to work around them as best I can in here.

Therefore: I think this is ready for final testing/review.

LawnGnome

comment created time in 5 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha 0a74343cb22d08981ecdacab05f3faf4d77b659b

doc: make webhook configuration consistent (#12529)

view details

push time in 5 days

delete branch sourcegraph/sourcegraph

delete branch : aharvey/unify-webhook-docs

delete time in 5 days

PR merged sourcegraph/sourcegraph

doc: make webhook configuration consistent docs team/campaigns

This was a challenge from @mrnugget to make all of the campaign-supported code hosts consistent in terms of documenting their webhook configuration.

+32 -27

0 comment

2 changed files

LawnGnome

pr closed time in 5 days

issue commentsourcegraph/sourcegraph

Design a GraphQL API that allows us to list campaigns on user and organization pages

I think this is reasonable, but I'll let @mrnugget do the honours in terms of closing it, since he's closer to the problem.

mrnugget

comment created time in 5 days

pull request commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

Just a word of caution though: before you fall into an optimisation rabbit hole, I'd pull back, create a ticket and revisit once we have the new workflow working.

I agree (as we talked about in our buddy sync). Filed #12565.

LawnGnome

comment created time in 5 days

issue openedsourcegraph/sourcegraph

campaigns: simplify SetDerivedState and history calculation

Within campaigns, we have a couple of sources of truth for external states (check state, review state, whether the PR/MR is open/closed/merged/whatever) at present: we have Changeset, which attempts to reflect the current state of the changeset, and ChangesetEvent, which attempts to track the history of every state transition that we've seen for the various event types that we care about.

The current state gets updated by SetDerivedState, which iterates over some or all of the changeset events multiple times in its various compute functions. Most notably, computeHistory() has to do a full iteration over the complete set of changeset events, which we use to some extent to short circuit the other computation functions (except computeCheckState() :man_shrugging:), but we could actually calculate the final states in that iteration. I'm also fairly suspicious of the short circuiting: it's dependent on the UpdatedAt field on the changeset, but that could change due to any number of background processes.

Here's roughly what I suggest:

  1. computeHistory() becomes the first thing SetDerivedState() does, and gets some new return values to capture the final set of check, review, and external states. We don't need those for the other use of computeHistory() (the GraphQL endpoint that powers the burndown chart), but they're almost free to calculate and return, so I'd argue for doing it anyway rather than having two functions that iterate over the changeset event collection that are only very slightly different.
  2. The other compute* functions in state.go essentially go away.

The main catch with this is that we have to be very careful that changeset events are created as needed every time a changeset is synced or reconciled, starting from the very beginning of Sourcegraph opening a PR/MR on the code host. The single source of truth for changeset external states becomes the changeset events, and the changeset fields really do just become a pure cache at that point.

created time in 5 days

pull request commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

  • Panic caused by close event, after first closed, then reopened and then the second close (and reopen also fails now). So basically from the third merge state webhook on, it fails.

This should be fixed by b39e54ed32b5674f4956815cfe86650e0e8180a5.

The underlying panic was also arguably a bug in our update code; 890f978dd0bfb9eac2c5d7a7d256300d2ec57de2 addresses that by at least turning it into an error we can report normally.

  • It doesn’t appear as merged in the changesetCountsOverTime resolver (still pending) (perhaps also other states not reflected, didn't test deeper here)

I fixed this one too, but I'm seeing some unreliability in how we invoke SetDerivedState(), specifically around the timestamp-based checks the compute functions (such as computeExternalState()) use to try to avoid iterating the full set of events.

The thing is: we already iterate over the full set of changeset events in computeHistory(). So I think we've actually overcomplicated this: as long as we synthesise the appropriate events on sync (which we have to anyway for the burndown chart), we could compute the external states we care about while we also calculate the history and get rid of computeExternalState() and computeReviewState() entirely.

I'm going to sketch this out (in a different PR) tomorrow morning.

LawnGnome

comment created time in 6 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha 890f978dd0bfb9eac2c5d7a7d256300d2ec57de2

Allow Update() to return an error rather than panic.

view details

Adam Harvey

commit sha b39e54ed32b5674f4956815cfe86650e0e8180a5

Wire up new event types.

view details

push time in 6 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha d535a9773620fd371f30de347337010dad908f0d

Update generated data.

view details

push time in 6 days

push eventsourcegraph/sourcegraph

uwedeportivo

commit sha 30f8af2388bce040be095a762c5492608a8db2f4

release captain 3.19 (#12505)

view details

Robert Lin

commit sha 37a8af898fffc0be990240f4979b7c1877b8fc3e

monitoring: migrate out-of-band alerts to the generator (#12391) Migrates all out-of-band alert rules defined by deploy-sourcegraph to the generator, such that each alert now has a panel as part of a service dashboard like all our other alerts, with a few changes and additions.

view details

Stephen Gutekanst

commit sha cff20a845c2263deacdcbf18e3b50e2b3c4af94c

Release Docker Compose v3.18.0-1 (#12509) * Release Docker Compose v3.18.0-1 * Release Docker Compose v3.18.0-1 * Release Docker Compose v3.18.0-1

view details

Robert Lin

commit sha da520cf2c4aa5396860e9569d8efe1f68e668f04

monitoring: log prom-wrapper attempts at connecting to the frontend (#12508)

view details

Robert Lin

commit sha 9db0d540d5dab475de834378c8a38988ef17ccd0

monitoring: use prometheus model.Duration (#12510)

view details

Stefan Hengl

commit sha d5b9e2ba04807b8555026027e43ce7eb9aec9103

web: update syntax highlighting for field content (#12512)

view details

Stefan Hengl

commit sha 888a7e74fa8edc76e3a57796d01b8f58e8ae8ca4

search: treat repo and rev separately when converting from glob to regex (#12492)

view details

Anukul Sangwan

commit sha fb3ab217aba970391f533a4e070e11c2a2f3c184

repo-updater: add HTML version of repo-updater-state page (#12498)

view details

Stefan Hengl

commit sha 66de79d60c9d6b31a4482ff557862638e14a6c58

web: update tooltips to be agnostic of regex and globbing (#12514)

view details

Keegan Carruthers-Smith

commit sha 2dcc237e22867e7d1c09edbce8458cbad6fd9977

frontend: speed up needs index site admin page (#12513) This page can be slow since we will only fetch a small number of repos from the db at a time. By fetching twice as many repos we have indexed we are gaurenteed to only need to fetch the list once from the DB.

view details

Marek

commit sha 4831eab61090dbcc4b2b2e4b827223c6ec169ce0

Improve and fix the browser extension readme (#12478)

view details

Robert Lin

commit sha 5d73f94a7ad9c8c5a5d30954f737e9dde5c2a8b5

monitoring: implement owner routing for alerts (#12491) Adds a new owners field to observability.alerts as proposed in #12010 for RFC-189. Only alerts that match the specified owners will be received by the associated notifier. Notifiers configured without the owners field will continue to receive all alerts of the specified level, even with notifiers that have owners present. Co-authored-by: Stephen Gutekanst <stephen.gutekanst@gmail.com>

view details

Keegan Carruthers-Smith

commit sha 5d25b0d8afd22c7e5a549ec800e9b06910f369e9

search: trace and observe each zoekt host (#12516) * search: trace and observe each zoekt host This commit moves tracing and observability from being around the aggregated zoekt client to being both the aggregated client and a trace per zoekt replica. To do this we adjust the prometheus metric to have a hostname label. We also change the category / family name to indicate if its a search against a specific zoekt replica or its the aggregation. * search: test NewMeteredSearcher

view details

Keegan Carruthers-Smith

commit sha 1897348061b97b4c97df6166bcc69dcad7de7d52

zoekt: update to ignore non-fatal ctag errors (#12517) We recently introduced hard failures when encountering a ctags error. However, there are a small number of repos (2 at the moment) which output a non-fatal error that we can ignore. For example: ignoring null tag in keybinding-emacs.js(line: 831)

view details

Thorsten Ball

commit sha ed5c7d5ca377bf46198265e286c0d7f7ed78858d

Check return value of computeDiffStat (#12521)

view details

Eric Fritz

commit sha 40b316180fd0821888f3f7d6fdae73acd42016b5

codeintel: Remove ErrMalformedBundle (#12497)

view details

Rijnard van Tonder

commit sha 7d6cafd040926f59c9dd58efa684a5da61f61734

remove replacer (#12480)

view details

Adam Harvey

commit sha c6db6b232639b991eb5746a9b90daf12f6133c7d

schema: allow personal repos on Bitbucket Server (#12504) Unlike GitHub and GitLab, personal repos are additionally namespaced with a tilde on Bitbucket Server. We should be able to specify these in the repository and exclusion configuration.

view details

Robert Lin

commit sha caf2339820f75d22e53933fa11afac613708d288

monitoring: increase alert repeat interval to 7d (#12484)

view details

Robert Lin

commit sha c81be13d383a3fa99eeee3aa4f6436d5a04d238c

monitoring: add for:7d to provisioning alerts (#12507)

view details

push time in 6 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

 We are actively collaborating with GitLab to improve our integration (e.g. the [ | [`GET /projects/:id`](https://docs.gitlab.com/ee/api/projects.html#get-single-project) | `api` | (1) If using GitLab OAuth and repository permissions, used to determine if a user has access to a given _project_; (2) Used to query repository metadata (e.g. description) for display on Sourcegraph. | | [`GET /projects/:id/repository/tree`](https://docs.gitlab.com/ee/api/repositories.html#list-repository-tree) | `api` | If using GitLab OAuth and repository permissions, used to verify a given user has access to the file contents of a repository within a project (i.e. does not merely have `Guest` permissions). | | (future) write access | `api` | graph site-admins (only) to perform large-scale code refactors, with Sourcegraph issuing and managing the merge requests on GitLab repositories, company-wide. |++## Webhooks++The `webhooks` setting allows specifying the webhook secrets necessary to authenticate incoming webhook requests to `/.api/gitlab-webhooks`.++```json+"webhooks": [+  {"secret": "verylongrandomsecret"}+]+```++Using webhooks is optional, but if configured on GitLab, they allow faster campaign updates than the normal background syncing (i.e. polling) which `repo-updater` provides.++The following [webhook events](https://docs.gitlab.com/ee/user/project/integrations/webhooks.html) are currently used:++- Merge request events+- Pipeline events++Webhooks must be configured on each project that you wish to monitor on GitLab. To set up a project webhook on GitLab, go to the Webhook settings page of the project.++Fill in the **URL** displayed after saving the `webhooks` Sourcegraph setting mentioned above and make sure it is publicly available.++Add the **Secret Token** that was configured within Sourcegraph: in the example above, that's `verylongrandomsecret`.++Select the event types you want to monitor. At a minimum, these should include **merge request events** and **pipeline events**.++Click on **Enable SSL verification** if you have configured SSL with a valid certificate in your Sourcegraph instance.++Finally, select **Add webhook** to create the webhook.

Done for GitLab in this PR; opened #12529 for the other code hosts.

LawnGnome

comment created time in 7 days

PR opened sourcegraph/sourcegraph

doc: make webhook configuration consistent docs team/campaigns

This was a challenge from @mrnugget to make all of the campaign-supported code hosts consistent in terms of documenting their webhook configuration.

+32 -27

0 comment

2 changed files

pr created time in 7 days

create barnchsourcegraph/sourcegraph

branch : aharvey/unify-webhook-docs

created branch time in 7 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha d58d32c7e9612fff4bd539f6db32c3ac0bd5bf71

Update GitLab documentation to match BitBucket Server.

view details

push time in 7 days

pull request commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

Just handling the things that weren't the main bug in @eseliger's excellent testing comment (thanks!):

  • Integrations settings page was misleading, maybe we should help in the docs, also added that comment inline above

Updated in 6b3cf11eb4be17e3d77b5aaa8874cf40c7eb1226.

  • All not supported events log errors, but they are frequent, I think we should just ignore them (or log in verbose) and also not fail the request since GitLab might retry.

Return values changed in 7e7753a2aa1e652038f3dfe0bd8c8aa987f6939c, but they're at info right now. Happy to change this if we think users are likely to configure event types that we can't handle.

  • Pending pipeline is not detected, stays in state “UNKNOWN” - it’s auto devops, maybe that’s the reason why? - Update: appeared after a manual sync, but was out of sync even though webhooks were enabled, not sure that's detectable(?)

This is the pipelines not including merge request issue noted in the initial PR description. I concur with your earlier inline comment that the GitLab issue should be reopened, and I'll do that when I have a good example to hand to provide.

  • Labelling events don’t work (501) - only appear after resync. I saw you added a label type that has the color set, does that mean we can go away from the hardcoded #000000?

Not yet. For consistency with syncs (where we don't get the colour), I haven't wired this up in this PR. When we switch to GraphQL, we can wire this up, but I figured we may as well unmarshal the fields now.

  • Cancelling the pipeline triggers an event which is successful (204), but the UI still shows pending, that was unexpected, but I’m not sure this is a fault of this PR. How do we handle cancelled otherwise? Failed? Unknown?

  • After merge it appeared closed in the UI, resync fixed.

Again, this would be the issue we have where we can't tie a pipeline event back to a specific merge request. It's annoying, to say the least.

Things I still have to investigate:

  • Closing yielded 204 but didn’t update the changeset in the UI (changeset event exists) - a resync works though

Need to investigate this. Could be related to @mrnugget's questions about how we calculate derived state.

  • Shows check state PENDING although no pipeline exists - also no auto devops pipeline

I think I know why this is happening; I don't (yet) know if we can detect it.

  • Panic caused by close event, after first closed, then reopened and then the second close (and reopen also fails now). So basically from the third merge state webhook on, it fails.

Will investigate.

  • It doesn’t appear as merged in the changesetCountsOverTime resolver (still pending) (perhaps also other states not reflected, didn't test deeper here)

Will investigate.

LawnGnome

comment created time in 7 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha 62709b87ea8bfa30f39cec7c31c10a62fc9c914a

Early return.

view details

Adam Harvey

commit sha 4a0778995f02e544c5c347df37a9c1aaf396c45c

Rename subtests.

view details

push time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

+package campaigns++import (+	"bytes"+	"context"+	"database/sql"+	"io"+	"io/ioutil"+	"net/http"+	"net/http/httptest"+	"strconv"+	"strings"+	"testing"+	"time"++	"github.com/google/go-cmp/cmp"+	"github.com/pkg/errors"+	"github.com/sourcegraph/sourcegraph/cmd/repo-updater/repos"+	"github.com/sourcegraph/sourcegraph/internal/api"+	"github.com/sourcegraph/sourcegraph/internal/campaigns"+	"github.com/sourcegraph/sourcegraph/internal/db/dbtest"+	"github.com/sourcegraph/sourcegraph/internal/db/dbutil"+	"github.com/sourcegraph/sourcegraph/internal/extsvc"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab/webhooks"+	"github.com/sourcegraph/sourcegraph/internal/repoupdater"+	"github.com/sourcegraph/sourcegraph/schema"+)++func testGitLabWebhook(db *sql.DB, userID int32) func(*testing.T) {+	return func(t *testing.T) {+		ctx := context.Background()++		t.Run("ServeHTTP", func(t *testing.T) {+			t.Run("missing external service", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, 12345, "https://example.com/")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				if have, want := rec.Result().StatusCode, http.StatusUnauthorized; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+			})++			t.Run("invalid external service", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)++				u := strings.ReplaceAll(extsvc.WebhookURL(extsvc.TypeGitLab, 12345, "https://example.com/"), "12345", "foo")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusInternalServerError; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "getting external service")+			})++			t.Run("malformed external service", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				es.Config = "invalid JSON"+				if err := rstore.UpsertExternalServices(ctx, es); err != nil {+					t.Fatal(err)+				}++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "not a valid secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusInternalServerError; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "validating the shared secret")+			})++			t.Run("missing secret", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusUnauthorized; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "shared secret is incorrect")+			})++			t.Run("incorrect secret", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "not a valid secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusUnauthorized; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "shared secret is incorrect")+			})++			t.Run("missing body", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusBadRequest; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "missing request body")+			})++			t.Run("unreadable body", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "secret")+				req.Body = &brokenReader{errors.New("foo")}++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusInternalServerError; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "reading payload")+			})++			t.Run("malformed body", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, bytes.NewBufferString("invalid JSON"))+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusInternalServerError; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "unmarshalling payload")+			})++			t.Run("invalid body", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				body := marshalJSON(t, &webhooks.EventCommon{+					ObjectKind: "unknown",+				})+				req, err := http.NewRequest("POST", u, bytes.NewBufferString(body))+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusNotImplemented; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "unknown object kind")+			})++			t.Run("error from handleEvent", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)+				repo := createGitLabRepo(t, ctx, rstore, es)+				changeset := createGitLabChangeset(t, ctx, store, repo)+				body := createMergeRequestPayload(t, repo, changeset, "close")++				// Remove the URL from the GitLab configuration.+				cfg, err := es.Configuration()+				if err != nil {+					t.Fatal(err)+				}+				conn := cfg.(*schema.GitLabConnection)+				conn.Url = ""+				es.Config = marshalJSON(t, conn)+				if err := rstore.UpsertExternalServices(ctx, es); err != nil {+					t.Fatal(err)+				}++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, bytes.NewBufferString(body))+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusInternalServerError; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "could not determine service id")+			})++			// The valid tests below are pretty "happy path": specific unit+			// tests for the utility methods on GitLabWebhook are below. We're+			// mostly just testing the routing here, since these are ServeHTTP+			// tests; however, these also act as integration tests. (Which,+			// considering they're ultimately invoked from TestIntegration,+			// seems fair.)++			t.Run("valid merge request approval events", func(t *testing.T) {+				for _, action := range []string{"approved", "unapproved"} {+					t.Run(action, func(t *testing.T) {+						store, rstore, clock := gitLabTestSetup(t, db)+						h := NewGitLabWebhook(store, rstore, clock.now)+						es := createGitLabExternalService(t, ctx, rstore)+						repo := createGitLabRepo(t, ctx, rstore, es)+						changeset := createGitLabChangeset(t, ctx, store, repo)+						body := createMergeRequestPayload(t, repo, changeset, "approved")++						u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+						req, err := http.NewRequest("POST", u, bytes.NewBufferString(body))+						if err != nil {+							t.Fatal(err)+						}+						req.Header.Add(webhooks.TokenHeaderName, "secret")++						repoupdater.MockEnqueueChangesetSync = func(ctx context.Context, ids []int64) error {+							if diff := cmp.Diff(ids, []int64{changeset.ID}); diff != "" {+								t.Errorf("unexpected changeset ID: %s", diff)+							}+							return nil+						}+						defer func() { repoupdater.MockEnqueueChangesetSync = nil }()++						rec := httptest.NewRecorder()+						h.ServeHTTP(rec, req)++						resp := rec.Result()+						if have, want := resp.StatusCode, http.StatusNoContent; have != want {+							t.Errorf("unexpected status code: have %d; want %d", have, want)+						}+					})+				}+			})++			t.Run("valid merge request state change events", func(t *testing.T) {+				for action, want := range map[string]campaigns.ChangesetEventKind{+					"close":  campaigns.ChangesetEventKindGitLabClosed,+					"merge":  campaigns.ChangesetEventKindGitLabMerged,+					"reopen": campaigns.ChangesetEventKindGitLabReopened,+				} {+					t.Run(action, func(t *testing.T) {+						store, rstore, clock := gitLabTestSetup(t, db)+						h := NewGitLabWebhook(store, rstore, clock.now)+						es := createGitLabExternalService(t, ctx, rstore)+						repo := createGitLabRepo(t, ctx, rstore, es)+						changeset := createGitLabChangeset(t, ctx, store, repo)+						body := createMergeRequestPayload(t, repo, changeset, action)++						u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+						req, err := http.NewRequest("POST", u, bytes.NewBufferString(body))+						if err != nil {+							t.Fatal(err)+						}+						req.Header.Add(webhooks.TokenHeaderName, "secret")++						rec := httptest.NewRecorder()+						h.ServeHTTP(rec, req)++						resp := rec.Result()+						if have, want := resp.StatusCode, http.StatusNoContent; have != want {+							t.Errorf("unexpected status code: have %d; want %d", have, want)+						}++						// Verify that the changeset event was upserted.+						assertChangesetEventForChangeset(t, ctx, store, changeset, want)+					})+				}+			})++			t.Run("valid pipeline events", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)+				repo := createGitLabRepo(t, ctx, rstore, es)+				changeset := createGitLabChangeset(t, ctx, store, repo)+				body := createPipelinePayload(t, repo, changeset, gitlab.Pipeline{+					ID:     123,+					Status: gitlab.PipelineStatusSuccess,+				})++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, bytes.NewBufferString(body))+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusNoContent; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}++				assertChangesetEventForChangeset(t, ctx, store, changeset, campaigns.ChangesetEventKindGitLabPipeline)+			})+		})++		t.Run("getExternalServiceFromRawID", func(t *testing.T) {+			// Since these tests don't write to the database, we can just share+			// the same database setup.+			store, rstore, clock := gitLabTestSetup(t, db)+			h := NewGitLabWebhook(store, rstore, clock.now)++			// Set up two GitLab external services.+			a := createGitLabExternalService(t, ctx, rstore)+			b := createGitLabExternalService(t, ctx, rstore)++			// Set up a GitHub external service.+			github := createGitLabExternalService(t, ctx, rstore)+			github.Kind = extsvc.KindGitHub+			if err := rstore.UpsertExternalServices(ctx, github); err != nil {+				t.Fatal(err)+			}++			t.Run("invalid ID", func(t *testing.T) {+				for _, id := range []string{"", "foo"} {+					t.Run(id, func(t *testing.T) {+						es, err := h.getExternalServiceFromRawID(ctx, "foo")+						if es != nil {+							t.Errorf("unexpected non-nil external service: %+v", es)+						}+						if err == nil {+							t.Error("unexpected nil error")+						}+					})+				}+			})++			t.Run("missing ID", func(t *testing.T) {+				for name, id := range map[string]string{+					"not found":  "12345",+					"wrong kind": strconv.FormatInt(github.ID, 10),+				} {+					t.Run(name, func(t *testing.T) {+						es, err := h.getExternalServiceFromRawID(ctx, id)+						if es != nil {+							t.Errorf("unexpected non-nil external service: %+v", es)+						}+						if want := errExternalServiceNotFound; err != want {+							t.Errorf("unexpected error: have %+v; want %+v", err, want)+						}+					})+				}+			})++			t.Run("valid ID", func(t *testing.T) {+				for id, want := range map[int64]*repos.ExternalService{+					a.ID: a,+					b.ID: b,+				} {+					sid := strconv.FormatInt(id, 10)+					t.Run(sid, func(t *testing.T) {+						have, err := h.getExternalServiceFromRawID(ctx, sid)+						if err != nil {+							t.Errorf("unexpected non-nil error: %+v", err)+						}+						if diff := cmp.Diff(have, want); diff != "" {+							t.Errorf("unexpected external service: %s", diff)+						}+					})+				}+			})+		})++		t.Run("getExternalServiceFromRawID database error", func(t *testing.T) {+			// This test is separate from the other unit tests for this+			// function above because it needs to set up a bad database+			// connection on the repo store.+			store, _, clock := gitLabTestSetup(t, db)+			rstore := repos.NewDBStore(&brokenDB{errors.New("foo")}, sql.TxOptions{})+			h := NewGitLabWebhook(store, rstore, clock.now)++			_, err := h.getExternalServiceFromRawID(ctx, "12345")+			if err == nil {+				t.Error("unexpected nil error")+			}+		})++		t.Run("database error", func(t *testing.T) {+			// We can induce an error with a broken database connection.+			store, rstore, clock := gitLabTestSetup(t, db)+			h := NewGitLabWebhook(store, rstore, clock.now)+			h.Store = NewStoreWithClock(&brokenDB{errors.New("foo")}, clock.now)++			es, err := h.getExternalServiceFromRawID(ctx, "12345")+			if es != nil {+				t.Errorf("unexpected non-nil external service: %+v", es)+			}+			if err == nil {+				t.Error("unexpected nil error")+			}+		})

You can definitely tell one of those test functions came well after the other. Updated.

LawnGnome

comment created time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

+import React from 'react'+import * as GQL from '../../../shared/src/graphql/schema'+import { CopyableText } from '../components/CopyableText'++interface Props {+    externalService: Pick<GQL.IExternalService, 'kind' | 'webhookURL'>+}++export const SiteAdminExternalServiceWebhook: React.FunctionComponent<Props> = props => {+    const { kind, webhookURL } = props.externalService++    if (webhookURL) {

Done. :smiley:

LawnGnome

comment created time in 7 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha 64f31aaabccee3a580acf13951ffda8bff371687

Directly load the external service.

view details

Adam Harvey

commit sha 32533e627e61ffef787d580d2fde3d41e46a1654

Explain the lack of Key()s.

view details

Adam Harvey

commit sha 3eba77aa986d6371f88156e17daf4048c54ea37d

Hopefully clarify MergeRequestEvent.

view details

push time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

+package webhooks++import (+	"fmt"+	"time"++	"github.com/pkg/errors"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab"+)++// MergeRequestEvent is the common type that underpins the types defined for+// specific merge request actions.+type MergeRequestEvent struct {+	EventCommon++	MergeRequest *gitlab.MergeRequest `json:"merge_request"`+	User         *gitlab.User         `json:"user"`+	Labels       *[]gitlab.Label      `json:"labels"`+}++type MergeRequestEventContainer interface {+	ToEvent() *MergeRequestEvent+}++type MergeRequestApprovedEvent struct{ MergeRequestEvent }+type MergeRequestCloseEvent struct{ MergeRequestEvent }+type MergeRequestMergeEvent struct{ MergeRequestEvent }+type MergeRequestReopenEvent struct{ MergeRequestEvent }+type MergeRequestUnapprovedEvent struct{ MergeRequestEvent }++func (e *MergeRequestApprovedEvent) ToEvent() *MergeRequestEvent   { return &e.MergeRequestEvent }+func (e *MergeRequestCloseEvent) ToEvent() *MergeRequestEvent      { return &e.MergeRequestEvent }+func (e *MergeRequestMergeEvent) ToEvent() *MergeRequestEvent      { return &e.MergeRequestEvent }+func (e *MergeRequestReopenEvent) ToEvent() *MergeRequestEvent     { return &e.MergeRequestEvent }+func (e *MergeRequestUnapprovedEvent) ToEvent() *MergeRequestEvent { return &e.MergeRequestEvent }++func (e *MergeRequestCloseEvent) Key() string  { return e.key("Close") }+func (e *MergeRequestMergeEvent) Key() string  { return e.key("Merge") }+func (e *MergeRequestReopenEvent) Key() string { return e.key("Reopen") }++func (e *MergeRequestEvent) key(prefix string) string {+	// We can't key solely off the merge request ID because it may be reopened+	// and closed multiple times. Instead, we'll use the CreatedAt field.+	return fmt.Sprintf("MergeRequest:%s:%d:%s", prefix, e.MergeRequest.IID, e.MergeRequest.CreatedAt.Format(time.RFC3339))+}++type mergeRequestEvent struct {

I've added some documentation, and renamed MergeRequestEvent to MergeRequestEventCommon. Please let me know if it's still unclear.

LawnGnome

comment created time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

+package webhooks++import (+	"fmt"+	"time"++	"github.com/pkg/errors"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab"+)++// MergeRequestEvent is the common type that underpins the types defined for+// specific merge request actions.+type MergeRequestEvent struct {+	EventCommon++	MergeRequest *gitlab.MergeRequest `json:"merge_request"`+	User         *gitlab.User         `json:"user"`+	Labels       *[]gitlab.Label      `json:"labels"`+}++type MergeRequestEventContainer interface {+	ToEvent() *MergeRequestEvent+}++type MergeRequestApprovedEvent struct{ MergeRequestEvent }+type MergeRequestCloseEvent struct{ MergeRequestEvent }+type MergeRequestMergeEvent struct{ MergeRequestEvent }+type MergeRequestReopenEvent struct{ MergeRequestEvent }+type MergeRequestUnapprovedEvent struct{ MergeRequestEvent }++func (e *MergeRequestApprovedEvent) ToEvent() *MergeRequestEvent   { return &e.MergeRequestEvent }+func (e *MergeRequestCloseEvent) ToEvent() *MergeRequestEvent      { return &e.MergeRequestEvent }+func (e *MergeRequestMergeEvent) ToEvent() *MergeRequestEvent      { return &e.MergeRequestEvent }+func (e *MergeRequestReopenEvent) ToEvent() *MergeRequestEvent     { return &e.MergeRequestEvent }+func (e *MergeRequestUnapprovedEvent) ToEvent() *MergeRequestEvent { return &e.MergeRequestEvent }++func (e *MergeRequestCloseEvent) Key() string  { return e.key("Close") }+func (e *MergeRequestMergeEvent) Key() string  { return e.key("Merge") }+func (e *MergeRequestReopenEvent) Key() string { return e.key("Reopen") }

Added.

LawnGnome

comment created time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

+package campaigns++import (+	"context"+	"fmt"+	"io/ioutil"+	"net/http"+	"strconv"+	"time"++	"github.com/inconshreveable/log15"+	"github.com/pkg/errors"+	"github.com/sourcegraph/sourcegraph/cmd/repo-updater/repos"+	"github.com/sourcegraph/sourcegraph/internal/campaigns"+	"github.com/sourcegraph/sourcegraph/internal/extsvc"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab/webhooks"+	"github.com/sourcegraph/sourcegraph/internal/repoupdater"+	"github.com/sourcegraph/sourcegraph/schema"+)++type GitLabWebhook struct{ *Webhook }++func NewGitLabWebhook(store *Store, repos repos.Store, now func() time.Time) *GitLabWebhook {+	return &GitLabWebhook{&Webhook{store, repos, now, extsvc.TypeGitLab}}+}++// ServeHTTP implements the http.Handler interface.+func (h *GitLabWebhook) ServeHTTP(w http.ResponseWriter, r *http.Request) {+	// Look up the external service.+	extSvc, err := h.getExternalServiceFromRawID(r.Context(), r.FormValue(extsvc.IDParam))+	if err == errExternalServiceNotFound {+		respond(w, http.StatusUnauthorized, err)+		return+	} else if err != nil {+		respond(w, http.StatusInternalServerError, errors.Wrap(err, "getting external service"))+		return+	}++	// 🚨 SECURITY: Verify the shared secret against the GitLab external service+	// configuration. If there isn't a webhook defined in the service with this+	// secret, or the header is empty, then we return a 401 to the client.+	if ok, err := validateGitLabSecret(extSvc, r.Header.Get(webhooks.TokenHeaderName)); err != nil {+		respond(w, http.StatusInternalServerError, errors.Wrap(err, "validating the shared secret"))+		return+	} else if !ok {+		respond(w, http.StatusUnauthorized, "shared secret is incorrect")+		return+	}++	// Parse the event proper.+	if r.Body == nil {+		respond(w, http.StatusBadRequest, "missing request body")+		return+	}+	payload, err := ioutil.ReadAll(r.Body)+	if err != nil {+		respond(w, http.StatusInternalServerError, errors.Wrap(err, "reading payload"))+		return+	}++	event, err := webhooks.UnmarshalEvent(payload)+	if err != nil {+		if errors.Is(err, webhooks.ErrObjectKindUnknown) {+			respond(w, http.StatusNotImplemented, err)+		} else {+			respond(w, http.StatusInternalServerError, errors.Wrap(err, "unmarshalling payload"))+		}+		return+	}++	// Route the request based on the event type.+	if err := h.handleEvent(r.Context(), extSvc, event); err != nil {+		respond(w, err.code, err)+	} else {+		respond(w, http.StatusNoContent, nil)+	}+}++var (+	errExternalServiceNotFound     = errors.New("external service not found")+	errExternalServiceWrongKind    = errors.New("external service is not of the expected kind")+	errPipelineMissingMergeRequest = errors.New("pipeline event does not include a merge request")+)++// getExternalServiceFromRawID retrieves the external service matching the+// given raw ID, which is usually going to be the string in the+// externalServiceID URL parameter.+//+// On failure, errExternalServiceNotFound is returned if the ID doesn't match+// any GitLab service.+func (h *GitLabWebhook) getExternalServiceFromRawID(ctx context.Context, raw string) (*repos.ExternalService, error) {+	id, err := strconv.ParseInt(raw, 10, 64)+	if err != nil {+		return nil, errors.Wrap(err, "parsing the raw external service ID")+	}++	es, err := h.Repos.ListExternalServices(ctx, repos.StoreListExternalServicesArgs{+		Kinds: []string{extsvc.KindGitLab},+	})+	if err != nil {+		return nil, errors.Wrap(err, "listing external services")+	}++	for _, e := range es {+		if e.ID == id {

That was indeed why: I wasn't sure if we had some sort of expectation that externalServiceID be optional across code hosts. I'll remove the loop.

LawnGnome

comment created time in 7 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha 1b43a90a4e3d3686286e42939b8e9b87c51920a4

Update enterprise/internal/campaigns/webhooks_gitlab.go Co-authored-by: Erik Seliger <erikseliger@me.com>

view details

push time in 7 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha dfd944ad384a1bc2117bf6be59d3217d87357a8e

Use UpdatedAt to avoid overwriting events.

view details

Adam Harvey

commit sha 6b3cf11eb4be17e3d77b5aaa8874cf40c7eb1226

Clarify where to find settings on old GitLab versions.

view details

Adam Harvey

commit sha 7e7753a2aa1e652038f3dfe0bd8c8aa987f6939c

Don't return error codes when it's our problem.

view details

Adam Harvey

commit sha ff02bd96fcb53c834fb60dfb6d59e94354428f08

Validate that a changeset is enqueued.

view details

push time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

+package campaigns++import (+	"bytes"+	"context"+	"database/sql"+	"io"+	"io/ioutil"+	"net/http"+	"net/http/httptest"+	"strconv"+	"strings"+	"testing"+	"time"++	"github.com/google/go-cmp/cmp"+	"github.com/pkg/errors"+	"github.com/sourcegraph/sourcegraph/cmd/repo-updater/repos"+	"github.com/sourcegraph/sourcegraph/internal/api"+	"github.com/sourcegraph/sourcegraph/internal/campaigns"+	"github.com/sourcegraph/sourcegraph/internal/db/dbtest"+	"github.com/sourcegraph/sourcegraph/internal/db/dbutil"+	"github.com/sourcegraph/sourcegraph/internal/extsvc"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab/webhooks"+	"github.com/sourcegraph/sourcegraph/internal/repoupdater"+	"github.com/sourcegraph/sourcegraph/schema"+)++func testGitLabWebhook(db *sql.DB, userID int32) func(*testing.T) {+	return func(t *testing.T) {+		ctx := context.Background()++		t.Run("ServeHTTP", func(t *testing.T) {+			t.Run("missing external service", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, 12345, "https://example.com/")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				if have, want := rec.Result().StatusCode, http.StatusUnauthorized; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+			})++			t.Run("invalid external service", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)++				u := strings.ReplaceAll(extsvc.WebhookURL(extsvc.TypeGitLab, 12345, "https://example.com/"), "12345", "foo")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusInternalServerError; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "getting external service")+			})++			t.Run("malformed external service", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				es.Config = "invalid JSON"+				if err := rstore.UpsertExternalServices(ctx, es); err != nil {+					t.Fatal(err)+				}++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "not a valid secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusInternalServerError; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "validating the shared secret")+			})++			t.Run("missing secret", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusUnauthorized; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "shared secret is incorrect")+			})++			t.Run("incorrect secret", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "not a valid secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusUnauthorized; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "shared secret is incorrect")+			})++			t.Run("missing body", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusBadRequest; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "missing request body")+			})++			t.Run("unreadable body", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, nil)+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "secret")+				req.Body = &brokenReader{errors.New("foo")}++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusInternalServerError; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "reading payload")+			})++			t.Run("malformed body", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, bytes.NewBufferString("invalid JSON"))+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusInternalServerError; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "unmarshalling payload")+			})++			t.Run("invalid body", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				body := marshalJSON(t, &webhooks.EventCommon{+					ObjectKind: "unknown",+				})+				req, err := http.NewRequest("POST", u, bytes.NewBufferString(body))+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusNotImplemented; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "unknown object kind")+			})++			t.Run("error from handleEvent", func(t *testing.T) {+				store, rstore, clock := gitLabTestSetup(t, db)+				h := NewGitLabWebhook(store, rstore, clock.now)+				es := createGitLabExternalService(t, ctx, rstore)+				repo := createGitLabRepo(t, ctx, rstore, es)+				changeset := createGitLabChangeset(t, ctx, store, repo)+				body := createMergeRequestPayload(t, repo, changeset, "close")++				// Remove the URL from the GitLab configuration.+				cfg, err := es.Configuration()+				if err != nil {+					t.Fatal(err)+				}+				conn := cfg.(*schema.GitLabConnection)+				conn.Url = ""+				es.Config = marshalJSON(t, conn)+				if err := rstore.UpsertExternalServices(ctx, es); err != nil {+					t.Fatal(err)+				}++				u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+				req, err := http.NewRequest("POST", u, bytes.NewBufferString(body))+				if err != nil {+					t.Fatal(err)+				}+				req.Header.Add(webhooks.TokenHeaderName, "secret")++				rec := httptest.NewRecorder()+				h.ServeHTTP(rec, req)++				resp := rec.Result()+				if have, want := resp.StatusCode, http.StatusInternalServerError; have != want {+					t.Errorf("unexpected status code: have %d; want %d", have, want)+				}+				assertBodyIncludes(t, resp.Body, "could not determine service id")+			})++			// The valid tests below are pretty "happy path": specific unit+			// tests for the utility methods on GitLabWebhook are below. We're+			// mostly just testing the routing here, since these are ServeHTTP+			// tests; however, these also act as integration tests. (Which,+			// considering they're ultimately invoked from TestIntegration,+			// seems fair.)++			t.Run("valid merge request approval events", func(t *testing.T) {+				for _, action := range []string{"approved", "unapproved"} {+					t.Run(action, func(t *testing.T) {+						store, rstore, clock := gitLabTestSetup(t, db)+						h := NewGitLabWebhook(store, rstore, clock.now)+						es := createGitLabExternalService(t, ctx, rstore)+						repo := createGitLabRepo(t, ctx, rstore, es)+						changeset := createGitLabChangeset(t, ctx, store, repo)+						body := createMergeRequestPayload(t, repo, changeset, "approved")++						u := extsvc.WebhookURL(extsvc.TypeGitLab, es.ID, "https://example.com/")+						req, err := http.NewRequest("POST", u, bytes.NewBufferString(body))+						if err != nil {+							t.Fatal(err)+						}+						req.Header.Add(webhooks.TokenHeaderName, "secret")++						repoupdater.MockEnqueueChangesetSync = func(ctx context.Context, ids []int64) error {+							if diff := cmp.Diff(ids, []int64{changeset.ID}); diff != "" {

Yes. Yes, we should.

LawnGnome

comment created time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

+package campaigns++import (+	"context"+	"fmt"+	"io/ioutil"+	"net/http"+	"strconv"+	"time"++	"github.com/inconshreveable/log15"+	"github.com/pkg/errors"+	"github.com/sourcegraph/sourcegraph/cmd/repo-updater/repos"+	"github.com/sourcegraph/sourcegraph/internal/campaigns"+	"github.com/sourcegraph/sourcegraph/internal/extsvc"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab/webhooks"+	"github.com/sourcegraph/sourcegraph/internal/repoupdater"+	"github.com/sourcegraph/sourcegraph/schema"+)++type GitLabWebhook struct{ *Webhook }++func NewGitLabWebhook(store *Store, repos repos.Store, now func() time.Time) *GitLabWebhook {+	return &GitLabWebhook{&Webhook{store, repos, now, extsvc.TypeGitLab}}+}++// ServeHTTP implements the http.Handler interface.+func (h *GitLabWebhook) ServeHTTP(w http.ResponseWriter, r *http.Request) {+	// Look up the external service.+	extSvc, err := h.getExternalServiceFromRawID(r.Context(), r.FormValue(extsvc.IDParam))+	if err == errExternalServiceNotFound {+		respond(w, http.StatusUnauthorized, err)+		return+	} else if err != nil {+		respond(w, http.StatusInternalServerError, errors.Wrap(err, "getting external service"))+		return+	}++	// 🚨 SECURITY: Verify the shared secret against the GitLab external service+	// configuration. If there isn't a webhook defined in the service with this+	// secret, or the header is empty, then we return a 401 to the client.+	if ok, err := validateGitLabSecret(extSvc, r.Header.Get(webhooks.TokenHeaderName)); err != nil {+		respond(w, http.StatusInternalServerError, errors.Wrap(err, "validating the shared secret"))+		return+	} else if !ok {+		respond(w, http.StatusUnauthorized, "shared secret is incorrect")+		return+	}++	// Parse the event proper.+	if r.Body == nil {+		respond(w, http.StatusBadRequest, "missing request body")+		return+	}+	payload, err := ioutil.ReadAll(r.Body)+	if err != nil {+		respond(w, http.StatusInternalServerError, errors.Wrap(err, "reading payload"))+		return+	}++	event, err := webhooks.UnmarshalEvent(payload)+	if err != nil {+		if errors.Is(err, webhooks.ErrObjectKindUnknown) {+			respond(w, http.StatusNotImplemented, err)

Yeah, that's a good point. I'll make the relevant updates for this and the below comment.

LawnGnome

comment created time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

+package webhooks++import (+	"fmt"+	"time"++	"github.com/pkg/errors"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab"+)++// MergeRequestEvent is the common type that underpins the types defined for+// specific merge request actions.+type MergeRequestEvent struct {+	EventCommon++	MergeRequest *gitlab.MergeRequest `json:"merge_request"`+	User         *gitlab.User         `json:"user"`+	Labels       *[]gitlab.Label      `json:"labels"`+}++type MergeRequestEventContainer interface {+	ToEvent() *MergeRequestEvent+}++type MergeRequestApprovedEvent struct{ MergeRequestEvent }+type MergeRequestCloseEvent struct{ MergeRequestEvent }+type MergeRequestMergeEvent struct{ MergeRequestEvent }+type MergeRequestReopenEvent struct{ MergeRequestEvent }+type MergeRequestUnapprovedEvent struct{ MergeRequestEvent }++func (e *MergeRequestApprovedEvent) ToEvent() *MergeRequestEvent   { return &e.MergeRequestEvent }+func (e *MergeRequestCloseEvent) ToEvent() *MergeRequestEvent      { return &e.MergeRequestEvent }+func (e *MergeRequestMergeEvent) ToEvent() *MergeRequestEvent      { return &e.MergeRequestEvent }+func (e *MergeRequestReopenEvent) ToEvent() *MergeRequestEvent     { return &e.MergeRequestEvent }+func (e *MergeRequestUnapprovedEvent) ToEvent() *MergeRequestEvent { return &e.MergeRequestEvent }++func (e *MergeRequestCloseEvent) Key() string  { return e.key("Close") }+func (e *MergeRequestMergeEvent) Key() string  { return e.key("Merge") }+func (e *MergeRequestReopenEvent) Key() string { return e.key("Reopen") }++func (e *MergeRequestEvent) key(prefix string) string {+	// We can't key solely off the merge request ID because it may be reopened+	// and closed multiple times. Instead, we'll use the CreatedAt field.+	return fmt.Sprintf("MergeRequest:%s:%d:%s", prefix, e.MergeRequest.IID, e.MergeRequest.CreatedAt.Format(time.RFC3339))

Actually, I take that back: it was meant to be the updated time. But it was still wrong. :smiley:

LawnGnome

comment created time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

 We are actively collaborating with GitLab to improve our integration (e.g. the [ | [`GET /projects/:id`](https://docs.gitlab.com/ee/api/projects.html#get-single-project) | `api` | (1) If using GitLab OAuth and repository permissions, used to determine if a user has access to a given _project_; (2) Used to query repository metadata (e.g. description) for display on Sourcegraph. | | [`GET /projects/:id/repository/tree`](https://docs.gitlab.com/ee/api/repositories.html#list-repository-tree) | `api` | If using GitLab OAuth and repository permissions, used to verify a given user has access to the file contents of a repository within a project (i.e. does not merely have `Guest` permissions). | | (future) write access | `api` | graph site-admins (only) to perform large-scale code refactors, with Sourcegraph issuing and managing the merge requests on GitLab repositories, company-wide. |++## Webhooks++The `webhooks` setting allows specifying the webhook secrets necessary to authenticate incoming webhook requests to `/.api/gitlab-webhooks`.++```json+"webhooks": [+  {"secret": "verylongrandomsecret"}+]+```++Using webhooks is optional, but if configured on GitLab, they allow faster campaign updates than the normal background syncing (i.e. polling) which `repo-updater` provides.++The following [webhook events](https://docs.gitlab.com/ee/user/project/integrations/webhooks.html) are currently used:++- Merge request events+- Pipeline events++Webhooks must be configured on each project that you wish to monitor on GitLab. To set up a project webhook on GitLab, go to the Webhook settings page of the project.

Thanks! I might play around with the wording a touch to indicate that, then.

LawnGnome

comment created time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

+package webhooks++import (+	"fmt"+	"time"++	"github.com/pkg/errors"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab"+)++// MergeRequestEvent is the common type that underpins the types defined for+// specific merge request actions.+type MergeRequestEvent struct {+	EventCommon++	MergeRequest *gitlab.MergeRequest `json:"merge_request"`+	User         *gitlab.User         `json:"user"`+	Labels       *[]gitlab.Label      `json:"labels"`+}++type MergeRequestEventContainer interface {+	ToEvent() *MergeRequestEvent+}++type MergeRequestApprovedEvent struct{ MergeRequestEvent }+type MergeRequestCloseEvent struct{ MergeRequestEvent }+type MergeRequestMergeEvent struct{ MergeRequestEvent }+type MergeRequestReopenEvent struct{ MergeRequestEvent }+type MergeRequestUnapprovedEvent struct{ MergeRequestEvent }++func (e *MergeRequestApprovedEvent) ToEvent() *MergeRequestEvent   { return &e.MergeRequestEvent }+func (e *MergeRequestCloseEvent) ToEvent() *MergeRequestEvent      { return &e.MergeRequestEvent }+func (e *MergeRequestMergeEvent) ToEvent() *MergeRequestEvent      { return &e.MergeRequestEvent }+func (e *MergeRequestReopenEvent) ToEvent() *MergeRequestEvent     { return &e.MergeRequestEvent }+func (e *MergeRequestUnapprovedEvent) ToEvent() *MergeRequestEvent { return &e.MergeRequestEvent }++func (e *MergeRequestCloseEvent) Key() string  { return e.key("Close") }+func (e *MergeRequestMergeEvent) Key() string  { return e.key("Merge") }+func (e *MergeRequestReopenEvent) Key() string { return e.key("Reopen") }++func (e *MergeRequestEvent) key(prefix string) string {+	// We can't key solely off the merge request ID because it may be reopened+	// and closed multiple times. Instead, we'll use the CreatedAt field.+	return fmt.Sprintf("MergeRequest:%s:%d:%s", prefix, e.MergeRequest.IID, e.MergeRequest.CreatedAt.Format(time.RFC3339))

Yes, it should be the event. Good catch!

LawnGnome

comment created time in 7 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha 25dd9445a844021e4c421b128de022f9405aa8f1

Update schema/gitlab.schema.json Co-authored-by: Erik Seliger <erikseliger@me.com>

view details

push time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

+package webhooks++import (+	"encoding/json"++	"github.com/pkg/errors"+	"github.com/sourcegraph/sourcegraph/internal/extsvc/gitlab"+)++// EventCommon contains fields that are common to all webhook event types.+type EventCommon struct {+	ObjectKind string               `json:"object_kind"`+	EventType  string               `json:"event_type"`+	Project    gitlab.ProjectCommon `json:"project"`+}++// Simple events that are simply unmarshalled and have no methods attached are defined below.+

I actually did this intentionally, since it's not a docblock for PipelineEvent as such. (But, equally, there used to more types than just PipelineEvent under this comment.)

Do you think I should change it to a more specific docblock?

LawnGnome

comment created time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

 We are actively collaborating with GitLab to improve our integration (e.g. the [ | [`GET /projects/:id`](https://docs.gitlab.com/ee/api/projects.html#get-single-project) | `api` | (1) If using GitLab OAuth and repository permissions, used to determine if a user has access to a given _project_; (2) Used to query repository metadata (e.g. description) for display on Sourcegraph. | | [`GET /projects/:id/repository/tree`](https://docs.gitlab.com/ee/api/repositories.html#list-repository-tree) | `api` | If using GitLab OAuth and repository permissions, used to verify a given user has access to the file contents of a repository within a project (i.e. does not merely have `Guest` permissions). | | (future) write access | `api` | graph site-admins (only) to perform large-scale code refactors, with Sourcegraph issuing and managing the merge requests on GitLab repositories, company-wide. |++## Webhooks++The `webhooks` setting allows specifying the webhook secrets necessary to authenticate incoming webhook requests to `/.api/gitlab-webhooks`.++```json+"webhooks": [+  {"secret": "verylongrandomsecret"}+]+```++Using webhooks is optional, but if configured on GitLab, they allow faster campaign updates than the normal background syncing (i.e. polling) which `repo-updater` provides.++The following [webhook events](https://docs.gitlab.com/ee/user/project/integrations/webhooks.html) are currently used:++- Merge request events+- Pipeline events++Webhooks must be configured on each project that you wish to monitor on GitLab. To set up a project webhook on GitLab, go to the Webhook settings page of the project.

(reposting with a better cropped/drawn screenshot)

On gitlab.com, "webhooks" is its own settings page:

image

Did you see something different elsewhere? I wonder if there's a version difference here.

LawnGnome

comment created time in 7 days

Pull request review commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

 We are actively collaborating with GitLab to improve our integration (e.g. the [ | [`GET /projects/:id`](https://docs.gitlab.com/ee/api/projects.html#get-single-project) | `api` | (1) If using GitLab OAuth and repository permissions, used to determine if a user has access to a given _project_; (2) Used to query repository metadata (e.g. description) for display on Sourcegraph. | | [`GET /projects/:id/repository/tree`](https://docs.gitlab.com/ee/api/repositories.html#list-repository-tree) | `api` | If using GitLab OAuth and repository permissions, used to verify a given user has access to the file contents of a repository within a project (i.e. does not merely have `Guest` permissions). | | (future) write access | `api` | graph site-admins (only) to perform large-scale code refactors, with Sourcegraph issuing and managing the merge requests on GitLab repositories, company-wide. |++## Webhooks++The `webhooks` setting allows specifying the webhook secrets necessary to authenticate incoming webhook requests to `/.api/gitlab-webhooks`.++```json+"webhooks": [+  {"secret": "verylongrandomsecret"}+]+```++Using webhooks is optional, but if configured on GitLab, they allow faster campaign updates than the normal background syncing (i.e. polling) which `repo-updater` provides.++The following [webhook events](https://docs.gitlab.com/ee/user/project/integrations/webhooks.html) are currently used:++- Merge request events+- Pipeline events++Webhooks must be configured on each project that you wish to monitor on GitLab. To set up a project webhook on GitLab, go to the Webhook settings page of the project.

Did you look at gitlab.com, or a private instance? On the former, it's a distinct page in the navigation:

image

Happy to clarify the wording, but I'm wondering if there's a version difference here somehow.

LawnGnome

comment created time in 7 days

Pull request review commentsourcegraph/sourcegraph

Add schema for importing existing changesets

         }       }     },+    "importChangesets": {+      "type": "array",+      "description": "Import existing changesets on code hosts.",+      "items": {+        "type": "object",+        "additionalProperties": false,+        "required": ["repository", "externalIDs"],+        "properties": {+          "repository": {

repository, or repo?

eseliger

comment created time in 7 days

Pull request review commentsourcegraph/sourcegraph

Add schema for importing existing changesets

         }       }     },+    "importChangesets": {+      "type": "array",+      "description": "Import existing changesets on code hosts.",+      "items": {+        "type": "object",+        "additionalProperties": false,+        "required": ["repository", "externalIDs"],+        "properties": {+          "repository": {+            "type": "string",+            "description": "The repository name as configured on your Sourcegraph instance."+          },+          "externalIDs": {+            "type": "array",+            "description": "The changesets to import from the code host. For GitHub this is the PR number, for Gitlab this is the MR number, for Bitbucket server this is the PR number.",
            "description": "The changesets to import from the code host. For GitHub this is the PR number, for GitLab this is the MR number, for Bitbucket Server this is the PR number.",

I wish everyone would pick a capitalisation scheme and stick to it. :smiley:

eseliger

comment created time in 7 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha c6db6b232639b991eb5746a9b90daf12f6133c7d

schema: allow personal repos on Bitbucket Server (#12504) Unlike GitHub and GitLab, personal repos are additionally namespaced with a tilde on Bitbucket Server. We should be able to specify these in the repository and exclusion configuration.

view details

push time in 7 days

delete branch sourcegraph/sourcegraph

delete branch : aharvey/allow-bitbucket-user-repos

delete time in 7 days

PR merged sourcegraph/sourcegraph

schema: allow personal repos on Bitbucket Server bitbucket bug team/campaigns

Unlike GitHub and GitLab, personal repos are additionally namespaced with a tilde on Bitbucket Server. We should be able to specify these in the repository and exclusion configuration.

<!-- Reminder: Have you updated the changelog and relevant docs (user docs, architecture diagram, etc) ? -->

+52 -10

0 comment

3 changed files

LawnGnome

pr closed time in 7 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha a95f5549737301162269bb82f3ea6283705df9b3

Remove a TODO. (For real, I wrote a 1k+ line test file instead of putting it inline, so this is _definitely_ done.)

view details

push time in 8 days

pull request commentsourcegraph/sourcegraph

campaigns: add GitLab webhook support

This is ready for review.

LawnGnome

comment created time in 8 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha e23fb43542487b94c0416998b3d00c63faaced00

Add webhook documentation.

view details

push time in 8 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha cf5105bec28880c4d7cf1ce7dcbfc970a367db4e

Add GitLab webhook support. This doesn't yet wire up a handler, but all the bits needed for configuration are in place.

view details

Adam Harvey

commit sha 69b7d003324943cb246cce92f322083542dd1d50

Add stub GitLab webhook implementation.

view details

Adam Harvey

commit sha a61b34501963fbbf24448ca76d45df2505f4863f

straggler test

view details

Adam Harvey

commit sha 72dae5c44925c646d411304cb49ac81adbeee85c

Add GitLab Label type.

view details

Adam Harvey

commit sha a274edce541fe086d12e477ccd83e05670ad0cce

Add initial GitLab webhook internals.

view details

Adam Harvey

commit sha 6edb1cb115d3fd5a1becad4c2e6ad1f77c3c0b96

WIP webhook handler stub.

view details

Adam Harvey

commit sha 91fe119cf14be3838b8fd6f19ba27dc18007b468

Make naming consistent.

view details

Adam Harvey

commit sha 3ea03d9e2337082673fc6b16d614851a3335784f

Harden time handling against webhooks.

view details

Adam Harvey

commit sha 1ab1da7e7caf7af4e4bbbeb0c8fbd4f5bb1724f7

Fix

view details

Adam Harvey

commit sha cc65d0754de393e6546db89457b9932ce79fb3af

Allow GitLab connections to be resolved.

view details

Adam Harvey

commit sha aea1602ff939febe99b15ba6d08d9bdf1fbdfb29

Stub WIP of note support. This might get rebased out tomorrow; GitLab is doing something hinky with system notes not coming through as note hooks reliably.

view details

Adam Harvey

commit sha 20f425d0062baee064f4459b2be58342bca0b19f

Implement approval support for real.

view details

Adam Harvey

commit sha 50986b62ddb0780782e5c79e866e4d550a6d4e34

Implement pipeline decoding.

view details

Adam Harvey

commit sha b6808f0ded64228777feb398e53c5bd1bf621c15

Split merge request webhooks into typed events.

view details

Adam Harvey

commit sha 1ab320d940c960fecd79788c47e406bb43bc1ade

Refactor GitLab webhook.

view details

Adam Harvey

commit sha 3e621edc92b8c3837d9077a9bbe2caf0c3920f12

Wire up GitLab changeset state events.

view details

Adam Harvey

commit sha 19cab8ca8515da42dde9851bdd31a96be86f90f6

Add router error handling tests.

view details

Adam Harvey

commit sha b368a4c176d35f5dee46811cb39bf73fa04656c6

Improve transaction support in GitLab webhook tests.

view details

Adam Harvey

commit sha 4694723b103930c3f0c788f816625c3dc27a9ffd

Fix erroneous assertion.

view details

Adam Harvey

commit sha c7710083aa9f82ef82fa09e387127efae0c1b638

Add valid merge request tests.

view details

push time in 8 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha f54cce669606a6708a91c435b0877b5188c607bf

Complete code coverage for webhooks.

view details

push time in 8 days

push eventsourcegraph/sourcegraph

Rijnard van Tonder

commit sha 3029f7449c0fb1198ffcbaee4ebdf25b8ee02ff7

search: log search inputs for differential testing parsers (#12479)

view details

Robert Lin

commit sha e208aa82343e82cf7a1b1bacc88b4d008dd0f537

monitoring: base alert_count only on state=firing alerts (#12483)

view details

github-actions[bot]

commit sha c6d280cec8513a8c2f56ae39185cba5acb955a6e

Update third-party licenses (#12488) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

view details

Thorsten Ball

commit sha 9a85a646af4887e3b9551bdcc43f8dac7f2614a2

Update example in basestore.Store docstring (#12432) I think the docstring was a bit outdated.

view details

Eric Fritz

commit sha 74f698b0c8fcb8cb247580580984cce550fed2c7

codeintel: Add CalculateVisibleUploads to store (#12408)

view details

Ryan Slade

commit sha 65d2eae2857724b745ad953f6b719a7bd1ee746c

go: Update out graphql-go dependency (#12459)

view details

Stefan Hengl

commit sha 255cb618c755026d801b2d2fd5bc9d30197c19a5

search: support negation for field content (#12412)

view details

Keegan Carruthers-Smith

commit sha a521dfbea179120227aae57e31720c998475eeaf

zoekt: prevent indexserver stalling (#12502) See https://github.com/sourcegraph/zoekt/compare/6ed80895db7e...a99799fdc041 - a99799f ctags: handle fatal ctags errors - 5fcad78 ctags: skip commands that are too large - cf36be0 ctags: use file basename - d4fecb4 indexserver: timeout if no output for 5m - 34a0dd6 indexserver: coarse 1h timeout on index job - fdc9c2c Merge remote-tracking branch 'gerrit/master' - b48e35d shards: a progress message every 10s when loading - 4481f7b shards: throttle loading in watcher

view details

Robert Lin

commit sha 434fd412bfeb7c2cdb0909797a31e09d1588944f

monitoring: fix alert state query, fix alert firing threshold (#12489)

view details

Adam Harvey

commit sha e6f08abaafb49abf563ea79f1170697162141569

schema: allow personal repos on Bitbucket Server Unlike GitHub and GitLab, personal repos are additionally namespaced with a tilde on Bitbucket Server. We should be able to specify these in the repository and exclusion configuration.

view details

Adam Harvey

commit sha 329a0eff6faad89078b1bc5c75bf475dc99741a0

Add GitLab webhook support. This doesn't yet wire up a handler, but all the bits needed for configuration are in place.

view details

Adam Harvey

commit sha 61f3867da3afd92cda6ab6993a66752aeeac8ed1

Add stub GitLab webhook implementation.

view details

Adam Harvey

commit sha 4cca2c848c90ffe0c7dfb2c69aab05eeb3781ba7

straggler test

view details

Adam Harvey

commit sha d66826dd621449ddf772a4cf636b7c18e38a13b6

Add GitLab Label type.

view details

Adam Harvey

commit sha c922622b6f52786eba833b18acc4d3bb6152c4a1

Add initial GitLab webhook internals.

view details

Adam Harvey

commit sha b4b6a572c6eaaca2523f111b03c657ca3aded432

WIP webhook handler stub.

view details

Adam Harvey

commit sha e297eb2db0cb8eaa2f9bf25189cdb4b8b4461059

Make naming consistent.

view details

Adam Harvey

commit sha 3294c80d30450a8b1ab554210432d96efbba2f54

Harden time handling against webhooks.

view details

Adam Harvey

commit sha 74be12f2b46dea8feff4f13ad185bb500945a4da

Fix

view details

Adam Harvey

commit sha 1d525314f6a0e86c6c1de61c8619f4bc8e8abdb6

Allow GitLab connections to be resolved.

view details

push time in 8 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha e6f08abaafb49abf563ea79f1170697162141569

schema: allow personal repos on Bitbucket Server Unlike GitHub and GitLab, personal repos are additionally namespaced with a tilde on Bitbucket Server. We should be able to specify these in the repository and exclusion configuration.

view details

push time in 8 days

PR opened sourcegraph/sourcegraph

schema: allow personal repos on Bitbucket Server bitbucket bug team/campaigns

Unlike GitHub and GitLab, personal repos are additionally namespaced with a tilde on Bitbucket Server. We should be able to specify these in the repository and exclusion configuration.

<!-- Reminder: Have you updated the changelog and relevant docs (user docs, architecture diagram, etc) ? -->

+52 -10

0 comment

3 changed files

pr created time in 8 days

create barnchsourcegraph/sourcegraph

branch : aharvey/allow-bitbucket-user-repos

created branch time in 8 days

push eventsourcegraph/sourcegraph

Rijnard van Tonder

commit sha 8af81ddfda37cfed89de48b65aaa12876d187f87

search: prevent nil deref for certain empty results (#12437)

view details

Erik Seliger

commit sha 3dd98c61c9b1eda660e4af2a762d8301ecc743c7

Fix monaco in storybooks (#12403) Adds webpack config and moves to telemetry service to make events suppressable.

view details

Robert Lin

commit sha 6ee7fef15b7b75f68678682e8a8eae10239dda57

docker-images(cadvisor): upgrade to v0.37.0 (#12426)

view details

Robert Lin

commit sha dd2b7b11d0374f2c1a28b14e48c55661ac7cc625

monitoring: alert owners (#12358) Adds a new required Observable.Owner field, with predefined owners based on 2021 engineering org, as proposed in #12010 for RFC-189. This field does not currently do anything. Many of these teams don't exist yet, but it seems the ones that don't are clearly covered by an existing team in the team page (for example, backend infrastructure => current Cloud team) Owners were assigned based on guesses for this first pass. Co-authored-by: ᴜɴᴋɴᴡᴏɴ <joe@sourcegraph.com>

view details

Stefan Hengl

commit sha 5b7dcce9983a8c1a25070d86d5c237e67f63d295

web: remove regex anchors if globbing is active (#12287)

view details

Ryan Slade

commit sha c70637d0ce3093ea8d1ad9d2ae71d93d1ecc183f

basestore: Fix doc comment (#12429)

view details

Farhan Attamimi

commit sha 3dd163b1925309b0d62fbff1f64acb9e31a612b0

Log click events on search homepage and repogroup page (#12405)

view details

Farhan Attamimi

commit sha ed9e75d750d782fc240c203610c6dac0338a8445

Fix: low contrast search examples in dark theme (#12462)

view details

Stefan Hengl

commit sha be8e5931216e63c103e7ad4f9cb2bab9324237a4

search: fix panic in globToRegex when input string has trailing backslash (#12463)

view details

Thorsten Ball

commit sha 6097255600f39e9427b14de9d7d57a6aa00486d4

Wrap basestore.Store in campaigns.Store (#12461) * Wrap basestore.Store in campaigns.Store * Remove now-unused Store.exec

view details

dependabot[bot]

commit sha 7fbcccff24d78af6b3b3bf0ae57502ffc6cc2e0d

build(deps): bump lodash in /packages/sourcegraph-extension-api (#12250) Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19. - [Release notes](https://github.com/lodash/lodash/releases) - [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.19) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

view details

Loïc Guychard

commit sha 88901b511b31ec7c95468da8be37a4bff7d3cda0

Fix private repository detection on GitHub.com (#12466)

view details

Keegan Carruthers-Smith

commit sha 9c70e49a5c8371a31bea82a291188bf9b869e805

zoekt: indexserver removes repos from queue (#12474) This includes several changes to zoekt indexserver which improve observability as well as make us behave better during rebalancing. In particular we would try to index all repos on a replica if we ever rebalanced. We will still do that, but as soon as the replica comes back up we will stop doing that. - 3e5f0f3 indexserver: track last index state as a metric - cfdfeff indexserver: metrics for queue - a453878 indexserver: stop indexing repositories if not listed

view details

uwedeportivo

commit sha 964d97bd4d5817949a5cf596e415c454525313fb

e2e regression test fixes (#12439) * e2e regression tests * e2e -> test in selectors * Update web/src/regression/extensions.test.ts Co-authored-by: Loïc Guychard <loic@sourcegraph.com> * prettier * eslint exception for innerText in core.test * eslint exception for innerText in core.test take 2 Co-authored-by: Loïc Guychard <loic@sourcegraph.com>

view details

uwedeportivo

commit sha 0ef436765fbe494af19506a6eaf42b86fc12ebf7

fix build (#12475)

view details

Felix Becker

commit sha 646ebbb62f5ed21ab1feec1e4fab19a0bb39845e

Use Dart Sass (#12329)

view details

Rijnard van Tonder

commit sha c6cfafb22cb2693695cadcffef5f06f219d8186f

search: fix field value scanner (#12446)

view details

Rijnard van Tonder

commit sha 1d5a67069d35c8840a075207aa77890ae61e7f89

search: error on breaking concat invariant (#12457)

view details

Adam Harvey

commit sha fa9c4e6c416f1f9d3f9ea0304facfbfae6d5e52b

Add GitLab webhook support. This doesn't yet wire up a handler, but all the bits needed for configuration are in place.

view details

Adam Harvey

commit sha 804101bfc68740fc837aabb490e14f255e3b6c03

Add stub GitLab webhook implementation.

view details

push time in 11 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha cb6fd9c981ee181e9c528626f56148604634b697

Improve transaction support in GitLab webhook tests.

view details

Adam Harvey

commit sha 8b706119c93dde2f5a823c6f6d5325cc7bdf4d78

Fix erroneous assertion.

view details

Adam Harvey

commit sha 926f5eebf388b77e196056f5f83f5f8226c5b9a2

Add valid merge request tests.

view details

Adam Harvey

commit sha 8ca3f21f50b05e25485c2c564dead2a77149507f

Add pipeline tests.

view details

push time in 11 days

push eventsourcegraph/sourcegraph

ᴜɴᴋɴᴡᴏɴ

commit sha 5ce8ae587d734252af7d8ab439510a8fddb4f421

gqltest: add docs to run and add tests (#12285) Co-authored-by: Rijnard van Tonder <rvantonder@gmail.com>

view details

Keegan Carruthers-Smith

commit sha 3cb30b5ff3f15ba152a9153873ea4b1ad3929519

honey: sample 1 in 16 events (#12317) Band-aid solution until we can investigate further. Again hitting very high usage. Likely due to recent scaling up of the indexed search cluster. Will require more investigation, but we should probably segment user request path.

view details

ᴜɴᴋɴᴡᴏɴ

commit sha 74159cd2319a43b08e7da75591fead5ba04d8758

authz: return empty `permissionsInfo` in OSS version (#12313)

view details

Ryan Slade

commit sha 9da26cb86fbf866625cd5133c23ee21c16dd124c

repo-updater: Sleep on error during syncClone (#12299) * repo-updater: Sleep on error during syncClone Instead of immediatley looping around and potentially putting more pressure on gitserver or the repo store. * Changes from review * Move err check * Refactor to remove sleep / continue pattern

view details

github-actions[bot]

commit sha 7fec1bcc70080af496621516526cd67851f6606e

Update third-party licenses (#12116) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

view details

ᴜɴᴋɴᴡᴏɴ

commit sha 4644536a09e4864073a60a812341b6d9342c418b

Update CHANGELOG.md (#12320)

view details

Keegan Carruthers-Smith

commit sha 2b195bf4f627fea8001aba68efbbf6bc2452b62c

doc: Update and add docs for multiple index search (#12295) Lots of changes have happened due to indexing multiple branches. We update the dev architecture docs to mention options (how we configure multiple branch search) as well as shallow cloning. We mention multiple revision search syntax. We mention how to configure multi branch indexing. We mention how to search HEAD explicitly.

view details

Keegan Carruthers-Smith

commit sha 42f81405fad86921e76e9b317d38ad5358e5dd34

ci: only run go steps if only changing go (#12321) This should significantly speed up Go PR checks.

view details

Thorsten Ball

commit sha d6d2fcdd1ab0ebfd50cb5431083b60853f2bfd04

Rename MockGitHubChangesetSync to more generic name (#12315) We've talked about this when the mock was introduced, but I think it got lost. So, here we go: this mock is not tied to GitHub, since its mocks are called in `SetDerivedState` when computing the `SyncState`, let's get rid of the `GitHub` in the name.

view details

Thorsten Ball

commit sha f18421a031104323cec4c830063efae1c6f28875

Remove unneeded context and error from CheckState method (#12316)

view details

Felix Becker

commit sha c6de674fa8b71600d28f4ab0028ccad8d97795b0

Automatically accept Chromatic baseline on master (#12325)

view details

Asdine El Hrychy

commit sha 90c8a30e6a4dbf48f23692658ca5be39e741df74

Replace gophermail by jordan-wright/email (#12239)

view details

Asdine El Hrychy

commit sha df326976fe6f99d344a1c6b66bf52f241370b998

Skip TLS Verify check if TLS is disabled (#12243)

view details

Erik Seliger

commit sha caa5695a399417f8f933ca30f55329e46a0368c4

Fix flacky chromatic snapshots (#12338)

view details

하광일 / CE / TC

commit sha 9161e3ef106ccc4ed1804b221f9db8fcd5d3fb96

Fix jaeger config in single docker container (#12312)

view details

Bunny

commit sha 219cc4d341d9fd293230a811d099bece7d777d9d

Revisions (#12217) * Revisions Updated the first paragraph and last sentence. Made a handful of other minor changes for readability * Removed redundancy about changeset * Fixed typo

view details

Keegan Carruthers-Smith

commit sha 1dffc994280aef9626250b1d64bc6b531e26636d

zoekt: improve logs and resource usage when loading (#12322) When an instance has a lot of shards it can take a while to start. This update includes two improvements to zoekt: - shards: throttle loading in watcher https://gerrit-review.googlesource.com/c/zoekt/+/275902 - shards: a progress message every 10s when loading https://gerrit-review.googlesource.com/c/zoekt/+/275903

view details

Dax McDonald

commit sha aa821d01117f83c7b5081f7f36e086655bbd7f2b

Update latest release to 3.18.0 (#12305)

view details

Felix Becker

commit sha 23b198aecbaf5dceb780176795d6dcc26cb116d2

Integration test improvements (#12324)

view details

Keegan Carruthers-Smith

commit sha 08e13f12caccde4960d2c49f98d9592639b2c1ac

src-expose: avoid git update-server-info in request path (#12342) We now only run git update-server-info if a repository has not been configured. This change exposed a bug in our hook setting being incorrect. This wasn't visible before since we would always run update-server-info when listing repos.

view details

push time in 12 days

push eventLawnGnome/waybar-custom-modules

Adam Harvey

commit sha 9f0d09cba3c3f5ad7e1be34c9e8416adcbf1933e

Remove unneeded struct field.

view details

push time in 13 days

push eventLawnGnome/waybar-custom-modules

Adam Harvey

commit sha cc09057d1ae88f377efcd6ac90fd440585c29bca

Add basic README.

view details

push time in 13 days

push eventLawnGnome/waybar-custom-providers

Adam Harvey

commit sha b12414ddae6bdf0d7fbba9e3621403445ccab7c8

Initial commit.

view details

push time in 13 days

create barnchLawnGnome/waybar-custom-providers

branch : master

created branch time in 13 days

created repositoryLawnGnome/waybar-custom-providers

An assortment of small(ish) custom modules for Waybar, including some optional sparkline support

created time in 13 days

push eventsourcegraph/src-cli

Adam Harvey

commit sha 83e31179da8ee039dd8dfc8585e85ce3fb181b0b

Add GitLab to the list of supported hosts (#249) * Add GitLab to the list of supported hosts. * Perform an explicit version check.

view details

push time in 13 days

PR merged sourcegraph/src-cli

Add GitLab to the list of supported hosts action-exec enhancement team/campaigns
+135 -3

0 comment

3 changed files

LawnGnome

pr closed time in 13 days

PR opened sourcegraph/sourcegraph

web: add storybook for SiteAdminExternalServicePage

As SiteAdminExternalServicePage is tightly coupled to fetching its data from GraphQL, I've used fetch-mock to pre-populate it with interesting external service configurations. I am currently unsure how I actually feel about this.

Unfortunately, I need the full page to be able to render and test the webhook configuration blocks, which is my actual goal, as part of #12139.

@eseliger, I'm particularly interested in your thoughts on this. Is this too icky to merge? Should I refactor the page to make it easier to test the bits I need? And, more broadly, since this is my very first time using storybooks, any other feedback would also be greatly appreciated. :smiley:

+119 -4

0 comment

3 changed files

pr created time in 14 days

create barnchsourcegraph/sourcegraph

branch : aharvey/site-admin-config-storybook

created branch time in 14 days

Pull request review commentsourcegraph/src-cli

Add GitLab to the list of supported hosts

 fragment repositoryFields on Repository { 		}  		// Skip repos from unsupported code hosts but don't report them explicitly.-		if !includeUnsupported && strings.ToLower(repo.ExternalRepository.ServiceType) != "github" && strings.ToLower(repo.ExternalRepository.ServiceType) != "bitbucketserver" {+		if !includeUnsupported && strings.ToLower(repo.ExternalRepository.ServiceType) != "github" && strings.ToLower(repo.ExternalRepository.ServiceType) != "bitbucketserver" && strings.ToLower(repo.ExternalRepository.ServiceType) != "gitlab" {

Yep, implemented.

LawnGnome

comment created time in 14 days

push eventsourcegraph/src-cli

Adam Harvey

commit sha 78de4698e6f2294e3807cd5dbbadabc22019f6b4

Perform an explicit version check.

view details

push time in 14 days

PR opened LawnGnome/forward-component

Update ESLint

Upgrade ESLint to the current 7.5.0.


This pull request was created by a Sourcegraph campaign. Click here to see the campaign.

+33 -13

0 comment

2 changed files

pr created time in 14 days

create barnchLawnGnome/forward-component

branch : update-eslint

created branch time in 14 days

PR opened sourcegraph/src-cli

Add GitLab to the list of supported hosts action-exec enhancement team/campaigns
+3 -1

0 comment

2 changed files

pr created time in 15 days

push eventsourcegraph/src-cli

Adam Harvey

commit sha 8b7aefd885a603c1e5cd2d96103daea11beca3ee

Add GitLab to the list of supported hosts.

view details

push time in 15 days

push eventsourcegraph/src-cli

Thorsten Ball

commit sha e6a62f3b4a6da760efd0b31312706e50fb2ef5c3

Move action execution related code to campaigns package (#230)

view details

Thorsten Ball

commit sha f567168183ff2344936087c9002990d94dffe5d5

Fix warnings produced by staticcheck (#231) I just ran [staticcheck.io](https://staticcheck.io) over the repository and these came up.

view details

Erik Seliger

commit sha ee088b9145ad2d21df611c7795ab3ed84239c38f

Allow yaml action files and make it the default format (#196)

view details

Eric Fritz

commit sha 0c602c12a60b879c5ab0c38a7a37dd56ae66ec38

Move shared functionality to codeintelutils (#208)

view details

Ryan Slade

commit sha 7d8389a631e01eba0cdb534a0144f1140e6f98e9

Revert "Remove -endpoint flag (#225)" (#234) * Revert "Remove -endpoint flag (#225)" This reverts commit a167443ba43d332f3dc6ee8b024c3e0d9b576779. This broke some existing users of src-cli. Another PR will make this a more graceful deprecation. * Remove unused package

view details

Adam Harvey

commit sha 037d69f03b7d51068a5d4177eec6bb2271ad825c

Update after landing sourcegraph/jsonx#8. (#233)

view details

Ryan Slade

commit sha fd5f1ac762a0c88dc88e0d02e48d3e668b4cf2ed

Deprecate -endpoint flag (#235) * Deprecate -endpoint flag This removes the documentation of the endpoint flag but still allows it to be used to override either the config or ennvironment variables. * Update changelog. * Update cmd/src/main.go Co-authored-by: Thorsten Ball <mrnugget@gmail.com> * Clean up CHANGELOG * Use t.Cleanup * Use Go 1.14 * Use Go 1.14 in Travis too * Try specific version of Go for Appveyor * Try example appveyor.yml * Revert "Try example appveyor.yml" This reverts commit 5f86ecef8a92ec5063428fa4a7458fee40e49923. * One more attempt, use different GOPATH Apparently Go 1.14.3 is installed by default: https://www.appveyor.com/docs/windows-images-software/#golang * Revert back to being Go 1.13 compatible Co-authored-by: Thorsten Ball <mrnugget@gmail.com>

view details

Erik Seliger

commit sha 522a55cfd4c1f5edbfe48a48e07abf2e56b3d93b

Visualize unsupported repos better (#236)

view details

Thorsten Ball

commit sha 12d0c5e7692bdb619c00c5faf8742c55aa44a0c1

Use Go 1.14 on AppVeyor (#240)

view details

Ryan Slade

commit sha 941986d6352947053f637bef721b154a324c349a

Travis should use Go 1.14 (#241)

view details

Ryan Slade

commit sha 3d0de6f6d701a7096fb87f8a45121d10215d6fec

Use testing.Cleanup (#242)

view details

Eric Fritz

commit sha d20515158205fb7db0874c0693b5237c445c7a66

Add progress bar to lsif upload command (#243)

view details

Eric Fritz

commit sha a7aec117ba86a3e1317d4e6a22bdb8046466a204

Allow users to supply arbitrary HTTP headers for requests (#239)

view details

Eric Fritz

commit sha aa0e80f75fdfaf8109514fa59bb81fb58cb61c2c

Add option to disable upload progress bar (#244)

view details

Eric Fritz

commit sha 0df5f2eb9e657b7f58dd7143229e6cca1980d5e9

Add instructions to update changelog on release. (#245)

view details

Eric Fritz

commit sha 83abb31584c043ea2de258384375c75a0696662e

Update changelog for 3.16 release.

view details

Eric Fritz

commit sha 59d8d22b39e5f7ae1a4901108a90bed0ef724361

Run go mod tidy.

view details

Asdine El Hrychy

commit sha b721a1270f64f64e17314182ac64d2053b831439

Remove clone-in-progress flag (#246)

view details

Thorsten Ball

commit sha c8eca18fb32839869196b7a5e95c1c6d637725d7

Fix progress bar when previously printed text has color (#247) * Remove unused dependency in ActionLogger * Fix typo * Show progress bar even when printed text has color

view details

renovate[bot]

commit sha 991b59d234b24fcd8142d384b4f90526c7199fe4

Add renovate.json (#26) Co-authored-by: Renovate Bot <bot@renovateapp.com>

view details

push time in 15 days

push eventLawnGnome/forward-component

Adam Harvey

commit sha f155a6b77111c5ac4150b7ef68c307419e4453cf

Create CI workflow

view details

Adam Harvey

commit sha cf3a3d713ba028ce9fc02bb3f79bd61b6cace279

Update node.js.yml

view details

Adam Harvey

commit sha 24ec2120afacab6c6598016a473b92bd86892997

Merge pull request #2 from LawnGnome/ci Create CI workflow

view details

push time in 15 days

push eventLawnGnome/forward-component

Adam Harvey

commit sha cf3a3d713ba028ce9fc02bb3f79bd61b6cace279

Update node.js.yml

view details

push time in 15 days

PR opened LawnGnome/forward-component

Create CI workflow
+28 -0

0 comment

1 changed file

pr created time in 15 days

create barnchLawnGnome/forward-component

branch : ci

created branch time in 15 days

push eventLawnGnome/forward-component

Adam Harvey

commit sha 2dbcabf7d42965258758a408a801d050fccf352d

Add eslint.

view details

push time in 15 days

create barnchLawnGnome/forward-component

branch : master

created branch time in 15 days

created repositoryLawnGnome/forward-component

Skeleton component to demonstrate Sourcegraph campaigns

created time in 15 days

issue commentsourcegraph/sourcegraph

Campaigns: 3.18 Tracking Issue

Weekly update: 2020-07-13 to 2020-07-17

Last week, I finalised GitLab support, then moved on to GitLab webhook support. This work has gone pretty well: most of the issues that have been hit have been related to idiosyncrasies in GitLab's webhook API, especially the approval/unapproval flow being different to the normal one we use in syncing (webhook approval events come through as merge request events, but syncing approvals happens via notes) and GitLab using a different date/time format for some webhooks. Nevertheless, the GitLab webhook work is now feature complete.

This week, I'll be finishing the testing of GitLab webhooks, including building a story file for the site admin component, producing GitLab-related collateral for our 3.18 blog post, and then moving on to... well, whatever we decide at tomorrow's iteration planning meeting, but most likely some src-cli work to support the new campaign flow.

mrnugget

comment created time in 15 days

push eventsourcegraph/sourcegraph

Keegan Carruthers-Smith

commit sha 5829ec0edee0517b562bbc45122c243ed1a507e1

codeowners: remove cloud from shared files (#12244) Cloud currently is the codeowner of many files that are modified by people all over Sourcegraph. As such our whole team is added to code reviews where our input really isn't required. There is discussions about removing code owners entirely. This is a step in that direction for the more egregerious examples of it for cloud. If you thought it was important the reviewed you did on something I removed ownership on, please add your own name (instead of team) to the corresponding directory.

view details

renovate[bot]

commit sha e3bcfd4dfbd96e05754b8e84aaa26b5fb82531e3

Update dependency @testing-library/react to ^10.4.7 (#11965) Co-authored-by: Renovate Bot <bot@renovateapp.com>

view details

renovate[bot]

commit sha f4ff73d7d3622802bcb474bdf6329f45fd0b9d47

Update dependency @slack/web-api to ^5.10.0 (#11143) Co-authored-by: Renovate Bot <bot@renovateapp.com>

view details

Keegan Carruthers-Smith

commit sha c132bff3065070519744a4b62092e6b944d72461

issue_template: typo in tracking_issue.md (#12279)

view details

Keegan Carruthers-Smith

commit sha f962fdb211f110a6dc937bbf752795ed3940a041

ci: run shared tests sooner (#12278) In particular I've seen client integration tests be the slowest build step.

view details

Rijnard van Tonder

commit sha ee8a1af850dbd14cd62a9fff20ac42284de066e6

search: unify parser routines (#12231)

view details

Stephen Gutekanst

commit sha 15c59b435636a00aeb4e5589266da08fb5ddc0e5

AWS: correct ulimit / fsnotify limit (#12275) Helps #11410

view details

Eric Fritz

commit sha 592bad183e57956f797f169f4e32c2301b2f0baf

codeintel: Refactor store (#12035)

view details

Stephen Gutekanst

commit sha 8ae83ab448128b649a884ee2a0d8e17f88452358

AWS: clarify that device mount point is important (#12282) Two customers have mistakenly set this to the default which is not `/dev/sdb` and the script has failed as a result. Helps https://github.com/sourcegraph/sourcegraph/issues/9829

view details

Dax McDonald

commit sha 9dd021a3365dc742c1fcf42e9c9b869ca1d0b125

Drop update_check.go (#12281) Co-authored-by: Stephen Gutekanst <stephen.gutekanst@gmail.com>

view details

AlicjaSuska

commit sha c8e4a41651cb7b17830f0d1bcc40187b0ef65f40

Add Figma URL for UserNavItem (#12290)

view details

Keegan Carruthers-Smith

commit sha f3d88f690a5767ef06d46796f8e976d951932bb0

trace: select http requests with trace=1 (#12291) Currently we parse trace=1 in our javascript layer. This adds trace selection to our endpoints that are directly requested via non-js endpoints (eg loading a page).

view details

Ryan Slade

commit sha 5cf616b29a9c53a6e6ab7784aaebe7596cc8a3a5

leader: Add a leader election package (#12289) * rcache: Add options param to TryAcquireMutex * leader: Add a leader package A small helper package that wraps our Redis based mutex to allow only a single instance of a function to run concurrently. * Update CODEOWNERS * Update internal/rcache/mutex_test.go Co-authored-by: Asdine El Hrychy <asdine.elhrychy@gmail.com> * No need for release func The outler leader loop can release when the worker function returns * Add jitter and respect ctx during sleep * Make worker fn to last parameter Makes it easier to use especially when fn is anonymous * Improve test Co-authored-by: Asdine El Hrychy <asdine.elhrychy@gmail.com>

view details

Loïc Guychard

commit sha eaa28952f62b9e7d3303b007b7f4400b9d327e53

Remove todo-security check (#12292) This check is old, very specific (anything other than `TODO(security)`, such as `todo security`, `TODO: security`, etc. will be missed), and is flaky: - https://buildkite.com/sourcegraph/sourcegraph/builds/68719#ef0d0bae-bdc7-4412-aa2a-dda5ac10aecb - https://sourcegraph.slack.com/archives/C07KZF47K/p1594989190353700?thread_ts=1594988538.353300&cid=C07KZF47K I feel like our code review process is a sufficient safeguard here.

view details

renovate[bot]

commit sha 35a4bb98ac463b91ff9934136f4aa9504d910a5b

Update dependency @types/puppeteer to v3 (#11197) Co-authored-by: Renovate Bot <bot@renovateapp.com> Co-authored-by: Felix Becker <felix.b@outlook.com>

view details

Eric Fritz

commit sha d39f0c72cc4c8379cc7b1c48619c29367729b461

codeintel: Extract generic worker process (#12262)

view details

Eric Fritz

commit sha c8c34394d09aa73ac47465d250167f386004c1a4

Update LSIF docs to link to example CI configurations (#12225)

view details

Keegan Carruthers-Smith

commit sha 5a1c63bec7bd289906cf340e3b7550800f9d452a

dev: only support watchman (#12252) Maintaining three different ways to watch files leads to unneccessary bugs. Instead lets pick just one.

view details

Chayim

commit sha e4a647a4e2c043828519cc712be00ef65e7c796e

Secure Token Schema (#12207) Creating the migrations for the token table, database schema change to accommodate key-value pairs.

view details

Felix Becker

commit sha 24ee7987c4e671d0df8ae3609652dc5b082b7cf9

Increase contrast of placeholders to meet WCAG (#12213)

view details

push time in 18 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha 3e771b1a928e866d4d1732087eef2546e78e0669

Implement approval support for real.

view details

Adam Harvey

commit sha 5a79acb6998070b8b5fcdc62dda71154da71d91e

Implement pipeline decoding.

view details

Adam Harvey

commit sha bafd01ee8413598620f4e5d9502dddbf13ab60f6

Split merge request webhooks into typed events.

view details

Adam Harvey

commit sha 0c3301cd52eda6ed2dca61a771f8999bae3a6ea2

Refactor GitLab webhook.

view details

Adam Harvey

commit sha 62dc07f1689a554d821d08edf7c8357182caf4f6

Wire up GitLab changeset state events.

view details

push time in 18 days

push eventsourcegraph/sourcegraph

Adam Harvey

commit sha 9b987daa80b686b1ae82d3c136ceae45695d2a02

Make naming consistent.

view details

Adam Harvey

commit sha bc083ece29bedb759f70e7604a6b65ba489a781f

Harden time handling against webhooks.

view details

Adam Harvey

commit sha e026bd25ea70516565d3a0eb696f75cdf92e9153

Fix

view details

Adam Harvey

commit sha 72092cf03d46352fc2414858cab8be8bb6f6a987

Allow GitLab connections to be resolved.

view details

Adam Harvey

commit sha 2ba95fe63d984b595906ce462afcdb7cbd7abdf3

Stub WIP of note support. This might get rebased out tomorrow; GitLab is doing something hinky with system notes not coming through as note hooks reliably.

view details

push time in 19 days

more