Syntect is a complex stack machine with lots of arbitrary regexp thrown into the mix, some files are going to take a really long time to highlight or get the stack machine stuck, unfortunately. This is also a known issue and is why we use a subprocess model in syntect-server to enable us to kill/escape those cases.

Don't we already have heuristics in the frontend client for syntect that prevents asking it to highlight some files? It seems completely reasonable to me to skip highlighting a file with a long line or that is large?

When developing and restarting the sourcegraph frontend, it will overwrite the configuration stored in the database. This setting prevents loading configuration from files, which in turn will prevent the overwriting behaviour.

Fixed a documentation inconsistency for the server-git command in README.md

This problem also tickled me. I took a quick look after what Tomas found. I remember the on-disk formats for appendonly.aof was quite simple, and I hoped the same was true for RDB. So I looked at a hexdump in both files around the bad key. (in redis-cache-backup)

From reading rdbSaveRawString strings are just encoded as len data. It has some heuristics, but it looks like it does the verbatim encoding since we can read the string. rdbSaveLen is quite complicated, so we need to guess where the length starts but we can reverse engineer. The string has length 85 so it seems it will be encoded into [2]byte (2 bits for int type, 14 bits for length). Quoting redis src:

01|XXXXXX XXXXXXXX => 01, the len is 14 byes, 6 bits + 8 bits of next byte

So this little bit of python can decode lengths:

f = lambda a, b: ((a&0x3F)<<8)|b
# >>> f(0x40, 0x55)
# 85

So the "@U" is the length of the string! It looks like the value is appearing after that, but I have just hit my wall and will go to sleep now. I'll leave checking out the length of the value (to see if we are doing overflows) as an exercise for the reader.

The key in dump.rdb

# strings -t d dump.rdb    | grep orgs/sourcegraph 
  83514 @Uv2:http:http://127.0.0.1:3180/orgs/sourcegraph/repos?sort=created&page=3&per_page=100
2572037 @Uv2:http:http://127.0.0.1:3180/orgs/sourcegraph/repos?sort=created&page=2&per_page=100
2797503 @Uv2:http:http://127.0.0.1:3180/orgs/sourcegraph/repos?sort=created&page=5&per_page=100
3406123 @Uv2:http:http://127.0.0.1:3180/orgs/sourcegraph/repos?sort=created&page=4&per_page=100
4275305 @Uv2:http:http://127.0.0.1:3180/orgs/sourcegraph/repos?sort=created&page=1&per_page=100

# hexdump -s 4275290 -n 200 -C dump.rdb 
00413c5a  c6 0c ad cb 73 01 00 00  f8 80 00 05 02 57 00 40  |....s........W.@|
00413c6a  55 76 32 3a 68 74 74 70  3a 68 74 74 70 3a 2f 2f  |Uv2:http:http://|
00413c7a  31 32 37 2e 30 2e 30 2e  31 3a 33 31 38 30 2f 6f  |127.0.0.1:3180/o|
00413c8a  72 67 73 2f 73 6f 75 72  63 65 67 72 61 70 68 2f  |rgs/sourcegraph/|
00413c9a  72 65 70 6f 73 3f 73 6f  72 74 3d 63 72 65 61 74  |repos?sort=creat|
00413caa  65 64 26 70 61 67 65 3d  31 26 70 65 72 5f 70 61  |ed&page=1&per_pa|
00413cba  67 65 3d 31 30 30 c3 80  00 01 4c 56 80 00 08 59  |ge=100....LV...Y|
00413cca  f5 1f 48 54 54 50 2f 31  2e 31 20 32 30 30 20 4f  |..HTTP/1.1 200 O|
00413cda  4b 0d 0a 54 72 61 6e 73  66 65 72 2d 45 6e 63 6f  |K..Transfer-Enco|
00413cea  64 69 1f 6e 67 3a 20 63  68 75 6e 6b 65 64 0d 0a  |di.ng: chunked..|
00413cfa  41 63 63 65 73 73 2d 43  6f 6e 74 72 6f 6c 2d 41  |Access-Control-A|
00413d0a  6c 6c 6f 0a 77 2d 4f 72  69 67 69 6e 3a 20 2a e0  |llo.w-Origin: *.|
00413d1a  08 1f 19 45 78 70 6f 73                           |...Expos|
00413d22

The key in appendonly.aof

# strings -t d appendonly.aof     | grep orgs/sourcegraph 
 151149 @Uv2:http:http://127.0.0.1:3180/orgs/sourcegraph/repos?sort=created&page=5&per_page=100
 745433 @Uv2:http:http://127.0.0.1:3180/orgs/sourcegraph/repos?sort=created&page=4&per_page=100
 982409 @Uv2:http:http://127.0.0.1:3180/orgs/sourcegraph/repos?sort=created&page=2&per_page=100
2756815 @Uv2:http:http://127.0.0.1:3180/orgs/sourcegraph/repos?sort=created&page=1&per_page=100
3184082 @Uv2:http:http://127.0.0.1:3180/orgs/sourcegraph/repos?sort=created&page=3&per_page=100

# hexdump -s 2756790 -n 200 -C appendonly.aof 
002a10b6  73 22 3a 31 30 36 7d 5d  7d fc c6 0c ad cb 73 01  |s":106}]}.....s.|
002a10c6  00 00 f8 80 00 05 02 59  00 40 55 76 32 3a 68 74  |.......Y.@Uv2:ht|
002a10d6  74 70 3a 68 74 74 70 3a  2f 2f 31 32 37 2e 30 2e  |tp:http://127.0.|
002a10e6  30 2e 31 3a 33 31 38 30  2f 6f 72 67 73 2f 73 6f  |0.1:3180/orgs/so|
002a10f6  75 72 63 65 67 72 61 70  68 2f 72 65 70 6f 73 3f  |urcegraph/repos?|
002a1106  73 6f 72 74 3d 63 72 65  61 74 65 64 26 70 61 67  |sort=created&pag|
002a1116  65 3d 31 26 70 65 72 5f  70 61 67 65 3d 31 30 30  |e=1&per_page=100|
002a1126  c3 80 00 01 4c 56 80 00  08 59 f5 1f 48 54 54 50  |....LV...Y..HTTP|
002a1136  2f 31 2e 31 20 32 30 30  20 4f 4b 0d 0a 54 72 61  |/1.1 200 OK..Tra|
002a1146  6e 73 66 65 72 2d 45 6e  63 6f 64 69 1f 6e 67 3a  |nsfer-Encodi.ng:|
002a1156  20 63 68 75 6e 6b 65 64  0d 0a 41 63 63 65 73 73  | chunked..Access|
002a1166  2d 43 6f 6e 74 72 6f 6c  2d 41 6c 6c 6f 0a 77 2d  |-Control-Allo.w-|
002a1176  4f 72 69 67 69 6e 3a 20                           |Origin: |
002a117e

@slimsag The latter. Personally I never want to load config from a file in dev. I prefer relying on the state in the DB. The number of times I have cursed when working on a new feature (behind a feature flag) and sourcegraph frontend starting up (due to start.sh or recompile) removing the feature flag is innumerable.

As it stands we have a "default dev" config hardcoded, we could mutate that if we wanted to change the default dev config. My guess is most devs would also prefer that behaviour?

When developing and restarting the sourcegraph frontend, it will overwrite the configuration stored in the database. This setting prevents loading configuration from files, which in turn will prevent the overwriting behaviour.

I'm not sure why you need a different sort function for and/or. Can't we just modify sortResults?

nit:

                // Prefer shorter file names (ie root files come first)
				if len(afile) != len(bfile) {
					return len(afile) < len(bfile)
				}

            // prefer exact matches
			return aMatch

I think you should use path here, not filepath. As a benefit path.Base is more performant due to assumptions it can make. We are abstracted away from the FS (I hope :P).

you don't need to check for nil here, you can do len on nil maps.

zoekt-indexserver still uses significant CPU, without seeming to affect the only other service metric available (average revision resolve duration)

Average revision resolve duration will is measuring an RPC call, so it is putting load on gitserver (via frontend). So lots of CPU use indicates it is likely indexing a lot. I would look at the recently added queue metrics.

zoekt-webserver still uses all of its memory, with frequently firing "50s+ indexed search request errors every 5m by code" alerts - this might be an issue of the alert being on a hard threshold rather than a ratio

The webserver uses a lot of memory even if it is not serving any requests. IE the memory use is dominated by the working set of indexes, not the results or number of them generated. At this scale we would need a lot of traffic for it to contribute to memory use over just holding the indexes in memory.

Here is the graph over the last 14d:

I'm not sure how this graph interacts with silences, but it seems that the day this issue was filed (24th) the warning alerts went up a bunch. The 29th I shipped some fixes and the big one on the 30th (and one or two more the next day or two). You can see the graph go mostly silent again. Then on the weekend we scaled up index search to be 200k instead of 100k repos and you can see on Monday (when the site has more activity) all the alerts start up again. I would suspect the root cause is that. cc @beyang

Thanks for the proposal. I think we would. It seems like the set of inputs we understand will only increase + the empty diff makes sense. @eseliger @mrnugget thoughts? Any concerns around the nil time param?

What if we sort exact matches by length (keeping alphabetical order for everything else)? This way exact matches on root level would always come first and we would not need to introduce a new identifier for root files.

I like this approach. In fact we implemented something like this a long time ago. IE we implemented some "ranking" on results based on input. So a match on a filename is ranked higher than matching on a dirname, etc. I think simple things here go a long way. I think our current approach of sorting is just naive, so as long as we are mostly deterministic then what we are doing is good.

Rijnard van Tonder notifications@github.com writes:

But I would suspect the issue isn't the speed of "grep", but that we are struggling to fetch an archive of ghdump. Our searcher code doesn't do per line searching, but uses the same technique in rg of searching the whole body and hydrating in line breaks.

Whoops, guess my diagnosis is off then. I'll remember that part of our searcher now :-) The downloaded zip (at master) is only about 200MB , which is about the size of the Linux and we don't have any issues with that IIRC (git clone of ghdump is 2.6GB). So seems like there's still something else going on.

Ok then it likely is us being slow at searching lol. We have integration benchmarks which can easily plug in any repo@hash. So if you feel like playing with our oldest search code have fun :D

"src serve-git" is the new and streamlined version of src-expose. We add an entry for it in our frontend for code host setup to simplify use of it.

Additionally this PR updates documentation around src-expose to instead point to or be about "src serve-git". Note: we still haven't done a tagged release of src including serve-git. I will ensure that is done before merging this document so the instructions are valid.

Alternatively we considered add a new code host type. But that is quite a heavyweight approach from a backend perspective.

Fixes https://github.com/sourcegraph/sourcegraph/issues/12363

This PR contains the following updates:

Package	Type	Update	New value	Sourcegraph
golang	stage	patch	1.14.6-alpine
golang	stage	minor	1.14-alpine

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:ghost: Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	References	Sourcegraph
history	resolutions	minor	4.10.1	source

Release Notes

<details> <summary>ReactTraining/history</summary>

v4.10.1

Compare Source

• Fixes some issues with using hash history on a page with a <base> tag (#​577, #​578)

v4.10.0

Compare Source

• Allows basename URL prefixes with special regex characters (#​566, #​544)

v4.9.0

Compare Source

The main change in this release is pre-bundled single-file builds for both development and production environments. This is not a breaking change, since we still support old-style require('history/createBrowserHistory') statements for now. However, these are deprecated in favor of importing/requireing from the main module.

Please see the release notes for 4.98.0-beta.0 for more info.

v4.7.2

Compare Source

v4.7.1

Compare Source

v4.7.0

Compare Source

v4.6.3

Compare Source

v4.6.2

Compare Source

v4.6.1

Compare Source

v4.6.0

Compare Source

</details>

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	References	Sourcegraph
@reach/menu-button	dependencies	patch	0.10.5	source

Release Notes

<details> <summary>reach/reach-ui</summary>

v0.10.5

Compare Source

🐛 Bugfixes

• alert, rect, utils
  • #​622 Fix issues with the rect observer between renders (@​chancestrickland)

✨ Enhancements

• dialog
  • Added dangerouslyBypassFocusLock prop for customizing focus lock behavior (@​gnapse)
• menu-button
  • #​619 Don’t try to use relatedTarget of the mousedown event (@​Andarist)
• utils
  • Improve as prop typing (@​chancestrickland)

🤜🤛 3 Committers

• Chance Strickland (@​chancestrickland)
• Ernesto García (@​gnapse)
• Mateusz Burzyński (@​Andarist)

v0.10.4

Compare Source

🐛 Bugfixes

• slider
  • #​583 fix(slider): prevent keyboard events when disabled (@​raunofreiberg)
• popover
  • #​609 [popover] Fix stuck focus (@​michaeldfoley)
  • [Combobox] Tab sequence gets stuck in popover interactive elements (@​indiesquidge)
• tabs
  • #​584 fix(tabs): wait for TabPanel to mount before setting hidden (@​raunofreiberg)

📝 Documentation

• #​597 Fix tabs.mdx animated example (@​homburg)

✨ Enhancements

• combobox
  • Ability to pass targetRef to ComboboxPopover just like a normal Popover (@​sanfilippopablo)

🤜🤛 6 Committers

• Austin Wood (@​indiesquidge)
• Chance Strickland (@​chancestrickland)
• Michael Foley (@​michaeldfoley)
• Pablo Sanfilippo (@​sanfilippopablo)
• Rauno Freiberg (@​raunofreiberg)
• Thomas B Homburg (@​homburg)

v0.10.3

Compare Source

🔨 Breaking Changes

• The only meaningful change to packages in this release is to the API for @reach/descendants. We do not document this module and don't generally recommend it for external use, but in case you're being bold, useDescendants has been renamed to useDescendantsInit. useDescendants is still used, but it will now access the actual descendant list from the appropriate context.

📝 Documentation

• #​581 Updated the README.md with new link of yarn install instructions (@​abhishekjakhar)
• #​587 Fix Dialog's initialFocusRef docs by changing the prop heading to the DialogOverlay component (@​CodingDive)

🏡 Housekeeping

• #​588 Build fix for CodeSandbox CI: Don't use Array.flatMap (@​raunofreiberg)
• #​599 Fix broken examples on the website (@​raunofreiberg)
• #​585 Remove redundant :root selector in website styles (@​abhishekjakhar)

🤜🤛 4 Committers

• Abhishek Jakhar (@​abhishekjakhar)
• Chance Strickland (@​chancestrickland)
• Mikey Stengel (@​CodingDive)
• Rauno Freiberg (@​raunofreiberg)

</details>

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	References	Sourcegraph
@reach/listbox	dependencies	patch	^0.10.5	source

Release Notes

<details> <summary>reach/reach-ui</summary>

v0.10.5

Compare Source

🐛 Bugfixes

• alert, rect, utils
  • #​622 Fix issues with the rect observer between renders (@​chancestrickland)

✨ Enhancements

• dialog
  • Added dangerouslyBypassFocusLock prop for customizing focus lock behavior (@​gnapse)
• menu-button
  • #​619 Don’t try to use relatedTarget of the mousedown event (@​Andarist)
• utils
  • Improve as prop typing (@​chancestrickland)

🤜🤛 3 Committers

• Chance Strickland (@​chancestrickland)
• Ernesto García (@​gnapse)
• Mateusz Burzyński (@​Andarist)

v0.10.4

Compare Source

🐛 Bugfixes

• slider
  • #​583 fix(slider): prevent keyboard events when disabled (@​raunofreiberg)
• popover
  • #​609 [popover] Fix stuck focus (@​michaeldfoley)
  • [Combobox] Tab sequence gets stuck in popover interactive elements (@​indiesquidge)
• tabs
  • #​584 fix(tabs): wait for TabPanel to mount before setting hidden (@​raunofreiberg)

📝 Documentation

• #​597 Fix tabs.mdx animated example (@​homburg)

✨ Enhancements

• combobox
  • Ability to pass targetRef to ComboboxPopover just like a normal Popover (@​sanfilippopablo)

🤜🤛 6 Committers

• Austin Wood (@​indiesquidge)
• Chance Strickland (@​chancestrickland)
• Michael Foley (@​michaeldfoley)
• Pablo Sanfilippo (@​sanfilippopablo)
• Rauno Freiberg (@​raunofreiberg)
• Thomas B Homburg (@​homburg)

v0.10.3

Compare Source

🔨 Breaking Changes

• The only meaningful change to packages in this release is to the API for @reach/descendants. We do not document this module and don't generally recommend it for external use, but in case you're being bold, useDescendants has been renamed to useDescendantsInit. useDescendants is still used, but it will now access the actual descendant list from the appropriate context.

📝 Documentation

• #​581 Updated the README.md with new link of yarn install instructions (@​abhishekjakhar)
• #​587 Fix Dialog's initialFocusRef docs by changing the prop heading to the DialogOverlay component (@​CodingDive)

🏡 Housekeeping

• #​588 Build fix for CodeSandbox CI: Don't use Array.flatMap (@​raunofreiberg)
• #​599 Fix broken examples on the website (@​raunofreiberg)
• #​585 Remove redundant :root selector in website styles (@​abhishekjakhar)

🤜🤛 4 Committers

• Abhishek Jakhar (@​abhishekjakhar)
• Chance Strickland (@​chancestrickland)
• Mikey Stengel (@​CodingDive)
• Rauno Freiberg (@​raunofreiberg)

v0.10.2

Compare Source

🐛 Bugfixes

• listbox
  • #​570 Properly hide the hidden input (@​ddanger)
• portal
  • #​574 Fix TypeError when using hot-loader (@​jylertones)

✨ Enhancements

• tabs
  • #​572 Use type button by default in Tab component (@​rubenmoya)

🤜🤛 3 Committers

• David Dangerfield (@​ddanger)
• Rubén Moya (@​rubenmoya)
• Tyler Jones (@​jylertones)

</details>

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	References	Sourcegraph
@reach/accordion	dependencies	patch	^0.10.5	source

Release Notes

<details> <summary>reach/reach-ui</summary>

v0.10.5

Compare Source

🐛 Bugfixes

• alert, rect, utils
  • #​622 Fix issues with the rect observer between renders (@​chancestrickland)

✨ Enhancements

• dialog
  • Added dangerouslyBypassFocusLock prop for customizing focus lock behavior (@​gnapse)
• menu-button
  • #​619 Don’t try to use relatedTarget of the mousedown event (@​Andarist)
• utils
  • Improve as prop typing (@​chancestrickland)

🤜🤛 3 Committers

• Chance Strickland (@​chancestrickland)
• Ernesto García (@​gnapse)
• Mateusz Burzyński (@​Andarist)

v0.10.4

Compare Source

🐛 Bugfixes

• slider
  • #​583 fix(slider): prevent keyboard events when disabled (@​raunofreiberg)
• popover
  • #​609 [popover] Fix stuck focus (@​michaeldfoley)
  • [Combobox] Tab sequence gets stuck in popover interactive elements (@​indiesquidge)
• tabs
  • #​584 fix(tabs): wait for TabPanel to mount before setting hidden (@​raunofreiberg)

📝 Documentation

• #​597 Fix tabs.mdx animated example (@​homburg)

✨ Enhancements

• combobox
  • Ability to pass targetRef to ComboboxPopover just like a normal Popover (@​sanfilippopablo)

🤜🤛 6 Committers

• Austin Wood (@​indiesquidge)
• Chance Strickland (@​chancestrickland)
• Michael Foley (@​michaeldfoley)
• Pablo Sanfilippo (@​sanfilippopablo)
• Rauno Freiberg (@​raunofreiberg)
• Thomas B Homburg (@​homburg)

v0.10.3

Compare Source

🔨 Breaking Changes

• The only meaningful change to packages in this release is to the API for @reach/descendants. We do not document this module and don't generally recommend it for external use, but in case you're being bold, useDescendants has been renamed to useDescendantsInit. useDescendants is still used, but it will now access the actual descendant list from the appropriate context.

📝 Documentation

• #​581 Updated the README.md with new link of yarn install instructions (@​abhishekjakhar)
• #​587 Fix Dialog's initialFocusRef docs by changing the prop heading to the DialogOverlay component (@​CodingDive)

🏡 Housekeeping

• #​588 Build fix for CodeSandbox CI: Don't use Array.flatMap (@​raunofreiberg)
• #​599 Fix broken examples on the website (@​raunofreiberg)
• #​585 Remove redundant :root selector in website styles (@​abhishekjakhar)

🤜🤛 4 Committers

• Abhishek Jakhar (@​abhishekjakhar)
• Chance Strickland (@​chancestrickland)
• Mikey Stengel (@​CodingDive)
• Rauno Freiberg (@​raunofreiberg)

</details>

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	Sourcegraph
alpine	final	minor	3.12
alpine	stage	minor	3.12
alpine	final	minor	3.12.0

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:ghost: Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	References	Sourcegraph
babel-jest	devDependencies	major	^26.1.0	source
jest (source)	devDependencies	major	^26.1.0	homepage, source

Release Notes

<details> <summary>facebook/jest</summary>

v26.1.0

Compare Source

Features

• [jest-mock] Export Mock, MockInstance, SpyInstance types (#​10138)
• [jest-config] Support config files exporting (async) functions (#​10001)
• [jest-cli, jest-core] Add --selectProjects CLI argument to filter test suites by project name (#​8612)
• [jest-cli, jest-init] Add coverageProvider to jest --init prompts (#​10044)

Fixes

• [jest-console] getConsoleOutput to receive global stack trace config and use it to format stack trace (#​10081)
• [jest-jasmine2] Stop adding : after an error that has no message (#​9990)
• [jest-diff] Control no diff message color with commonColor in diff options (#​9997)
• [jest-snapshot] Fix TypeScript compilation (#​10008)

Chore & Maintenance

• [docs] Correct confusing filename in enableAutomock example (#​10055)
• [jest-core] 🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉 (#​10000)
• [jest-core, jest-reporters, jest-test-result, jest-types] Cleanup displayName type (#​10049)
• [jest-runtime] Jest-internal sandbox escape hatch (#​9907)
• [jest-fake-timers] Update now param type to support Date in addition to number. (#​10169)
• [docs] Add param to setSystemTime docs and remove preceding period from it and getRealSystemTime (#​10169)
• [jest-snapshot, jest-util] Replace make-dir with fs.mkdir (#​10136)
• [docs] Added parcel-bundler documentation inside readme.md file

Performance

• [jest-core, jest-transform, jest-haste-map] Improve Jest startup time and test runtime, particularly when running with coverage, by caching micromatch and avoiding recreating RegExp instances (#​10131)

v26.0.1

Compare Source

Fixes

• [jest-circus] Backward compatibility for deprecated DescribeBlock.tests to not break e.g. Detox reporter

v26.0.0

Compare Source

Features

• [jest-environment-jsdom] [BREAKING] Upgrade jsdom to v16 (#​9606)
• [@jest/fake-timers] Add possibility to use a modern implementation of fake timers, backed by @sinonjs/fake-timers (#​7776)
• [jest-runtime] Add createMockFromModule as an alias for genMockFromModule (#​9962)

Fixes

• [babel-jest] Handle null being passed to createTransformer (#​9955)
• [jest-circus, jest-console, jest-jasmine2, jest-reporters, jest-util, pretty-format] Fix time durating formatting and consolidate time formatting code (#​9765)
• [jest-circus] [BREAKING] Fail tests if a test takes a done callback and have return values (#​9129)
• [jest-circus] [BREAKING] Throw a proper error if a test / hook is defined asynchronously (#​8096)
• [jest-circus] Throw more descriptive error if hook is defined inside test (#​9957)
• [jest-circus] [BREAKING] Align execution order of tests to match jasmine's top to bottom order (#​9965)
• [jest-config, jest-resolve] [BREAKING] Remove support for browser field (#​9943)
• [jest-haste-map] Stop reporting files as changed when they are only accessed (#​7347)
• [jest-resolve] Show relative path from root dir for module not found errors (#​9963)
• [jest-runtime] Fix absolute path moduleNameMapper + jest.mock bug (#​8727)

Chore & Maintenance

• [*] [BREAKING] TypeScript definitions requires a minimum of TypeScript v3.8 (#​9823)
• [*] [BREAKING] Drop support for Node 8 (#​9423)
• [*] Upgrade to chalk@4 (#​9752)
• [*] Remove usage of realpath-native (#​9952)
• [docs] Fix example reference implementation to use Jest with Phabricator (#​8662)
• [docs] Added default compiler to tranform (#​8583)
• [docs] Updated Testing Frameworks guide with React; make it generic (#​9106)
• [expect, jest-mock, pretty-format] [BREAKING] Remove build-es5 from package (#​9945)
• [@jest/fake-timers, @​jest/environment] [BREAKING] Rename LolexFakeTimers to ModernFakeTimers (#​9960)
• [jest-haste-map] [BREAKING] removed providesModuleNodeModules (#​8535)
• [jest-runtime] [BREAKING] Remove long-deprecated require.requireActual and require.requireMock methods (#​9854)

</details>

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: Ignore: Close this PR and you won't be reminded about these updates again.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	Sourcegraph
golang	stage	digest	1.14-alpine

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	Sourcegraph
alpine	final	digest	3.11

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	Sourcegraph
prom/prometheus	stage	digest	v2.16.0

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	References	Sourcegraph
monaco-editor	dependencies	minor	^0.20.0	source
monaco-editor-webpack-plugin	devDependencies	minor	^1.9.0	source

Release Notes

<details> <summary>Microsoft/monaco-editor</summary>

v0.20.0

Compare Source

New & Noteworthy

• The editor can now be hosted inside a Shadow Root.
• There is new API to read the editor's content width and height.
• New editor options:
  • renderValidationDecorations - render validation decorations even in read only editors
  • wrappingStrategy - delegate wrapping points computation to the browser
  • comments.insertSpace - insert a space around comments when running commenting commands
  • foldingHighlight - highlight folded regions
  • peekWidgetDefaultFocus - focus the inline editor or the tree when opening peek view

Breaking changes

• Renamed onCompositionStart, onCompositionEnd to onDidCompositionStart, onDidCompositionEnd
• Changed the argument passed in to onDidPaste
• WorkspaceEdit.edits has now changed its shape such that edits must no longer be grouped by resource.
• The Monaco Editor no longer supports IE 11. The last version that was tested in IE 11 is 0.18.1.

Thank you

Contributions to monaco-editor:

• Josh Goldberg (@​JoshuaKGoldberg): Added section in Monarch docs for Inspect Tokens development helper PR #​1807

Contributions to monaco-typescript:

• Elizabeth Craig (@​ecraig12345): Add types for TypeScriptWorker and missing LanguageServiceDefaults methods PR #​54

Contributions to monaco-languages:

• alan.invents (@​ALANVF): Highlight function definitions better PR #​79
• @​nrayburn-tech: Add support for multiline comments in Swift PR #​80

v0.19.3

Compare Source

• brings back a way to get resolved editor options - #​1734

Thank you

Contributions to monaco-editor:

• Brijesh Bittu (@​brijeshb42): Playground: Add keyboard shortcut to run playground code PR #​1756

Contributions to monaco-languages:

• Rikki Schulte (@​acao): add tokenizer for graphql language variables PR #​78

v0.19.2

Compare Source

• fixes issue with default value of autoIndent - #​1726

v0.19.1

Compare Source

• fixes issue with .d.ts file in the ESM distribution - #​1729
• adds types for global editor options (such as wordBasedSuggestions) - #​1746
• adds support for reStructuredText.

Thank you

Contributions to monaco-editor:

• Lars Hvam (@​larshp)
  • Playground: add ABAP sample PR #​1737
  • Playground: fix codelens provider example PR #​1738

Contributions to monaco-languages:

• Changwon Choe (@​qwefgh90): add support for reStructuredText PR #​77

v0.19.0

Compare Source

New & Noteworthy

• It is now possible to pass in a dimension in the editor construction options in order to avoid a synchronous layout.
• There is new API to provide semantic tokens.
• New options:
  • multiCursorPaste: define how to distribute paste in case of multi-cursor
  • matchBrackets: control if enclosing brackets should be highlighted
• Fixes for tokenization in: TypeScript, JavaScript, Handlebars, Kotlin and VB.

Breaking changes

• getConfiguration() is replaced by getRawOptions(), which returns the passed in editor options.
• Starting with this version, the Monaco Editor no longer supports IE 11. The last version that was tested in IE 11 is 0.18.1.

Thank you

Contributions to monaco-editor:

• Lars Hvam (@​larshp)
  • contributing: add details for running website locally PR #​1617
  • playground: update symbols-provider-example PR #​1616
• Remy Suen (@​rcjsuen): Add CompletionItem with snippet support to the example PR #​1703

Contributions to monaco-editor-webpack-plugin:

• Dominik Moritz (@​domoritz): Bump to 0.16 PR #​62
• Mike Greiling (@​mikegreiling): Fix webpack_public_path within getWorkerUrl method PR #​63
• Roman Krasiuk (@​rkrasiuk): Bump to 0.17.0 and Add graphql support PR #​67
• Niklas Mollenhauer (@​nikeee): Add loader-utils and make @​types/webpack a dev dependency PR #​74
• James Diefenderfer (@​jimmydief)
  • Add support for plugin-specific public path PR #​81
  • Add support for dynamic filenames PR #​83

Contributions to monaco-languages:

• Maksym Bykovskyy (@​mbykovskyy): Adds cameligo language support PR #​75
• Steven Degutis (@​sdegutis): Adds Markdown Table syntax highlighting PR #​73
• Sergey Romanov (@​Serhioromano): Improvements to ST language PR #​76
• Sebastian Pahnke (@​spahnke): [JS/TS] Add support for the nullish-coalesce operator PR #​74

Contributions to monaco-typescript:

• Denys Vuika (@​DenysVuika): register multiple extra libs at once PR #​24
• Elizabeth Craig (@​ecraig12345)
  • Generate and publish typings for package PR #​50
  • Remove another require call PR #​49
• @​katis: Update TypeScript to 3.7.2 PR #​51
• Tamas Kiss (@​kisstkondoros): Add documentation to signature help PR #​52
• Lars Hvam (@​larshp): fix typo PR #​45
• Sebastian Pahnke (@​spahnke)
  • Provide related information to diagnostics PR #​48
• Alessandro Fragnani (@​alefragnani): Add Pascal samples PR #​1358
  • Adopt monaco.MarkerTag API PR #​47
  • Add support to ignore certain diagnostics PR #​46

</details>

<details> <summary>Microsoft/monaco-editor-webpack-plugin</summary>

v1.9.0

Compare Source

</details>

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:ghost: Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	References	Sourcegraph
jsdom	devDependencies	major	^16.2.2	source
@types/jsdom	devDependencies	major	16.2.0	source

Release Notes

<details> <summary>jsdom/jsdom</summary>

v16.2.2

Compare Source

• Updated StyleSheetList for better spec compliance; notably it no longer inherits from Array.prototype. (ExE-Boss)
• Fixed requestAnimationFrame() from preventing process exit. This likely regressed in v16.1.0.
• Fixed setTimeout() to no longer leak the closures passed in to it. This likely regressed in v16.1.0. (AviVahl)
• Fixed infinite recursion that could occur when calling click() on a <label> element, or one of its descendants.
• Fixed getComputedStyle() to consider inline style="" attributes. (eps1lon)
• Fixed several issues with <input type="number">'s stepUp() and stepDown() functions to be properly decimal-based, instead of floating point-based.
• Fixed various issues where updating selectEl.value would not invalidate properties such as selectEl.selectedOptions. (ExE-Boss)
• Fixed <input>'s src property, and <ins>/<del>'s cite property, to properly reflect as URLs.
• Fixed window.addEventLister, window.removeEventListener, and window.dispatchEvent to properly be inherited from EventTarget, instead of being distinct functions. (ExE-Boss)
• Fixed errors that would occur if attempting to use a DOM object, such as a custom element, as an argument to addEventListener.
• Fixed errors that would occur when closing a window with outstanding requests to data: URLs.
• Fixed sporadic issues with the value of <input type="month"> that could occur in some time zones and for some times.
• Fixed document.implementation.createDocument() to return an XMLDocument, instead of a Document. (ExE-Boss)
• Fixed running jsdom in a browser to detect globals more reliably. (ExE-Boss)

v16.2.1

Compare Source

• Updated saxes, to bring in some BOM-related fixes.
• Updated Acorn-related packages to squelch npm audit warnings.

v16.2.0

Compare Source

• Added support for custom elements! Congratulations and thanks to @​pmdartus for making this happen, after ten months of hard work and lots of effort poured into the complex architectural prerequisites in jsdom and supporting packages.
• Fixed some issues when trying to use Attr as a Node, e.g. by checking its baseURI property or calling attr.cloneNode().
• Fixed a memory leak during parsing that was introduced in v14.0.0.
• Fixed edge cases in number/string conversion used for certain element properties that reflected integer attributes.

v16.1.0

Compare Source

• Added console.timeLog().
• Changed Attr to extend Node, to align with specifications. (ExE-Boss)
• Changed <noscript> children to be parsed as nodes, instead of as text, when runScripts is left as the default of undefined. (ACHP)
• Upgraded cssstyle to v2.1.0, which brings along fixes to handling of rgba() and hsl() colors. (kraynel)
• Fixed some selection-related issues when manipulating the value of <input>s and <textarea>s. (Matthew-Goldberg)
• Fixed various issues with setTimeout(), setInterval(), and requestAnimationFrame(), particularly around window closing and recursive calls.

v16.0.1

Compare Source

• Fixed Node v10 and v11 support when runScripts was set.
• Fixed the behavior when changing an <input>'s type="" attribute.
• Fixed input validation behavior for <input type="range"> when max="" is less than min="".

v16.0.0

Compare Source

For this release we'd like to welcome @​pmdartus to the core team. Among other work, he's driven the heroic effort of constructor prototype and reform in jsdom and its dependencies over the last few months, to allow us to move away from shared constructors and prototypes, and set the groundwork for custom elements support (coming soon!).

Breaking changes:

• Node v10 is now the minimum supported version.
• The dom.runVMScript() API has been replaced with the more general dom.getInternalVMContext() API.
• Each jsdom Window now creates new instances of all the web platform globals. That is, our old shared constructor and prototypes caveat is no longer in play.
• Each jsdom Window now exposes all JavaScript-spec-defined globals uniformly. When runScripts is disabled, it exposes them as aliases of the ones from the outer Node.js environment. Whereas when runScripts is enabled, it exposes fresh copies of each global from the new scripting environment. (Previously, a few typed array classes would always be aliased, and with runScripts disabled, the other classes would not be exposed at all.)

Other changes:

• Added the AbstractRange, Range, StaticRange, Selection, and window.getSelection() APIs.
• Added working constructors for Comment, Text, and DocumentFragment.
• Added valueAsDate, valueAsNumber, stepUp() and stepDown() to <input> elements. (kraynel)
• Added window.origin.
• Removed document.origin.
• Fixed <template> to work correctly inside XML documents.
• Fixed some bugs which would cause jsdom to choose the wrong character encoding because it was failing to detect <meta charset> or <meta http-equiv="charset"> elements.
• Fixed input.type to default to "text". (connormeredith)
• Fixed incorrect validation errors for <input> with fractional values for their step="" attribute. (kontomondo)
• Fixed incorrect validation errors on readonly <input> elements.
• Fixed <input type="email" multiple pattern="..."> validation.
• Fixed fileReader.readAsDataURL() to always base64-encode the result. (ytetsuro)
• Fixed inserting <img> elements into documents without a browsing context to no longer crash when the canvas package is installed.
• Fixed a memory leak when using window.setTimeout() or window.setInterval().
• Improved the performance of getComputedStyle(). (eps1lon)

</details>

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: Ignore: Close this PR and you won't be reminded about these updates again.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	References	Sourcegraph
@octokit/rest	devDependencies	minor	^16.43.1	source

Release Notes

<details> <summary>octokit/rest.js</summary>

v16.43.1

Compare Source

Bug Fixes

• TypeScript: named { Octokit } export (486eadd)

v16.43.0

Compare Source

Features

• deprecate octokit.gitdata.*, octokit.pullRequests.* (#​1585) (989afc1)

v16.42.2

Compare Source

Bug Fixes

• recover Octokit.plugin (7b316d8)

v16.42.1

Compare Source

Bug Fixes

• do not throw if using deprecated Octokit default export without options (dec7334)

v16.42.0

Compare Source

Features

• deprecate const Octokit = require("@​octokit/rest") in favor of const { Octokit } = require("@​octokit/rest") (bbc3e48)

v16.41.2

Compare Source

Bug Fixes

• package: update @​octokit/plugin-rest-endpoint-methods to version 2.3.0 (a10ef70), closes #​1571

v16.41.1

Compare Source

Bug Fixes

• package: update @​octokit/plugin-rest-endpoint-methods to version 2.2.0 (3539118)
• Typescript: since parameter type is integer, not string (6285b6c)
• TypeScript: file parameter is no data for .repos.uploadReleaseAsset() (dc2cc51)

v16.41.0

Compare Source

Features

• typescript: types for latest Action endpoint methods (#​1567) (0758d61)

v16.40.2

Compare Source

Bug Fixes

• typescript: new auth strategy options (0c7b109)

v16.40.1

Compare Source

Bug Fixes

• typescript: descripion & response types fixes (#​1561) (7c64d9c)

v16.40.0

Compare Source

Features

• Actions endpoint methdos (a706b0d)

v16.39.0

Compare Source

Features

• accessing response.data[namespacekey] in .paginate(options, mapFn) map function now logs a deprecation (3caa24a)

v16.38.3

Compare Source

Bug Fixes

• typescript: .git.getTree types (#​1556) (39cdce8)

v16.38.2

Compare Source

Bug Fixes

• everest preview graduated (441c452)
• add .endpoint() method to all deprecated endpoint methods (9701151)
• bring back octokit.teams.removeMember() (d439ca7)
• bring back `octokit.teams.removeMembership() (dadfe9a)
• correct deprecation message for octokit.apps.* OAuth authorization methods' (5f5e62e)

v16.38.1

Compare Source

Bug Fixes

• typescript: authStrategy key in Octokit constructor options (fb19f11)

v16.38.0

Compare Source

Features

• All authentication strategies from @octokit/auth are now supported. Setting auth to an object or a basic authentication string is now deprecated and will be removed in v17 (4573ee2)

v16.37.0

Compare Source

Features

• .apps.revokeInstallationToken() (ed31137)
• .migrations.listReposForOrg(), .migrations.listReposForUser() (8777073)
• .repos.listForOrg()'s type parameter can now be set to "internal" (GitHub Enterprise Cloud only) (9c71d18)
• visibility parameter for .repos.createForAuthenticatedUser() and .repos.createInOrg() (31b094b)
• new delete_branch_on_merge parameter for .repos.createForAuthenticatedUser(), .repos.createInOrg(), .repos.update() (c783249)
• new parameters for .orgs.update(): members_can_create_internal_repositories, members_can_create_private_repositories, members_can_create_public_repositories (7be5174)
• new parameters for .repos.updateBranchProtection(): allow_deletions, allow_force_pushes, required_linear_history (c5632ff)
• rename & deprecation of endpoint methods using /teams/:team_id* path internally. The are being replaced by *ForOrg methods that use /orgs/:org/teams/:team_slug* internally. With the next breaking version, the *ForOrg suffix will be removed using another deprecation. (faecc99)

Bug Fixes

• graduated previews removed: barred-rock, echo, hagar (53b8021)

</details>

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	Sourcegraph
alpine	final	digest	3.10
alpine	stage	digest	3.10

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: Ignore: Close this PR and you won't be reminded about these updates again.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

This PR contains the following updates:

Package	Type	Update	New value	Sourcegraph
wrouesnel/postgres_exporter	final	minor	v0.8.0

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or if you tick the rebase/retry checkbox below.

:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

• [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

This PR has been generated by WhiteSource Renovate. View repository job log here.

We are going to do this for reasons mentioned in this thread: https://twitter.com/mislav/status/1270388510684598272

More context in https://github.com/sourcegraph/sourcegraph/issues/11389#issuecomment-642877899

This PR contains the following updates:

Package	Type	Update	New value	References	Sourcegraph
yarn	engines	minor	^1.21.0	source

Release Notes

<details> <summary>yarnpkg/yarn</summary>

v1.21.0

Compare Source

v1.19.2

Compare Source

• Folders like .cache won't be pruned from the node_modules after each install.

  #​7699 - Maël Nison

• Correctly installs workspace child dependencies when workspace child not symlinked to root.

  #​7289 - Daniel Tschinder

• Makes running scripts with Plug'n Play possible on node 13.

  #​7650 - Sander Verweij

• Change run command to check cwd/node_modules/.bin for commands. Fixes run in workspaces.

  #​7151 - Jeff Valore

v1.19.1

Compare Source

Important: This release contains a cache bump. It will cause the very first install following the upgrade to take slightly more time, especially if you don't use the Offline Mirror feature. After that everything will be back to normal.

• Computes the --modules-folder & friends paths based on the cwd.

  #​7607 - mbpreble

• Stores the sha512 in the cache even when not provided by the server.

  #​7591 - Maël Nison / #​7595 - Michael

• Uses the right Node binary when using yarn-path.

  #​7592 - Maël Nison

v1.19.0

Compare Source

Important: This release contains a cache bump. It will cause the very first install following the upgrade to take slightly more time, especially if you don't use the Offline Mirror feature. After that everything will be back to normal.

• Fixes a potential vulnerability regarding how the build artifacts are stored

  Reported by ChALkeR, fixed by Maël Nison

v1.18.0

Compare Source

• Suggests using the Yarn 2 development trunk on PnP-enabled projects

  #​7512 - Maël Nison

• Preserves linked packages when calling yarn create

  #​7543 - Nick McCurdy

• Fixes the offline mirror filenames when using Verdaccio

  #​7499 - xv2

• Fixes using link:. to refer to the package folder

  #​7512 - Maël Nison

• Runs the prepare lifecycle of git dependencies even if NODE_ENV is set to production.

  #​7398 - John Firebaugh

• Fixes the postversion lifecycle method not being called when using --no-git-tag-version.

  #​7154 - Hampus Tågerud

• Ignores potentially large vscode keys in package.json to avoid E2BIG errors.

  #​7419 - Eric Amodio

• Enforces https for the Yarn and npm registries.

  #​7393 - Maël Nison

• Adds support for reading yarnPath from v2-produced .yarnrc.yml files.

  #​7350 - Maël Nison

v1.17.3

Compare Source

v1.17.2

Compare Source

v1.17.1

Compare Source

v1.17.0

Compare Source

• Adds prereleases flags and prerelease identifier to yarn version.

  #​7336 - Daniel Seijo

• Fixes audits when used with yarn add & yarn upgrade

  #​7326 - David Sanders

• Adds support for the --offline flag to yarn global add

  #​7330 - Francis Crick

• Yarn will tolerate Yaml at parse time. Full support isn't ready yet and will only come at the next major.

  #​7300 - Maël Nison

• Fixes a bug when using the link: protocol with a folder that doesn't contain a package.json

  #​7337 - Maël Nison

v1.16.0

Compare Source

• Retries downloading a package on yarn install when we get a ETIMEDOUT error.

  #​7163 - Vincent Bailly

• Implements yarn audit --level [severity] flag to filter the audit command's output.

  #​6716 - Rogério Vicente

• Implements yarn audit --groups group_name [group_name ...].

  #​6724 - Tom Milligan

• Exposes the script environment variables to yarn create spawned processes.

  #​7127 - Eli Perelman

• Prevents EPIPE errors from being printed.

  #​7194 - Abhishek Reddy

• Adds support for the npm enterprise URLs when computing the offline mirror filenames.

  #​7200 - John Millikin

• Tweaks the lockfile parser logic to parse a few extra cases

  #​7210 - Maël Nison

v1.15.2

Compare Source

The 1.15.1 doesn't exist due to a release hiccup.

• Reverts a behavior causing boggus interactions between PowerShell and yarn global

  #​6954 - briman0094

• Fixes a bug where non-zero exit codes were converted to a generic 1 when running yarn run

  #​6926 - Kyle Fang

• Fixes production / development reporting when running yarn audit

  #​6970 - Adam Richardson

v1.15.1

Compare Source

v1.15.0

Compare Source

• Removes --scripts-prepend-node-path as Yarn's default behavior makes this obsolete

  #​7057 - Jason Grout

• Fixes the advisory link printed by yarn audit

  #​7091 - Jakob Krigovsky

• Fixes npm_config_ environment variable parsing to support those prefixed with underscore (ex: _auth)

  #​7070 - Nicholas Boll

• Fixes yarn upgrade --latest for dependencies using > or >= range specifier

  #​7080 - Xukai Wu

• Fixes --modules-folder handling in several places (ex: yarn check now respects --modules-folder)

  #​6850 - Jeff Valore

• Removes rootModuleFolders (internal variable which wasn't used anywhere)

  #​6846 - Jeff Valore

• Adds support for setting global-folder from .yarnrc files

  #​7056 - Hsiao-nan Cheung

• Makes yarn version cancellable via ctrl-c or empty string

  #​7064 - Olle Lauri Boström

• Adds support for yarn policies set-version berry

  #​7041 - Maël Nison

• Fixes yarn upgrade --scope when using exotic (github) dependencies

  #​7017 - Jeff Valore

• Fixes occasionally mismatching upper/lowecases of drive letters in win32 pnp check

  #​7007 - Christoph Werner

• Fixes the error reporting for non-HTTP network errors (such as invalid certificates)

  #​6968 - Chih-Hsuan Yen

• Changes the location where the --require ./.pnp.js flag gets added into NODE_OPTIONS: now at the front (bis)

  #​6951 - John-David Dalton

• Packages won't be auto-unplugged anymore if ignore-scripts is set in the yarnrc file

  #​6983 - Micha Reiser

• Enables displaying Emojis on Terminus by default

  #​7093 - David Refoua

• Run the engines check before executing run scripts.

  #​7013 - Eloy Durán

v1.14.0

Compare Source

• Improves PnP compatibility with Node 6

  #​6871 - Robert Jackson

• Fixes PnP detection with workspaces (installConfig is now read at the top-level)

  #​6878 - Maël Nison

• Fixes an interaction between yarn pack and bundled dependencies

  #​6908 - Travis Hoover

• Adds support for GITHUB_TOKEN in yarn policies set-version

  #​6912 - Billy Vong

• Fixes an issue where resolve would forward an incomplete basedir to the PnP hook

  #​6882 - Zoran Regvart

• Fixes the command that yarn unlink recommends to run as a followup (now yarn install --force)

  #​6931 - Justin Sacbibit

• Changes the location where the --require ./.pnp.js flag gets added into NODE_OPTIONS: now at the front

  #​6942 - John-David Dalton

• Fixes a bug where os and platform requirements weren't properly checked when engines was missing

  #​6976 - Micha Reiser

v1.13.0

Compare Source

• Implements a new package.json field: peerDependenciesMeta

  #​6671 - Maël Nison

• Adds an optional settings to peerDependenciesMeta to silence missing peer dependency warnings

  #​6671 - Maël Nison

• Implements yarn policies set-version [range]. Check the documentation for usage & tips.

  #​6673 - Maël Nison

• Fixes a resolution issue when a package had an invalid main entry

  #​6682 - Maël Nison

• Decreases the size of the generated $PATH environment variable for a better Windows support

  #​6683 - Rowan Lonsdale

• Fixes postinstall scripts for third-party packages when they were referencing a binary from their own dependencies

  #​6712 - Maël Nison

• Fixes yarn audit exit code overflow

  #​6748 - Andrey Vetlugin

• Stops automatically unplugging packages with postinstall script when running under --ignore-scripts

  #​6820 - Maël Nison

• Adds transparent support for the resolve package when using Plug'n'Play

  #​6816 - Maël Nison

• Properly reports the error codes when the npm registry throws 500's

  #​6817 - Maël Nison

v1.12.3

Compare Source

Important: This release contains a cache bump. It will cause the very first install following the upgrade to take slightly more time, especially if you don't use the Offline Mirror feature. After that everything will be back to normal.

• Fixes an issue with yarn audit when using workspaces

  #​6625 - Jeff Valore

• Uses NODE_OPTIONS to instruct Node to load the PnP hook, instead of raw CLI arguments

  Caveat: This change might cause issues for PnP users having a space inside their cwd (cf nodejs/node#​24065)

  #​6479 - Maël Nison

• Fixes Gulp when used with Plug'n'Play

  #​6623 - Maël Nison

• Fixes an issue with yarn audit when the root package was missing a name

  #​6611 - Jack Zhao

• Fixes an issue with yarn audit when a package was depending on an empty range

  #​6611 - Jack Zhao

• Fixes an issue with how symlinks are setup into the cache on Windows

  #​6621 - Yoad Snapir

• Upgrades inquirer, fixing upgrade-interactive for users using both Node 10 and Windows

  #​6635 - Philipp Feigl

• Exposes the path to the PnP file using require.resolve('pnpapi')

  #​6643 - Maël Nison

v1.12.1

Compare Source

• Ensures the engine check is ran before showing the UI for upgrade-interactive

  #​6536 - Orta Therox

• Restores Node v4 support by downgrading cli-table3

  #​6535 - Mark Stacey

• Prevents infinite loop when parsing corrupted lockfiles with unterminated strings

  #​4965 - Ryan Hendrickson

• Environment variables now have to start with YARN_ (instead of just contain it) to be considered

  #​6518 - Michael Gmelin

• Fixes the extensions option when used by resolveRequest

  #​6479 - Maël Nison

• Fixes handling of empty string entries for bin in package.json

  #​6515 - Ryan Burrows

• Adds support for basic auth for registries with paths, such as artifactory

  #​5322 - Karolis Narkevicius

• Adds 2FA (Two Factor Authentication) support to publish & alike

  #​6555 - Krzysztof Zbudniewek

• Fixes how the files property is interpreted to bring it in line with npm

  #​6562 - Bertrand Marron

• Fixes Yarn invocations on Darwin when the yarn binary was symlinked

  #​6568 - Hidde Boomsma

• Fixes require.resolve when used together with the paths option

  #​6565 - Maël Nison

v1.12.0

Compare Source

• Adds initial support for PnP on Windows

  #​6447 - John-David Dalton

• Adds yarn audit (and the --audit flag for all installs)

  #​6409 - Jeff Valore

• Adds a special logic to PnP for ESLint compatibility (temporary, until eslint/eslint#​10125 is fixed)

  #​6449 - Maël Nison

• Makes the PnP hook inject a process.versions.pnp variable when setup (equals to VERSIONS.std)

  #​6464 - Maël Nison

• Disables by default (configurable) the automatic migration of the integrity field. It will be re-enabled in 2.0.

  #​6465 - Maël Nison

• Fixes the display name of the faulty package when the NPM registry returns corrupted data

  #​6455 - Grey Baker

• Prevents crashes when running yarn outdated and the NPM registry forgets to return the latest tag

  #​6454 - mad-mike

• Fixes yarn run when used together with workspaces and PnP

  #​6444 - Maël Nison

• Fixes an edge case when peer dependencies were resolved multiple levels deep (webpack-dev-server)

  #​6443 - Maël Nison

v1.11.1

Compare Source

v1.11.0

Compare Source

</details>

Renovate configuration

:date: Schedule: "on the 1st through 7th day of the month" in timezone America/Los_Angeles.

:vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

:recycle: Rebasing: Whenever PR becomes conflicted, or if you modify the PR title to begin with "rebase!".

:no_bell: Ignore: Close this PR and you won't be reminded about this update again.

• [x] <!-- renovate-rebase -->If you want to rebase/retry this PR, check this box

Newsflash: Renovate has joined WhiteSource, and is now free for all use. Learn more or view updated terms and privacy policies.

Reported by https://app.hubspot.com/contacts/2762526/company/768958891

See Slack thread at https://sourcegraph.slack.com/archives/CTJCMDCCS/p1595352999461200?thread_ts=1595350296.457900&cid=CTJCMDCCS

2020/07/21 17:35:51 error indexing <$REPO>@HEAD=<$SHA_BRANCH_LIST>: command [git -C /data/index/.indexserver.tmp/tmpfriend-6-554088670/<$REPO.git> -c protocol.version=2 fetch --depth=1 http://sourcegraph-frontend-internal/.internal/git/<$REPO> <$SHA LIST>] failed: exit status 128
OUT:
ERR: warning: redirecting to http://gitserver-0.gitserver:3178/git/<$REPO>/
error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
fatal: the remote end hung up unexpectedly

When the user reduced the number of branches from double digits to just 1, it worked just fine. He then increased up to 5, and got the same error.

• Kubernetes
• Fetching code from Gerritt

Haven't really heard back from the customer, nor have I heard this happen to others. I'm going to close this out. Feel free to re-open if we need to investigate more.

FYI this was the script I ran to update all our open PRs to merge into main. It was modified from the action to run locally + use netrc for auth.

#!/bin/sh -l

PREVIOUS_DEFAULT="master"
NEW_DEFAULT="main"
GITHUB_API_URL="https://api.github.com"
GITHUB_REPOSITORY="sourcegraph/sourcegraph"

echo "Migrating from '$PREVIOUS_DEFAULT' to '$NEW_DEFAULT'"
echo "Repo Name: $GITHUB_REPOSITORY"

# Check the API to see what the current default branch is
CURRENT_DEFAULT=$(curl -n --silent $GITHUB_API_URL/repos/$GITHUB_REPOSITORY | jq -r .default_branch)
echo "Default branch: $CURRENT_DEFAULT"

if [ "$CURRENT_DEFAULT" = "$NEW_DEFAULT" ]; then
	echo "Default has changed to $NEW_DEFAULT"
	# Check all existing PRs to see if we should change their base
	PRS=$(curl -n --silent "$GITHUB_API_URL/repos/$GITHUB_REPOSITORY/pulls?base=$PREVIOUS_DEFAULT&state=open")

	# TODO: PATCH /repos/:owner/:repo/pulls/:pull_number
	for row in $(echo $PRS | jq -r 'map(select(.locked == false)) | .[].url'); do
		echo "Attempting to update $row"
		curl -n --silent --show-error -w "Status Code: %{http_code}\n" --request PATCH --data '{"base": "'$NEW_DEFAULT'"}' "$row"
	done

	exit
fi;

This mostly updates our dev CI/helpers which hardcoded master to instead hardcode main. In the case of CI we treat both main and master as the default branch.

Part of https://github.com/sourcegraph/sourcegraph/issues/11464

Because it sounded like your previous suggestion was to exclude these before logging? Saves piping the noise over the wire. I plan to delete this tomorrow, if you're worried about overhead?

Oops, my bad :P Yeah whatever, its all just logs. I've been finding it useful to look at every search query. Really helps drive home how many more search queries we do for symbols (thanks to code intel) vs interactive input from users.

I think the sort of magic which responds to the results makes things quite complicated to understand. Displaying different results differently makes a lot of sense to me and can be done purely in the code that collects results so that sounds much more reliable.

Another suggestion, which I think I have seen done before. If your glob doesn't contain a / or ** there is an implicit transormation to **G** where G was the original glob. If we have a way to reference root files (/ prefix?) this allows all the same power with the same convenience.

We don't seem to be getting searcher traces. But I would suspect the issue isn't the speed of "grep", but that we are struggling to fetch an archive of ghdump. Our searcher code doesn't do per line searching, but uses the same technique in rg of searching the whole body and hydrating in line breaks.

I'm guessing this is a bit tricky around consistency and convenience. ie I like that I can just do -file:_test.go. But I suppose it needs to be **_test.go?

@rvantonder mentioned a similar example. Right now, we try to be very predictable and consistent. IE -file:_test.go will exclude the file _test.go in the project's root folder, while -file:**_test.go will exclude all test files. Rijnard already wrote down some ideas on how to help users in #12467. I wonder whether it just takes a couple of searches to get used to or whether it is really inconvenient and an entry barrier.

Agreed, if we guide the user that makes a lot of sense. I agree that once you learn to include ** it becomes very natural and much more exact.

I ended up working late last night so got a late start today. I'm gonna do this tomorrow morning eu-timezones rather.

Last Week

Main focus was search indexing and performance at scale. I first focussed on a large customer, but then then morphed into search performance for sourcegraph.com. Ended up looking into it after hack sessions with Robert, Tomas, Stefan and Ryan. Biggest win was custom marshalling of the list of repo/branches to search we sent to Zoekt. Quoting my posts on #progress:

Our large scale instance can now index every repo. This makes it possible to do an exhaustive search (eg find accidental commits of passwords) which was a signing hurdle. As an extra bonus the page which tells you which repos need indexing now works (a long time complaint at scale).

Searching all indexed repos on Sourcegraph.com is now ~0.4s faster (about 25% faster overall). When doing an indexed search we transfer a list of repos to search to Zoekt (our indexed searcher). This is a cost we pay every search no matter how many results it finds. We optimized how we create the list and it now takes about 75ms to transfer 100k repos instead of 500ms on sourcegraph.com. More details here sourcegraph#12546. For the most eye melting benchmark results look here zoekt#53.

Shaved off another 350ms with Ryan. Searching on sourcegraph.com across 100k repos now takes less than half a second on average.

See sourcegraph#12557 for how we tackled that. We are nearly at the point where search time is dominated by actual source code searching instead of our infrastructure around zoekt. Stefan and I explored the final slowdown now and will likely try tackle it next week.

A lot of this work also ended required observability to find what to do. So there have been lots of improvements made to logs in zoekt and jaeger tracing around zoekt in the frontend.

Ended up not journaling daily last week, regrets. But was more transparent in #progress so could use that to track high level work.

I ended up not doing my final work for src-expose. Will spend a day on it this week.

This Week

First half of the week is smaller tasks I've been meaning to get to + src-expose. Thursday I'll start work on streaming search. My focus will be purely experimental. I've been meaning to create an alternative graphql/JS free search UI. I'll likely create a dumb UI for this and use it as a playground for streaming work. So that will be my main goal this week.

Explicit non-goal: Over the weekend we doubled the number of indexed repos on Sourcegraph.com (100k -> 200k) leading to performance + OOM issues. I'm going to let other look into that, and instead focus on other work.

Next Week

Monday is a public holiday, so 4 days. Will hopefully have a very fast streaming search PoC that we can than look into integrating into the main product. Depending on where the code takes me I imagine two paths: streaming for searcher or integrating streaming into main search path.

Looked at the places we used context.Background in graphqlbackend. Pretty much all are invalid.

merging this now since a commit I landed earlier has an occasional race condition fixed in this PR.

Looked at the places we used context.Background in graphqlbackend. Pretty much all are invalid.

Seeing the output of the rest repository listing API may also help us. We don't list specific repositories, but rather list via the repositories endpoint. The customer seems to be using repositoryQuery: public, so this should be somewhat equivalent to what we actually do:

curl \
  -H "Authorization: token token3" \
  -H "Accept: application/vnd.github.jean-grey-preview+json,application/vnd.github.mercy-preview+json" \
  -H "Accept: application/vnd.github.machine-man-preview+json" \
  'https://github.example.com/api/v3/repositories?per_page=100' | jq .

Added the extra headers since we include them: https://github.com/sourcegraph/sourcegraph/blob/0a19e43b9ea405f2ba4aa54ce7aaf6a39a121bf4/internal/extsvc/github/client.go#L257

I took a quick look at this, I'm not sure why it isn't working. Most likely reason is to me is still syncing isn't running for some reason, but it could be something else. @unknwon do you have bandwidth to dive deeper?

In the thread the updated_at time is 2020-06-08 18:01:39.318373+00 which indicates this repo hasn't changed in a while. But we don't update unless we think it has changed. I investigated the following and came up short:

1. We correctly unmarshal rest archived. I tested with a mini-program using the structs we unmarshal with against the user supplied example response.
2. We detect archived changing in the syncer. https://github.com/sourcegraph/sourcegraph/blob/0a19e43b9ea405f2ba4aa54ce7aaf6a39a121bf4/cmd/repo-updater/repos/types.go#L627
3. We handle fork the same way we handle archived in store.go

I wish we had a simple way to tell the last time a full sync ran, I don't think we log that. This is a somewhat hacky way to find when we last updated a repo when it wasn't an insert. (we special case inserting so syncing could still be broken even though new repos are added):

select max(updated_at) from repo where updated_at = created_at;

I am guessing for the archived repo including all sql columns may help debugging.

Mini-program to test unmarshalling:

<details>

package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
	"log"
)

type restRepositoryPermissions struct {
	Admin bool `json:"admin"`
	Push  bool `json:"push"`
	Pull  bool `json:"pull"`
}

type restRepository struct {
	ID          string `json:"node_id"` // GraphQL ID
	DatabaseID  int64  `json:"id"`
	FullName    string `json:"full_name"` // same as nameWithOwner
	Description string
	HTMLURL     string `json:"html_url"` // web URL
	Private     bool
	Fork        bool
	Archived    bool
	Permissions restRepositoryPermissions `json:"permissions"`
}

func main() {
	b, err := ioutil.ReadFile("/tmp/foo.json")
	if err != nil {
		log.Fatal(err)
	}
	var r restRepository
	if err := json.Unmarshal(b, &r); err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%#+v\n", r)
	fmt.Println(r.Archived)
}

</details>

Structural search expects to only search HEAD (currently). However, it was using a RepoSet which will return results for any branch we have indexed. I believe this would result in potential benign false positives for files to search. We switch to instead passing in the computed RepoBranches (which should be limited to HEAD).

Part of https://github.com/sourcegraph/sourcegraph/issues/11902

I noticed we sometimes didn't get zoekt spans in Jaeger traces. Turns out it's due to the "no context deadline" logic we use when count or timeout fields are specified. This is because we use a new context. We now set the trace on the new context.

I noticed we sometimes didn't get zoekt spans in Jaeger traces. Turns out it's due to the "no context deadline" logic we use when count or timeout fields are specified. This is because we use a new context. We now set the trace on the new context.

Structural search expects to only search HEAD (currently). However, it was using a RepoSet which will return results for any branch we have indexed. I believe this would result in potential benign false positives for files to search. We switch to instead passing in the computed RepoBranches (which should be limited to HEAD).

Part of https://github.com/sourcegraph/sourcegraph/issues/11902

Yes, I'll update the plan above to limit pushes to master to just me :) Any other admin can add themselves to that list if needs be.

Plan for tomorrow morning:

1. Merge this.
2. Push latest master to main.
3. Update GitHub settings to make main the default.
4. Mark main branch as protected.
5. Run this script to update open PRs to point to main.
6. Announce in #dev-announce: Action Required: Our default branch has been renamed from master to main. Please update your local clones by running git fetch && git checkout origin/main && ./dev/2020-08-update-default-branch-to-main.sh. Report any issues you find in #dev-chat.
7. I'll look for things that may have broke.
8. In a day or two I'll look for any accidental commits to master and rebase and push to main.

There is no step 9. delete master. I think I'll leave it there since it doesn't hurt to keep around for old links.

This is what we did for lightstep and it is quite convenient. We couldn't do this when we originally added Jaeger since we didn't proxy it. However, now that we do we can provide a direct link.

Several things that I came across while trying to change this code. Includes minor fixes and refactorings to make another PR cleaner.