Hugo plugin for the asdf version manager
issue openedsourcegraph/sourcegraph
deploy-sourcegraph-dhall: incrementally migrate sourcegraph.com services over to dhall
In order to gather more feedback from @sourcegraph/distribution about the following aspects of deploy-sourcegraph-dhall:
-
what it's like to make larger changes to deploy-sourcegraph-dhall's components (adding new templating logic for more configuration options, adding new fields to the global configuration type)
-
what the pure-yaml -> dhall migration experience is like
I propose the following:
A member of @sourcegraph/distribution (other than @ggilmore or @uwedeportivo) will migrate the following subset of sourcegraph.com services to deploy-sourcegraph-dhall:
- Frontend (still thinking about this one - this is the riskiest since it sees the most configuration changes/commits by far)
- Redis
- Prometheus
In the end state:
- The above components will have their folders / K8s YAML files removed from 'base/' on deploy-sourcegraph-dot-com
- deploy-sourcegraph-dot-com will contain a dhall configuration pipeline that outputs a K8s YAML file that's semantically equivalent to what was in
base/before. This YAML file will be checked into the repository. - deploy-sourcegraph-dot-com's CI will kubectl apply this new K8s YAML from dhall in addition to the remaining K8s YAML from services that haven't been migrated
This change should be entirely transparent since the K8s YAML that deploy-sourcegraph-dhall generates will be semantically identical to what was there before.
Note: Unlike https://github.com/sourcegraph/sourcegraph/issues/11830, deploy-sourcegraph-dot-com doesn't have a clean diff that clearly shows the transformations that will need to be made to the base configuration. This is a known problem that will need to be solved (see https://github.com/sourcegraph/sourcegraph/issues/12110).
To start off with, the implementer will rely on the git diff output between deploy-sourcegraph-dot-com and deploy-sourcegraph. If this proves to be insufficient, then @ggilmore will use the implementer's feedback to figure which of the solutions from https://github.com/sourcegraph/sourcegraph/issues/12110 we should invest in.
created time in 3 days
issue openedsourcegraph/sourcegraph
deploy-sourcegraph-dhall: implement migration path from pure-yaml -> dhall
We should have a story for capturing and translating the modifications that users built on top of pure-yaml deploy-sourcegraph over to deploy-sourcegraph-dhall.
Here are a few ideas for this:
- Ask for the
git diffbetween their fork and the upstream deploy-sourcegraph, and then do the translation by hand in the pipeline file. - Use
kubectl diffagainst the output of deploy-sourcegraph-dhall and iteratively tweak the pipeline configuration until the diff is a no-op (not sure if the kubectl diff output is very clean - might include runtime information). - Write scripts / programs to read the existing cluster configuration (either from the YAML or from what the kubectl api server returns) and translate it into a dhall record that's structured in the same way that https://github.com/sourcegraph/deploy-sourcegraph-dhall/blob/master/src/base/component.dhall specifies.
dhall diffcan give us a fine-grained diff that can be used to iterate on the pipeline configuration until the diff is empty. - Compare the YAML from the existing deployment and the dhall pipeline's output. The YAML will need to be cleaned and normalized (collapsed into single kubernetes list, keys recusively sorted, whitespace trimmed, etc.) so that the diff is clean.
created time in 4 days
issue openedsourcegraph/sourcegraph
deploy-sourcegraph-dhall: grafana: implement new configuration logic for k8s.sgdev.org
Right now the generate function in Grafana https://github.com/sourcegraph/deploy-sourcegraph-dhall/blob/master/src/base/grafana/generate.dhall doesn't actually read the configuration that's passed to it .
https://github.com/sourcegraph/deploy-sourcegraph-dogfood-k8s/blob/ca1a4d1a7e678f6fa65abd2fdab5047fff31000e/generated/params.sh#L38-L44
This issue tracks the work that's needed to implement the k8s.sgdev.org transformations specified for Redis in the above file.
- new configuration fields will need to be added to https://github.com/sourcegraph/deploy-sourcegraph-dhall/blob/master/src/base/grafana/configuration.dhall
- new logic will need to be written in https://github.com/sourcegraph/deploy-sourcegraph-dhall/blob/master/src/base/grafana/generate.dhall to support these new transformations.
The PR implementing this functionality will include an example pipeline configuration in the description that declares the same transformations as the ones linked above.
created time in 4 days
issue openedsourcegraph/sourcegraph
deploy-sourcegraph-dhall: redis: implement configuration logic for k8s.sgdev.org
Right now the generate function in Redis https://github.com/sourcegraph/deploy-sourcegraph-dhall/blob/master/src/base/redis/generate.dhall doesn't actually read the configuration that's passed to it .
https://github.com/sourcegraph/deploy-sourcegraph-dogfood-k8s/blob/ca1a4d1a7e678f6fa65abd2fdab5047fff31000e/generated/params.sh#L92-L98
This issue tracks the work that's needed to implement the k8s.sgdev.org transformations specified for Redis in the above file.
- new configuration fields will need to be added to https://github.com/sourcegraph/deploy-sourcegraph-dhall/blob/master/src/base/redis/configuration.dhall
- new logic will need to be written in https://github.com/sourcegraph/deploy-sourcegraph-dhall/blob/master/src/base/redis/generate.dhall to support these new transformations.
The PR implementing this functionality will include an example pipeline configuration in the description that declares the same transformations as the ones linked above.
created time in 4 days
issue openedsourcegraph/sourcegraph
deploy-sourcegraph-dhall: frontend: implement new configuration logic for k8s.sgdev.org
Right now the generate function in Frontend https://github.com/sourcegraph/deploy-sourcegraph-dhall/blob/master/src/base/frontend/generate.dhall doesn't support all the transformations that are currently specified in the k8s.sgdev.org transformation pipeline:
https://github.com/sourcegraph/deploy-sourcegraph-dogfood-k8s/blob/ca1a4d1a7e678f6fa65abd2fdab5047fff31000e/generated/params.sh#L13-L25
New configuration fields will need to be added to https://github.com/sourcegraph/deploy-sourcegraph-dhall/blob/master/src/base/frontend/configuration.dhall and new logic will need to be written in https://github.com/sourcegraph/deploy-sourcegraph-dhall/blob/master/src/base/frontend/generate.dhall to support these new transformations.
The PR implementing this functionality will include an example pipeline configuration in the description that declares the same transformations as the ones linked above.
created time in 4 days
issue openedsourcegraph/sourcegraph
RFC 141: prepare k8s.sgdev.org infrastructure for dhall-based pipeline
https://github.com/sourcegraph/deploy-sourcegraph-dogfood-k8s is currently running the old bash-based prototype. The following work needs to be done to transition over to the new deploy-sourcegraph-dhall-based pipeline for dogfooding.
CI
- [ ] the buildkite agents will need to have the
dhall-asdfplugin baked into their docker images - Various scripts will need to be written:
- [ ] a script to render the dhall configuration pipleine into a single kubernetes yaml file
- [ ] a script apply the kubernetes yaml file from the step above to the cluster
- [ ] a script that checks that the kubernetes yaml file that's checked into the repository is in sync with what the dhall-pipeline renders
Infrastructure
- [ ] deploy-sourcegraph-dogfood-k8s's existing CI pipeline seems to be broken as a result of the testing for https://github.com/sourcegraph/sourcegraph/issues/10455. In addition, it's possible that there will be some downtime on k8s.sgdev.org during this bash->dhall transition period. k8s.sgdev.org is used for demos (this will need to be fixed separately), so it's important that it stay up. To "fix" both of these and move as quickly as possible, I propose to create a clone of the existing k8s.sgdev.org cluster under a separate domain (k8s-dhall.sgdev.org). The postgres taable frok k8s.sgdev.org will need to be dumped and restored on the new cluster.
created time in 4 days
delete branch sourcegraph/deploy-sourcegraph-dhall
delete branch : revert-4-ci-change-package
delete time in 5 days
delete branch sourcegraph/deploy-sourcegraph-dhall
delete branch : simplify.sh
delete time in 5 days
create barnchsourcegraph/deploy-sourcegraph-dhall
created branch time in 5 days
delete branch sourcegraph/deploy-sourcegraph-dhall
delete branch : simplify.sh
delete time in 5 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 5986f86c64b87160778fb391cf37da1ca131918e
remove boilerplate from raw yaml-to-dhall output (#8)
push time in 5 days
PR merged sourcegraph/deploy-sourcegraph-dhall
Thanks to https://github.com/dhall-lang/dhall-haskell/pull/1902 and @Gabriel439, I was able to use a preview build of dhall schemas to partially automate the use of K8s types instead of the raw yaml-to-dhall output.
pr closed time in 5 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha f1182559ec5476739bbf572d0d60d25bc7b76738
rm simple.dhall helper files
commit sha dd34acf2848d8368375e999b0d1f18fd04a1c5cd
rm simplify.sh helper script
push time in 5 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 9f220246da113bc4108d555b5070548606fc4c7b
simplify redis
commit sha 359a20ab640b3d03ecafeb9ebdf723c884919e6d
simplify replacer
commit sha 8bc649ba1065ab9393366bb9a810df11fb1fb4ab
simplify repo-updater
commit sha eea41f2f07ebd064e90c2460793870782d7e39dd
simple searcher
commit sha 72bf14290309dafdb7185d8e6db0dc07adb590fa
simple symbols
commit sha 84a081f65051af017f63eb508293f48859eeca16
simple syntax highlighter
push time in 5 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha c80c69464bc3c8c7666679f3b07ebd1a95ce0669
simplify prometheus
commit sha ed57a3414ef07ed343617bde0410cd742c23c4ee
simplify query runner
push time in 5 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 30d74c02351fae398104d74a329d6747aad70482
simplify indexed-search
commit sha 4973e5d8de9fbe396d6431a714814a482b9f267d
simplify jaeger
commit sha 4c9dce9f84fea786502933477a6e38bf612bf3fb
simplify precise-code-intel
push time in 5 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 19740a4a82b1ff9dd02bb98e8c03445e89cd8026
simplify grafana
push time in 5 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 31d1eb60fef344c6ee115f541b33630865f4db88
simplify cadvisor
commit sha 03308add46807f02f728921ffcc089490213227d
simpliify github proxy
push time in 5 days
PR opened sourcegraph/deploy-sourcegraph-dhall
pr created time in 5 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha aef740df0594a18d4adeac8f50497e55eea96f1f
query-runner
commit sha 1d2a4e8568d885209b1d5253102e1ac07850e0b4
redis
commit sha 119e78366106cecf58b413ae7c0b6aae251930c6
redis
commit sha cca4bded66b100847ef69949c63bde614841ad2d
replacer
commit sha 03c009941541029decc48609a2a3ad992b150a1e
repo-updater
commit sha c85a5efff9c4087d1f46c057f6d4450d08190482
searcher
commit sha 7ddd0c3eae6577ee1604b2377fa3b954f67a51e8
symbols
commit sha 9342571fb4641ff59b72bf9a7a7e54dc024da454
syntax highlighter
push time in 6 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha f992af6ce4c042ef27fc962d63f6426ce53bf187
correct import
commit sha 2dd0a2b7289b9355bd0a0e3a97b5b97457124861
jaeger
commit sha ad7f560937547b3f909039b62e13695864a12687
postgres
commit sha ff2e1a069b6f61a9061e093c1d7077eac7c03245
precise-code-intel.dhall
commit sha e240b0cac78340d03c1f873c69c2fca729080417
prometheus
push time in 6 days
create barnchsourcegraph/deploy-sourcegraph-dhall
created branch time in 6 days
delete branch sourcegraph/deploy-sourcegraph-dhall
delete branch : more-config
delete time in 6 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha cbf603bba06a63f1b295992925faac63cdd34b4e
update services to more recent deploy-sourcegraph configuration (#7)
push time in 6 days
PR merged sourcegraph/deploy-sourcegraph-dhall
Based off of https://github.com/sourcegraph/deploy-sourcegraph/commit/9fae2fbcb3d619e4679bef41cc5aae03bcebe69c
Fixes https://github.com/sourcegraph/sourcegraph/issues/11829
pr closed time in 6 days
issue closedsourcegraph/sourcegraph
deploy-sourcegraph-dhall: bring resources up-to-date with latest deploy-sourcegraph commits
https://github.com/sourcegraph/deploy-sourcegraph-dhall is based off an old commit of deploy-sourcegraph: https://github.com/sourcegraph/deploy-sourcegraph/tree/dhall-rfc-141
The configuration needs to be caught up to date with https://github.com/sourcegraph/deploy-sourcegraph/commit/3437ce590576d46ac275b19849f2fea8ed6f4ac0.
This should be fairly straightforward: it's mostly just a lot of copy pasting.
The following rough script creates a dhall record of all the resources (keyed by the resource's kind when run in the root of a component's folder (example: https://github.com/sourcegraph/deploy-sourcegraph/tree/3437ce590576d46ac275b19849f2fea8ed6f4ac0/base/cadvisor).
#!/usr/bin/env bash
cd "$(dirname "${BASH_SOURCE[0]}")"
set -euo pipefail
record="{=}"
for f in *.yaml; do
kind=$(yq read "$f" 'kind')
dhallExpression=$(printf "let kubernetes = https://raw.githubusercontent.com/dhall-lang/dhall-kubernetes/master/1.17/package.dhall in kubernetes.%s.Type" "$kind")
k8sDhall="$(yaml-to-dhall --file "$f" --records-loose "${dhallExpression}")"
record="$record /\ { $kind = ${k8sDhall} }"
done
echo "$record" | dhall --explain --output upstream.dhall
[Note that won't work off-the-shelf for every compoment (see sourcegraph-frontend's two services) - but it's easy enough to hack the above script for those purposes)].
The upstream.dhall file can be compared against the current component with dhall diff.
Repeat this copy-paste process for each component in https://github.com/sourcegraph/deploy-sourcegraph-dhall
closed time in 6 days
ggilmore
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 9181284fee4526339698ea252e02fb5516791c19
diff.dhall generate-plain.dhall
commit sha 65b90fb3199bf48141c78898f65bd9468e7b55d4
adjust ci check
push time in 6 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 1ddc8bac2d312968bb78caffe30913de83c3dd5d
remove freeze from pipeline
push time in 6 days
push eventsourcegraph/deploy-sourcegraph
commit sha 9fae2fbcb3d619e4679bef41cc5aae03bcebe69c
wip
push time in 6 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 1e767bcd168f4bdc9217a4f0fe8297da1cad491c
ci: add generate to typecheck stop
push time in 6 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha fa5da0bfb799074d8f1db4d04dc77a53bad198ab
remove diff.sh
push time in 6 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 651bd672711a7cebb6481787417ff966fa3fcea2
remove upstream files
push time in 6 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 2966ea4f4476e8439d169e774f074dcf88a49f16
update all components
push time in 6 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 566def8a580a23735d4da2f8a1536034278a8a1a
remove generate from package.dhall
push time in 7 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 0f76cde3b6e44f1155eb173e4a753839aec0257e
extend refactor to render.dhall
push time in 8 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha ca1e7c19a7168867ae515609007d5fb15f65401f
add all render functions
commit sha b33ffbd19a7e6bcd239e34495c52776baac033bf
lint
push time in 8 days
push eventggilmore/tep-alumni-website
commit sha bf068a85c966452b20f09bd23c87f89a694116cf
ci: run suite on every commit (#8)
push time in 9 days
push eventggilmore/tep-alumni-website
commit sha 813bbd07a49571933ad0fcfdb73162724a835535
dhall 1.33.1
commit sha 97f6f0851c4debf228f010ee658638807900dc0a
switch back to upstream github-actions
push time in 9 days
push eventggilmore/tep-alumni-website
commit sha fdf7d3905abe7abb707892e9c5a2f6043defaa06
add donation buttons back (#7)
push time in 9 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 8eec7fba8f8421c377268ad49922d15c80729a37
lint
push time in 12 days
PR opened sourcegraph/deploy-sourcegraph-dhall
pr created time in 12 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha d4497bbe2e736fed7878d59129caef863ff7027f
ci: run checks on every commit (#5)
commit sha 289839b6efdb7c23a53ca254d19a379b431c51dd
dev: use watchexec instead of entr (#6)
commit sha de1f3cf37378ad2d949eac3b4a6a54e8b1197080
Merge branch 'master' into more-config
commit sha 200c2e515225dc5aebbdc3bc3abe26e5e3ecc7af
Merge remote-tracking branch 'origin/master' into more-config
commit sha 48094aa65bd0ce9d6c5653135e96eacb89391b79
wip
push time in 12 days
delete branch sourcegraph/deploy-sourcegraph-dhall
delete branch : use-watchexec
delete time in 13 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 289839b6efdb7c23a53ca254d19a379b431c51dd
dev: use watchexec instead of entr (#6)
push time in 13 days
PR merged sourcegraph/deploy-sourcegraph-dhall
pr closed time in 13 days
PR opened sourcegraph/deploy-sourcegraph-dhall
pr created time in 13 days
create barnchsourcegraph/deploy-sourcegraph-dhall
created branch time in 13 days
delete branch sourcegraph/deploy-sourcegraph-dhall
delete branch : ci-all-commits
delete time in 13 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha d4497bbe2e736fed7878d59129caef863ff7027f
ci: run checks on every commit (#5)
push time in 13 days
PR merged sourcegraph/deploy-sourcegraph-dhall
pr closed time in 13 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha f1e8bcb55fa61644f549e7a601586ed25cc5a010
rename some checks
push time in 13 days
PR opened sourcegraph/deploy-sourcegraph-dhall
pr created time in 13 days
create barnchsourcegraph/deploy-sourcegraph-dhall
created branch time in 13 days
create barnchsourcegraph/deploy-sourcegraph-dhall
created branch time in 13 days
issue openedsourcegraph/sourcegraph
deploy-sourcegraph-dhall: extend configuration layer for subset of services on k8s.sgdev.org
https://github.com/sourcegraph/deploy-sourcegraph-dogfood-k8s/blob/4669f10e139feccd1e8a68490faecd9c5b76437d/generated/params.sh#L11-L44
Right now, k8s.sgdev.org is configured via a bash-based configuration pipeline as a result RFC 141. This will be replaced by https://github.com/sourcegraph/deploy-sourcegraph-dhall as part of the Dhall dogfooding process.
Part of this will be done by @ggilmore to lay the groundwork. The rest will be done by another member of the distribution team to spread knowledge and solicit feedback. This other team member will need to need to go through the dhall tutorial + pair with me in order to get up to speed about the Dhall language itself and https://github.com/sourcegraph/deploy-sourcegraph-dhall's architecture.
@ggilmore proposes that he'll replicate frontend, grafana, and redis from https://github.com/sourcegraph/deploy-sourcegraph-dogfood-k8s/blob/release/generated/params.sh. The other distribution teammate will handle the other services
The work involved here includes:
- writing scripts to invoke dhall in the k8s.sgdev.org pipeline
- refactoring k8s.sgdev.org's bash-based configuration scripts to merge the output of the dhall-based-pipeline can be deployed alongside the legacy bash-based one
- Writing more dhall code in deploy-sourcegraph-dhall to handle new configuration needs (such as increasing the storage capacity of a persistent volume claim - needed for gitserver, etc.)
- Wiring up the configuration layer for all services in deploy-sourcegraph-dhall, not just the ones listed in https://github.com/sourcegraph/sourcegraph/issues/10936.
created time in 14 days
issue openedsourcegraph/sourcegraph
deploy-sourcegraph-dhall: bring resources up-to-date with latest deploy-sourcegraph commits
https://github.com/sourcegraph/deploy-sourcegraph-dhall is based off an old commit of deploy-sourcegraph: https://github.com/sourcegraph/deploy-sourcegraph/tree/dhall-rfc-141
The configuration needs to be caught up to date with https://github.com/sourcegraph/deploy-sourcegraph/commit/3437ce590576d46ac275b19849f2fea8ed6f4ac0.
This should be fairly straightforward: it's mostly just a lot of copy pasting.
The following rough script creates a dhall record of all the resources (keyed by the resource's kind when run in the root of a component's folder (example: https://github.com/sourcegraph/deploy-sourcegraph/tree/3437ce590576d46ac275b19849f2fea8ed6f4ac0/base/cadvisor).
#!/usr/bin/env bash
cd "$(dirname "${BASH_SOURCE[0]}")"
set -euo pipefail
record="{=}"
for f in *.yaml; do
kind=$(yq read "$f" 'kind')
dhallExpression=$(printf "let kubernetes = https://raw.githubusercontent.com/dhall-lang/dhall-kubernetes/master/1.17/package.dhall in kubernetes.%s.Type" "$kind")
k8sDhall="$(yaml-to-dhall --file "$f" --records-loose "${dhallExpression}")"
record="$record /\ { $kind = ${k8sDhall} }"
done
echo "$record" | dhall --explain --output upstream.dhall
[Note that won't work off-the-shelf for every compoment (see sourcegraph-frontend's two services) - but it's easy enough to hack the above script for those purposes)].
The upstream.dhall file can be compared against the current component with dhall diff.
Repeat this copy-paste process for each component in https://github.com/sourcegraph/deploy-sourcegraph-dhall
created time in 14 days
create barnchsourcegraph/deploy-sourcegraph-dhall
branch : revert-4-ci-change-package
created branch time in 14 days
delete branch sourcegraph/deploy-sourcegraph-dhall
delete branch : ci-change-package
delete time in 14 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha e35a476a0edb9e99ccc60322872bd278fda4b955
ci: switch to upsteam package instead of my fork (#4)
push time in 14 days
PR merged sourcegraph/deploy-sourcegraph-dhall
pr closed time in 14 days
PR opened sourcegraph/deploy-sourcegraph-dhall
pr created time in 14 days
create barnchsourcegraph/deploy-sourcegraph-dhall
created branch time in 14 days
issue commentdhall-lang/dhall-haskell
performance advice for dhall-kubernetes based package
Hey @Gabriel439 thanks for looking into this! A few responses:
- I am using dhall
1.33.1for the benchmark https://github.com/sourcegraph/deploy-sourcegraph-dhall/blob/285b1515ade18682d1fac33f8e70bb32100dd67c/.tool-versions#L1. - 🤦 I realize now that I was freezing everything besides
pipeline.dhallfor the benchmark. Here are my benchmark numbers after freezing that file:~/dev/go/src/github.com/sourcegraph/ds-dhall master* 28s ❯ bench 'dhall-to-yaml --file pipeline.dhall' benchmarking dhall-to-yaml --file pipeline.dhall time 8.996 s (8.203 s .. NaN s) 0.995 R² (0.991 R² .. 1.000 R²) mean 8.468 s (8.285 s .. 8.781 s) std dev 299.8 ms (45.39 ms .. 371.0 ms) variance introduced by outliers: 19% (moderately inflated)- I still find it surprising that there is a ~ 14 second speedup after I freeze the
pipeline.dhallfile alone. Can you explain why this is the case?
- I still find it surprising that there is a ~ 14 second speedup after I freeze the
-
One of the ways I'm exploring to reduce the size of the cached expression is to see if we can encode things more compactly by specifying the union type only once
- Ah, I thought it'd make more sense for each component to construct its own list of resources so that it'd be easier to concatenate them later without having to know all the resources in each component.
comment created time in 15 days
issue commentsourcegraph/sourcegraph
Distribution: 3.18 Tracking issue
This week I mainly focused on https://github.com/sourcegraph/deploy-sourcegraph-dhall for RFC 141:
- I imported the rest of the deploy-sourcegraph services
- Wrote scripts to set up a proper dev environment (linting, formatting, compiling on save)
- Added CI (also rendered in Dhall)
- Added benchmarks and opened https://github.com/dhall-lang/dhall-haskell/issues/1890 to ask for help investigating performance issues
I'm in the process of:
- incorporating the configuration layer into the newly added services
- generating a diff between https://github.com/sourcegraph/deploy-sourcegraph-dogfood-k8s and https://github.com/sourcegraph/deploy-sourcegraph-dhall so that we can migrate over and do proper dogfooding
comment created time in 17 days
issue openeddhall-lang/dhall-haskell
performance advice with dhall-kubernetes based package
Forked from https://functionalprogramming.slack.com/archives/C82N1L0MB/p1592337474256100
My team is investigating Dhall to see if it'd be a good way to ship + customize our complex Kubernetes application on our internal clusters and for our customers to use.
https://github.com/sourcegraph/deploy-sourcegraph-dhall is my PoC implementation of our Kubernetes configuration. I like what I see so far, but I'm wondering how I can improve the render times that I'm seeing.
I've posted benchmark's in the https://github.com/sourcegraph/deploy-sourcegraph-dhall 's README: https://github.com/sourcegraph/deploy-sourcegraph-dhall#benchmarks
Going off of https://discourse.dhall-lang.org/t/figuring-out-performance-bottlenecks/251/5,one thing that I want to call out is that my normal form is apparently ~300x the size of the Prelude.
I know that dhall-kubernetes has a lot of types, but is this size expected?
- Are there non-ideal things things that I'm doing in https://github.com/sourcegraph/deploy-sourcegraph-dhall that cause this bloat?
- Are there some tools that can visualize the normal form and identify common patterns?
created time in 18 days
delete branch sourcegraph/deploy-sourcegraph-dhall
delete branch : editor-condig
delete time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha c002f47fe8508c95f823e8cc3aa688e002cb8423
ci: add editorconfig for shfmt (#3)
push time in 18 days
PR merged sourcegraph/deploy-sourcegraph-dhall
pr closed time in 18 days
PR opened sourcegraph/deploy-sourcegraph-dhall
pr created time in 18 days
create barnchsourcegraph/deploy-sourcegraph-dhall
created branch time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha a3f32062b10e08435c6ad73085a5df8c4c30c00d
Update README.md
push time in 18 days
delete branch sourcegraph/deploy-sourcegraph-dhall
delete branch : add-pipeline-check
delete time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 285b1515ade18682d1fac33f8e70bb32100dd67c
ci: add step to see if package compiles correctly (#2)
push time in 18 days
PR merged sourcegraph/deploy-sourcegraph-dhall
pr closed time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha dd0aa9425437d8ed6a9466c3a8e76eb3978d8783
fix case sensitivity issue
push time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha b69a86f996601e9dad5871c9dfe02c6d6e8051d2
debug
push time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 3e479effa86e3cea55c17444e3ab30cabd859b65
debug
push time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha a3537b2b5a562e431aff692fab9df538e243698b
more debug
push time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha ba603911b00609a29d81a3ca11a3695b1fefbe7a
try removing submodules
push time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 01dd79267770bc5bf18bb13e92c6b78ff6167b46
remove themes?
push time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 60e3dd2a14de44c17ae1c090343f88b019682597
debug
push time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha bd755bd760a9fc9e85a40bdbc411e807823988bc
more debug
push time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha cf6eb991334fe6e164d7dda625a9fb06e2c80814
debug
push time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 43e1be10b87ffc48034dac935bb03eabd00b36f2
debug
push time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 8ef8ebf9989446311d12c0179e5222a9fb99c3eb
try fixiing script
push time in 18 days
PR opened sourcegraph/deploy-sourcegraph-dhall
pr created time in 18 days
create barnchsourcegraph/deploy-sourcegraph-dhall
created branch time in 18 days
push eventsourcegraph/deploy-sourcegraph-dhall
commit sha 6ebfad4852b42f46ed38282795ff2e9b7d75d18c
add rest of the deploy-sourcegraph services (#1)
push time in 18 days