profile
viewpoint
Daniel Chen chenbh @vmware Canada

chenbh/dotnvim 1

neovim configs

chenbh/Sumobot 1

Arduino robot programmed to find and push another robot off a ring similar to a sumo wrestling match.

chenbh/atc 0

demo purposes

chenbh/Chazm 0

Java game that requires the player to answer science questions in order to advance in a maze. Supports printing and high score keeping.

chenbh/ColourUs-BackEnd 0

Java program that identified the dominate emotion (out of happy, sad, anger, love, surprised, fear) in a tweet and displays the results by regional base.

chenbh/concourse 0

Concourse is a container-based continuous thing-doer written in Go and Elm.

chenbh/concourse-bosh-deployment 0

A toolchain for deploying Concourse with BOSH.

chenbh/concourse-bosh-release 0

Concourse BOSH release

chenbh/concourse-loop-finder 0

Finds circular concourse pipelines

push eventconcourse/concourse

Aidan Oldershaw

commit sha 805fb859db192d6ef4394e68a80289bd1b1dc114

containerd: behaviour: return exe not found error concourse/concourse#6098 doesn't work for containerd because our runtime doesn't try to detect exe not found errors. This was copied from gdn: https://github.com/cloudfoundry/guardian/blob/3da697a62154c6d1f797aede3a0fb23f2297e6cd/rundmc/runcontainerd/runcontainerd.go#L201-L221 Also, use fake iptables in the network test so I can run it on mac Signed-off-by: Aidan Oldershaw <aoldershaw@pivotal.io>

view details

Aidan Oldershaw

commit sha 7908bc73aaf6822d2cb8582a82b2a574a0c1d36e

testflight: add coverage for default shell fallback It's a feature that requires cooperation between worker, ATC, and fly, and is an easy thing to miss without higher-level coverage. Signed-off-by: Aidan Oldershaw <aoldershaw@pivotal.io>

view details

Aidan Oldershaw

commit sha 9104ed6a18089ebb7349a2e9b509e661f0bb4ff1

containerd: behaviour: fix exe not found check Should be bound to proc.Start rather than task.Exec, which doesn't execute the process Signed-off-by: Aidan Oldershaw <aoldershaw@pivotal.io>

view details

Aidan Oldershaw

commit sha 0b4a952e1f6f3a4871d56ea5f54a45ae84d65a53

Merge pull request #6304 from concourse/sh-fallback-containerd `fly intercept` falls back to `sh` when `bash` is missing (containerd runtime)

view details

push time in 40 minutes

delete branch concourse/concourse

delete branch : sh-fallback-containerd

delete time in 40 minutes

PR merged concourse/concourse

`fly intercept` falls back to `sh` when `bash` is missing (containerd runtime) bug misc

<!-- Hi there! Thanks for submitting a pull request to Concourse!

The title of your pull request will be used to generate the release notes. Please provide a brief sentence that describes the PR, using the imperative mood. Please refrain from adding prefixes like 'feature:', and don't include a period at the end.

Examples: "Add feature to doohickey", "Fix panic during spline reticulation"

We will edit the title if needed so don't worry about getting it perfect!

To help us review your PR, please fill in the following information. -->

What does this PR accomplish?

<!-- Choose all that apply. Also, mention the linked issue here. This will magically close the issue once the PR is merged. --> Bug Fix | Feature | Documentation

#6098 changed the behaviour of fly intercept to fallback to sh as the default command if the container doesn't have bash. However, this requires cooperation with the runtime to return a specific garden error, which containerd did not follow.

Changes proposed by this PR:

<!-- Tell the reviewer What changed, Why, and How were you able to accomplish that? -->

  • Return a garden.ExecutableNotFoundError when the executable is not found (logic copied from Guardian)
  • Add testflight coverage

Notes to reviewer:

<!-- Leave a message to whoever is going to review this PR. Mainly, pointers to review the PR, and how they can test it. -->

Contributor Checklist

<!-- Most of the PRs should have the following added to them, this doesn't apply to all PRs, so it is helpful to tell us what you did. -->

Reviewer Checklist

<!-- This section is intended for the reviewers only, to track review progress. -->

  • [ ] Code reviewed
  • [ ] Tests reviewed
  • [ ] Documentation reviewed
  • [ ] Release notes reviewed
  • [ ] PR acceptance performed
  • [ ] New config flags added? Ensure that they are added to the BOSH and Helm packaging; otherwise, ignored for the integration tests (for example, if they are Garden configs that are not displayed in the --help text).
+128 -11

0 comment

5 changed files

aoldershaw

pr closed time in 40 minutes

pull request commentconcourse/concourse-bosh-release

add godebug configs

This is kind of annoying but it seems reasonable to allow this for now just so folks aren't blocked on things outside of their control. 👍

@xtremerui I wouldn't go as far as proactively adding a param for this everywhere - I think it's better for folks to run into it first so they know that something needs to be done. At least with the Helm chart users won't be blocked because you can always add the env var yourself after generating the template.

I'm not keen to add this in to all the resource types, either - that's a lot of churn for something that's deprecated. I'm more inclined to have folks build their own forks instead until they can fix their certs.

xtremerui

comment created time in 2 hours

push eventconcourse/ci

Alex Suraci

commit sha 8824d85e1cee93e8883a3e02c29736455871a734

resource pipelines: use oci-build-task Signed-off-by: Alex Suraci <suraci.alex@gmail.com>

view details

push time in 3 hours

push eventconcourse/mock-resource

git

commit sha 621f403f55857c560579db405ddd50f7abbeeaae

bump to 0.11.0

view details

push time in 3 hours

created tagconcourse/mock-resource

tagv0.11.0

a resource for testing; reflects the version it's told, and is able to mirror itself

created time in 3 hours

release concourse/mock-resource

v0.11.0

released time in 3 hours

issue commentconcourse/mock-resource

Are there any docs?

nope - it really shouldn't be used in production, it's only bundled in for testing Concourse itself so that testflight etc. don't have to use the network.

docs would be nice but they'd only be there to support other developers writing Concourse tests. if/when we do add them we should take care to add a big scary warning so that no one uses this for anything serious.

jchesterpivotal

comment created time in 3 hours

push eventconcourse/ci

Aidan Oldershaw

commit sha 99d8d50fe424420f99f99d147ba3ec21048cc6c2

bump windows boot disk to 64GB Signed-off-by: Aidan Oldershaw <aoldershaw@pivotal.io>

view details

push time in 4 hours

pull request commentconcourse/concourse-bosh-release

add godebug configs

Totally agree, i came to a PR recetnly https://github.com/concourse/docker-image-resource/pull/317, which also adds the same env var for go 1.15.

We should come up with a more systematic way to allow folks ignore the warning (indeed there is case that the cert is out of their control).

xtremerui

comment created time in 4 hours

pull request commentconcourse/concourse-bosh-release

add godebug configs

Hm I'm going to pull in @vito here for some insight, because my initial reaction is that this additional flag is added just so that someone that has an outdated certificate is trying to continue using it even though it is no longer supported. I'm just hesitant on adding flags to temporarily fix one person's use case, when it can be solved through generating a proper cert. But I also have very little knowledge of certificates, AD servers, etc and if this will be a more widespread problem that many of our users will run into, so I could be totally wrong.

xtremerui

comment created time in 5 hours

pull request commentconcourse/concourse

Speed up database queries by adding a `job_id` column to build image resource caches table and adding an index for ordering builds of a job

Whenever we create a new build for a check, it will delete any past builds created for that resource check and this will cascade to deleting the build_image_resource_cache for that build.

Ah, right - that makes me feel better about it!

If you are fine with it, I think it would really help with the performance of the query

It's fine with me - it seems like any alternatives also have their downsides, and I don't think a little denormalization is so scary in this particular case, given that the denormalized state is effectively immutable and the build_image_resource_caches table shouldn't ever be so massive. So I think the benefits outweigh the risks

clarafu

comment created time in 5 hours

push eventconcourse/concourse

Aidan Oldershaw

commit sha d606a5c71b5524987184c31bdad65d709283cf20

fly: set-pipeline prints pipeline name + instance vars When testing instanced pipelines, someone accidentally tried to multiple instance vars in a single flag, e.g. ``` fly sp ... -i foo=123,bar=456 ``` which actually results in the following instance vars: ``` foo: "123,bar=456" ``` This commit prints the pipeline name and pipeline instance vars to stdout before prompting the user to confirm so they know what they're setting is correct. Signed-off-by: Aidan Oldershaw <aoldershaw@pivotal.io>

view details

Aidan Oldershaw

commit sha 4bb6ddc1e219f8849eda68b59039978efce755d1

Merge pull request #6300 from concourse/set-pipeline-info `fly set-pipeline` prints pipeline name and instance vars

view details

push time in 5 hours

delete branch concourse/concourse

delete branch : set-pipeline-info

delete time in 5 hours

PR merged concourse/concourse

`fly set-pipeline` prints pipeline name and instance vars enhancement

<!-- Hi there! Thanks for submitting a pull request to Concourse!

The title of your pull request will be used to generate the release notes. Please provide a brief sentence that describes the PR, using the imperative mood. Please refrain from adding prefixes like 'feature:', and don't include a period at the end.

Examples: "Add feature to doohickey", "Fix panic during spline reticulation"

We will edit the title if needed so don't worry about getting it perfect!

To help us review your PR, please fill in the following information. -->

What does this PR accomplish?

<!-- Choose all that apply. Also, mention the linked issue here. This will magically close the issue once the PR is merged. --> Bug Fix | Feature | Documentation

To reduce the risk of users making mistakes when setting pipelines, fly set-pipeline prints:

$ fly -t dev sp -p some-pipeline -c pipeline.yml
... (diff)

pipeline name: some-pipeline

apply configuration? [yN]:

When there are instance vars set, it also prints those:

$ fly -t dev sp -p some-pipeline -c pipeline.yml -i foo=123 -i bar=456
... (diff)

pipeline name: some-pipeline
pipeline instance vars:
  foo: 123
  bar: 456

apply configuration? [yN]:

Changes proposed by this PR:

<!-- Tell the reviewer What changed, Why, and How were you able to accomplish that? -->

  • Print info prompt before confirming with the user

Notes to reviewer:

<!-- Leave a message to whoever is going to review this PR. Mainly, pointers to review the PR, and how they can test it. -->

Contributor Checklist

<!-- Most of the PRs should have the following added to them, this doesn't apply to all PRs, so it is helpful to tell us what you did. -->

Reviewer Checklist

<!-- This section is intended for the reviewers only, to track review progress. -->

  • [ ] Code reviewed
  • [ ] Tests reviewed
  • [ ] Documentation reviewed
  • [ ] Release notes reviewed
  • [ ] PR acceptance performed
  • [ ] New config flags added? Ensure that they are added to the BOSH and Helm packaging; otherwise, ignored for the integration tests (for example, if they are Garden configs that are not displayed in the --help text).
+46 -2

0 comment

2 changed files

aoldershaw

pr closed time in 5 hours

Pull request review commentconcourse/concourse

Group instanced pipelines on UI

 urlUpdate : Routes.Route -> Model -> ( Model, List Effect ) urlUpdate route model =     let         ( newSubmodel, subEffects ) =-            if route == model.route then+            if route == model.session.route then                 ( model.subModel, [] )              else if routeMatchesModel route model then                 SubPage.urlUpdate-                    { from = model.route+                    { from = model.session.route                     , to = route                     }                     ( model.subModel, [] )              else                 SubPage.init model.session route++        oldSession =+            model.session++        newSession =+            { oldSession | route = route }

Two good catches, thanks! Fixed

aoldershaw

comment created time in 5 hours

push eventconcourse/ci

Alex Suraci

commit sha cba5c714b0292f932405f7c8d6c5b13103be8523

resources: update params for saving in OCI format Signed-off-by: Alex Suraci <suraci.alex@gmail.com>

view details

push time in 5 hours

PR opened concourse/concourse

fly: structure: avoid recompiling in tests

Changes proposed by this PR:

Rather than re-compiling fly with the version var overridden, allow faking out the client version via an env var. This speeds up the fly integration suite (11m48s -> 6m18s) and should reduce timeout flakiness in CI on the Windows/Darwin workers.

Contributor Checklist

<!-- Most of the PRs should have the following added to them, this doesn't apply to all PRs, so it is helpful to tell us what you did. -->

Reviewer Checklist

<!-- This section is intended for the reviewers only, to track review progress. -->

  • [ ] Code reviewed
  • [ ] Tests reviewed
  • [ ] Documentation reviewed
  • [ ] Release notes reviewed
  • [ ] PR acceptance performed
  • [ ] New config flags added? Ensure that they are added to the BOSH and Helm packaging; otherwise, ignored for the integration tests (for example, if they are Garden configs that are not displayed in the --help text).
+76 -55

0 comment

6 changed files

pr created time in 5 hours

issue openedconcourse/concourse

Pull Windows worker terraform config into Greenpeace

I set this up as a one-off for now, but it should be pulled into Greenpeace:

https://github.com/concourse/ci/tree/master/deployments/windows-worker

We'll probably want to parameterize the URL of the concourse.zip that it downloads and figure out when it should be deployed/updated. Maybe every time we ship? Maybe we should have a job to ship from master too, for when the worker version bumps?

note: @xtremerui noticed that if you ever RDP into the machine, Chocolatey stops working. So watch out for that. :laugh:

created time in 5 hours

push eventconcourse/concourse

Aidan Oldershaw

commit sha 0b274f3cf1e429fbc15d3b0756d506c1bd4b4e80

web: behaviour: fix "no instance vars" header padding Signed-off-by: Aidan Oldershaw <aoldershaw@pivotal.io>

view details

Aidan Oldershaw

commit sha 94af3c750b0ccc28931054605a81fc0972e6c2e3

web: behaviour: clear hover on route change This way, tooltips won't persist on page navigation if you don't hover anything else Signed-off-by: Aidan Oldershaw <aoldershaw@pivotal.io>

view details

push time in 6 hours

push eventconcourse/ci

Alex Suraci

commit sha 07909e77866eeccd8ce68b606d2900c336e40828

remove pass-release-notes-only-change release notes are generated now! Signed-off-by: Alex Suraci <suraci.alex@gmail.com>

view details

Alex Suraci

commit sha 5a6e2d1cc047191ac87a15651e06c8fcb1c1efe8

remove build-prerelease-notes no longer used Signed-off-by: Alex Suraci <suraci.alex@gmail.com>

view details

Alex Suraci

commit sha df06ec513736acb8de7721366b5377fc561c861c

Revert "Revert "use variant param in registry-image and alias bumping"" this was reverted because it caused an infinite loop, but that has since been fixed in master This reverts commit 0c315ae3bfba0eeee5aa84303e81abf82c7c968e.

view details

push time in 6 hours

Pull request review commentconcourse/docker-image-resource

Added docker_config_json parameter

 LOG_FILE=${LOG_FILE:-/tmp/docker.log} SKIP_PRIVILEGED=${SKIP_PRIVILEGED:-false} STARTUP_TIMEOUT=${STARTUP_TIMEOUT:-120} +# Otherwise we get "certificate relies on legacy Common Name field"+export GODEBUG="x509ignoreCN=0"

In what user case that you need this env var? Are you seeing the error when check/in/out?

From what I understand in this resource, only the check cmd is using Go, so in/out that uses bash script should not be affected?

markround

comment created time in 6 hours

pull request commentconcourse/concourse

Speed up database queries by adding a `job_id` column to build image resource caches table and adding an index for ordering builds of a job

@aoldershaw That's a really good point! I think right now build_image_resource_caches for check builds created for resources and resource types are most likely always being deleted during the cascade delete of the build. Whenever we create a new build for a check, it will delete any past builds created for that resource check and this will cascade to deleting the build_image_resource_cache for that build. So the gc for build_image_resource_caches or the build finishing delete query should only be for builds created by a job.

I think Jamie was mostly concerned with how far we want to go with the denormalizing things for performance reasons and if we could find another way that didn't involve denormalizing. The current way the database is modelled is very much normalized, and I kind of broke that when I added the successful_build_outputs table in order to improve performance. 😅 I think it might just be something we have to get aligned with, since previously we always followed the normalized database model. I also think its fine if we want to denormalize the database in order to improve performance, as long as it makes sense and doesn't cause problems with database disk usage.

If you are fine with it, I think it would really help with the performance of the query. I've tried many different things to try to have the same result as just adding the job_id to the table, but nothing really worked because it will always need to do a join to the builds table in order to fetch the job_id. I thought about maybe partitioning the builds table by job_id, which might help with the speed of querying to the builds but that would be a huge change (and Im still not sure if I want to go that route because it does have it's tradeoffs). One minor weird thing is that if we do denormalize the job_id, now that we have builds that are tied to resource_id or resource_type_id, there will be plenty of rows in the build_image_resource_caches table that won't have the job_id column filled out. That being said, this is also the same as before when we can have one-off builds which also wouldn't have their job_id column set.

clarafu

comment created time in 6 hours

pull request commentconcourse/concourse

Group instanced pipelines on UI

If we don't want to display the instance group name in the top header, maybe it could be within the filter search bar? That way there is indication that this is just a filtered search of the dashboard?

Yup, the plan is for clicking the instance group to apply a filter like this:

review

which implies improvements to filtering (chip creation + multiple filters), which is extra work that I think is better left for a follow up PR. p.s. that's just a crappy mockup I made quickly, but Matthew made a nicer one somewhere on Figma I believe

aoldershaw

comment created time in 6 hours

Pull request review commentconcourse/docker-image-resource

Added docker_config_json parameter

 start_docker \ 	"${max_concurrent_uploads}" \ 	"$insecure_registries" \ 	"$registry_mirror"-log_in "$username" "$password" "$registry"+  +if [ -z "$docker_config_json" ]; then+  log_in "$username" "$password" "$registry"+else+  docker_config_json_to_file "$docker_config_json"+fi

Could you add an out test to cover this feature?

markround

comment created time in 6 hours

Pull request review commentconcourse/docker-image-resource

Added docker_config_json parameter

 Note: docker registry must be [v2](https://docs.docker.com/registry/spec/api/). * `aws_access_key_id`: *Optional.* AWS access key to use for acquiring ECR   credentials. +* `docker_config_json` : *Optional.* The raw `config.json` file used for authenticating with Docker registries. If specified, `username` and `password` parameters will be ignored. You may find this useful if you need to be authenticated against multiple registries (e.g. pushing to a private registry, but you also also need to pull authenticate to pull images from Docker Hub without being rate-limited).
* `docker_config_json` : *Optional.* The raw `config.json` file used for authenticating with Docker registries. If specified, `username` and `password` parameters will be ignored. You may find this useful if you need to be authenticated against multiple registries (e.g. pushing to a private registry, but you also need to pull authenticate to pull images from Docker Hub without being rate-limited).
markround

comment created time in 6 hours

Pull request review commentconcourse/concourse

feat(atc): support chained container placement strategies.

 func (strategy *LimitActiveTasksPlacementStrategy) Choose(logger lager.Logger, w 		} 	} -	leastBusyWorkers := workersByWork[minActiveTasks]-	if len(leastBusyWorkers) < 1 {-		return nil, nil-	}-	return leastBusyWorkers[strategy.rand.Intn(len(leastBusyWorkers))], nil+	return workersByWork[minActiveTasks], nil } -func (strategy *LimitActiveTasksPlacementStrategy) ModifiesActiveTasks() bool {+func (strategy *LimitActiveTasksPlacementStrategyNode) ModifiesActiveTasks() bool { 	return true } -type RandomPlacementStrategy struct {+type RandomPlacementStrategyNode struct { 	rand *rand.Rand } -func NewRandomPlacementStrategy() ContainerPlacementStrategy {-	return &RandomPlacementStrategy{+func newRandomPlacementStrategyNode() ContainerPlacementStrategyChainNode {+	return &RandomPlacementStrategyNode{ 		rand: rand.New(rand.NewSource(time.Now().UnixNano())), 	} } -func (strategy *RandomPlacementStrategy) Choose(logger lager.Logger, workers []Worker, spec ContainerSpec) (Worker, error) {-	return workers[strategy.rand.Intn(len(workers))], nil+func (strategy *RandomPlacementStrategyNode) Choose(logger lager.Logger, workers []Worker, spec ContainerSpec) ([]Worker, error) {+	return []Worker{workers[strategy.rand.Intn(len(workers))]}, nil

Perhaps this should return all workers and let containerPlacementStrategy.Choose() deal with selecting at random.

Though there's no good reason to do it, if you specified CONCOURSE_CONTAINER_PLACEMENT_STRATEGY= random,volume-locality, the volume-locality strategy would have no effect, since we already filtered the list down to a single worker.

So, random is in a bit of a weird spot. It might be more valuable to view these as XYZPlacementStrategyNodes as filters - in which case, RandomPlacementStrategyNode is really an IdentityFilter or something like that - i.e. it just returns the full list of workers that it's provided.

evanchaoli

comment created time in 7 hours

Pull request review commentconcourse/concourse

feat(atc): support chained container placement strategies.

 func (strategy *VolumeLocalityPlacementStrategy) Choose(logger lager.Logger, wor 		} 	} -	highestLocalityWorkers := workersByCount[highestCount]--	return highestLocalityWorkers[strategy.rand.Intn(len(highestLocalityWorkers))], nil+	return workersByCount[highestCount], nil } -func (strategy *VolumeLocalityPlacementStrategy) ModifiesActiveTasks() bool {+func (strategy *VolumeLocalityPlacementStrategyNode) ModifiesActiveTasks() bool { 	return false } -type FewestBuildContainersPlacementStrategy struct {+type FewestBuildContainersPlacementStrategyNode struct { 	rand *rand.Rand

Looks like we can get rid of rand here, as well as in LimitActiveTasksPlacementStrategyNode and possibly RandomPlacementStrategyNode (see my comment there)

evanchaoli

comment created time in 7 hours

pull request commentconcourse/concourse

Group instanced pipelines on UI

Ah, good point - I hadn't really thought of that. Perhaps fly rename-pipeline could work the same way fly order-pipelines works as of 0e68698 (i.e. it takes a pipeline name rather than a pipeline ref)? Though, that would mean you can't move a pipeline instance between instance groups - not sure if that's a common use-case, though.

Ya I think it would make sense for it to work the same way as fly order-pipelines with using a pipeline name. I thought about it not being able to move a pipeline instance between groups but I thought that it might be weird being able to do that anyways because the instance vars should be tied to things set in the pipeline, and if you want to change those var values then it should produce a whole new instance instead of keeping the history of the builds produced by the old vars.

This was a deliberate decision after some discussion with @matthewpereira a while back, but I don't remember 100% why (removed in e08105e). I think it's at least in part related to #5921 (comment) - if viewing an instance group is a special case of filtering, we're still conceptually on the dashboard. That said, it's not obvious currently that it is a special case of filtering yet, so idk

I see, I think I wanted to make that suggestion because when I was playing around with instance groups I found myself getting lost sometimes. Usually it was when I was clicking a lot between the dashboard and instance groups, since the pages are widely the same and there is nothing that is indicating whether I am looking at the full dashboard or just the instance group view (other than the fact that it only shows the instances within that group). If we don't want to display the instance group name in the top header, maybe it could be within the filter search bar? That way there is indication that this is just a filtered search of the dashboard? This is just one option though, I would also be fine keeping it the way it is now too. 😁

aoldershaw

comment created time in 7 hours

more