profile
viewpoint
Boris Popovschi Zyqsempai Pentalog Chisinau, Moldova https://www.linkedin.com/in/boris-popovschi ContainerD, Firecracker, Kubernetes contributor and evangelist.

Zyqsempai/agent 0

Virtual Machine agent for hardware virtualized containers

Zyqsempai/aks-engine 0

AKS Engine: Units of Kubernetes on Azure!

Zyqsempai/amazon-vpc-cni-k8s 0

Networking plugin repository for pod networking in Kubernetes using Elastic Network Interfaces on AWS

Zyqsempai/athens 0

A Go module datastore and proxy

Zyqsempai/aufs 0

AUFS Snapshotter for containerd

Zyqsempai/btrfs 0

Btrfs bindings for Go

Zyqsempai/build-pipeline 0

A K8s-native Pipeline resource.

Zyqsempai/cgroups 0

cgroups package for Go

Zyqsempai/cli 0

The Docker CLI

PR opened containerd/cgroups

Only append Hugetlb in Subsystems list when available

This is to fix failed tests when system doesn't have hugetlb.

In test, the Subsystems() function is used,

https://github.com/containerd/cgroups/blob/4cbc285b33272039927a0b066d3412799db8de14/cgroup_test.go#L47-L52

However the mockCgroup creates the subsystems from defaults, which only adds hugetls when available,

https://github.com/containerd/cgroups/blob/4cbc285b33272039927a0b066d3412799db8de14/mock_test.go#L29-L41

https://github.com/containerd/cgroups/blob/4cbc285b33272039927a0b066d3412799db8de14/utils.go#L143-L146

+4 -1

0 comment

1 changed file

pr created time in 3 hours

pull request commentcontainerd/containerd

Fix some typos and grammars

Build succeeded.

ekrecker

comment created time in 4 hours

pull request commentcontainerd/containerd

Fix some typos and grammars

Hi @ekrecker. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

<details>

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. </details>

ekrecker

comment created time in 4 hours

PR opened containerd/containerd

Fix some typos and grammars

I've fixed typos and grammars

+4 -4

0 comment

1 changed file

pr created time in 4 hours

issue commentcontainerd/containerd

Windows: Roadmap to WCOW

I've rebased the first PR (#4395) but haven't rebased the rest of the series as they'll need to be rebased on #4395 anyway, and the changes themselves shouldn't be heavily affected, there are no conflicts showing except the vendor.conf conflict from #4395 before I rebased it.

darstahl

comment created time in 9 hours

pull request commentcontainerd/containerd

Use go-winio for applying tarballs

Build succeeded.

TBBle

comment created time in 9 hours

pull request commentcontainerd/containerd

Use go-winio for applying tarballs

Rebased to current master, and go-winio revendored to v0.4.15 (no relevant changes compared to the previous commit-SHA I was using).

TBBle

comment created time in 9 hours

issue commentcontainerd/stargz-snapshotter

Create tagged releases (with binaries)

https://github.com/containerd/stargz-snapshotter/releases/tag/v0.1.0

AkihiroSuda

comment created time in 11 hours

issue openedcontainerd/containerd

Add priority flag for containerd on Windows hosts

What is the problem you're trying to solve

On Windows hosts, there is a chance that the containerd process won't get enough CPU cycles because of the default priority set for the Windows process.

xref: https://github.com/kubernetes/kubernetes/issues/95735#issuecomment-719083689

Describe the solution you'd like

Having an additional flag to containerd similar to what we did for kubelet might be helpful.

Let me know if this is a reasonable approach. I can work on it.

Additional context xref: https://github.com/kubernetes/kubernetes/issues/95735

cc @marosset @jsturtevant @aravindhp

created time in 12 hours

issue commentcontainerd/containerd

ctr image pull operation not permitted: failed to mount /var/lib/containerd/tmpmounts/containerd-mount895935883 & operation not permitted: unknown

@fuweid I've run a container with the following Dockerfile. When I run the example code, it gives the output:

# go run test.go
go: downloading github.com/containerd/containerd v1.4.1
go: downloading github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635
go: downloading github.com/opencontainers/runc v0.1.1
go: downloading google.golang.org/grpc v1.33.2
go: downloading github.com/containerd/continuity v0.0.0-20200928162600-f2cc35102c2a
go: downloading github.com/gogo/protobuf v1.3.1
go: downloading github.com/opencontainers/image-spec v1.0.1
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/opencontainers/runtime-spec v1.0.2
go: downloading golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9
go: downloading github.com/containerd/typeurl v1.0.1
go: downloading golang.org/x/net v0.0.0-20201110031124-69a78807bb2b
go: downloading github.com/gogo/googleapis v1.4.0
go: downloading github.com/opencontainers/selinux v1.6.0
go: downloading github.com/containerd/fifo v0.0.0-20201026212402-0724c46b320c
go: downloading golang.org/x/sys v0.0.0-20201113221540-83cfaa298f31
go: downloading github.com/google/uuid v1.1.2
go: downloading google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013
go: downloading google.golang.org/protobuf v1.25.0
go: downloading github.com/containerd/ttrpc v1.0.2
go: downloading github.com/golang/protobuf v1.4.1
go: downloading github.com/sirupsen/logrus v1.7.0
go: downloading github.com/opencontainers/go-digest v1.0.0
go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c
go: downloading golang.org/x/text v0.3.3
&{0xc000412000 {docker.io/library/redis:alpine map[] {application/vnd.docker.distribution.manifest.list.v2+json sha256:b0e84b6b92149194d99953e44f7d1fa1f470a769529bb05b4164eae60d8aea6c 1645 [] map[] <nil>} {120444700 63742094649 <nil>} {120444700 63742094649 <nil>}} {0xc00029b140}}
new container error
2020/11/27 17:24:09 parent snapshot sha256:39c9befdce218aaca5472217a6bc3dc97dd3ac299f92aa5884186c5b6f684107 does not exist: not found
exit status 1

Dockerfile:

FROM golang

RUN apt-get update && \
    apt-get install -y unzip libbtrfs-dev libseccomp-dev btrfs-tools btrfs-progs

RUN go get github.com/containerd/containerd
RUN wget -c https://github.com/google/protobuf/releases/download/v3.11.4/protoc-3.11.4-linux-x86_64.zip && \ 
	unzip protoc-3.11.4-linux-x86_64.zip -d /usr/local

RUN go get github.com/opencontainers/runc && cd $GOPATH/src/github.com/opencontainers/runc && \
	make && make install

RUN cd $GOPATH/src/github.com/containerd/containerd && \
	make && make install

RUN mkdir -p /etc/cni/net.d

COPY mynet.conf /etc/cni/net.d/mynet.conf

CMD "containerd"

Code:

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/containerd/containerd"
	"github.com/containerd/containerd/namespaces"
	"github.com/containerd/containerd/oci"
)

func main() {
	if err := redisExample(); err != nil {
		log.Fatal(err)
	}
}

func redisExample() error {
	client, err := containerd.New("/run/containerd/containerd.sock")
	if err != nil {
		return err
	}
	defer client.Close()
	ctx := namespaces.WithNamespace(context.Background(), "example")

	image, err := client.Pull(ctx, "docker.io/library/redis:alpine")
	if err != nil {
		fmt.Println("pull error")
		return err
	}

	fmt.Println(image)

	container, err := client.NewContainer(
		ctx,
		"redis-server",
		containerd.WithNewSnapshot("redis-server-snapshot", image),
		containerd.WithNewSpec(oci.WithImageConfig(image)),
	)

	if err != nil {
		fmt.Println("new container error")
		return err
	}
	defer container.Delete(ctx, containerd.WithSnapshotCleanup)

	return nil
}

I pass privileged flag:

docker run --privileged -it -v "$PWD":/go/containerd-test golang-containerd

Do you have any idea why is that happening?

mstrYoda

comment created time in a day

created tagcontainerd/stargz-snapshotter

tagv0.1.0

Fast docker image distribution plugin for containerd, based on CRFS/stargz

created time in a day

PR opened containerd/imgcrypt

minify manifest json

fix: #28

+1 -1

0 comment

1 changed file

pr created time in a day

push eventcontainerd/stargz-snapshotter

ktock

commit sha cdca641fb019e3d6830b2cf010b790f35aa90738

Bump containerd to 1.4.2 Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>

view details

ktock

commit sha a41d13f932d52fe7759afc18c31c24f90573389a

Add `disable_snapshot_annotations = false` to config Since 1.4.2, `disable_snapshot_annotations = false` is required for enabling remote snapshots. Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>

view details

ktock

commit sha 15cb911baa46bb10d474682d706fcbfd9b3b2d92

Remove unused Dockerfile Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>

view details

Akihiro Suda

commit sha 9c1306360c51373135ce18a6cc521d948f9b26c5

Merge pull request #180 from ktock/containerd-1.4.2 Bump containerd/containerd to 1.4.2

view details

push time in a day

PR merged containerd/stargz-snapshotter

Bump containerd/containerd to 1.4.2

Notable change: https://github.com/containerd/containerd/pull/4665

+6 -28

0 comment

5 changed files

ktock

pr closed time in a day

issue openedcontainerd/imgcrypt

minify/compress manifest json

https://github.com/containerd/imgcrypt/blob/36840ad3a650af64eb6e8a7428256823df05ac67/images/encryption/encryption.go#L320

Was the indentation used before for testing? maybe we can compress it to save a little bit size 🤔

created time in a day

issue commentcontainerd/containerd

Windows: Roadmap to WCOW

@kevpar - PTAL. @TBBle - I am no longer working in the same area but Kevin or someone on the team should be able to help you out getting these reviews in. Sorry for any inconvenience.

darstahl

comment created time in a day

pull request commentcontainerd/containerd

[cri/config] : fix range iterator issue in ValidatePluginConfig

Build succeeded.

gaurav1086

comment created time in a day

pull request commentcontainerd/containerd

[cri/config] : fix range iterator issue in ValidatePluginConfig

Hi @gaurav1086. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

<details>

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. </details>

gaurav1086

comment created time in a day

PR opened containerd/containerd

[cri/config] : fix range iterator issue in ValidatePluginConfig

Go uses the same address variable while iterating in a range, so use a copy when using its address.

Sample code to highlight the issue: https://play.golang.com/p/2rTrumogQ5I

+1 -1

0 comment

1 changed file

pr created time in a day

startedevansm7/vftool

started time in 2 days

PR opened containerd/stargz-snapshotter

Bump containerd/containerd to 1.4.2

Notable change: https://github.com/containerd/containerd/pull/4665

+6 -28

0 comment

5 changed files

pr created time in 2 days

issue commentcontainerd/containerd

Containerd reports image names as null for the images have no tag and digest

@dmcgowan Yes, this is a workaround and can be removed after the bug is fixed in k8s. The bug https://github.com/kubernetes/kubernetes/pull/79018 was reported in k8s but it was closed without a fix yet.

Missxiaoguo

comment created time in 2 days

push eventcontainerd/stargz-snapshotter

ktock

commit sha 712cff193c6e90655d6c0e012622025d11a10521

Add document about optimization with `ctr-remote` Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>

view details

Akihiro Suda

commit sha 4629943d9e01bebe1f6c5bf9deab8f89ff1a4d7f

Merge pull request #179 from ktock/o-doc Add document about optimization with `ctr-remote`

view details

push time in 2 days

PR merged containerd/stargz-snapshotter

Add document about optimization with `ctr-remote`

This includes updates for the demo environment according to the contents of the doc.

+300 -3

0 comment

6 changed files

ktock

pr closed time in 2 days

Pull request review commentcontainerd/stargz-snapshotter

Add document about optimization with `ctr-remote`

+# Optimize Images with `ctr-remote image optimize`++This doc describes example usages of `ctr-remote image optimize` command for converting images into eStargz.++`ctr-remote images optimize` command (call `ctr-remote` in this doc) enables users to convert an image into eStargz.+The converted image can be lazily pulled by Stargz Snapshotter which can speed up the container startup.+Because this image is backward compatible to OCI/Docker image, this can be also run by other runtimes that don't support lazy pull (e.g. Docker).++Though lazy pull speeds up the container's startup, it's possible, especially with slow NW, that the runtime performance becomes lower because reading files can induce remotely downloading file contents.+For mitigating this, `ctr-remote` also allows to *optimize* the image against the *workload* the image runs.+Here, workload means the configuration of the container that runs from that image, including the entrypoint program, environment variables, user etc.+This optimization is done by baking the information about files that are likely accessed during runtime (called *prioritized files*), to the image.+On runtime, Stargz Snapshotter prefetches these prioritized files before mounting the layer for making sure these files are locally accessible.+This can avoid downloading chunks on every file read and mitigate the runtime performance drawbacks.++For more details about eStargz and its optimization, refer also to the [doc about image formats](/docs/stargz-estargz.md).++## Requirements++- fusermount: Installable through `fuse` (or `fuse3`) using package manager. e.g. `apt install fuse` on ubuntu.+- runc: Release binaries are available on https://github.com/opencontainers/runc/. Or you should already have this if you are using container runtimes (containerd, Docker, etc.) on your host.+- CNI plugins (if network connection is needed during optimization): Release binaries are available on https://github.com/containernetworking/plugins.++`ctr-remote` requires the root privilege, so run this command by root or with `sudo`.+Rootless execution of this command is still WIP.++For trying the examples described in this doc, you can also use the docker-compose-based demo environment.+You can setup this environment as the following commands.+*Note that this runs privileged containers on your host.*++```+$ cd ${GOPATH}/src/github.com/containerd/stargz-snapshotter/script/demo+$ docker-compose build containerd_demo+$ docker-compose up -d+$ docker exec -it containerd_demo /bin/bash+(inside container) # ./script/demo/run.sh+```++## Optimizing an image++The following command optimizes an (non-eStargz) image `ghcr.io/stargz-containers/golang:1.15.3-buster-org` (this is a copy of `golang:1.15.3-buster`) and pushes the result eStargz image into `registry2:5000/golang:1.15.3-esgz`.+This doesn't append workload-related configuration options (e.g. `--entrypoint`) so this optimizes the image against the default configurations baked to the image e.g. through Dockefile instructions (`ENTRYPOINT`, etc) when building the original image.++Here, `registry2:5000` in the demo environment serves images with HTTP (not HTTPS) so we need to tell it to the `ctr-remote` command using `--plain-http` option and `http://` prefix.++```+ctr-remote image optimize --plain-http \+           ghcr.io/stargz-containers/golang:1.15.3-buster-org \+           http://registry2:5000/golang:1.15.3-esgz+```++When you run this command, `ctr-remote` runs the source image (`ghcr.io/stargz-containers/golang:1.15.3-buster-org`) as a container and profiles all file accesses during the execution.+Then these accessed files are marked as "prioritized" files and will be prefetched on runtime.++You can lazy-pull this image with Stargz Snapshotter.+The following example lazily pulls this image to containerd, using `ctr-remote image rpull` command (`http://` prefix isn't needed here).++```console+# ctr-remote image rpull --plain-http registry2:5000/golang:1.15.3-esgz+egistry2:5000/golang:1.15.3-esgz+fetching sha256:bd2e5983... application/vnd.oci.image.index.v1+json+fetching sha256:55aef317... application/vnd.docker.distribution.manifest.v2+json+fetching sha256:d1123476... application/vnd.docker.container.image.v1+json+# ctr-remote run --rm -t --snapshotter=stargz registry2:5000/golang:1.15.3-esgz test echo hello+hello+```++## Optimizing an image with custom configuration++You can also specify the custom workload configuration that the image is optimized against.+The following example optimizes the image against the workload running `go version` on `/bin/bash`.++```+ctr-remote image optimize --plain-http \+           --entrypoint='[ "/bin/bash", "-c" ]' --args='[ "go version" ]' \+           ghcr.io/stargz-containers/golang:1.15.3-buster-org \+           http://registry2:5000/golang:1.15.3-esgz-go-version+```++Other options are also available for configuring the workload.++|Option|Description|+---|---+|`--entrypoint`|Entrypoint of the container (in JSON array)|+|`--args`|Arguments for the entrypoint (in JSON array)|+|`--env`|Environment variables in the container|+|`--user`|User name to run the process in the container|+|`--cwd`|Working directory|+|`--period`|The time seconds during profiling the file accesses|+|`-t`or`--terminal`|Enable to interactively control the container|++## Mounting files from the host++There are several cases where sharing files from host to the container during optimization is useful.+This includes when we want to optimize an image against building a binary using compilers.++For these use-cases, you can mount files on the hosts to the container using `--mount` option.+The following example optimizes the image against the workload where Go compiler compiles "hello world" program.++First, create the following Go source file at `/etc/hello.go`.

Exactly.

ktock

comment created time in 2 days

Pull request review commentcontainerd/stargz-snapshotter

Add document about optimization with `ctr-remote`

+# Optimize Images with `ctr-remote image optimize`++This doc describes example usages of `ctr-remote image optimize` command for converting images into eStargz.++`ctr-remote images optimize` command (call `ctr-remote` in this doc) enables users to convert an image into eStargz.+The converted image can be lazily pulled by Stargz Snapshotter which can speed up the container startup.+Because this image is backward compatible to OCI/Docker image, this can be also run by other runtimes that don't support lazy pull (e.g. Docker).++Though lazy pull speeds up the container's startup, it's possible, especially with slow NW, that the runtime performance becomes lower because reading files can induce remotely downloading file contents.+For mitigating this, `ctr-remote` also allows to *optimize* the image against the *workload* the image runs.+Here, workload means the configuration of the container that runs from that image, including the entrypoint program, environment variables, user etc.+This optimization is done by baking the information about files that are likely accessed during runtime (called *prioritized files*), to the image.+On runtime, Stargz Snapshotter prefetches these prioritized files before mounting the layer for making sure these files are locally accessible.+This can avoid downloading chunks on every file read and mitigate the runtime performance drawbacks.++For more details about eStargz and its optimization, refer also to the [doc about image formats](/docs/stargz-estargz.md).++## Requirements++- fusermount: Installable through `fuse` (or `fuse3`) using package manager. e.g. `apt install fuse` on ubuntu.+- runc: Release binaries are available on https://github.com/opencontainers/runc/. Or you should already have this if you are using container runtimes (containerd, Docker, etc.) on your host.+- CNI plugins (if network connection is needed during optimization): Release binaries are available on https://github.com/containernetworking/plugins.++`ctr-remote` requires the root privilege, so run this command by root or with `sudo`.+Rootless execution of this command is still WIP.++For trying the examples described in this doc, you can also use the docker-compose-based demo environment.+You can setup this environment as the following commands.+*Note that this runs privileged containers on your host.*++```+$ cd ${GOPATH}/src/github.com/containerd/stargz-snapshotter/script/demo+$ docker-compose build containerd_demo+$ docker-compose up -d+$ docker exec -it containerd_demo /bin/bash+(inside container) # ./script/demo/run.sh+```++## Optimizing an image++The following command optimizes an (non-eStargz) image `ghcr.io/stargz-containers/golang:1.15.3-buster-org` (this is a copy of `golang:1.15.3-buster`) and pushes the result eStargz image into `registry2:5000/golang:1.15.3-esgz`.+This doesn't append workload-related configuration options (e.g. `--entrypoint`) so this optimizes the image against the default configurations baked to the image e.g. through Dockefile instructions (`ENTRYPOINT`, etc) when building the original image.++Here, `registry2:5000` in the demo environment serves images with HTTP (not HTTPS) so we need to tell it to the `ctr-remote` command using `--plain-http` option and `http://` prefix.++```+ctr-remote image optimize --plain-http \+           ghcr.io/stargz-containers/golang:1.15.3-buster-org \+           http://registry2:5000/golang:1.15.3-esgz+```++When you run this command, `ctr-remote` runs the source image (`ghcr.io/stargz-containers/golang:1.15.3-buster-org`) as a container and profiles all file accesses during the execution.+Then these accessed files are marked as "prioritized" files and will be prefetched on runtime.++You can lazy-pull this image with Stargz Snapshotter.+The following example lazily pulls this image to containerd, using `ctr-remote image rpull` command (`http://` prefix isn't needed here).++```console+# ctr-remote image rpull --plain-http registry2:5000/golang:1.15.3-esgz+egistry2:5000/golang:1.15.3-esgz+fetching sha256:bd2e5983... application/vnd.oci.image.index.v1+json+fetching sha256:55aef317... application/vnd.docker.distribution.manifest.v2+json+fetching sha256:d1123476... application/vnd.docker.container.image.v1+json+# ctr-remote run --rm -t --snapshotter=stargz registry2:5000/golang:1.15.3-esgz test echo hello+hello+```++## Optimizing an image with custom configuration++You can also specify the custom workload configuration that the image is optimized against.+The following example optimizes the image against the workload running `go version` on `/bin/bash`.++```+ctr-remote image optimize --plain-http \+           --entrypoint='[ "/bin/bash", "-c" ]' --args='[ "go version" ]' \+           ghcr.io/stargz-containers/golang:1.15.3-buster-org \+           http://registry2:5000/golang:1.15.3-esgz-go-version+```++Other options are also available for configuring the workload.++|Option|Description|+---|---+|`--entrypoint`|Entrypoint of the container (in JSON array)|+|`--args`|Arguments for the entrypoint (in JSON array)|+|`--env`|Environment variables in the container|+|`--user`|User name to run the process in the container|+|`--cwd`|Working directory|+|`--period`|The time seconds during profiling the file accesses|+|`-t`or`--terminal`|Enable to interactively control the container|++## Mounting files from the host++There are several cases where sharing files from host to the container during optimization is useful.+This includes when we want to optimize an image against building a binary using compilers.++For these use-cases, you can mount files on the hosts to the container using `--mount` option.+The following example optimizes the image against the workload where Go compiler compiles "hello world" program.++First, create the following Go source file at `/etc/hello.go`.++```golang+package main++import "fmt"++func main() {+     fmt.Println("hello world")+}+```++Then you can build it inside the container by bind-mounting the file `/tmp/hello.go` on the host to this container.++```+ctr-remote image optimize --plain-http \+           --mount=type=bind,source=/tmp/hello.go,destination=/hello.go,options=bind:ro \+           --entrypoint='[ "/bin/bash", "-c" ]' --args='[ "go build -o /hello /hello.go && /hello" ]' \+           ghcr.io/stargz-containers/golang:1.15.3-buster-org \+           http://registry2:5000/golang:1.15.3-esgz-hello-world+```++The syntax of the `--mount` option is conpatible to containerd's `ctr` tool and [corresponds to the OCI Runtime Spec](https://github.com/opencontainers/runtime-spec/blob/v1.0.2/config.md#mounts).+You need to specify the following key-value pairs with comma separators.++|Field|Description|+---|---+|`type`|The type of the filesystem to be mounted|+|`destination`|The absolute path to the destination mount point in the container|+|`source`|The source of the mount|+|`options`|Mount options (separated by `:`) of the filesystem|++## Enabling CNI-based Networking++You can also gain network connection in the container during optimization, using CNI plugins.+Once you configure CNI plugins on the host, CNI-based networking can be enabled using `--cni` option.++The following example accesses to https://example.com from the container, using `bridge` CNI plugin installed in the demo environment.+```+ctr-remote image optimize --plain-http \+           --cni \+           --entrypoint='[ "/bin/bash", "-c" ]' \+           --args='[ "curl example.com" ]' \+           ghcr.io/stargz-containers/golang:1.15.3-buster-org \+           http://registry2:5000/golang:1.15.3-esgz-curl+```++If CNI plugins and configurations are installed to locations other than well-known paths (/opt/cni/bin and /etc/cni/net.d), you can tell it to `ctr-remote` using the following options.++|Option|Description|+---|---+|`--cni-plugin-config-dir`|Path to the directory where CNI plugins configurations are stored|+|`--cni-plugin-dir`|Path to the directory where CNI plugins binaries are installed|++By default, `/etc/hosts` and `/etc/resolv.conf` are configured as the following.++/etc/hosts++```+127.0.0.1	localhost+::1	localhost ip6-localhost ip6-loopback

indent

ktock

comment created time in 2 days

Pull request review commentcontainerd/stargz-snapshotter

Add document about optimization with `ctr-remote`

+# Optimize Images with `ctr-remote image optimize`++This doc describes example usages of `ctr-remote image optimize` command for converting images into eStargz.++`ctr-remote images optimize` command (call `ctr-remote` in this doc) enables users to convert an image into eStargz.+The converted image can be lazily pulled by Stargz Snapshotter which can speed up the container startup.+Because this image is backward compatible to OCI/Docker image, this can be also run by other runtimes that don't support lazy pull (e.g. Docker).++Though lazy pull speeds up the container's startup, it's possible, especially with slow NW, that the runtime performance becomes lower because reading files can induce remotely downloading file contents.

NW -> network

ktock

comment created time in 2 days

Pull request review commentcontainerd/stargz-snapshotter

Add document about optimization with `ctr-remote`

+# Optimize Images with `ctr-remote image optimize`++This doc describes example usages of `ctr-remote image optimize` command for converting images into eStargz.++`ctr-remote images optimize` command (call `ctr-remote` in this doc) enables users to convert an image into eStargz.+The converted image can be lazily pulled by Stargz Snapshotter which can speed up the container startup.+Because this image is backward compatible to OCI/Docker image, this can be also run by other runtimes that don't support lazy pull (e.g. Docker).++Though lazy pull speeds up the container's startup, it's possible, especially with slow NW, that the runtime performance becomes lower because reading files can induce remotely downloading file contents.+For mitigating this, `ctr-remote` also allows to *optimize* the image against the *workload* the image runs.+Here, workload means the configuration of the container that runs from that image, including the entrypoint program, environment variables, user etc.+This optimization is done by baking the information about files that are likely accessed during runtime (called *prioritized files*), to the image.+On runtime, Stargz Snapshotter prefetches these prioritized files before mounting the layer for making sure these files are locally accessible.+This can avoid downloading chunks on every file read and mitigate the runtime performance drawbacks.++For more details about eStargz and its optimization, refer also to the [doc about image formats](/docs/stargz-estargz.md).++## Requirements++- fusermount: Installable through `fuse` (or `fuse3`) using package manager. e.g. `apt install fuse` on ubuntu.+- runc: Release binaries are available on https://github.com/opencontainers/runc/. Or you should already have this if you are using container runtimes (containerd, Docker, etc.) on your host.+- CNI plugins (if network connection is needed during optimization): Release binaries are available on https://github.com/containernetworking/plugins.++`ctr-remote` requires the root privilege, so run this command by root or with `sudo`.+Rootless execution of this command is still WIP.++For trying the examples described in this doc, you can also use the docker-compose-based demo environment.+You can setup this environment as the following commands.+*Note that this runs privileged containers on your host.*++```+$ cd ${GOPATH}/src/github.com/containerd/stargz-snapshotter/script/demo+$ docker-compose build containerd_demo+$ docker-compose up -d+$ docker exec -it containerd_demo /bin/bash+(inside container) # ./script/demo/run.sh+```++## Optimizing an image++The following command optimizes an (non-eStargz) image `ghcr.io/stargz-containers/golang:1.15.3-buster-org` (this is a copy of `golang:1.15.3-buster`) and pushes the result eStargz image into `registry2:5000/golang:1.15.3-esgz`.+This doesn't append workload-related configuration options (e.g. `--entrypoint`) so this optimizes the image against the default configurations baked to the image e.g. through Dockefile instructions (`ENTRYPOINT`, etc) when building the original image.++Here, `registry2:5000` in the demo environment serves images with HTTP (not HTTPS) so we need to tell it to the `ctr-remote` command using `--plain-http` option and `http://` prefix.++```+ctr-remote image optimize --plain-http \+           ghcr.io/stargz-containers/golang:1.15.3-buster-org \+           http://registry2:5000/golang:1.15.3-esgz+```++When you run this command, `ctr-remote` runs the source image (`ghcr.io/stargz-containers/golang:1.15.3-buster-org`) as a container and profiles all file accesses during the execution.+Then these accessed files are marked as "prioritized" files and will be prefetched on runtime.++You can lazy-pull this image with Stargz Snapshotter.+The following example lazily pulls this image to containerd, using `ctr-remote image rpull` command (`http://` prefix isn't needed here).++```console+# ctr-remote image rpull --plain-http registry2:5000/golang:1.15.3-esgz+egistry2:5000/golang:1.15.3-esgz+fetching sha256:bd2e5983... application/vnd.oci.image.index.v1+json+fetching sha256:55aef317... application/vnd.docker.distribution.manifest.v2+json+fetching sha256:d1123476... application/vnd.docker.container.image.v1+json+# ctr-remote run --rm -t --snapshotter=stargz registry2:5000/golang:1.15.3-esgz test echo hello+hello+```++## Optimizing an image with custom configuration++You can also specify the custom workload configuration that the image is optimized against.+The following example optimizes the image against the workload running `go version` on `/bin/bash`.++```+ctr-remote image optimize --plain-http \+           --entrypoint='[ "/bin/bash", "-c" ]' --args='[ "go version" ]' \+           ghcr.io/stargz-containers/golang:1.15.3-buster-org \+           http://registry2:5000/golang:1.15.3-esgz-go-version+```++Other options are also available for configuring the workload.++|Option|Description|+---|---+|`--entrypoint`|Entrypoint of the container (in JSON array)|+|`--args`|Arguments for the entrypoint (in JSON array)|+|`--env`|Environment variables in the container|+|`--user`|User name to run the process in the container|+|`--cwd`|Working directory|+|`--period`|The time seconds during profiling the file accesses|+|`-t`or`--terminal`|Enable to interactively control the container|++## Mounting files from the host++There are several cases where sharing files from host to the container during optimization is useful.+This includes when we want to optimize an image against building a binary using compilers.++For these use-cases, you can mount files on the hosts to the container using `--mount` option.+The following example optimizes the image against the workload where Go compiler compiles "hello world" program.++First, create the following Go source file at `/etc/hello.go`.

I guess you meant /tmp/hello.go here

ktock

comment created time in 2 days

more