profile
viewpoint

RHEmployee/fmf 0

Flexible Metadata Format

RHEmployee/kernel_settings 0

Role to use to set kernel tuning parameters

RHEmployee/tmt 0

Test Management Tool

RHEmployee/tuned-1 0

Tuning Profile Delivery Mechanism for Linux

pull request commentpsss/tmt

Add support for ssh multiplexing

Hi. Thank you very much! Great inspirative changes.

RHEmployee

comment created time in 3 days

push eventRHEmployee/tmt

Robin Hack

commit sha 53335cecd6aa65a81829b67f5da4b40cbefc2f79

SSH multiplexing support.

view details

push time in a month

Pull request review commentpsss/tmt

WIP: SSH multiplexing support.

 def _ssh_options(self, join=False):                 options.extend(['-i', shlex.quote(key) if join else key])         if self.password:             options.extend(['-oPasswordAuthentication=yes'])++        if not self._ssh_socket_path:+            self._ssh_socket_path = f"/run/user/{os.getuid()}/tmt/" + next(+                tempfile._get_candidate_names())

Hi. Looks like traceback is gone with last update. Race condition again. Would you please take a look if traceback is really gone in your setup? Thank you.

RHEmployee

comment created time in a month

PullRequestReviewEvent

push eventRHEmployee/tmt

Robin Hack

commit sha 17c6ae6b60894dd91d4c79ae61afd56da539f5af

SSH multiplexing support.

view details

push time in a month

push eventRHEmployee/tmt

Robin Hack

commit sha 41a023798e18d6e49db0681baa7134bb2644333e

SSH multiplexing support.

view details

push time in a month

push eventRHEmployee/tuned-1

Robin Hack

commit sha 0ed7e72ea960660c4eb6ee54744a9c6659959571

beakerlibtests: add new test. Add slowdown for centos stream - systemd failure: Active: failed (Result: start-limit-hit)

view details

push time in a month

Pull request review commentpsss/tmt

WIP: SSH multiplexing support.

 def _ssh_options(self, join=False):                 options.extend(['-i', shlex.quote(key) if join else key])         if self.password:             options.extend(['-oPasswordAuthentication=yes'])++        if not self._ssh_socket_path:+            self._ssh_socket_path = f"/run/user/{os.getuid()}/tmt/" + next(+                tempfile._get_candidate_names())

Yes. Background switch has been removed according to discussion.

RHEmployee

comment created time in a month

PullRequestReviewEvent

push eventRHEmployee/tmt

Robin Hack

commit sha c17c0a98959aa2fe675950b1336d09e2b7406739

SSH multiplexing support.

view details

push time in a month

Pull request review commentpsss/tmt

WIP: SSH multiplexing support.

 def _ssh_options(self, join=False):                 options.extend(['-i', shlex.quote(key) if join else key])         if self.password:             options.extend(['-oPasswordAuthentication=yes'])++        if not self._ssh_socket_path:+            self._ssh_socket_path = f"/run/user/{os.getuid()}/tmt/" + next(+                tempfile._get_candidate_names())++        if force_init_ssh_plex:+            options.extend(['-S', self._ssh_socket_path, '-MNnT'])+        else:+            options.extend(['-S', self._ssh_socket_path])+         return ' '.join(options) if join else options -    def _ssh_command(self, join=False):+    def _ssh_establish_master_connectoin(self):+        self._ssh_multiplex_init_done = True+        os.makedirs(f"/run/user/{os.getuid()}/tmt", exist_ok=True)+        command = self._ssh_command(+            join=True, force_init_ssh_plex=True) + ' ' + self._ssh_guest()+        self.run(command, background=True)++    def _ssh_command(self, join=False, force_init_ssh_plex=False):

_ssh_options() is called twice during establishing master connection. When master connection is not established, then _ssh_command() is invoked within _ssh_establish_master_connection(). This means that some signaling must be in place otherwise _ssh_establish_master_connection() will be called again (if check for _ssh_multiplex_init_done).

Then _ssh_command() continues and reaches point when master connection is established with special switches. I exploiting _ssh_option() for this purpuse to inject parameters to ssh.

Then master connection is established and we are back inside original _ssh_command which continues but now without any special magic passed to ssh_options().

Scheme:

  1. _ssh_command() is called from code (doesn't matter from where)
  2. check for self._ssh_multiplex_init_done is done and it's false for first call
  3. first call: we call _ssh_establish_master_connection()
  4. inside _ssh_establish_matster_connection(): we call _ssh_command() again but now we change way how _ssh_options() should behave
  5. _ssh_command: we pass first check because we already changed _ssh_multiplex_init_done. But now we need to tell _ssh_options what we want to return and in this case we just want arguments for establishing master connection.
  6. _ssh_options(force_init_ssh_plex=true) returns switches for master connection itself.
  7. now ssh connection is established (should be :)).
  8. then we are back where we started: inside _ssh_command() from point 1. And we continue like nothing happened... So we just indicate, that we no longer need parameters for master connection.
RHEmployee

comment created time in 2 months

PullRequestReviewEvent

Pull request review commentpsss/tmt

WIP: SSH multiplexing support.

 def _ssh_options(self, join=False):                 options.extend(['-i', shlex.quote(key) if join else key])         if self.password:             options.extend(['-oPasswordAuthentication=yes'])++        if not self._ssh_socket_path:+            self._ssh_socket_path = f"/run/user/{os.getuid()}/tmt/" + next(+                tempfile._get_candidate_names())

Hi. I'm able to reproduce it with tmt run provision -h virtual login finish but it still looks like different problem. And I'm not sure where ssh port number comes from so I can investigate it further.

RHEmployee

comment created time in 2 months

PullRequestReviewEvent

Pull request review commentpsss/tmt

WIP: SSH multiplexing support.

 def _ssh_options(self, join=False):                 options.extend(['-i', shlex.quote(key) if join else key])         if self.password:             options.extend(['-oPasswordAuthentication=yes'])++        if not self._ssh_socket_path:+            self._ssh_socket_path = f"/run/user/{os.getuid()}/tmt/" + next(+                tempfile._get_candidate_names())++        if force_init_ssh_plex:+            options.extend(['-S', self._ssh_socket_path, '-MNnT'])+        else:+            options.extend(['-S', self._ssh_socket_path])+         return ' '.join(options) if join else options -    def _ssh_command(self, join=False):+    def _ssh_establish_master_connectoin(self):+        self._ssh_multiplex_init_done = True+        os.makedirs(f"/run/user/{os.getuid()}/tmt", exist_ok=True)+        command = self._ssh_command(+            join=True, force_init_ssh_plex=True) + ' ' + self._ssh_guest()+        self.run(command, background=True)++    def _ssh_command(self, join=False, force_init_ssh_plex=False):

This part is not nice. However there is chicken egg problem.

When _ssh_command is invoked, there is need to establish master connection first. However, when _ssh_establish_master_connection is invoked it needs again to call _ssh_command which then needs to skip another call to _ssh_establish_master_connection (endless loop) however there is catch within _ssh_options which just needs to provide right parameters when initial ssh master connection is established and after that it's just need to ignore it.

There is question if we want to have possibility of not to use ssh multiplexing so then self.ssh_socket_path should be used in _ssh_options, to detect if there multiplexing has been used.

RHEmployee

comment created time in 2 months

PullRequestReviewEvent

push eventRHEmployee/tmt

Robin Hack

commit sha f2ec5864fb3477f1437413a80f5a9e502accf346

SSH multiplexing support.

view details

push time in 2 months

PullRequestReviewEvent

Pull request review commentpsss/tmt

WIP: SSH multiplexing support.

 def _ssh_options(self, join=False):                 options.extend(['-i', shlex.quote(key) if join else key])         if self.password:             options.extend(['-oPasswordAuthentication=yes'])++        if not self._ssh_socket_path:+            self._ssh_socket_path = f"/run/user/{os.getuid()}/tmt/" + next(+                tempfile._get_candidate_names())

tmt run provision login


But got a traceback:

File "/home/psss/git/tmt/tmt/steps/provision/init.py", line 280, in _ssh_command self._ssh_establish_master_connectoin() File "/home/psss/git/tmt/tmt/steps/provision/init.py", line 275, in _ssh_establish_master_connectoin self.run(command, background=True) File "/home/psss/git/tmt/tmt/utils.py", line 447, in run return self._run( File "/home/psss/git/tmt/tmt/utils.py", line 332, in _run descriptors = [process.stdout.fileno(), process.stderr.fileno()] AttributeError: 'NoneType' object has no attribute 'fileno'


Could you have a look a this? Please, also rebase on the latest master. Thanks.

Hi. It's kinda strange. With: $ PYTHONPATH="$(pwd)" ./bin/tmt -vvv -ddd run provision -h minute login

$ PYTHONPATH="$(pwd)" tmt run provision login I just did rebase to current master without any issues.

So it looks like bug of tmt itself maybe discovered by different use. When machine just doesn't have ssh port (tcp port 22) open, then this happens.

Example: Try to boot a new openstack machine. Wait for an ssh connection to the machine. Execute command 'whoami' on guest '10.0.142.176'. Run command 'ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oIdentitiesOnly=yes -i /usr/share/qa-tools/1minutetip/1minutetip -S /run/user/20787/tmt/7igj8rbe -MNnT root@10.0.142.176'. Write file '/var/tmp/tmt/run-034/plans/features/advanced/provision/step.yaml'. Write file '/var/tmp/tmt/run-034/plans/features/advanced/provision/guests.yaml'. Traceback (most recent call last): File "/home/rhack/devel/tmt/./bin/tmt", line 8, in <module> tmt.cli.main()

Then I figured out that machine have ssh closed and this raised. When I used: $ tmt run provision login and have openssh running, it worked fine.

Is using of an internal method tempfile._get_candidate_names() safe? I guess the under-the-hood implemenation is not granted and could change? It depends, which safety do you need. I found _get_candidate_names() used in wild and because I just need to generate some random name for socket file which is later handled by openssh itself, I'm not much take care about atomicity of file operations as such. If you know better alternative how not to use "semi" private _python stuff, I will be happy to use it :). But I can take a look again.

RHEmployee

comment created time in 2 months

push eventRHEmployee/tmt

Petr Šplíchal

commit sha d5ba2214d136cc1b52b5388fc6ac2c594cfc0450

Extend .gitignore with a few more common patterns

view details

František Nečas

commit sha 8ff4648d4dddaf507c7ea4a28726d39d76be1d96

Add more ignored files, categorize gitignore more Signed-off-by: František Nečas <fnecas@redhat.com>

view details

František Nečas

commit sha 7341ab1de0abb4b62360e59e8222bbd2b5c6a5a5

Produce better errors for unsupported plugins Previously, if an unsupported plugin was used on the command line along with its options (e.g. tmt run provision -h virtual -i image), the plugin option would be assigned by click to provision instead resulting in a confusing message (Unknown option for provision). This was an issue with provision and report since they have installable plugins. Give installation hints also when the step method is passed through CLI, not just through L2 metadata. Signed-off-by: František Nečas <fnecas@redhat.com>

view details

Petr Šplíchal

commit sha c3c549e448329776f7ad39ea6e82ec62cb18b20b

Simplify the search for step method options Stop at the first argument which looks like the next run subcommand. We cannot properly check for possible other plugin options as the step method is still to be decided.

view details

Petr Šplíchal

commit sha 677fb7b38198c228e498db500b971d3eb43001cf

Second chapter of the Guide: Under The Hood Describe the essential terms such as tests, plans, stories. Explain the core attributes common across all levels. Demonstrate on a couple of simple examples. Resolves #739.

view details

Miroslav Vadkerti

commit sha 395c677caa116d77ec5859c6ec3021ec4b564ce4

Install `rsync` before guest.pull() @mruprich reported that after running installability test the `tmt` fails on missing rsync when pulling out artifacts. After investiation, it turned out, installability for `rsync` removes `rsync` from the test machine, but `tmt` still expects it to be around. This tries to fix the problem and make sure `rsync` is still installed on the machine before trying to pull out the artifacts. Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>

view details

Petr Šplíchal

commit sha 19ebb1a93267b523bbc4763e9c56b3557a617127

Adjust the check for rsync before pull and push Fix the test not to run the whole test suite and clean up. Let's use a private method for the rsync check. Plus a couple of minor style adjustments.

view details

Petr Šplíchal

commit sha affe79f536ddb8b322b37a7fdd0b39d682b40cff

Improve fetching remote environment files Currently using a remote file to fetch environment variables would only work if preceeded by a local file because `full_path` was not initialized. Fix the bug and also improve error handling for urls.

view details

František Nečas

commit sha c0d4e9476e5004583f7b06401c70de6ebb03b5eb

Add Github Action for PyPI releases Signed-off-by: František Nečas <fnecas@redhat.com>

view details

Lukas Zachar

commit sha 9c469078eddf3c85a99fa79ee9ec2099c65c85cb

Explicitly mention '.' as special value for names

view details

Petr Šplíchal

commit sha c253c615a9670c636fcde7ddffc39cb3b510118f

Document how to create a new minor/major release Covers all steps needed to create and new minor or major `tmt` release for the GitHub project, pypi, quay and Fedora repos.

view details

Lukas Zachar

commit sha cd0561ad88c9d4266cd876ffedbf0834bac7bb39

Remove the minute.obsolete provision plugin Hint with internal plugin rpm name is printed

view details

František Nečas

commit sha 845bf70549d315c82d7368003616350a37ea19b8

Rename soon-to-be deprecated resultcallback Click 8.1 will remove resultcallback in favour of result_callback. However, click 8.0 is not available in Fedora 33 and 34 yet and will not be so both versions should be supported. Signed-off-by: František Nečas <fnecas@redhat.com>

view details

Petr Šplíchal

commit sha a21483016cff4ac6959e3294078c16301c9a9778

Increase duration for tests using containers Sometimes a network glitch causes these tests take a little bit more than 5 minutes to complete (fetch image and dnf metadata).

view details

František Nečas

commit sha a2de52a0e1df5615e3835c75d7550ffdc299ecf0

Make run --force behave more expectably Previously, tmt run -f behaved differently from the other common flags (such as verbose). Not only was it inehrited into children (steps), but it also removed the run workdir which could lead to surprising the user if they only wanted to rerun the selected steps, without having to specify -f to all of them manually. Make --force behaviour more in line with the other flags and add a new option --scratch that follows the behaviour of former --force (remove the whole workdir). Signed-off-by: František Nečas <fnecas@redhat.com>

view details

Ondrej Moris

commit sha 160e2db4825e640c76a15da6bcaec078b238b85b

Add the multihost testing specification Previously, specifications for plans supported only single-host testing scenario. This work adds support also for multi-host testing. Namely, provision specification was extended in order to express provisioning of various roles and both prepare and execute specifications were extended so that it can be (optionally) expressed on which multi-host role that step should happen (by default on all). Signed-off-by: Ondrej Moris <omoris@redhat.com>

view details

Petr Šplíchal

commit sha 0dd218804d0cf4c591373189179dfc0914768754

Adjust the multihost test specification Clarify the usage of `name` and `role` for selecting guests. Add a couple more illustrative examples. Move all changes under the new stories only to prevent user confusion about what is already implemented and what is still a draft.

view details

Ondrej Mosnacek

commit sha f4f1e397ed2318b4b1452939bec17b7d9a3da841

Clarify and update spec for the discover step Commit 679e444913c4 ("Use the default branch in the discover fmf plugin") changed some defaults from "master" to the default branch of the origin repository, but the documentation in specs/ wasn't updated. Correct this and also clarify the semantics of modified-url and modified-ref and use the modified-* fields in the example. Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>

view details

František Nečas

commit sha 34074268db4d5d02f0c7a2916241c61b22aa3ac0

Implement option to exit after first failure Signed-off-by: František Nečas <fnecas@redhat.com>

view details

Denis Karpelevich

commit sha 29d7cf6d4706fb2b393aed8174dbe3884e6f4bf2

Use current git remote for url in fmf-id #806 Previously fmf-id always used git remote origin regardless the real remote of the current branch. The updated version will use current remote. Signed-off-by: Denis Karpelevich <dkarpele@redhat.com>

view details

push time in 2 months

push eventRHEmployee/tuned-1

Robin Hack

commit sha 5e5dd15b3d4fc315db59b3b82abb79911cd66405

beakerlibtests: add new test.

view details

push time in 2 months

push eventRHEmployee/tuned-1

Robin Hack

commit sha 00a9d5cde4a9ad2d6e71f8ed5259881695bf4c69

Tests: Be in sync with upstreaming guide.

view details

push time in 2 months

Pull request review commentredhat-performance/tuned

Tests: Be in sync with upstreaming guide.

 # # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #-#   Copyright (c) 2019 Red Hat, Inc.+#   Copyright Red Hat 

Hi Jarda. Thank you for heads up. This documentation is private so I can share url only in private.

RHEmployee

comment created time in 2 months

PullRequestReviewEvent

PR opened redhat-performance/tuned

beakerlibtests: add new test.

Add another test which seems suitable to upstream.

+92 -0

0 comment

2 changed files

pr created time in 2 months

create barnchRHEmployee/tuned-1

branch : add-tests

created branch time in 2 months

create barnchRHEmployee/tuned-1

branch : sync-with-upstr-guide

created branch time in 2 months

fork RHEmployee/tuned-1

Tuning Profile Delivery Mechanism for Linux

fork in 2 months

push eventRHEmployee/tuned

RHEmployee

commit sha 5b462372a1000c2e9b0caac35d1dd738275bf309

Merge pull request #1 from RHEmployee/master Another initial commit which contains library itself.

view details

Petr Šplíchal

commit sha 0f7b4880236a8dae999c7d76f7dda36872d749ae

Fix docs, remove generated error from README

view details

RHEmployee

commit sha 53c67ac9c4c64d4d2639a2c513bee2a036b26ef5

Merge pull request #2 from beakerlib/docs Fix docs, remove generated error from README

view details

push time in 2 months

more