profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/jtratner/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

berdario/pew 1108

A tool to manage multiple virtual environments written in pure python

counsyl/stor 30

A cross-compatible CLI and Python API for accessing block and object storage

jtratner/dotfiles 4

jtratner's dotfiles, feat. vim and zsh -- borrowing heavily from awesome dotfiles makers like @holman, sontek and natw

jtratner/django-turtle-shell 1

An easy little wrapper for shell scripts :)

jtratner/dotvim 1

easy-to-use, powerful (and maybe fun?) vim setup [mirror-ish of jtratner's dotfiles]

counsyl/schematics 0

Python Data Structures for Humans™.

counsyl/staged-recipes 0

A place to submit conda recipes before they become fully fledged conda-forge feedstocks

jtratner/angular-chart.js 0

Reactive, responsive, beautiful charts for AngularJS using Chart.js: http://jtblin.github.io/angular-chart.js

jtratner/AttrDict 0

A dictionary that allows attribute-style access.

jtratner/baya 0

Hierarchical LDAP groups for authorization in django

push eventjtratner/django-turtle-shell

Jeff Tratner

commit sha bd892ed090004919dd744dc21d028007b8364098

Update README for better graphql explanation

view details

push time in a month

pull request commentdnanexus/dx-toolkit

Apps 646 dx archive/unarchive

What happens if you have multiple projects listed? Throw an error?

Do folders get resolved to individual files? Listed in folders key? What about if there is both a file AND a folder at the same name in DNAnexus?

On Thu, Jul 29, 2021 at 3:11 PM Yuxin Shi ***@***.***> wrote:

**@. commented on this pull request.

In src/python/dxpy/scripts/dx.py https://github.com/dnanexus/dx-toolkit/pull/695#discussion_r679418785:

  •    paths = [split_unescaped(':', path, include_empty_strings=True) for path in args.path]
    
  •    possible_projects = set()
    
  •    possible_objects = set()
    
  •    for p in paths:
    
  •        if len(p)>2:
    
  •            err_exit("Path '{}' is invalid. Please check the inputs or check --help for example inputs.".format(p), code=3)
    
  •        elif len(p) == 2:
    
  •            possible_projects.add(p[0])
    
  •        possible_objects.add(p[-1])
    

Updates on getting the paths:

  • parse all paths and throws errors in there is an invalid one like "proj-xxx:mistyped:filename"
  • get one single project ID as the target. If a path is given as ":file-id", assume the project to be the current project context. Return NONE if we could not find one (will throw an error in later validation).
  • gather objects in the target project, resolve filenames if needed

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dnanexus/dx-toolkit/pull/695#pullrequestreview-718445010, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMGHK4TWX5RSIVXHICDAT3T2GRXLANCNFSM464WIVXQ .

YuxinShi0423

comment created time in 2 months

Pull request review commentdnanexus/dx-toolkit

Apps 646 dx archive/unarchive

 def publish(args):     except:         err_exit() +def archive(args):+    dx_archive_errors = [InvalidState, ResourceNotFound, PermissionDenied]++    # resolve paths+    target_project = None    +    target_folder = None+    target_files = []+    for path in args.path:+        try: +            project_id, folderpath, file_results = resolve_existing_path(path, expected_classes=None, allow_mult=True, all_mult=args.all)+        except ResolutionError as e:+            err_exit('Could not resolve "' + path + '": ' + e.msg, code=3, arg_parser=parser_archive)

tho that's only if python 3.6+

YuxinShi0423

comment created time in 2 months

PullRequestReviewEvent

Pull request review commentdnanexus/dx-toolkit

Apps 646 dx archive/unarchive

 def publish(args):     except:         err_exit() +def archive(args):+    dx_archive_errors = [InvalidState, ResourceNotFound, PermissionDenied]++    # resolve paths+    target_project = None    +    target_folder = None+    target_files = []+    for path in args.path:+        try: +            project_id, folderpath, file_results = resolve_existing_path(path, expected_classes=None, allow_mult=True, all_mult=args.all)+        except ResolutionError as e:+            err_exit('Could not resolve "' + path + '": ' + e.msg, code=3, arg_parser=parser_archive)

Just nit here

            err_exit(f'Could not resolve {repr(path)}:  {type(e).__name__}:{e}', code=3, arg_parser=parser_archive)
YuxinShi0423

comment created time in 2 months

PullRequestReviewEvent

issue openeddnanexus/dx-toolkit

get_download_url should never raise SystemExit

Raising a systemexit is really problematic because hard to catch. Really should throw an exception that the dx CLI tool converts to a systemexit.

How to reproduce:

  1. Get file ID and project ID that you do not have access to
  2. dxpy.DXFile(file_id, project_id).get_download_url()
home/deploy/app/config/2021-07-01.005634_010aac/repo/.venv/lib/python3.6/site-packages/dxpy/cli/__init__.py in try_call(func, *args, **kwargs)
     35     try:
---> 36         return func(*args, **kwargs)
     37     except:

/home/deploy/app/config/2021-07-01.005634_010aac/repo/.venv/lib/python3.6/site-packages/dxpy/__init__.py in DXHTTPRequest(resource, data, method, headers, auth, timeout, use_compression, jsonify_data, want_full_response, decode_response_body, prepend_srv, session_handler, max_retries, always_retry, **kwargs)
    681                         raise exceptions.HTTPError(response.status, content)
--> 682                     raise error_class(content, response.status, time_started, req_id)
    683                 else:

PermissionDenied: VIEW permission required in project-FbPGg5j06GZ3Ybx4KY7100g5 to perform this action, code 401. Request Time=1625101595.2516103, Request ID=1625101595488-323642

During handling of the above exception, another exception occurred:

SystemExit                                Traceback (most recent call last)
<ipython-input-12-cf196c92df9b> in <module>()
----> 1 dxf.get_download_url()

/home/deploy/app/config/2021-07-01.005634_010aac/repo/.venv/lib/python3.6/site-packages/dxpy/bindings/dxfile.py in get_download_url(self, duration, preauthenticated, filename, project, **kwargs)
    778         if project is None and 'DX_JOB_ID' not in os.environ:
    779             project_from_handler = self.get_proj_id()
--> 780             if project_from_handler and object_exists_in_project(self.get_id(), project_from_handler):
    781                 project = project_from_handler
    782

/home/deploy/app/config/2021-07-01.005634_010aac/repo/.venv/lib/python3.6/site-packages/dxpy/utils/resolver.py in object_exists_in_project(obj_id, proj_id)
    209     if not is_container_id(proj_id):
    210         raise ValueError('Expected %r to be a container ID' % (proj_id,))
--> 211     return try_call(dxpy.DXHTTPRequest, '/' + obj_id + '/describe', {'project': proj_id})['project'] == proj_id
    212
    213

/home/deploy/app/config/2021-07-01.005634_010aac/repo/.venv/lib/python3.6/site-packages/dxpy/cli/__init__.py in try_call(func, *args, **kwargs)
     36         return func(*args, **kwargs)
     37     except:
---> 38         try_call_err_exit()
     39
     40 def prompt_for_yn(prompt_str, default=None):

/home/deploy/app/config/2021-07-01.005634_010aac/repo/.venv/lib/python3.6/site-packages/dxpy/cli/__init__.py in try_call_err_exit()
     30
     31 def try_call_err_exit():
---> 32     err_exit(expected_exceptions=default_expected_exceptions + (DXError,))
     33
     34 def try_call(func, *args, **kwargs):

/home/deploy/app/config/2021-07-01.005634_010aac/repo/.venv/lib/python3.6/site-packages/dxpy/exceptions.py in err_exit(message, code, expected_exceptions, arg_parser, ignore_sigpipe, exception)
    259         raise exc
    260     elif isinstance(exc, expected_exceptions):
--> 261         exit_with_exc_info(EXPECTED_ERR_EXIT_STATUS, message, print_tb=dxpy._DEBUG > 0, exception=exception)
    262     elif ignore_sigpipe and isinstance(exc, IOError) and getattr(exc, 'errno', None) == errno.EPIPE:
    263         if dxpy._DEBUG > 0:

/home/deploy/app/config/2021-07-01.005634_010aac/repo/.venv/lib/python3.6/site-packages/dxpy/exceptions.py in exit_with_exc_info(code, message, print_tb, exception)
    218     if message != '' and not message.endswith('\n'):
    219         sys.stderr.write('\n')
--> 220     sys.exit(code)
    221
    222 network_exceptions = (requests.packages.urllib3.exceptions.ProtocolError,

created time in 3 months

pull request commentdnanexus/dx-toolkit

Apps 646 dx archive/unarchive

Even if API doesn't have --dry-run in it, you can still fake out --dry-run as you've said, more about consistency of the CLI command that you're presenting others

YuxinShi0423

comment created time in 3 months

pull request commentdnanexus/dx-toolkit

Apps 646 dx archive/unarchive

small thought: why not drop the --dry-run flag, have -y --yes flag for both and then if you run either archive or unarchive, run --dry-run first if -y isn't specified and then prompt? That way default is to do nothing, and normal path is interactive prompt:

$ dx archive project-123:/folder
Would archive 256 files totalling 500GB - continue? [Y/n] Y
Archived 256 files.
$ dx unarchive project-123:/folder
Would unarchive 256 files at a cost of $X - continue? [Y/n] Y
Unarchived 256 files...

or:

dx archive -y project-123:/folder
Archived 256 files
YuxinShi0423

comment created time in 3 months