profile
viewpoint

karrtikr/code-bank 0

My day to day implementations in Data structures, Bash, Verilog, Octave, Latex and more..

karrtikr/depends 0

The script powering http://apps.himanshumishra.in/depends

karrtikr/documentation 0

Auto-generated documentation builds from the networkx/networkx/doc Sphinx sources.

karrtikr/ete 0

ETE is a Python programming toolkit for building, comparing, annotating, manipulating and visualising trees. It provides both a comprehensive API and a collection of command line tools, including utilities to work with the NCBI taxonomy tree.

karrtikr/fos-proposals 0

:gift_heart: :penguin: Expressing love for Open Source

issue commentmicrosoft/vscode

API: Allow to use the file watcher for aribitrary folders

Is https://github.com/microsoft/vscode/issues/100870 a dup of this?

innerlee

comment created time in 11 hours

issue commentmicrosoft/vscode

Support file watching outside of the workspace

Is https://github.com/microsoft/vscode/issues/3025 dup of this?

stamblerre

comment created time in 11 hours

delete branch karrtikr/vscode-python

delete branch : completeles

delete time in 11 hours

push eventmicrosoft/vscode-python

Kartik Raj

commit sha 25fd710af55bc572310c94e6e96dbb688a831c5b

Update the initialization of the PythonEnvInfoCache to pass in a function that checks for env info completeness (#14446) * Update the initialization of the PythonEnvInfoCache to pass in a function that checks for env info completeness * Update interface

view details

push time in 11 hours

PR merged microsoft/vscode-python

Update the initialization of the PythonEnvInfoCache to pass in a function that checks for env info completeness skip news

EnvCache needs to know if interpreterInfo.py has been run or not.

+43 -2

2 comments

3 changed files

karrtikr

pr closed time in 11 hours

issue commentmicrosoft/vscode-python

Python extension not loading on remote-ssh (on windows only)

Okay some progress. The initial error message states that the extension is apparently not installed/enabled as suspected. And you say that it was disabled by default on remote, which is likely why you got the issue when debugging.

The errors you get after enabling the extension seems to because you're unable to connect to remote os and has nothing to do with the extension. Please open up an issue on vscode-remote where you may get help regarding that.

Once you get that working, and the extension is enabled successfully, let me know. FWIW for me the following appears on Remote SSH and the extension is working fine. You can notice Python extension is installed in both tabs.

image

aabbas90

comment created time in 13 hours

push eventkarrtikr/vscode-python

Kartik Raj

commit sha 99dcb2ec56435cc3027115bb130be2b282e61879

Update interface

view details

push time in a day

create barnchkarrtikr/vscode-python

branch : completeles

created branch time in a day

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import { chain, iterable } from '../../../../common/utils/async';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'envs'));+        dirPaths.push(path.join(homeDir, '.direnv'));+        dirPaths.push(path.join(homeDir, '.venvs'));+        dirPaths.push(path.join(homeDir, '.virtualenvs'));+        dirPaths.push(path.join(homeDir, '.local', 'share', 'virtualenvs'));+    }++    const exists = await Promise.all(dirPaths.map(pathExists));+    exists.forEach((v, i) => {+        if (v) {+            venvDirs.push(dirPaths[i]);+        }+    });++    return venvDirs;+}++/**+ * Gets the virtual environment kind for a given interpreter path.+ * This only checks for environments created using venv, virtualenv,+ * and virtualenvwrapper based environments.+ * @param interpreterPath: Absolute path to the interpreter paths.+ */+async function getVirtualEnvKind(interpreterPath:string): Promise<PythonEnvKind> {+    if (await isVenvEnvironment(interpreterPath)) {+        return PythonEnvKind.Venv;+    }++    if (await isVirtualenvwrapperEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnvWrapper;+    }++    if (await isVirtualenvEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnv;+    }++    return PythonEnvKind.Unknown;+}++/**+ * Takes absolute path to the interpreter and environment kind to build+ * environment info.+ * @param {string} interpreterPath: Absolute path to interpreter.+ * @param {PythonEnvKind} kind: Kind for the given environment.+ */+async function buildEnvInfo(interpreterPath:string, kind:PythonEnvKind): Promise<PythonEnvInfo> {+    let version:PythonVersion;+    try {+        version = parseVersion(path.basename(interpreterPath));+    } catch (ex) {+        traceError(`Failed to parse version from path: ${interpreterPath}`, ex);+        version = UNKNOWN_PYTHON_VERSION;+    }+    return {+        name: '',+        location: '',+        kind,+        executable: {+            filename: interpreterPath,+            sysPrefix: '',+            ...(await getFileInfo(interpreterPath)),+        },+        version,+        arch: Architecture.Unknown,+        distro: { org: '' },+    };+}++/**+ * Finds and resolves virtual environments created in known global locations.+ */+export class GlobalVirtualEnvironmentLocator extends PythonEnvsWatcher implements ILocator {+    private virtualEnvKinds = [+        PythonEnvKind.Venv,+        PythonEnvKind.VirtualEnv,+        PythonEnvKind.VirtualEnvWrapper,+    ];++    public constructor(private readonly searchDepth?:number) {

I just recalled we also have a python.venvFolders setting which allows users to define additional virtual env dirs they wish to search in. How to incorporate that?

karthiknadig

comment created time in 2 days

PullRequestReviewEvent

issue commentmicrosoft/vscode-python

Python extension not loading on remote-ssh (on windows only)

I see, this seems really weird. Can you please try deleting the extension directory folder for VSCode in both your remote machine and host OS machine (for eg. /Users/***/.vscode/extensions/ms-python.python-2020.***/), and try reinstalling the Python extension on both machines?

aabbas90

comment created time in 2 days

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import { chain, iterable } from '../../../../common/utils/async';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'envs'));+        dirPaths.push(path.join(homeDir, '.direnv'));+        dirPaths.push(path.join(homeDir, '.venvs'));+        dirPaths.push(path.join(homeDir, '.virtualenvs'));+        dirPaths.push(path.join(homeDir, '.local', 'share', 'virtualenvs'));+    }++    const exists = await Promise.all(dirPaths.map(pathExists));+    exists.forEach((v, i) => {+        if (v) {+            venvDirs.push(dirPaths[i]);+        }+    });++    return venvDirs;+}++/**+ * Gets the virtual environment kind for a given interpreter path.+ * This only checks for environments created using venv, virtualenv,+ * and virtualenvwrapper based environments.+ * @param interpreterPath: Absolute path to the interpreter paths.+ */+async function getVirtualEnvKind(interpreterPath:string): Promise<PythonEnvKind> {+    if (await isVenvEnvironment(interpreterPath)) {+        return PythonEnvKind.Venv;+    }++    if (await isVirtualenvwrapperEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnvWrapper;+    }++    if (await isVirtualenvEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnv;+    }++    return PythonEnvKind.Unknown;+}++/**+ * Takes absolute path to the interpreter and environment kind to build+ * environment info.+ * @param {string} interpreterPath: Absolute path to interpreter.+ * @param {PythonEnvKind} kind: Kind for the given environment.+ */+async function buildEnvInfo(interpreterPath:string, kind:PythonEnvKind): Promise<PythonEnvInfo> {

Could possibly be a general method and not specific to this locator.

karthiknadig

comment created time in 2 days

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import { chain, iterable } from '../../../../common/utils/async';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'envs'));+        dirPaths.push(path.join(homeDir, '.direnv'));+        dirPaths.push(path.join(homeDir, '.venvs'));+        dirPaths.push(path.join(homeDir, '.virtualenvs'));+        dirPaths.push(path.join(homeDir, '.local', 'share', 'virtualenvs'));

I only mentioned path.join(homeDir, '.local', 'share', 'virtualenvs') in the spike because I saw it contained Pipenv environments in Linux😅 If we're not including Pipenv, this directory might be to miscellaneous too check for this locator.

karthiknadig

comment created time in 2 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import { chain, iterable } from '../../../../common/utils/async';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'envs'));

We should also push path.join(homeDir, 'Envs') on non-Windows. (The default value of workOnHome is ~/Envs) All these are mentioned in the spike.

karthiknadig

comment created time in 2 days

PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import { chain, iterable } from '../../../../common/utils/async';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;

Given the algorithm, depth 4 can be really heavy. Note we assumed earlier that all the environments were found exactly at depth 2, and it found most (possibly all?) of the environments for us.

Doing a depth search of 3 for a directory like say Lib seems really heavy and also wasteful. I suggest changing default depth to 2.

karthiknadig

comment created time in 2 days

PullRequestReviewEvent

issue commentmicrosoft/vscode-python

python.autoComplete.extraPaths not resolve on same namespace prefix

Thanks for the bug report! We investigate issues in order based on priority and severity, which includes the impact it has on your ability to use the extension to do productive work, and the number of people affected. If other users come forward and leave a comment demonstrating they are seeing/reproducing the problem then we will raise this issue's priority.

If you think your issue is more of a question or configuration problem rather than a bug, please ask on Stack Overflow with the visual-studio-code and python labels for help.

Thanks for your understanding and patience!

lmeguira

comment created time in 2 days

issue commentmicrosoft/vscode-python

Python extension not loading on remote-ssh (on windows only)

@aabbas90 I think you don't Python extension installed on the remote Linux machine you have. IIRC it's mentioned in the docs you need to install it. Click the marketplace icon to install.

aabbas90

comment created time in 2 days

push eventkarrtikr/vscode-python

Rich Chiodo

commit sha b517aa0531ced1e6ac14876200c925fbc91fe87c

Escaping fix broke a number of things (#14145) * Fixes for escaping * Push a comment ot start PR again * Cache task is failing * Remove cache task * Not fixing so just put back cache task

view details

Don Jayamanne

commit sha d24bfc50242d2ff48e94b902fbf0fbbf3893551c

Auto assign datascience issues (#14147)

view details

Don Jayamanne

commit sha b685ef3dc3bb245e6272c6e9f21bb3d662d429c9

Change name of event used to activate extension (#14153)

view details

Don Jayamanne

commit sha 8e0e622b2cacdd72730a00f1a3d136de6fed482f

Fixes to streamed output in native notebooks (#14158) For #13611 * Fixes to streamed output * added tests

view details

Don Jayamanne

commit sha 50c3fd3d7a278d49fbf63190b4d11492dd72b3bf

Fixes to for escaping of output in native notebooks (#14159)

view details

Don Jayamanne

commit sha 6332b811dff577b383bc9715d3a34ec22df2eed3

Look for `data science` labels when auto assigning issues (#14161)

view details

Kim-Adeline Miguel

commit sha 81b6d8f11b82fb83f1ae3bc6261ccf2cecae4da3

Environment info cache class (#14065) * Add persistent storage external deps * PythonEnvInfoCache class + tests * Instantiate & initialize cache class in createAPI * Add extra test for flush() and initialize() * Env cache fixes: storage key + find() result check * Update src/client/pythonEnvironments/common/externalDependencies.ts Co-authored-by: Kartik Raj <karraj@microsoft.com> * Use areSameEnvironment in getEnv * Don't ping persistent storage for every initialize * No need to export CompleteEnvInfoFunction * PythonEnvInfoCache doc comment * Rename createGlobalPersistentStoreStub to get... * Preemptively drop id key (#14051) * Return deep copies * IPersistentStore wrapper around IPersistentState * Use correct areSameEnvironment + fix stub * Remove obsolete comment * getEnv -> filterEnvs * Remove stubbing of areSameEnvironment * Update areSameEnv * Move IPersistentStateFactory registration to registerForIOC * Revert "Move IPersistentStateFactory registration to registerForIOC" This reverts commit edc6ce5e418e9a0d5f4afd074e54d5dc0180ae3f. * Don't instantiate nor initialize cache for now Co-authored-by: Kartik Raj <karraj@microsoft.com>

view details

Brett Cannon

commit sha 411d1d4ae7e3884a8ef9294a0aa1688dc05d0f37

Update Component Governance notes in the release plan

view details

David Kutugata

commit sha 5bc4eab5f7fd0e038643cd21fe85bc7acb5bd472

Merge point release back into main (#14171) * Port ipykernel install fix to release (#13975) * Fix installing ipykernel into interpreters for raw kernel (#13959) * update news Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> * Merge in changes to release (#13976) * Up release version for new release (#13928) * Up release version * Update changelog * Update changelog * Workaround test issue (#13930) * Try different version of VS code in release * Change to make it use the actual variable * Use a real version * More tests failing with gpu error (#13935) * Try different version of VS code in release * Change to make it use the actual variable * Use a real version * Two more version changes * Fix kernel and server name missing in certain situations (#13974) * Fix kernel name and server name * Fixup server name for remote situations * Add some functional tests * Add news entry * Delete news file * Port two fixes to the release branch (#13995) * Disable split views of custom editors (#13985) * Fix backup storage by looking at the options correctly (#13983) * Fix backup storage by looking at the options correctly * Fix backup by being more explicit * Only linux tests are failing. Hopefully fix them * Fixup changelog Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> * add jedi-language-server to 3rd party notices (#13977) * add jedi-language-server to 3rd party notices * move license from distribution to repository file * disable test_discover_complex_default and (#14024) test_discover_complex_doctest * Upgrade isort to 5.5.3 (#14035) (#14037) * prepare release (#14042) * fixed annoying warnings (#14049) * Cherry pick to address path issues. (#14125) * Do not quote isolated in exec module (#14108) * Do not quote isolated in exec module * Revert "Do not quote isolated in exec module" This reverts commit b9fa04c06be0876861017eff5ee032ca71acda3d. * Revert "IPyKernel install issue with windows paths (#13667)" This reverts commit 23725abd4249811f24475e215c2d78ed5e508e79. * Fix unit test broken by recent revert (#14122) * Port escape fix to release branch (#14133) * Fix HTML escaping to match what Jupyter does (#14038) * Basic idea * add some functional tests * Add news entry * Fix functional tests * Update changelog * update version and changelog (#14139) * Escaping fix broke a number of things (#14145) (#14154) * Fixes for escaping * Push a comment ot start PR again * Cache task is failing * Remove cache task * Not fixing so just put back cache task * remove leftovers * import pytest * stop skipping python tests Co-authored-by: Ian Huff <ianhu@microsoft.com> Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> Co-authored-by: Rich Chiodo <rchiodo@users.noreply.github.com> Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> Co-authored-by: Kartik Raj <karraj@microsoft.com> Co-authored-by: Karthik Nadig <kanadig@microsoft.com>

view details

Ian Huff

commit sha cc093b02ffea0f89ce58b671e50c8487c6025067

NB Convert 6.0 support for export (#14177)

view details

Karthik Nadig

commit sha cfe12a7855d1e05d2b46365a40910337f97ce434

Windows store locator (#14162) * Initial commit for windows store locator * More tests * Simplify locator * Tweaks * Test fixes * Fix tests

view details

Rich Chiodo

commit sha afc155b47a0ceaadc452cba03dc7c817c2147a88

A different way of fixing escaping (#14186) * Move escaping to just output * Add some tests to verify round tripping * Fixup test for rountrip and make roundtripping actually work * Add news entry * Add to manual test file * Fix streaming problem and add more to the test * Fix traceback unit test * Fix problem caught by functional tests :) * Another functional test catch

view details

Don Jayamanne

commit sha 146dae0564553d1924061179445d22d8b9735d7b

Fixes to escaping of output in native notebooks (#14199)

view details

Don Jayamanne

commit sha 716e61c7ea75d87c1373b0fde74d7fb3f442c3f2

Display description in kernel picker (#14196)

view details

Brett Cannon

commit sha 0a36df0b4704923f098301bf59ddb9cd288596a7

Bump node-fetch to 2.6.1 (#14181)

view details

dependabot[bot]

commit sha a83c93d0968c28208ba52628f9cf6c0f155adce4

Bump pytest from 6.0.1 to 6.1.0 in /news (#14190) Bumps [pytest](https://github.com/pytest-dev/pytest) from 6.0.1 to 6.1.0. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/master/CHANGELOG.rst) - [Commits](https://github.com/pytest-dev/pytest/compare/6.0.1...6.1.0) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

view details

Kim-Adeline Miguel

commit sha ae84247cf8d18c72054788a54c8522d8ce3e78e5

Envs cache initialization (#14180) * Move commonRegisterTypes to initializeComponents * Instantiate and initialize envsCache * Use a separate initializeCommon function

view details

Kim-Adeline Miguel

commit sha 192b2e285e422ae714b2b13790aa6f06a1e30f95

Failing tests with spaces in path: " Activation Environment" venv tests (#14207) * Use toCommandArgument() * Run venv tests in PR validation * Add venv to mac stage * Undo pipeline changes * Custom GHA workflow to run venv tests * Move to directory with spaces * Move defaults outside (apply to entire workflow) * mark env variables as env * run tests in working directory * Delete test workflow

view details

Mojtaba Samimi

commit sha 80894e4be86c1e7d1261837208d82416a06671c4

bump plotly.js-dist v1.56.0 (#14183)

view details

Don Jayamanne

commit sha 74a4d9926a8ebee8c8d456286acffae929c3a9a0

Select kernel based on metadata in notebook (#14217) * Fix picking kernel based on metadata

view details

push time in 2 days

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as fsapi from 'fs-extra';+import * as path from 'path';+import { getOSType, OSType } from '../../common/utils/platform';+import { isPosixPythonBin } from './posixUtils';+import { isWindowsPythonExe } from './windowsUtils';++export async function* findInterpretersInDir(root:string, recurseLevels?:number): AsyncGenerator<string> {+    const dirContents = (await fsapi.readdir(root)).map((c) => path.join(root, c));+    const os = getOSType();++    for (const item of dirContents) {

https://github.com/microsoft/vscode-python/pull/14416#discussion_r506755080

karthiknadig

comment created time in 4 days

PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as fsapi from 'fs-extra';+import * as path from 'path';+import { getOSType, OSType } from '../../common/utils/platform';+import { isPosixPythonBin } from './posixUtils';+import { isWindowsPythonExe } from './windowsUtils';++export async function* findInterpretersInDir(root:string, recurseLevels?:number): AsyncGenerator<string> {+    const dirContents = (await fsapi.readdir(root)).map((c) => path.join(root, c));+    const os = getOSType();++    for (const item of dirContents) {+        const stat = await fsapi.lstat(item);+        if (stat.isDirectory()) {+            if (recurseLevels && recurseLevels > 0) {+                const subItems = findInterpretersInDir(item, recurseLevels - 1);+                for await (const subItem of subItems) {+                    yield subItem;+                }+            }+        } else if (+            (os === OSType.Windows && isWindowsPythonExe(item))+            || (os !== OSType.Windows && isPosixPythonBin(item))+        ) {+            yield item;+        }+    }+}
export async function* findInterpretersInDir(root:string, recurseLevels?:number): AsyncIterableIterator<string> {
    const dirContents = (await fsapi.readdir(root)).map((c) => path.join(root, c));
    const os = getOSType();
    const generators = dirContents.map((item) => {
        async function* generator() {
            const stat = await fsapi.lstat(item);
            if (stat.isDirectory()) {
                if (recurseLevels && recurseLevels > 0) {
                    const subItems = findInterpretersInDir(item, recurseLevels - 1);
                    for await (const subItem of subItems) {
                        yield subItem;
                    }
                }
            } else if (
                (os === OSType.Windows && isWindowsPythonExe(item))
                || (os !== OSType.Windows && isPosixPythonBin(item))
            ) {
                yield item;
            }
        }
        return generator();
    });
    yield* iterable(chain(generators));
}
karthiknadig

comment created time in 4 days

PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as fsapi from 'fs-extra';+import * as path from 'path';+import { getOSType, OSType } from '../../common/utils/platform';+import { isPosixPythonBin } from './posixUtils';+import { isWindowsPythonExe } from './windowsUtils';++export async function* findInterpretersInDir(root:string, recurseLevels?:number): AsyncGenerator<string> {+    const dirContents = (await fsapi.readdir(root)).map((c) => path.join(root, c));+    const os = getOSType();++    for (const item of dirContents) {

You can constructor a iterator out of each loop body and then chain the iterators. Does that make sense? I'll leave an example.

karthiknadig

comment created time in 4 days

PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'Envs'));

What about the others?

karthiknadig

comment created time in 4 days

PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'Envs'));+        dirPaths.push(path.join(homeDir, '.venvs'));+        dirPaths.push(path.join(homeDir, '.virtualenvs'));+    }++    const exists = await Promise.all(dirPaths.map(pathExists));+    exists.forEach((v, i) => {+        if (v) {+            venvDirs.push(dirPaths[i]);+        }+    });++    return venvDirs;+}++/**+ * Gets the virtual environment kind for a given interpreter path.+ * This only checks for environments created using venv, virtualenv,+ * and virtualenvwrapper based environments.+ * @param interpreterPath: Absolute path to the interpreter paths.+ */+async function getVirtualEnvKind(interpreterPath:string): Promise<PythonEnvKind> {+    if (await isVenvEnvironment(interpreterPath)) {+        return PythonEnvKind.Venv;+    }++    if (await isVirtualenvwrapperEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnvWrapper;+    }++    if (await isVirtualenvEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnv;+    }++    return PythonEnvKind.Unknown;+}++/**+ * Takes absolute path to the interpreter and environment kind to build+ * environment info.+ * @param {string} interpreterPath: Absolute path to interpreter.+ * @param {PythonEnvKind} kind: Kind for the given environment.+ */+async function buildEnvInfo(interpreterPath:string, kind:PythonEnvKind): Promise<PythonEnvInfo> {+    let version:PythonVersion;+    try {+        version = parseVersion(path.basename(interpreterPath));+    } catch (ex) {+        traceError(`Failed to parse version from path: ${interpreterPath}`, ex);+        version = UNKNOWN_PYTHON_VERSION;+    }+    return {+        name: '',+        location: '',+        kind,+        executable: {+            filename: interpreterPath,+            sysPrefix: '',+            ...(await getFileInfo(interpreterPath)),+        },+        version,+        arch: Architecture.Unknown,+        distro: { org: '' },+    };+}++/**+ * Finds and resolves virtual environments created in known global locations.+ */+export class GlobalVirtualEnvironmentLocator extends PythonEnvsWatcher implements ILocator {+    private virtualEnvKinds = [+        PythonEnvKind.Venv,+        PythonEnvKind.VirtualEnv,+        PythonEnvKind.VirtualEnvWrapper,+    ];++    public constructor(private readonly searchDepth?:number) {+        super();+    }++    public iterEnvs(): IPythonEnvsIterator {+        // Number of levels of sub-directories to recurse when looking for+        // interpreters+        const searchDepth = this.searchDepth ?? DEFAULT_SEARCH_DEPTH;++        const { virtualEnvKinds } = this;

Btw you can avoid this if you use something like https://github.com/microsoft/vscode-python/blob/218dd8de3ac990a3587d2cc157e6f44d18cd9dd7/src/client/pythonEnvironments/base/locators/composite/environmentsResolver.ts#L49

karthiknadig

comment created time in 5 days

PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'Envs'));+        dirPaths.push(path.join(homeDir, '.venvs'));+        dirPaths.push(path.join(homeDir, '.virtualenvs'));+    }++    const exists = await Promise.all(dirPaths.map(pathExists));+    exists.forEach((v, i) => {+        if (v) {+            venvDirs.push(dirPaths[i]);+        }+    });++    return venvDirs;+}++/**+ * Gets the virtual environment kind for a given interpreter path.+ * This only checks for environments created using venv, virtualenv,+ * and virtualenvwrapper based environments.+ * @param interpreterPath: Absolute path to the interpreter paths.+ */+async function getVirtualEnvKind(interpreterPath:string): Promise<PythonEnvKind> {+    if (await isVenvEnvironment(interpreterPath)) {+        return PythonEnvKind.Venv;+    }++    if (await isVirtualenvwrapperEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnvWrapper;+    }++    if (await isVirtualenvEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnv;+    }++    return PythonEnvKind.Unknown;+}++/**+ * Takes absolute path to the interpreter and environment kind to build+ * environment info.+ * @param {string} interpreterPath: Absolute path to interpreter.+ * @param {PythonEnvKind} kind: Kind for the given environment.+ */+async function buildEnvInfo(interpreterPath:string, kind:PythonEnvKind): Promise<PythonEnvInfo> {+    let version:PythonVersion;+    try {+        version = parseVersion(path.basename(interpreterPath));+    } catch (ex) {+        traceError(`Failed to parse version from path: ${interpreterPath}`, ex);+        version = UNKNOWN_PYTHON_VERSION;+    }+    return {+        name: '',+        location: '',+        kind,+        executable: {+            filename: interpreterPath,+            sysPrefix: '',+            ...(await getFileInfo(interpreterPath)),+        },+        version,+        arch: Architecture.Unknown,+        distro: { org: '' },+    };+}++/**+ * Finds and resolves virtual environments created in known global locations.+ */+export class GlobalVirtualEnvironmentLocator extends PythonEnvsWatcher implements ILocator {+    private virtualEnvKinds = [+        PythonEnvKind.Venv,+        PythonEnvKind.VirtualEnv,+        PythonEnvKind.VirtualEnvWrapper,+    ];++    public constructor(private readonly searchDepth?:number) {+        super();+    }++    public iterEnvs(): IPythonEnvsIterator {+        // Number of levels of sub-directories to recurse when looking for+        // interpreters+        const searchDepth = this.searchDepth ?? DEFAULT_SEARCH_DEPTH;++        const { virtualEnvKinds } = this;

Are you running TSLint/ESLint? I don't think this is allowed.

karthiknadig

comment created time in 5 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'Envs'));+        dirPaths.push(path.join(homeDir, '.venvs'));+        dirPaths.push(path.join(homeDir, '.virtualenvs'));+    }++    const exists = await Promise.all(dirPaths.map(pathExists));+    exists.forEach((v, i) => {+        if (v) {+            venvDirs.push(dirPaths[i]);+        }+    });++    return venvDirs;+}++/**+ * Gets the virtual environment kind for a given interpreter path.+ * This only checks for environments created using venv, virtualenv,+ * and virtualenvwrapper based environments.+ * @param interpreterPath: Absolute path to the interpreter paths.+ */+async function getVirtualEnvKind(interpreterPath:string): Promise<PythonEnvKind> {+    if (await isVenvEnvironment(interpreterPath)) {+        return PythonEnvKind.Venv;+    }++    if (await isVirtualenvwrapperEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnvWrapper;+    }++    if (await isVirtualenvEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnv;+    }++    return PythonEnvKind.Unknown;+}++/**+ * Takes absolute path to the interpreter and environment kind to build+ * environment info.+ * @param {string} interpreterPath: Absolute path to interpreter.+ * @param {PythonEnvKind} kind: Kind for the given environment.+ */+async function buildEnvInfo(interpreterPath:string, kind:PythonEnvKind): Promise<PythonEnvInfo> {+    let version:PythonVersion;+    try {+        version = parseVersion(path.basename(interpreterPath));+    } catch (ex) {+        traceError(`Failed to parse version from path: ${interpreterPath}`, ex);+        version = UNKNOWN_PYTHON_VERSION;+    }+    return {+        name: '',+        location: '',+        kind,+        executable: {+            filename: interpreterPath,+            sysPrefix: '',+            ...(await getFileInfo(interpreterPath)),+        },+        version,+        arch: Architecture.Unknown,+        distro: { org: '' },+    };+}++/**+ * Finds and resolves virtual environments created in known global locations.+ */+export class GlobalVirtualEnvironmentLocator extends PythonEnvsWatcher implements ILocator {+    private virtualEnvKinds = [+        PythonEnvKind.Venv,+        PythonEnvKind.VirtualEnv,+        PythonEnvKind.VirtualEnvWrapper,+    ];++    public constructor(private readonly searchDepth?:number) {+        super();+    }++    public iterEnvs(): IPythonEnvsIterator {+        // Number of levels of sub-directories to recurse when looking for+        // interpreters+        const searchDepth = this.searchDepth ?? DEFAULT_SEARCH_DEPTH;++        const { virtualEnvKinds } = this;+        const iterator = async function* () {+            const envRootDirs = await getGlobalVirtualEnvDirs();++            for (const envRootDir of envRootDirs) {+                traceVerbose(`Searching for global virtual envs in: ${envRootDir}`);++                const generator = findInterpretersInDir(envRootDir, searchDepth);++                for await (const env of generator) {+                    // We only care about python.exe (on windows) and python (on linux/mac)+                    // Other version like python3.exe or python3.8 are often symlinks to+                    // python.exe or python in the same directory in the case od virtual+                    // environments.+                    const name = path.basename(env).toLowerCase();+                    if (name === 'python.exe' || name === 'python') {+                        // We should extract the kind here to avoid doing is*Environment()+                        // check multiple times. Those checks are file system heavy and+                        // we can use the kind to determine this anyway.+                        const kind = await getVirtualEnvKind(env);++                        if (virtualEnvKinds.includes(kind)) {+                            traceVerbose(`Global Virtual Environment: [added] ${env}`);++                            yield buildEnvInfo(env, kind);+                        } else {+                            traceVerbose(`Global Virtual Environment: [skipped] ${env}`);+                        }+                    } else {+                        traceVerbose(`Global Virtual Environment: [skipped] ${env}`);+                    }+                }+            }+        };++        return iterator();+    }++    public async resolveEnv(env: string | PythonEnvInfo): Promise<PythonEnvInfo | undefined> {+        const executablePath = typeof env === 'string' ? env : env.executable.filename;

This will all go away right? As I pointed out earlier, resolveEnv should only build on top of env. Only env.executable.filename is collected and the rest are discarded which is wrong. The returned environment should only append additional information.

But if we plan to remove these later it doesn't matter.

karthiknadig

comment created time in 5 days

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'Envs'));+        dirPaths.push(path.join(homeDir, '.venvs'));+        dirPaths.push(path.join(homeDir, '.virtualenvs'));+    }++    const exists = await Promise.all(dirPaths.map(pathExists));+    exists.forEach((v, i) => {+        if (v) {+            venvDirs.push(dirPaths[i]);+        }+    });++    return venvDirs;+}++/**+ * Gets the virtual environment kind for a given interpreter path.+ * This only checks for environments created using venv, virtualenv,+ * and virtualenvwrapper based environments.+ * @param interpreterPath: Absolute path to the interpreter paths.+ */+async function getVirtualEnvKind(interpreterPath:string): Promise<PythonEnvKind> {+    if (await isVenvEnvironment(interpreterPath)) {

If you check the findings of Spike: analyze deficiencies: Locator for pipenv, we may as well merge the pipenv locator in this one.

karthiknadig

comment created time in 5 days

PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'Envs'));+        dirPaths.push(path.join(homeDir, '.venvs'));+        dirPaths.push(path.join(homeDir, '.virtualenvs'));+    }++    const exists = await Promise.all(dirPaths.map(pathExists));+    exists.forEach((v, i) => {+        if (v) {+            venvDirs.push(dirPaths[i]);+        }+    });++    return venvDirs;+}++/**+ * Gets the virtual environment kind for a given interpreter path.+ * This only checks for environments created using venv, virtualenv,+ * and virtualenvwrapper based environments.+ * @param interpreterPath: Absolute path to the interpreter paths.+ */+async function getVirtualEnvKind(interpreterPath:string): Promise<PythonEnvKind> {+    if (await isVenvEnvironment(interpreterPath)) {

I realize it'll eventually be resolved, but what's the reason for separating Pipenv locator in the first place? AFAICS it does the same thing. Also, the file name globalVirtualEnvronmentLocator.ts is misleading then because it doesn't cover all global virtual environments.

karthiknadig

comment created time in 5 days

PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as fsapi from 'fs-extra';+import * as path from 'path';+import { getOSType, OSType } from '../../common/utils/platform';+import { isPosixPythonBin } from './posixUtils';+import { isWindowsPythonExe } from './windowsUtils';++export async function* findInterpretersInDir(root:string, recurseLevels?:number): AsyncGenerator<string> {+    const dirContents = (await fsapi.readdir(root)).map((c) => path.join(root, c));+    const os = getOSType();++    for (const item of dirContents) {+        const stat = await fsapi.lstat(item);+        if (stat.isDirectory()) {+            if (recurseLevels && recurseLevels > 0) {+                const subItems = findInterpretersInDir(item, recurseLevels - 1);+                for await (const subItem of subItems) {+                    yield subItem;+                }+            }+        } else if (os === OSType.Windows && isWindowsPythonExe(item)) {

Oh I missed that the locator filters the environments further.

karthiknadig

comment created time in 5 days

PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as fsapi from 'fs-extra';+import * as path from 'path';+import { getOSType, OSType } from '../../common/utils/platform';+import { isPosixPythonBin } from './posixUtils';+import { isWindowsPythonExe } from './windowsUtils';++export async function* findInterpretersInDir(root:string, recurseLevels?:number): AsyncGenerator<string> {+    const dirContents = (await fsapi.readdir(root)).map((c) => path.join(root, c));+    const os = getOSType();++    for (const item of dirContents) {+        const stat = await fsapi.lstat(item);+        if (stat.isDirectory()) {+            if (recurseLevels && recurseLevels > 0) {+                const subItems = findInterpretersInDir(item, recurseLevels - 1);+                for await (const subItem of subItems) {+                    yield subItem;+                }+            }+        } else if (os === OSType.Windows && isWindowsPythonExe(item)) {

Note isWindowsPythonExe also looks for python3.8.exe (/^python(\d+(.\d+)?)?\.exe$/) so you may find duplicates, same for isPosixPythonBin. We should either only look for python.exe and python, or eliminate duplicates by resolving symlinks.

karthiknadig

comment created time in 5 days

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'Envs'));

More directories to look into: 'envs', '.pyenv', '.direnv', '~/.local/share/virtualenvs'. Please double check the spike analyzing deficiencies of old implementation and implementation itself to make sure we don't regress.

karthiknadig

comment created time in 5 days

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'Envs'));+        dirPaths.push(path.join(homeDir, '.venvs'));+        dirPaths.push(path.join(homeDir, '.virtualenvs'));+    }++    const exists = await Promise.all(dirPaths.map(pathExists));+    exists.forEach((v, i) => {+        if (v) {+            venvDirs.push(dirPaths[i]);+        }+    });++    return venvDirs;+}++/**+ * Gets the virtual environment kind for a given interpreter path.+ * This only checks for environments created using venv, virtualenv,+ * and virtualenvwrapper based environments.+ * @param interpreterPath: Absolute path to the interpreter paths.+ */+async function getVirtualEnvKind(interpreterPath:string): Promise<PythonEnvKind> {+    if (await isVenvEnvironment(interpreterPath)) {

Global virtual environments can also be of type Pipenv. Check missing. Otherwise we're currently detecting the wrong types for some environments found in .virtualenvs for example.

karthiknadig

comment created time in 5 days

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'Envs'));+        dirPaths.push(path.join(homeDir, '.venvs'));+        dirPaths.push(path.join(homeDir, '.virtualenvs'));+    }++    const exists = await Promise.all(dirPaths.map(pathExists));+    exists.forEach((v, i) => {+        if (v) {+            venvDirs.push(dirPaths[i]);+        }+    });++    return venvDirs;+}++/**+ * Gets the virtual environment kind for a given interpreter path.+ * This only checks for environments created using venv, virtualenv,+ * and virtualenvwrapper based environments.+ * @param interpreterPath: Absolute path to the interpreter paths.+ */+async function getVirtualEnvKind(interpreterPath:string): Promise<PythonEnvKind> {+    if (await isVenvEnvironment(interpreterPath)) {

It may make sense to merge Pyenv locator in this as well as the logic is almost the same.

karthiknadig

comment created time in 5 days

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as fsapi from 'fs-extra';+import * as path from 'path';+import { getOSType, OSType } from '../../common/utils/platform';+import { isPosixPythonBin } from './posixUtils';+import { isWindowsPythonExe } from './windowsUtils';++export async function* findInterpretersInDir(root:string, recurseLevels?:number): AsyncGenerator<string> {+    const dirContents = (await fsapi.readdir(root)).map((c) => path.join(root, c));+    const os = getOSType();++    for (const item of dirContents) {

The second iteration of this loop need not wait for the first one to complete. We can parallelize this as well especially because the loop has a lstat call.

karthiknadig

comment created time in 5 days

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as fsapi from 'fs-extra';+import * as path from 'path';+import { getOSType, OSType } from '../../common/utils/platform';+import { isPosixPythonBin } from './posixUtils';+import { isWindowsPythonExe } from './windowsUtils';++export async function* findInterpretersInDir(root:string, recurseLevels?:number): AsyncGenerator<string> {+    const dirContents = (await fsapi.readdir(root)).map((c) => path.join(root, c));+    const os = getOSType();++    for (const item of dirContents) {+        const stat = await fsapi.lstat(item);+        if (stat.isDirectory()) {+            if (recurseLevels && recurseLevels > 0) {+                const subItems = findInterpretersInDir(item, recurseLevels - 1);+                for await (const subItem of subItems) {+                    yield subItem;+                }+            }+        } else if (os === OSType.Windows && isWindowsPythonExe(item)) {+            yield item;+        } else if (os !== OSType.Windows && isPosixPythonBin(item)) {+            yield item;+        }
        } else if (
            (os === OSType.Windows && isWindowsPythonExe(item))
            || (os !== OSType.Windows && isPosixPythonBin(item))
        ) {
            yield item;
        }

Also maybe the OS check should go inside isWindowsPythonExe() and isPosixPythonBin().

karthiknadig

comment created time in 5 days

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'Envs'));+        dirPaths.push(path.join(homeDir, '.venvs'));+        dirPaths.push(path.join(homeDir, '.virtualenvs'));+    }++    const exists = await Promise.all(dirPaths.map(pathExists));+    exists.forEach((v, i) => {+        if (v) {+            venvDirs.push(dirPaths[i]);+        }+    });++    return venvDirs;+}++/**+ * Gets the virtual environment kind for a given interpreter path.+ * This only checks for environments created using venv, virtualenv,+ * and virtualenvwrapper based environments.+ * @param interpreterPath: Absolute path to the interpreter paths.+ */+async function getVirtualEnvKind(interpreterPath:string): Promise<PythonEnvKind> {+    if (await isVenvEnvironment(interpreterPath)) {+        return PythonEnvKind.Venv;+    }++    if (await isVirtualenvwrapperEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnvWrapper;+    }++    if (await isVirtualenvEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnv;+    }++    return PythonEnvKind.Unknown;+}++/**+ * Takes absolute path to the interpreter and environment kind to build+ * environment info.+ * @param {string} interpreterPath: Absolute path to interpreter.+ * @param {PythonEnvKind} kind: Kind for the given environment.+ */+async function buildEnvInfo(interpreterPath:string, kind:PythonEnvKind): Promise<PythonEnvInfo> {+    let version:PythonVersion;+    try {+        version = parseVersion(path.basename(interpreterPath));+    } catch (ex) {+        traceError(`Failed to parse version from path: ${interpreterPath}`, ex);+        version = UNKNOWN_PYTHON_VERSION;+    }+    return {+        name: '',+        location: '',+        kind,+        executable: {+            filename: interpreterPath,+            sysPrefix: '',+            ...(await getFileInfo(interpreterPath)),+        },+        version,+        arch: Architecture.Unknown,+        distro: { org: '' },+    };+}++/**+ * Finds and resolves virtual environments created in known global locations.+ */+export class GlobalVirtualEnvironmentLocator extends PythonEnvsWatcher implements ILocator {+    private virtualEnvKinds = [+        PythonEnvKind.Venv,+        PythonEnvKind.VirtualEnv,+        PythonEnvKind.VirtualEnvWrapper,+    ];++    public constructor(private readonly searchDepth?:number) {+        super();+    }++    public iterEnvs(): IPythonEnvsIterator {+        // Number of levels of sub-directories to recurse when looking for+        // interpreters+        const searchDepth = this.searchDepth ?? DEFAULT_SEARCH_DEPTH;++        const { virtualEnvKinds } = this;+        const iterator = async function* () {+            const envRootDirs = await getGlobalVirtualEnvDirs();++            for (const envRootDir of envRootDirs) {+                traceVerbose(`Searching for global virtual envs in: ${envRootDir}`);++                const generator = findInterpretersInDir(envRootDir, searchDepth);++                for await (const env of generator) {

Nice and clean👍

karthiknadig

comment created time in 5 days

Pull request review commentmicrosoft/vscode-python

Implement global virtual environment locator

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as path from 'path';+import { traceError, traceVerbose } from '../../../../common/logger';+import {+    Architecture, getEnvironmentVariable, getUserHomeDir,+} from '../../../../common/utils/platform';+import {+    PythonEnvInfo, PythonEnvKind, PythonVersion, UNKNOWN_PYTHON_VERSION,+} from '../../../base/info';+import { parseVersion } from '../../../base/info/pythonVersion';+import { ILocator, IPythonEnvsIterator } from '../../../base/locator';+import { PythonEnvsWatcher } from '../../../base/watcher';+import { findInterpretersInDir } from '../../../common/commonUtils';+import { getFileInfo, pathExists } from '../../../common/externalDependencies';+import { isVenvEnvironment, isVirtualenvEnvironment, isVirtualenvwrapperEnvironment } from './virtualEnvironmentIdentifier';++const DEFAULT_SEARCH_DEPTH = 4;++/**+ * Gets all default virtual environment locations. This uses WORKON_HOME,+ * and user home directory to find some known locations where global virtual+ * environments are often created.+ */+async function getGlobalVirtualEnvDirs(): Promise<string[]> {+    const venvDirs:string[] = [];+    const dirPaths:string[] = [];++    const workOnHome = getEnvironmentVariable('WORKON_HOME');+    if (workOnHome) {+        dirPaths.push(workOnHome);+    }++    const homeDir = getUserHomeDir();+    if (homeDir) {+        dirPaths.push(path.join(homeDir, 'Envs'));+        dirPaths.push(path.join(homeDir, '.venvs'));+        dirPaths.push(path.join(homeDir, '.virtualenvs'));+    }++    const exists = await Promise.all(dirPaths.map(pathExists));+    exists.forEach((v, i) => {+        if (v) {+            venvDirs.push(dirPaths[i]);+        }+    });++    return venvDirs;+}++/**+ * Gets the virtual environment kind for a given interpreter path.+ * This only checks for environments created using venv, virtualenv,+ * and virtualenvwrapper based environments.+ * @param interpreterPath: Absolute path to the interpreter paths.+ */+async function getVirtualEnvKind(interpreterPath:string): Promise<PythonEnvKind> {+    if (await isVenvEnvironment(interpreterPath)) {+        return PythonEnvKind.Venv;+    }++    if (await isVirtualenvwrapperEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnvWrapper;+    }++    if (await isVirtualenvEnvironment(interpreterPath)) {+        return PythonEnvKind.VirtualEnv;+    }++    return PythonEnvKind.Unknown;+}++/**+ * Takes absolute path to the interpreter and environment kind to build+ * environment info.+ * @param {string} interpreterPath: Absolute path to interpreter.+ * @param {PythonEnvKind} kind: Kind for the given environment.+ */+async function buildEnvInfo(interpreterPath:string, kind:PythonEnvKind): Promise<PythonEnvInfo> {+    let version:PythonVersion;+    try {+        version = parseVersion(path.basename(interpreterPath));+    } catch (ex) {+        traceError(`Failed to parse version from path: ${interpreterPath}`, ex);+        version = UNKNOWN_PYTHON_VERSION;+    }+    return {+        name: '',+        location: '',+        kind,+        executable: {+            filename: interpreterPath,+            sysPrefix: '',+            ...(await getFileInfo(interpreterPath)),+        },+        version,+        arch: Architecture.Unknown,+        distro: { org: '' },+    };+}++/**+ * Finds and resolves virtual environments created in known global locations.+ */+export class GlobalVirtualEnvironmentLocator extends PythonEnvsWatcher implements ILocator {+    private virtualEnvKinds = [+        PythonEnvKind.Venv,+        PythonEnvKind.VirtualEnv,+        PythonEnvKind.VirtualEnvWrapper,+    ];++    public constructor(private readonly searchDepth?:number) {+        super();+    }++    public iterEnvs(): IPythonEnvsIterator {+        // Number of levels of sub-directories to recurse when looking for+        // interpreters+        const searchDepth = this.searchDepth ?? DEFAULT_SEARCH_DEPTH;++        const { virtualEnvKinds } = this;+        const iterator = async function* () {+            const envRootDirs = await getGlobalVirtualEnvDirs();++            for (const envRootDir of envRootDirs) {

The second iteration of this loop need not wait for the first one to complete. We can parallelize this as well.

karthiknadig

comment created time in 5 days

PullRequestReviewEvent
PullRequestReviewEvent

issue closedmicrosoft/vscode-python

No “Run Python File in Terminal” icon in menu bar when using “Remote - WSL” extension

<!-- Please search existing issues to avoid creating duplicates. -->

  • VS Code version: 1.49.0
  • Extension version (available under the Extensions sidebar): 2020.8.109390
  • OS and version: Windows 10 1903
  • Python version (& distribution if applicable, e.g. Anaconda): 3.8
  • Type of virtual environment used (N/A | venv | virtualenv | conda | ...): venv

Please see my question on StackOverflow. I don't know whether this is a bug or not.

https://stackoverflow.com/questions/63920819/no-run-python-file-in-terminal-when-using-remote-wsl-extension

closed time in 5 days

DavidA2014

issue commentmicrosoft/vscode-python

No “Run Python File in Terminal” icon in menu bar when using “Remote - WSL” extension

Closing as it's not reproducible nor common among users.

DavidA2014

comment created time in 5 days

issue commentmicrosoft/vscode-python

Confusing error "Could not load unit test config from launch.json" if version is missing

Hi there thanks for suggesting this! A more specific error message would help.

cdleonard

comment created time in 5 days

issue commentmicrosoft/vscode-python

Extension host crashes

Please upload the CPU profile which will help us track down the issue. Follow the steps here: https://github.com/Microsoft/vscode/wiki/Performance-Issues#profile-the-running-extensions

cyrusdaboo

comment created time in 6 days

Pull request review commentmicrosoft/vscode-python

Add CachingLocator.

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import '../../common/extensions';+import { createDeferred } from '../../common/utils/async';+import { PythonEnvInfo } from '../base/info';+import {+    ILocator,+    IPythonEnvsIterator,+    PythonLocatorQuery,+} from '../base/locator';+import { getEnvs, getQueryFilter } from '../base/locatorUtils';+import { PythonEnvsWatcher } from '../base/watcher';++interface IEnvsCache {+    initialize(): Promise<void>;+    listAll(): Promise<PythonEnvInfo[] | undefined>;+    getEnv(env: Partial<PythonEnvInfo>): Promise<PythonEnvInfo | undefined>;+    setAll(envs: PythonEnvInfo[]): Promise<void>;+    flush(): Promise<void>;+}++export class CachingLocator extends PythonEnvsWatcher implements ILocator {+    private readonly initializing = createDeferred<void>();++    constructor(+        private readonly cache: IEnvsCache,+        private readonly locator: ILocator,+    ) {+        super();+        locator.onChanged((event) => {+            this.refresh()+                .then(() => this.fire(event))+                .ignoreErrors();+        });+    }++    public async initialize(): Promise<void> {+        await this.cache.initialize();+        const envs = await this.cache.listAll();+        if (envs !== undefined) {+            this.initializing.resolve();+            await this.refresh();+        } else {+            // There is nothing in the cache, so we must wait for the+            // initial refresh to finish before allowing iteration.+            await this.refresh();+            this.initializing.resolve();+        }+    }++    public iterEnvs(query?: PythonLocatorQuery): IPythonEnvsIterator {+        return this.iterFromCache(query);+    }++    public async resolveEnv(env: string | PythonEnvInfo): Promise<PythonEnvInfo | undefined> {+        // If necessary we could be more aggressive about invalidating+        // the cached value.+        const envForKey = typeof env === 'string'+            ? { executable: { filename: env, sysPrefix: '', ctime: -1, mtime: -1 }}+            : env;+        const cached = await this.cache.getEnv(envForKey);+        if (cached !== undefined) {+            return cached;+        }+        // Fall back to the underlying locator.+        const envs = await this.cache.listAll();+        const resolved = await this.locator.resolveEnv(env);+        if (resolved !== undefined) {+            envs!.push(resolved);+            await this.update(envs!);+        }+        return resolved;+    }++    private async* iterFromCache(query?: PythonLocatorQuery): IPythonEnvsIterator {+        // XXX For now we wait for the initial refresh to finish...+        await this.initializing.promise;

Note if we aren't doing this, user will have to wait for the refresh to finish in order to see any environments. We should create a separate work item for this in case we don't forget.

ericsnowcurrently

comment created time in 20 days

PullRequestReviewEvent

pull request commentmicrosoft/vscode-python

Add CachingLocator.

@ericsnowcurrently @kimadeline I just had one comment regarding potential correctness https://github.com/microsoft/vscode-python/pull/14020#discussion_r498459009 I leave the rest upto you guys😄

ericsnowcurrently

comment created time in 20 days

Pull request review commentmicrosoft/vscode-python

Add CachingLocator.

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import '../../common/extensions';+import { createDeferred } from '../../common/utils/async';+import { PythonEnvInfo } from '../base/info';+import {+    ILocator,+    IPythonEnvsIterator,+    PythonLocatorQuery,+} from '../base/locator';+import { getEnvs, getQueryFilter } from '../base/locatorUtils';+import { PythonEnvsWatcher } from '../base/watcher';++interface IEnvsCache {+    initialize(): Promise<void>;+    listAll(): Promise<PythonEnvInfo[] | undefined>;+    getEnv(env: Partial<PythonEnvInfo>): Promise<PythonEnvInfo | undefined>;+    setAll(envs: PythonEnvInfo[]): Promise<void>;+    flush(): Promise<void>;+}++export class CachingLocator extends PythonEnvsWatcher implements ILocator {+    private readonly initializing = createDeferred<void>();++    constructor(+        private readonly cache: IEnvsCache,+        private readonly locator: ILocator,+    ) {+        super();+        locator.onChanged((event) => {+            this.refresh()+                .then(() => this.fire(event))+                .ignoreErrors();+        });+    }++    public async initialize(): Promise<void> {+        await this.cache.initialize();+        const envs = await this.cache.listAll();+        if (envs !== undefined) {

https://github.com/microsoft/vscode-python/pull/14020#discussion_r498459009

ericsnowcurrently

comment created time in 20 days

PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Add CachingLocator.

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import '../../common/extensions';+import { createDeferred } from '../../common/utils/async';+import { PythonEnvInfo } from '../base/info';+import {+    ILocator,+    IPythonEnvsIterator,+    PythonLocatorQuery,+} from '../base/locator';+import { getEnvs, getQueryFilter } from '../base/locatorUtils';+import { PythonEnvsWatcher } from '../base/watcher';++interface IEnvsCache {+    initialize(): Promise<void>;+    listAll(): Promise<PythonEnvInfo[] | undefined>;

An empty array means we didn't find any environments "we know there are no environments so don't waste any effort trying to find them"

Empty array signifies nothing was found in the previous session, it does not mean that there are no new environments to be found. User may have created environments without the extension being activated, or user might have refreshed in the previous session before discovery was completed, so the cache wasn't populated.

So regardless of whether it's undefined or empty, we ideally should be refreshing in both cases.

ericsnowcurrently

comment created time in 20 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Add CachingLocator.

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import '../../common/extensions';+import { createDeferred } from '../../common/utils/async';+import { PythonEnvInfo } from '../base/info';+import {+    ILocator,+    IPythonEnvsIterator,+    PythonLocatorQuery,+} from '../base/locator';+import { getEnvs, getQueryFilter } from '../base/locatorUtils';+import { PythonEnvsWatcher } from '../base/watcher';++interface IEnvsCache {+    initialize(): Promise<void>;+    listAll(): Promise<PythonEnvInfo[] | undefined>;

The distinction impacts what the caching locator does during initialization

That's where I differ. I see the caching locator doing the same thing in both the cases.

ericsnowcurrently

comment created time in 21 days

PullRequestReviewEvent

issue closedmicrosoft/vscode-python

notebook problems with running conda envs

Issue Type: <b>Bug</b> Related: https://github.com/microsoft/vscode-python/issues/13071 install python extenstion open a .ipynb file set the conda env on run:

Command failed: . activate myenv && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python /home/<user>/.vscode/extensions/ms-python.python-2020.9.111407/pythonFiles/pyvsc-run-isolated.py /home/<user>/.vscode/extensions/ms-python.python-2020.9.111407/pythonFiles/printEnvVariables.py bash: activate: Datei oder Verzeichnis nicht gefunden

Extension version: 2020.9.111407 VS Code version: Code 1.49.2 (e5e9e69aed6e1984f7499b7af85b3d05f9a6883a, 2020-09-24T16:26:09.944Z) OS version: Linux x64 5.8.11-arch1-1

<details> <summary>System Info</summary>

Item Value
CPUs Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz (4 x 2294)
GPU Status 2d_canvas: enabled<br>flash_3d: enabled<br>flash_stage3d: enabled<br>flash_stage3d_baseline: enabled<br>gpu_compositing: enabled<br>multiple_raster_threads: enabled_on<br>oop_rasterization: disabled_off<br>opengl: enabled_on<br>protected_video_decode: unavailable_off<br>rasterization: disabled_software<br>skia_renderer: enabled_on<br>video_decode: unavailable_off<br>vulkan: disabled_off<br>webgl: enabled<br>webgl2: enabled
Load (avg) 1, 1, 1
Memory (System) 7.48GB (2.81GB free)
Process Argv --no-sandbox --unity-launch --crash-reporter-id 1b09f273-3ad9-4d17-b97b-06cad50c25cb
Screen Reader no
VM 0%
DESKTOP_SESSION xfce
XDG_CURRENT_DESKTOP XFCE
XDG_SESSION_DESKTOP undefined
XDG_SESSION_TYPE x11

</details> <!-- generated by issue reporter -->

closed time in 21 days

KoSv

issue commentmicrosoft/vscode-python

notebook problems with running conda envs

Awesome

KoSv

comment created time in 21 days

issue commentmicrosoft/vscode-python

Disable import sorting for virtualenv

Oh I see what you mean, thanks for explaining! So currently you don't have a way to skip files if we're continuing with the stdin route.

I think the only way we can avoid using stdin is if we go the old route & create a temporary file in the workspace folder as suggested here: https://github.com/microsoft/vscode-python/issues/4891#issuecomment-564472540 cc @PeterJCLaw do you see any problems with that?

pikeas

comment created time in 21 days

PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Windows store locator

 export async function getWindowsStorePythonExes(): Promise<string[]> {         .filter(isWindowsPythonExe); } -// tslint:disable-next-line: no-suspicious-comment-// TODO: The above APIs will be consumed by the Windows Store locator class when we have it.+export class WindowsStoreLocator extends PythonEnvsWatcher implements ILocator {+    private readonly kind:PythonEnvKind = PythonEnvKind.WindowsStore;++    public iterEnvs(): IPythonEnvsIterator {+        const buildEnvInfo = (exe:string) => this.buildEnvInfo(exe);+        const iterator = async function* () {+            const exes = await getWindowsStorePythonExes();+            yield* exes.map(buildEnvInfo);+        };+        return iterator();+    }++    public async resolveEnv(env: string | PythonEnvInfo): Promise<PythonEnvInfo | undefined> {+        const executablePath = typeof env === 'string' ? env : env.executable.filename;+        if (await isWindowsStoreEnvironment(executablePath)) {+            return this.buildEnvInfo(executablePath);+        }+        return undefined;+    }++    private async buildEnvInfo(exe:string): Promise<PythonEnvInfo> {+        let version:PythonVersion;+        try {+            version = parseVersion(path.basename(exe));

Wait, parseVersion is only meant to parse version strings, like "3.8.1b1", no?

karthiknadig

comment created time in 21 days

Pull request review commentmicrosoft/vscode-python

Windows store locator

 export async function getWindowsStorePythonExes(): Promise<string[]> {         .filter(isWindowsPythonExe); } -// tslint:disable-next-line: no-suspicious-comment-// TODO: The above APIs will be consumed by the Windows Store locator class when we have it.+export class WindowsStoreLocator extends PythonEnvsWatcher implements ILocator {+    private readonly kind:PythonEnvKind = PythonEnvKind.WindowsStore;++    public iterEnvs(): IPythonEnvsIterator {+        const buildEnvInfo = (exe:string) => this.buildEnvInfo(exe);+        const iterator = async function* () {+            const exes = await getWindowsStorePythonExes();+            yield* exes.map(buildEnvInfo);+        };+        return iterator();+    }++    public async resolveEnv(env: string | PythonEnvInfo): Promise<PythonEnvInfo | undefined> {

We'll be removing this right? If not, this doesn't seem right. resolveEnv should always add to env. So you might wanna clone and replace on existing fields on passed in environments. Right now I see we're returning empty fields there.

karthiknadig

comment created time in 21 days

PullRequestReviewEvent

issue commentmicrosoft/vscode-python

notebook problems with running conda envs

Actually it's the bin directory that contains the activation script, not condabin, so can you try adding /opt/miniconda3/bin to PATH (or wherever the bin directory is) and see if it works?

KoSv

comment created time in 21 days

Pull request review commentmicrosoft/vscode-python

Environment info cache class

 export function registerTypes(serviceManager: IServiceManager) {     serviceManager.addSingleton<IInterpreterPathService>(IInterpreterPathService, InterpreterPathService);     serviceManager.addSingleton<IExtensions>(IExtensions, Extensions);     serviceManager.addSingleton<IRandom>(IRandom, Random);-    serviceManager.addSingleton<IPersistentStateFactory>(IPersistentStateFactory, PersistentStateFactory);     serviceManager.addSingleton<ITerminalServiceFactory>(ITerminalServiceFactory, TerminalServiceFactory);

however the latter was being added to the service manager later in the process

I think that should be changed. We need to ensure that we do all the registrations before initializing the python envs component. Basically time to do the TODO item, https://github.com/microsoft/vscode-python/blob/56903927cf9ec400f4b36d765424b6b1a2ca963d/src/client/extensionActivation.ts#L82-L87

Put them all before line 43 here, https://github.com/microsoft/vscode-python/blob/6bad0ef3fa233b477f6a9300c90a37a4eaa2ae8d/src/client/extensionInit.ts#L38-L45

kimadeline

comment created time in 21 days

PullRequestReviewEvent

issue commentmicrosoft/vscode-python

Run selection indentation error

Spike results

We insert blank lines between global statements, for eg. the following selection has two global statements: the if-else block and print.


if True:
    print('true')
else:
    print('')

print('')

So we insert an extra character between the two, hence it's run as two different statements image

The bug is that starting from somewhere around Python 3.7.5, the following

if a:
    b
elif c:
    d

is being considered as two global statements and so sent separately,

if a:
    b

&

elif c:
    d

so sending the second elif block fails.

The ast library changed, and the older logic we use to detect global statements is no longer working correctly. https://github.com/microsoft/vscode-python/blob/a58eeffd1b64498e2afe5f11597888dfd1c8699c/pythonFiles/normalizeForInterpreter.py#L48-L57

We either can modify the logic or how we use ast library, or can use regexes instead.

DougPlumley

comment created time in 21 days

push eventkarrtikr/vscode-python

Kartik Raj

commit sha 34ef50db123e078d0040e2e499fd5ef3ece34b91

And?

view details

push time in 21 days

Pull request review commentmicrosoft/vscode-python

Update reducer to use new implementation of merge environments

 function checkIfFinishedAndNotify(     }
 }
 
-export function mergeEnvironments(environment: PythonEnvInfo, other: PythonEnvInfo): PythonEnvInfo {
-    const result = cloneDeep(environment);
-    // Preserve type information.
-    // Possible we identified environment as unknown, but a later provider has identified env type.
-    if (environment.kind === PythonEnvKind.Unknown && other.kind && other.kind !== PythonEnvKind.Unknown) {
-        result.kind = other.kind;
-    }
-    const props: (keyof PythonEnvInfo)[] = [
-        'version',
-        'kind',
-        'executable',
-        'name',
-        'arch',
-        'distro',
-        'defaultDisplayName',
-        'searchLocation',
+function resolveEnvCollision(oldEnv: PythonEnvInfo, newEnv: PythonEnvInfo): PythonEnvInfo {
+    const [env, other] = sortEnvInfoByPriority(oldEnv, newEnv);
+    return mergeEnvironments(env, other);
+}
+
+/**
+ * Selects an environment based on the environment selection priority. This should
+ * match the priority in the environment identifier.
+ */
+function sortEnvInfoByPriority(...envs: PythonEnvInfo[]): PythonEnvInfo[] {
+    // tslint:disable-next-line: no-suspicious-comment
+    // TODO: When we consolidate the PythonEnvKind and EnvironmentType we should have
+    // one location where we define priority and

We have a separate work item (565) for that 📃

karrtikr

comment created time in 21 days

PullRequestReviewEvent

issue commentmicrosoft/vscode-python

notebook problems with running conda envs

Yes, it should work with both, source should also be able to find the activation script given it's in PATH. So I would think you have a corrupt install.

KoSv

comment created time in 21 days

issue commentmicrosoft/vscode-python

Disable import sorting for virtualenv

Is there a reason code formatters are called by filename but import sorting is called by stdin?

Yes, I think it was so that isort works for unsaved files: https://github.com/microsoft/vscode-python/issues/4891

No, because the only option that I can see is to create a second isort config file just for VS Code Python, which skips everything.

Why though? I don't see why you can't simply point the setting path to the original isort config file (which I assume has this configuration to skip such files), without creating a second isort config file.

pikeas

comment created time in 21 days

issue commentmicrosoft/vscode-python

notebook problems with running conda envs

Thanks that helps, looks like the activation command failed for you. conda activate is the best way to do this on new versions of conda, but source activate should also work provided that it's done in the right directory - and . activate is a shorthand for that.

I think it might be that you don't have the conda bin directory in PATH, and so bash can't find the activate script. Can you please run the following which prints environment variables before activation?

python /home//.vscode/extensions/ms-python.python-2020.9.111407/pythonFiles/pyvsc-run-isolated.py /home//.vscode/extensions/ms-python.python-2020.9.111407/pythonFiles/printEnvVariables.py

I'm particularly interested in PATH, and if it has condabin directory on PATH.

KoSv

comment created time in 21 days

issue commentmicrosoft/vscode-python

Disable import sorting for virtualenv

it means that an additonal isort config file needs to be maintained just for use by the VS Code Python extension

Sorry I don't get this part, an additional isort config file? Can't you simply use the existing config file?

pikeas

comment created time in 21 days

Pull request review commentmicrosoft/vscode-python

Environment info cache class

 export function registerTypes(serviceManager: IServiceManager) {     serviceManager.addSingleton<IInterpreterPathService>(IInterpreterPathService, InterpreterPathService);     serviceManager.addSingleton<IExtensions>(IExtensions, Extensions);     serviceManager.addSingleton<IRandom>(IRandom, Random);-    serviceManager.addSingleton<IPersistentStateFactory>(IPersistentStateFactory, PersistentStateFactory);     serviceManager.addSingleton<ITerminalServiceFactory>(ITerminalServiceFactory, TerminalServiceFactory);

IPersistentStateFactory is not specific to python environments, why're we moving it?

kimadeline

comment created time in 21 days

PullRequestReviewEvent
PullRequestReviewEvent

issue commentmicrosoft/vscode-python

notebook problems with running conda envs

@KoSv Thanks for trying it out? Can you try running the command in the error in bash terminal and see if it works for you? I assume you're on Unix.

. activate myenv && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python /home//.vscode/extensions/ms-python.python-2020.9.111407/pythonFiles/pyvsc-run-isolated.py /home//.vscode/extensions/ms-python.python-2020.9.111407/pythonFiles/printEnvVariables.py

The error Datei oder Verzeichnis nicht gefunden in German translates to File or directory not found, I'm not sure which file this is referring to.

KoSv

comment created time in 22 days

Pull request review commentmicrosoft/vscode-python

Environment info cache class

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as assert from 'assert';+import * as sinon from 'sinon';+import { PythonEnvInfoCache } from '../../../client/pythonEnvironments/base/envsCache';+import { PythonEnvInfo, PythonEnvKind } from '../../../client/pythonEnvironments/base/info';+import * as envInfo from '../../../client/pythonEnvironments/base/info/env';+import * as externalDependencies from '../../../client/pythonEnvironments/common/externalDependencies';++suite('Environment Info cache', () => {+    let getGlobalPersistentStoreStub: sinon.SinonStub;+    let areSameEnvironmentStub: sinon.SinonStub;+    let updatedValues: PythonEnvInfo[] | undefined;++    const allEnvsComplete = () => true;+    const envInfoArray = [+        {+            kind: PythonEnvKind.Conda, name: 'my-conda-env', defaultDisplayName: 'env-one',+        },+        {+            kind: PythonEnvKind.Venv, name: 'my-venv-env', defaultDisplayName: 'env-two',+        },+        {+            kind: PythonEnvKind.Pyenv, name: 'my-pyenv-env', defaultDisplayName: 'env-three',+        },+    ] as PythonEnvInfo[];++    setup(() => {+        areSameEnvironmentStub = sinon.stub(envInfo, 'areSameEnvironment');

Reasoning: https://github.com/microsoft/vscode-python/pull/14019#discussion_r493773721

kimadeline

comment created time in 22 days

PullRequestReviewEvent

Pull request review commentmicrosoft/vscode-python

Environment info cache class

+// Copyright (c) Microsoft Corporation. All rights reserved.+// Licensed under the MIT License.++import * as assert from 'assert';+import * as sinon from 'sinon';+import { PythonEnvInfoCache } from '../../../client/pythonEnvironments/base/envsCache';+import { PythonEnvInfo, PythonEnvKind } from '../../../client/pythonEnvironments/base/info';+import * as envInfo from '../../../client/pythonEnvironments/base/info/env';+import * as externalDependencies from '../../../client/pythonEnvironments/common/externalDependencies';++suite('Environment Info cache', () => {+    let getGlobalPersistentStoreStub: sinon.SinonStub;+    let areSameEnvironmentStub: sinon.SinonStub;+    let updatedValues: PythonEnvInfo[] | undefined;++    const allEnvsComplete = () => true;+    const envInfoArray = [+        {+            kind: PythonEnvKind.Conda, name: 'my-conda-env', defaultDisplayName: 'env-one',+        },+        {+            kind: PythonEnvKind.Venv, name: 'my-venv-env', defaultDisplayName: 'env-two',+        },+        {+            kind: PythonEnvKind.Pyenv, name: 'my-pyenv-env', defaultDisplayName: 'env-three',+        },+    ] as PythonEnvInfo[];++    setup(() => {+        areSameEnvironmentStub = sinon.stub(envInfo, 'areSameEnvironment');

You shouldn't need to stub this. We're trying to use real implementations where we can.

kimadeline

comment created time in 22 days

PullRequestReviewEvent
PullRequestReviewEvent

issue commentmicrosoft/vscode-python

Disable import sorting for virtualenv

Can you paste your settings and configuration file (and it's location)? I assume you've some way of telling isort which files are to be skipped: https://pycqa.github.io/isort/docs/configuration/options/#skip

You can use "python.sortImports.args" to specify the settings path which contains this skip configuration as suggested here: https://github.com/PyCQA/isort/issues/1500#issuecomment-699583316

pikeas

comment created time in 22 days

issue commentmicrosoft/vscode-python

No “Run Python File in Terminal” icon in menu bar when using “Remote - WSL” extension

Thanks for the info. However I don't see any errors related to Python extension here (You can ask vscode-remote for the errors regarding vscode remote resource.). Can you also paste your User & workspace settings?

Unfortunately as I'm unable to reproduce it, not much can be done here. If other users come forward and leave a comment demonstrating they are seeing/reproducing the problem then we will raise this issue's priority.

DavidA2014

comment created time in 22 days

push eventkarrtikr/vscode-python

Kartik Raj

commit sha d99acd7b767980be9c507389964861e002dac4fe

Undo unintentional formatting changes

view details

push time in 22 days

push eventkarrtikr/vscode-python

Eric Snow

commit sha 218dd8de3ac990a3587d2cc157e6f44d18cd9dd7

Add PythonEnvInfo-related helpers. (#14051) This PR adds some basic helpers that we use in a subsequent PR. The following small drive-by changes are also included: * drop PythonEnvInfo.id property * make some internal helpers public

view details

Eric Snow

commit sha 98c03870a1de5723e2e60c93362978e62232893e

Add PythonEnvInfo-related helpers. (#14051) This PR adds some basic helpers that we use in a subsequent PR. The following small drive-by changes are also included: * drop PythonEnvInfo.id property * make some internal helpers public

view details

push time in 22 days

push eventkarrtikr/vscode-python

Eric Snow

commit sha dc5ef1a3dc8c690349da40d69f36841c27fc4967

Add PythonEnvInfo-related helpers. (#14051) This PR adds some basic helpers that we use in a subsequent PR. The following small drive-by changes are also included: * drop PythonEnvInfo.id property * make some internal helpers public

view details

push time in 22 days

more