profile
viewpoint

Ask questionsAllow test suites to report errors like tests

Currently addressing an issue with the Jest extension that I've been working on. Sometimes there are issues parsing the test files - so I know what suites and tests are present. I'd like to be able to show errors for suites as well as tests so I can advise the user of what went wrong.

The *Info types are as follows:

export interface TestSuiteInfo {
  type: 'suite';
  id: string;
  label: string;
  description?: string;
  tooltip?: string;
  file?: string;
  line?: number;
  children: (TestSuiteInfo | TestInfo)[];
}

export interface TestInfo {
  type: 'test';
  id: string;
  label: string;
  description?: string;
  tooltip?: string;
  file?: string;
  line?: number;
  skipped?: boolean;
}

In both cases I can display the label, description and tooltip for both suites and tests. So there is no way to indicate that there is a problem to the user (apart from using TestLoadFinishedEvent.errorMessage). It would be cool to have some way to show tests and suites that have issues at this stage.

The *Event types are as follows:

export interface TestSuiteEvent {
  type: 'suite';
  suite: string | TestSuiteInfo;
  state: 'running' | 'completed';
  description?: string;
  tooltip?: string;
  file?: string;
  line?: number;
}

export interface TestEvent {
  type: 'test';
  test: string | TestInfo;
  state: 'running' | 'passed' | 'failed' | 'skipped' | 'errored';
  message?: string;
  decorations?: TestDecoration[];
  description?: string;
  tooltip?: string;
  file?: string;
  line?: number;
}

Here I can display the label, description and tooltips as before for both. However I can for tests, display a message and indicate an error state. It would be cool if we could show error state and a message for suites as well.

I realise that there might be issues with making suites errored when there are passing tests within them as well as many other issues. Feel free to discuss, or point me to previous discussion or advise me on alternatives to solve my original problem. I'm not sure if I can convert a suite to a test and vice versa as errors come and go...

hbenl/vscode-test-explorer

Answer questions kondratyev-nv

In Python Test Explorer, I create nodes for tests/files that have invalid tests (for example, file parsing issues), and I also fire a test event with errored state and a stacktrace in message for them just after finished event. By clicking on such test full stacktrace is shown in Output panel. Parent nodes are already maked errored as well in this case. image

IMO, there is no need to have anything special for such cases. Tooltips are inconvenient for errors and showing errors from all invalid tests by clicking on a suite would look messy in the Output.

useful!

Related questions

No questions were found.
Github User Rank List