profile
viewpoint

Ask questionsAllow test suites to report errors like tests

Currently addressing an issue with the Jest extension that I've been working on. Sometimes there are issues parsing the test files - so I know what suites and tests are present. I'd like to be able to show errors for suites as well as tests so I can advise the user of what went wrong.

The *Info types are as follows:

export interface TestSuiteInfo {
  type: 'suite';
  id: string;
  label: string;
  description?: string;
  tooltip?: string;
  file?: string;
  line?: number;
  children: (TestSuiteInfo | TestInfo)[];
}

export interface TestInfo {
  type: 'test';
  id: string;
  label: string;
  description?: string;
  tooltip?: string;
  file?: string;
  line?: number;
  skipped?: boolean;
}

In both cases I can display the label, description and tooltip for both suites and tests. So there is no way to indicate that there is a problem to the user (apart from using TestLoadFinishedEvent.errorMessage). It would be cool to have some way to show tests and suites that have issues at this stage.

The *Event types are as follows:

export interface TestSuiteEvent {
  type: 'suite';
  suite: string | TestSuiteInfo;
  state: 'running' | 'completed';
  description?: string;
  tooltip?: string;
  file?: string;
  line?: number;
}

export interface TestEvent {
  type: 'test';
  test: string | TestInfo;
  state: 'running' | 'passed' | 'failed' | 'skipped' | 'errored';
  message?: string;
  decorations?: TestDecoration[];
  description?: string;
  tooltip?: string;
  file?: string;
  line?: number;
}

Here I can display the label, description and tooltips as before for both. However I can for tests, display a message and indicate an error state. It would be cool if we could show error state and a message for suites as well.

I realise that there might be issues with making suites errored when there are passing tests within them as well as many other issues. Feel free to discuss, or point me to previous discussion or advise me on alternatives to solve my original problem. I'm not sure if I can convert a suite to a test and vice versa as errors come and go...

hbenl/vscode-test-explorer

Answer questions kondratyev-nv

What you describe is a good workaround, but there shouldn't be a need for adapter authors to come up with workarounds. And since the ability to mark a suite as "errored" keeps coming up, I guess it's time to support it.

Sorry, I did not mean that doing nothing is the way. It would be amazing if tests and suites can be conveniently marked as errored.

However, I believe that suites should not be in errored state by themselves. Suites should always aggregate children states, this behavior is expected and already works. For example, errored suite makes no sense if all children in it can pass. So, IMO view and behavior should be similar to what is now achieved with workarounds. If there would be a more straightforward way to do so - awesome!

useful!

Related questions

No questions were found.
Github User Rank List