profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/tkremenek/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

apple/swift-format 1192

Formatting technology for Swift source code

tkremenek/swift 5

The Swift Programming Language

tkremenek/swift-corelibs-foundation 1

The Foundation Project, providing core utilities, internationalization, and OS independence

tkremenek/swift-package-manager 1

The Package Manager for the Swift Programming Language

tkremenek/rouge 0

A pure Ruby code highlighter that is compatible with Pygments

tkremenek/swift-corelibs-xctest 0

The XCTest Project, A Swift core library for providing unit test support

tkremenek/swift-lldb 0

This is the version of LLDB that supports the Swift programming language & REPL.

tkremenek/swift-source-compat-suite 0

The infrastructure and project index comprising the Swift source compatibility suite.

pull request commentapple/swift-corelibs-xctest

Sr 1872 using runloopsource in wait for expectation

@stmontgomery @millenomi @compnerd Could you please provide your inputs on this?

@stmontgomery @millenomi @compnerd Could you please provide your inputs on this?

pranavshenoy

comment created time in 8 hours

pull request commentapple/swift-collections

Add missing final keyword

@lorentey first of all, sorry for long feedback.

What @kylemacomber said should be correct (at least according to this and this).

Which should mean that changes are only useful form code-style perspective.

But I did some additional performance testing for Deque (because _DequeBuffer was changed to final) to make sure that all works as expected. Did comparison of Deque with final _DequeBuffer vs non-final.

First, simply tested append operation with performance test (using measure).

func testAppend() {
  var deque = Deque<Int>()
    
  let options = XCTMeasureOptions()
  options.iterationCount = 100
    
  measure {
    (0...1000000).forEach { _ in deque.append(1) }
  }
}

Results:

  • Without WMO.
    • final - 0.775
    • non-final - 0.778 sec
  • With WMO (-O)
    • final - 0.024
    • non-final - 0.026 sec

Then, benchmark test append (max-size - 16M, cycles - 20). 01 Deque: append

Difference is quite small (and can be just accuracy error). Now for me it seems that changes are only useful to keep same style and do things explicit.

So @lorentey if you think that this PR is unnecessary, please feel free to close it 😀

DimaMishchenko

comment created time in 9 hours

Pull request review commentapple/swift-syntax

Add Buildable gyb

+%{+  from gyb_syntax_support import *+  from gyb_syntax_support.kinds import lowercase_first_word+  from gyb_syntax_support.kinds import SYNTAX_BASE_KINDS+  # -*- mode: Swift -*-+  # Ignore the following admonition it applies to the resulting .swift file only++  def syntax_buildable_child_type(type_name, syntax_kind, is_token, is_optional=False):+    if syntax_kind in SYNTAX_BASE_KINDS:+      buildable_type = syntax_kind + 'Buildable'+    elif not is_token:+      buildable_type = syntax_kind+    else:+      buildable_type = type_name++    if is_optional:+      buildable_type += '?'++    return buildable_type+}%+//// Automatically Generated From DeclBuildables.swift.gyb.+//===----------------------------------------------------------------------===//+//+// This source file is part of the Swift.org open source project+//+// Copyright (c) 2014 - 2021 Apple Inc. and the Swift project authors+// Licensed under Apache License v2.0 with Runtime Library Exception+//+// See https://swift.org/LICENSE.txt for license information+// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors+//+//===----------------------------------------------------------------------===//++import SwiftSyntax++// MARK: Protocols++% for kind in SYNTAX_BASE_KINDS:+%   if kind != 'SyntaxCollection':+%     build_kind = 'Syntax' if kind == 'Syntax' else kind + 'Syntax'+%     if kind == 'Syntax':+public protocol ${kind}ListBuildable {+%     else:+public protocol ${kind}ListBuildable: SyntaxListBuildable {+%     end+  func build${kind}List(format: Format, leadingTrivia: Trivia?) -> [${build_kind}]+}++%     if kind == 'Syntax':+public protocol ${kind}Buildable: ${kind}ListBuildable {+%     else:+public protocol ${kind}Buildable: SyntaxBuildable, ${kind}ListBuildable {+%     end+  func build${kind}(format: Format, leadingTrivia: Trivia?) -> ${build_kind}+}++extension ${kind}Buildable {+%     if kind != 'Syntax':+  func build${kind}(format: Format) -> ${build_kind} {+    build${kind}(format: format, leadingTrivia: nil)+  }+%     end+  public func buildSyntax(format: Format) -> Syntax {+    buildSyntax(format: format, leadingTrivia: nil)+  }++  public func buildSyntax(format: Format, leadingTrivia: Trivia?) -> Syntax {+    Syntax(build${kind}(format: format, leadingTrivia: leadingTrivia))+  }++  public func build${kind}List(format: Format) -> [${build_kind}] {+    build${kind}List(format: format, leadingTrivia: nil)+  }++  public func build${kind}List(format: Format, leadingTrivia: Trivia?) -> [${build_kind}] {+    [build${kind}(format: format, leadingTrivia: leadingTrivia)]+  }+}++%   end+% end++// MARK: - Buildables++% for node in SYNTAX_NODES:+%   if node.is_buildable():+public struct ${node.syntax_kind}: ${node.base_kind}Buildable {

And maybe also for the init and all the children?

kimdv

comment created time in 15 hours

pull request commentapple/swift-syntax

Fix typo

Whoops... Accidentally also deleted the remote branch. 🤦‍♂️

kimdv

comment created time in 15 hours

PullRequestEvent

PR closed apple/swift-syntax

Fix typo

Fixes a small typo

+192 -192

0 comment

2 changed files

kimdv

pr closed time in 15 hours

pull request commentapple/swift-syntax

Add Buildable gyb

I have address more less all off them, I have some small questions and added some comments.

I’ve added some comments inline but they are all mostly stylistic now, nothing major.

Once you’ve addressed them, I will give the PR another review, run the test suite and get this merged. I’ve lost track but are there any related. If there are any, could you also push any pending changes to your apple/swift PR for that?

Yes, there is one here: https://github.com/apple/swift/pull/36726

kimdv

comment created time in 15 hours

Pull request review commentapple/swift-syntax

Add Buildable gyb

+%{+  from gyb_syntax_support import *+  from gyb_syntax_support.kinds import lowercase_first_word+  from gyb_syntax_support.kinds import SYNTAX_BASE_KINDS+  # -*- mode: Swift -*-+  # Ignore the following admonition it applies to the resulting .swift file only++  def syntax_buildable_child_type(syntax_kind, is_token=False, is_optional=False):+    if syntax_kind in SYNTAX_BASE_KINDS:+      buildable_type = syntax_kind + 'Buildable'+    elif not is_token:+      buildable_type = syntax_kind+    elif 'List' in syntax_kind:+      buildable_type = syntax_kind + 'Syntax'+    else:+      buildable_type = 'TokenSyntax'++    if is_optional:+      buildable_type += '?'++    return buildable_type+}%+//// Automatically Generated From DeclBuildables.swift.gyb.+//===----------------------------------------------------------------------===//+//+// This source file is part of the Swift.org open source project+//+// Copyright (c) 2014 - 2021 Apple Inc. and the Swift project authors+// Licensed under Apache License v2.0 with Runtime Library Exception+//+// See https://swift.org/LICENSE.txt for license information+// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors+//+//===----------------------------------------------------------------------===//++import SwiftSyntax++// MARK: Protocols++% for kind in SYNTAX_BASE_KINDS:+%   if kind != 'SyntaxCollection':+%     build_kind = 'Syntax' if kind == 'Syntax' else kind + 'Syntax'+%     if kind == 'Syntax':+public protocol ${kind}ListBuildable {+%     else:+public protocol ${kind}ListBuildable: SyntaxListBuildable {+%     end+  func build${kind}List(format: Format, leadingTrivia: Trivia?) -> [${build_kind}]+}++%     if kind == 'Syntax':+public protocol ${kind}Buildable: ${kind}ListBuildable {+%     else:+public protocol ${kind}Buildable: SyntaxBuildable, ${kind}ListBuildable {+%     end+  func build${kind}(format: Format, leadingTrivia: Trivia?) -> ${build_kind}+}++extension ${kind}Buildable {+  public func buildSyntax(format: Format, leadingTrivia: Trivia?) -> Syntax {+    Syntax(build${kind}(format: format, leadingTrivia: leadingTrivia))+  }++  public func build${kind}List(format: Format, leadingTrivia: Trivia?) -> [${build_kind}] {+    [build${kind}(format: format, leadingTrivia: leadingTrivia)]+  }+}++%   end+% end++// MARK: - Buildables++% for node in SYNTAX_NODES:+%   if node.is_buildable():+%     return_type = 'Syntax'+%     if node.base_kind != 'Syntax':+%       return_type = node.base_kind + 'Syntax'+%     end+public struct ${node.syntax_kind}: ${node.base_kind}Buildable {+%     child_params = []+%     for child in node.children:+%       param_type = syntax_buildable_child_type(child.syntax_kind, child.is_token(), child.is_optional)+%       child_params.append("%s: %s" % (child.swift_name, param_type))+%     end+%     for child_param in child_params:+  let ${child_param}+%     end++  public init(+    ${',\n    '.join(child_params)}+  ) {+%     for child in node.children:+    self.${child.swift_name} = ${child.swift_name}+%     end+  }+  +  func build${node.syntax_kind}(format: Format, leadingTrivia: Trivia?) -> ${node.syntax_kind}Syntax {+%     child_params = []+%     for child in node.children:+%       param_name = child.swift_name+%       if child.is_token() and child.requires_leading_newline:+%         param_name = param_name + '.withLeadingTrivia(.newlines(1) + format._makeIndent())'+%       elif child.syntax_kind in SYNTAX_BASE_KINDS or not child.is_token():+%         if child.is_optional:+%           param_name = param_name + "?"+%         end+%         format = 'format'+%         if child.is_indented:+%           format += '._indented()'+%         end            +%         param_name = param_name + ".build" + child.syntax_kind + "(format: " + format + ", leadingTrivia: nil)"+%       end+%       child_params.append("%s: %s" % (child.swift_name, param_name))+%     end+    let ${node.swift_syntax_kind} = SyntaxFactory.make${node.syntax_kind}(+      ${',\n      '.join(child_params)}+    )+    +    if let leadingTrivia = leadingTrivia {+      return ${node.swift_syntax_kind}+        .withLeadingTrivia(leadingTrivia)+    }++    return ${node.swift_syntax_kind}+  }++  public func build${node.base_kind}(format: Format, leadingTrivia: Trivia?) -> ${return_type} {

I have added a comment. Looked a bit on how it was done in the SwiftSyntax. Is it good enough or should there be more? 😁

kimdv

comment created time in 16 hours

PR opened apple/swift-syntax

Fix typo
+192 -192

0 comment

2 changed files

pr created time in 16 hours

Pull request review commentapple/swift-syntax

Add Buildable gyb

+%{+  from gyb_syntax_support import *+  from gyb_syntax_support.kinds import lowercase_first_word+  from gyb_syntax_support.kinds import SYNTAX_BASE_KINDS+  # -*- mode: Swift -*-+  # Ignore the following admonition it applies to the resulting .swift file only++  def syntax_buildable_child_type(type_name, syntax_kind, is_token, is_optional=False):+    if syntax_kind in SYNTAX_BASE_KINDS:+      buildable_type = syntax_kind + 'Buildable'+    elif not is_token:+      buildable_type = syntax_kind+    else:+      buildable_type = type_name++    if is_optional:+      buildable_type += '?'++    return buildable_type+}%+//// Automatically Generated From DeclBuildables.swift.gyb.+//===----------------------------------------------------------------------===//+//+// This source file is part of the Swift.org open source project+//+// Copyright (c) 2014 - 2021 Apple Inc. and the Swift project authors+// Licensed under Apache License v2.0 with Runtime Library Exception+//+// See https://swift.org/LICENSE.txt for license information+// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors+//+//===----------------------------------------------------------------------===//++import SwiftSyntax++// MARK: Protocols++% for kind in SYNTAX_BASE_KINDS:+%   if kind != 'SyntaxCollection':+%     build_kind = 'Syntax' if kind == 'Syntax' else kind + 'Syntax'+%     if kind == 'Syntax':+public protocol ${kind}ListBuildable {+%     else:+public protocol ${kind}ListBuildable: SyntaxListBuildable {+%     end+  func build${kind}List(format: Format, leadingTrivia: Trivia?) -> [${build_kind}]+}++%     if kind == 'Syntax':+public protocol ${kind}Buildable: ${kind}ListBuildable {+%     else:+public protocol ${kind}Buildable: SyntaxBuildable, ${kind}ListBuildable {+%     end+  func build${kind}(format: Format, leadingTrivia: Trivia?) -> ${build_kind}+}++extension ${kind}Buildable {+%     if kind != 'Syntax':+  func build${kind}(format: Format) -> ${build_kind} {+    build${kind}(format: format, leadingTrivia: nil)+  }+%     end+  public func buildSyntax(format: Format) -> Syntax {+    buildSyntax(format: format, leadingTrivia: nil)+  }++  public func buildSyntax(format: Format, leadingTrivia: Trivia?) -> Syntax {+    Syntax(build${kind}(format: format, leadingTrivia: leadingTrivia))+  }++  public func build${kind}List(format: Format) -> [${build_kind}] {+    build${kind}List(format: format, leadingTrivia: nil)+  }++  public func build${kind}List(format: Format, leadingTrivia: Trivia?) -> [${build_kind}] {+    [build${kind}(format: format, leadingTrivia: leadingTrivia)]+  }+}++%   end+% end++// MARK: - Buildables++% for node in SYNTAX_NODES:+%   if node.is_buildable():+public struct ${node.syntax_kind}: ${node.base_kind}Buildable {

@ahoppen would it also make sense to add documentation here`

%     for line in dedented_lines(node.description):
/// ${line}
%     end
kimdv

comment created time in 16 hours

Pull request review commentapple/swift-syntax

Add Buildable gyb

+import XCTest+import SwiftSyntax+import SwiftSyntaxBuilder++final class EnumCaseElementTests: XCTestCase {+  func test_enumCaseElementInit() {+    let leadingTrivia = Trivia.garbageText("␣")+    let string = SyntaxFactory.makeStringSegment("Hello World")+    let segment = StringSegment(content: string)+    let segments = StringLiteralSegments([segment])++    let stringLiteralExpr: ExprBuildable = StringLiteralExpr(openDelimiter: nil,+                                                             openQuote: SyntaxFactory.makeStringQuoteToken(),

It is not public. We could make them public and I think actually we should do that

kimdv

comment created time in 16 hours

Pull request review commentapple/swift-syntax

Add Buildable gyb

+%{+  from gyb_syntax_support import *+  from gyb_syntax_support.kinds import lowercase_first_word+  from gyb_syntax_support.kinds import SYNTAX_BASE_KINDS+  # -*- mode: Swift -*-+  # Ignore the following admonition it applies to the resulting .swift file only++  def syntax_buildable_child_type(syntax_kind, is_token=False, is_optional=False):+    if syntax_kind in SYNTAX_BASE_KINDS:+      buildable_type = syntax_kind + 'Buildable'+    elif not is_token:+      buildable_type = syntax_kind+    elif 'List' in syntax_kind:+      buildable_type = syntax_kind + 'Syntax'+    else:+      buildable_type = 'TokenSyntax'++    if is_optional:+      buildable_type += '?'++    return buildable_type+}%+//// Automatically Generated From DeclBuildables.swift.gyb.+//===----------------------------------------------------------------------===//+//+// This source file is part of the Swift.org open source project+//+// Copyright (c) 2014 - 2021 Apple Inc. and the Swift project authors+// Licensed under Apache License v2.0 with Runtime Library Exception+//+// See https://swift.org/LICENSE.txt for license information+// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors+//+//===----------------------------------------------------------------------===//++import SwiftSyntax++// MARK: Protocols++% for kind in SYNTAX_BASE_KINDS:+%   if kind != 'SyntaxCollection':+%     build_kind = 'Syntax' if kind == 'Syntax' else kind + 'Syntax'+%     if kind == 'Syntax':+public protocol ${kind}ListBuildable {+%     else:+public protocol ${kind}ListBuildable: SyntaxListBuildable {+%     end+  func build${kind}List(format: Format, leadingTrivia: Trivia?) -> [${build_kind}]+}++%     if kind == 'Syntax':+public protocol ${kind}Buildable: ${kind}ListBuildable {+%     else:+public protocol ${kind}Buildable: SyntaxBuildable, ${kind}ListBuildable {+%     end+  func build${kind}(format: Format, leadingTrivia: Trivia?) -> ${build_kind}+}++extension ${kind}Buildable {+  public func buildSyntax(format: Format, leadingTrivia: Trivia?) -> Syntax {+    Syntax(build${kind}(format: format, leadingTrivia: leadingTrivia))+  }++  public func build${kind}List(format: Format, leadingTrivia: Trivia?) -> [${build_kind}] {+    [build${kind}(format: format, leadingTrivia: leadingTrivia)]+  }+}++%   end+% end++// MARK: - Buildables++% for node in SYNTAX_NODES:+%   if node.is_buildable():+%     return_type = 'Syntax'+%     if node.base_kind != 'Syntax':+%       return_type = node.base_kind + 'Syntax'+%     end

Moved it down. But I think we should still call it return_type then, because it is only used for that now.

kimdv

comment created time in 16 hours

Pull request review commentapple/swift-syntax

Add Buildable gyb

+%{+  from gyb_syntax_support import *+  from gyb_syntax_support.kinds import lowercase_first_word+  from gyb_syntax_support.kinds import SYNTAX_BASE_KINDS+  # -*- mode: Swift -*-+  # Ignore the following admonition it applies to the resulting .swift file only++  def syntax_buildable_child_type(syntax_kind, is_token=False, is_optional=False):+    if syntax_kind in SYNTAX_BASE_KINDS:+      buildable_type = syntax_kind + 'Buildable'+    elif not is_token:+      buildable_type = syntax_kind+    elif 'List' in syntax_kind:+      buildable_type = syntax_kind + 'Syntax'+    else:+      buildable_type = 'TokenSyntax'++    if is_optional:+      buildable_type += '?'++    return buildable_type+}%+//// Automatically Generated From DeclBuildables.swift.gyb.+//===----------------------------------------------------------------------===//+//+// This source file is part of the Swift.org open source project+//+// Copyright (c) 2014 - 2021 Apple Inc. and the Swift project authors+// Licensed under Apache License v2.0 with Runtime Library Exception+//+// See https://swift.org/LICENSE.txt for license information+// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors+//+//===----------------------------------------------------------------------===//++import SwiftSyntax++// MARK: Protocols++% for kind in SYNTAX_BASE_KINDS:+%   if kind != 'SyntaxCollection':+%     build_kind = 'Syntax' if kind == 'Syntax' else kind + 'Syntax'+%     if kind == 'Syntax':+public protocol ${kind}ListBuildable {+%     else:+public protocol ${kind}ListBuildable: SyntaxListBuildable {+%     end+  func build${kind}List(format: Format, leadingTrivia: Trivia?) -> [${build_kind}]

The reason is that if we pass .zero is replaces the leading Trivia.

We expect ␣import SwiftSyntax or

␣struct TestStruct {
}

But get

but get ␣importSwiftSyntax

␣struct TestStruct{
}
kimdv

comment created time in 16 hours

Pull request review commentapple/swift-syntax

Add Buildable gyb

+%{+  from gyb_syntax_support import *+  from gyb_syntax_support.kinds import lowercase_first_word+  from gyb_syntax_support.kinds import SYNTAX_BASE_KINDS+  # -*- mode: Swift -*-+  # Ignore the following admonition it applies to the resulting .swift file only++  def syntax_buildable_child_type(syntax_kind, is_token=False, is_optional=False):+    if syntax_kind in SYNTAX_BASE_KINDS:+      buildable_type = syntax_kind + 'Buildable'+    elif not is_token:+      buildable_type = syntax_kind+    elif 'List' in syntax_kind:

I have just tried this, and ended up with something that we wanted.

Look at the example below:

public struct AvailabilityVersionRestriction: SyntaxBuildable {
  let platform: TokenSyntax
  let version: VersionTupleSyntax?

  public init(
    platform: TokenSyntax,
    version: VersionTupleSyntax?
  ) {
    self.platform = platform
    self.version = version
  }

  ...
}

But we want

public struct AvailabilityVersionRestriction: SyntaxBuildable {
  let platform: TokenSyntax
  let version: VersionTuple?

  public init(
    platform: TokenSyntax,
    version: VersionTuple?
  ) {
    self.platform = platform
    self.version = version
  }

  ...
}
kimdv

comment created time in 17 hours

issue closedapple/ml-qrecc

Explicit Annotations of the Source of the Conversations

Thank you for the great work, and for open-sourcing the dataset.

The paper mentions that QReCC contains 13,598 dialogues, out of which 9.3K dialogues are based on the questions from QuAC, 80 are from TREC CAsT, and 4.4K are from NQ.

Do you have the annotations of the source of conversations? Specifically, I am looking for where a conversation with a specific "Conversation_no" is from QuAC, TREC CAsT, or NQ.

According to my exploration, the first 80 conversations are from TREC CAsT, and I could match ~5k conversations from QuAC by exact match. However, I think many conversations have some corrections or are shorter than the original QuAC conversations. It will be very helpful to have an explicit mapping .

closed time in 21 hours

vaibhavad

issue commentapple/ml-qrecc

Explicit Annotations of the Source of the Conversations

We added the conversation source information. You can use the field "Conversation_source".

vaibhavad

comment created time in 21 hours

push eventapple/ml-qrecc

RavitejaAnantha

commit sha 3ce0cb29e05b07b5e0a38df02de09233a1d031b1

Add conversation source information

view details

push time in 21 hours

issue openedapple/swift-algorithms

Using Github Bots?

<!-- Thanks for contributing to Swift Algorithms!

Before you submit your issue, please replace the paragraph
below with information about your proposed feature.

-->

Github has lots of really handy bots that integrate with pushed commits and PR actions, as well as Github Actions CI. I have used Github's bots in my own projects and found them very secure, helpful and easy to integrate.

I have raised #133 re: [ImgBot] but I am open to your feedback on more general Bot integrations as well.

Thanks.

created time in a day

push eventapple/swift-evolution

Joe Groff

commit sha 62a1dfd86e904e16d3fd6ca335e57e32a9e87025

Begin review of SE-0309

view details

push time in a day

Pull request review commentapple/swift-tools-support-core

improve process state management

 public final class Process: ObjectIdentifierProtocol {         if outputRedirection.redirectsOutput {             let stdoutPipe = Pipe()             let stderrPipe = Pipe()++            var pending: [UInt8]?+            let pendingLock = Lock()+             stdoutPipe.fileHandleForReading.readabilityHandler = { (fh : FileHandle) -> Void in                 let contents = fh.readDataToEndOfFile()                 self.outputRedirection.outputClosures?.stdoutClosure([UInt8](contents))-                if case .success(let data) = self.stdout.result {-                    self.stdout.result = .success(data + contents)+                pendingLock.withLock {+                    if let stderr = pending {+                        self.stateLock.withLock {+                            self?.state = .outputReady(stdout: .success(contents), stderr: .success(stderr))+                        }+                    } else {+                        pending = contents+                    }                 }             }             stderrPipe.fileHandleForReading.readabilityHandler = { (fh : FileHandle) -> Void in                 let contents = fh.readDataToEndOfFile()                 self.outputRedirection.outputClosures?.stderrClosure([UInt8](contents))-                if case .success(let data) = self.stderr.result {-                    self.stderr.result = .success(data + contents)+                pendingLock.withLock {+                    if let stdout = pending {+                        self.stateLock.withLock {+                            self?.state = .outputReady(stdout: .success(stdout), stderr: .success(contents))
                            self.state = .outputReady(stdout: .success(stdout), stderr: .success(contents.withUnsafeBytes { Array<UInt8>($0) }))
tomerd

comment created time in a day

Pull request review commentapple/swift-tools-support-core

improve process state management

 public final class Process: ObjectIdentifierProtocol {         if outputRedirection.redirectsOutput {             let stdoutPipe = Pipe()             let stderrPipe = Pipe()++            var pending: [UInt8]?+            let pendingLock = Lock()+             stdoutPipe.fileHandleForReading.readabilityHandler = { (fh : FileHandle) -> Void in                 let contents = fh.readDataToEndOfFile()                 self.outputRedirection.outputClosures?.stdoutClosure([UInt8](contents))-                if case .success(let data) = self.stdout.result {-                    self.stdout.result = .success(data + contents)+                pendingLock.withLock {+                    if let stderr = pending {+                        self.stateLock.withLock {+                            self?.state = .outputReady(stdout: .success(contents), stderr: .success(stderr))
                            self.state = .outputReady(stdout: .success(contents.withUnsafeBytes { Array<UInt8>($0) }), stderr: .success(stderr))
tomerd

comment created time in a day

Pull request review commentapple/swift-tools-support-core

improve process state management

 public final class Process: ObjectIdentifierProtocol {         if outputRedirection.redirectsOutput {             let stdoutPipe = Pipe()             let stderrPipe = Pipe()++            var pending: [UInt8]?+            let pendingLock = Lock()+             stdoutPipe.fileHandleForReading.readabilityHandler = { (fh : FileHandle) -> Void in                 let contents = fh.readDataToEndOfFile()                 self.outputRedirection.outputClosures?.stdoutClosure([UInt8](contents))-                if case .success(let data) = self.stdout.result {-                    self.stdout.result = .success(data + contents)+                pendingLock.withLock {+                    if let stderr = pending {+                        self.stateLock.withLock {+                            self?.state = .outputReady(stdout: .success(contents), stderr: .success(stderr))+                        }+                    } else {+                        pending = contents+                    }                 }             }             stderrPipe.fileHandleForReading.readabilityHandler = { (fh : FileHandle) -> Void in                 let contents = fh.readDataToEndOfFile()                 self.outputRedirection.outputClosures?.stderrClosure([UInt8](contents))-                if case .success(let data) = self.stderr.result {-                    self.stderr.result = .success(data + contents)+                pendingLock.withLock {+                    if let stdout = pending {+                        self.stateLock.withLock {+                            self?.state = .outputReady(stdout: .success(stdout), stderr: .success(contents))+                        }+                    } else {+                        pending = contents
                        pending = contents.withUnsafeBytes { Array<UInt8>($0) }
tomerd

comment created time in a day

Pull request review commentapple/swift-tools-support-core

improve process state management

 public final class Process: ObjectIdentifierProtocol {         if outputRedirection.redirectsOutput {             let stdoutPipe = Pipe()             let stderrPipe = Pipe()++            var pending: [UInt8]?+            let pendingLock = Lock()+             stdoutPipe.fileHandleForReading.readabilityHandler = { (fh : FileHandle) -> Void in                 let contents = fh.readDataToEndOfFile()                 self.outputRedirection.outputClosures?.stdoutClosure([UInt8](contents))-                if case .success(let data) = self.stdout.result {-                    self.stdout.result = .success(data + contents)+                pendingLock.withLock {+                    if let stderr = pending {+                        self.stateLock.withLock {+                            self?.state = .outputReady(stdout: .success(contents), stderr: .success(stderr))+                        }+                    } else {+                        pending = contents
                        pending = contents.withUnsafeBytes { Array<UInt8>($0) }
tomerd

comment created time in a day

pull request commentapple/swift-tools-support-core

improve process state management

This definitely doesn't seem to build on Windows:

S:\SourceCache\swift-tools-support-core\Sources\TSCBasic\Process.swift:419:33: error: cannot use optional chaining on non-optional value of type 'Process'
                            self?.state = .outputReady(stdout: .success(contents), stderr: .success(stderr))
                            ~~~~^

S:\SourceCache\swift-tools-support-core\Sources\TSCBasic\Process.swift:419:73: error: cannot convert value of type 'Data' to expected argument type '[UInt8]'
                            self?.state = .outputReady(stdout: .success(contents), stderr: .success(stderr))
                                                                        ^
S:\SourceCache\swift-tools-support-core\Sources\TSCBasic\Process.swift:422:35: error: cannot assign value of type 'Data' to type '[UInt8]'
                        pending = contents
                                  ^~~~~~~~
S:\SourceCache\swift-tools-support-core\Sources\TSCBasic\Process.swift:432:33: error: cannot use optional chaining on non-optional value of type 'Process'
                            self?.state = .outputReady(stdout: .success(stdout), stderr: .success(contents))
                            ~~~~^

S:\SourceCache\swift-tools-support-core\Sources\TSCBasic\Process.swift:432:99: error: cannot convert value of type 'Data' to expected argument type '[UInt8]'
                            self?.state = .outputReady(stdout: .success(stdout), stderr: .success(contents))
                                                                                                  ^
S:\SourceCache\swift-tools-support-core\Sources\TSCBasic\Process.swift:435:35: error: cannot assign value of type 'Data' to type '[UInt8]'
                        pending = contents
                                  ^~~~~~~~
S:\SourceCache\swift-tools-support-core\Sources\TSCBasic\Process.swift:660:16: error: value of type 'UnsafeMutablePointer<FILE>' (aka 'UnsafeMutablePointer<_iobuf>') has no member 'thread'
        stdout.thread?.join()
        ~~~~~~ ^~~~~~
S:\SourceCache\swift-tools-support-core\Sources\TSCBasic\Process.swift:661:16: error: value of type 'UnsafeMutablePointer<FILE>' (aka 'UnsafeMutablePointer<_iobuf>') has no member 'thread'
        stderr.thread?.join()
        ~~~~~~ ^~~~~~
S:\SourceCache\swift-tools-support-core\Sources\TSCBasic\Process.swift:667:28: error: value of type 'UnsafeMutablePointer<FILE>' (aka 'UnsafeMutablePointer<_iobuf>') has no member 'result'
            output: stdout.result,
                    ~~~~~~ ^~~~~~
S:\SourceCache\swift-tools-support-core\Sources\TSCBasic\Process.swift:668:34: error: value of type 'UnsafeMutablePointer<FILE>' (aka 'UnsafeMutablePointer<_iobuf>') has no member 'result'
            stderrOutput: stderr.result
                          ~~~~~~ ^~~~~~
tomerd

comment created time in a day

issue closedapple/ml-hypersim

2D 3D alignment

Hi @mikeroberts3000, what would be a good way to check the 3D geometry's alignment with the generated 2D images? Is there any code that can demonstrate how to parse the scene information and align 3D geometry with the images using a different renderer such as blender or pytorch3d?

closed time in a day

umyta

issue commentapple/ml-hypersim

2D 3D alignment

Great question! It is always a good idea to experiment with the mesh data and camera data, to make sure you're interpreting it all correctly.

Let's assume you obtained your own copy of the source assets, exported an OBJ file for a particular scene, and then ran our code/python/tools/generate_mesh_from_obj.py code for that scene. In this case, you will have the following files in the _detail/mesh folder for that scene:

  • mesh_vertices.hdf5 contains a Vx3 array of vertex positions for the entire scene
  • mesh_faces_vi.hdf5 contains an Fx3 array of faces for the entire scene; this array stores integer IDs that refer to the values in the mesh_vertices array
    • If mesh_faces_vi[i] == [a b c], it means that the ith face in the scene is a triangle with vertices (v0, v1, v2) == (mesh_vertices[a], mesh_vertices[b], mesh_vertices[c])
  • mesh_faces_oi.hdf5 contains an Fx1 array with the "object ID" of each face; note that these "object IDs" are the most fine-grained segmentation available in our data, and are more fine-grained than our semantic instance IDs; in other words, each "semantic instance" in our data is composed of several "object IDs"
  • mesh_objects_sii.hdf5 contains a Bx1 array with the semantic instance ID of each object ID
  • mesh_objects_si.hdf5 contains a Bx1 array with the NYU40 semantic label for each object ID

To understand how to project this mesh data onto our pre-rendered images, here are some steps that will be helpful, in increasing order of difficulty:

  • Load the mesh data into your own interactive viewer to make sure you understand the mesh format. For this step, don't worry about aligning the mesh data to any particular camera pose. I implement something like this in code/python/tools/visualize_mesh.py that you can use as a reference. In my code, I use mayavi for visualizing the mesh, but any interactive viewer library should work.

  • Select a single semantic instance from the mesh data and visualize only that instance. Manually choose an instance that is visible in a pre-rendered image, get the semantic instance ID from the segmentation ground truth image, and use the semantic instance ID to select the appropriate vertices and faces from the mesh data. In my code (code/python/tools/visualize_mesh.py), I render each semantic instance separately in a different color, so there is already the necessary logic in this file to select a single semantic instance from the mesh data.

  • Once you're familiar with the mesh format, try projecting the mesh vertices for a single instance onto one of our pre-rendered images. The projected vertices should align pixel-perfectly with the pre-rendered image. This step will require using the camera ground truth data for that particular image. I implement something like this in code/python/tools/scene_generate_images_bounding_box.py that you can use as a reference. In my code, I project each 3D corner of the semantic instance's bounding box (8 corners in total), but you could just as easily project mesh vertices instead of bounding box corners. In my code, there is a bit of extra logic to render lines rather than points, and to render everything in the correct depth order. You can ignore all of this additional logic, since the goal here is simply to project a collection of points in a way that aligns with a pre-rendered image (i.e., you don't need to worry about rendering lines, and you don't need to worry about rendering things in the correct depth order). At this point, you have re-implemented the complete geometric transformation pipeline in Hypersim.

Give these steps a try, and post here if you run into any problems 😄

umyta

comment created time in a day

issue openedapple/ml-hypersim

2D 3D alignment

Hi @mikeroberts3000, what would be a good way to check the 3D geometry's alignment with the generated 2D images? Is there any code that can demonstrate how to parse the scene information and align 3D geometry with the images using a different renderer such as blender or pytorch3d?

created time in a day

pull request commentapple/swift-algorithms

Fix contribution guidelines link in PR templates

@mdznr Do we need to keep the link correct for the PR, file viewer or both?

iSame7

comment created time in 2 days

pull request commentapple/swift-algorithms

[ImgBot] Optimize images

Sorry, who are you?

LemonSpike

comment created time in 2 days

created tagapple/swift-tools-support-core

tagswift-DEVELOPMENT-SNAPSHOT-2021-04-16-a

Contains common infrastructural code for both SwiftPM and llbuild.

created time in 2 days