profile
viewpoint

apple/swift 53958

The Swift Programming Language

apple/swift-lldb 645

This is the version of LLDB that supports the Swift programming language & REPL.

marian-nmt/marian-dev 114

Fast Neural Machine Translation in C++ - development repository

rxwei/cuda-swift 72

Parallel Computing Library for Linux and macOS & NVIDIA CUDA Wrapper

param087/swiftML 51

Swift library for Machine Learning

Ayush517/S4TF-Tutorials 45

Repository for reviews and evaluations of converted Swift Notebooks as part of GSoC '19 program

dlvm-team/CoreTensor 36

Dynamic and ranked tensor data structures

rxwei/DeepLearning 14

DEPRECATED. See tensorflow/swift-apis

PullRequestReviewEvent
PullRequestReviewEvent

delete branch rxwei/swift-evolution-staging

delete branch : optional-keypath-iterable

delete time in 11 days

push eventapple/swift-evolution-staging

Richard Wei

commit sha 43dfbb438aef25f64376686ec90618294369472c

Conform 'Optional' to 'KeyPathIterable'. (#5)

view details

push time in 11 days

pull request commentapple/swift-evolution-staging

Conform 'Optional' to 'KeyPathIterable'.

Merging now to unblock things. Additional feedback welcome.

rxwei

comment created time in 11 days

pull request commentapple/swift-evolution-staging

Conform 'Optional' to 'KeyPathIterable'.

@Azoy could you review?

rxwei

comment created time in 11 days

create barnchrxwei/swift-evolution-staging

branch : optional-keypath-iterable

created branch time in 11 days

pull request commentapple/swift

[AutoDiff] [Sema] Limit implicit `@differentiable` attribute creation.

This has broken some code where there are multiple @differentiable attributes on a protocol requirement. Implicit attributes should always be created when the declaration being type-checked has a differentiable attribute that covers a superset of differentiable parameters.

public protocol P: Differentiable {
    @differentiable(wrt: self)
    @differentiable(wrt: (self, x))
    func foo(_ x: Float) -> Float
}

// In a different file:
public struct S: P {
    // Should generate implicit `@differentiable(wrt: self)` here.
    @differentiable(wrt: (self, x))
    public func foo(_ x: Float) -> Float {
        x
    }
}

I'll fix this.

dan-zheng

comment created time in 11 days

startedapple/swift-algorithms

started time in 19 days

PullRequestReviewEvent
PullRequestReviewEvent

PR closed apple/swift

[AutoDiff] WIP: Implement the Differentiable Curry

This PR contains my ongoing implementation of The Differentiable Curry.

Umbrella issue: SR-11882

cc @dan-zheng @marcrasi @dimitriv

TODOs:

  • Type calculation
    • [x] [SIL] Change SILFunctionType::getAutoDiffDerivativeFunctionType to include closure context tangents in the result linear map type.
    • [x] [Sema] Add an option to AnyFunctionType::getAutoDiffDerivativeFunctionType to include closure context tangents in the result linear map type, for calculating the result linear map type of Builtin.applyDerivative*.
  • Standard library and builtins
    • [x] [AST] Change Builtin.differentiableFunction*'s type to include closure context tangents.
    • [x] [stdlib] Update differential operators to use the updated AD builtins.
  • SIL generation
    • [x] [SILGen] Change derivative registration linear map thunks (SILGenFunction::getThunkedAutoDiffLinearMap) to include closure context tangents.
    • [x] [SILGen] Change @differentiable reabstraction thunk generation (createAutoDiffThunk in SILGenPoly.cpp) to include closure context tangents in derivative type calculation.
  • Differentiation transform
    • [x] [Main pass] Handle closure context tangents in subset parameters thunks.
    • [ ] [ActivityAnalysis] Propagate variedness and usefulness for function values when visiting apply.
    • [ ] [DifferentialEmitter] Differentiate apply with respect to the function, forward-propagating closure context tangents.
    • [ ] [PullbackEmitter] Differentiate apply with respect to the function, producing closure context tangents of AnyDerivative type in the pullback.
    • [ ] [PullbackEmitter] Differentiate partial_apply, casting closure context tangents from tan[f] to tan[args].
+55030 -767

3 comments

490 changed files

rxwei

pr closed time in 24 days

startedapple/swift-atomics

started time in 25 days

PR closed apple/swift

[Sema] Always consider outer alternatives when resolving a decl ref.

Currently, when a type member has the same base name as a global declaration, one cannot refer to the global member from a method context, and the current diagnostics force the user to use a fully qualified name. This is a long-standing issue with the use of top-level min and max in an extension, and is causing unnecessarily verbosity in math algorithms in extensions for ElementaryFunctions-conforming types.

More context:

  1. Forum thread: ElementaryFunctions and ambiguity in extensions
  2. SR-456: Confusing build error when calling 'max' function within 'extension Int'.
  3. SR-1772: File-level function with the same name as instance function not picked up by compiler.

This PR makes UnresolvedDeclRefExpr resolution always consider alternative candidates from the outer context. The following code will no longer trigger any error or warning.

import Foundation // which exports a top-level `sin(_:)`.

extension Float {
    func foo() {
        // Refers to top-level `sin(_:)`, not static method `Float.sin(_:)`.
        _ = sin(self)
        _ = sin(_:) as (Float) -> Float
    }
}

extension Sequence {
    func foo() {
        // Refers to top-level `max(_:_:)`.
        _ = max(1, 2)
    }
}
+15 -44

23 comments

3 changed files

rxwei

pr closed time in 25 days

pull request commentapple/swift

[Sema] Always consider outer alternatives when resolving a decl ref.

Closing since I'm no longer pursuing this.

rxwei

comment created time in 25 days

delete branch rxwei/swift

delete branch : autodiff-find-original

delete time in a month

push eventapple/swift

Richard Wei

commit sha a845c5aff3528ad522034d01d42ed222fc0b862e

[AutoDiff] NFC: Rename AutoDiff-specific method and improve documentation. (#34112) `findAbstractFunctionDecl` in TypeCheckAttr.cpp has a very general name but the functionality is specific to AutoDiff. This patch renames it to `findAutoDiffOriginalFunctionDecl` and introduces minor improvements to the documentation.

view details

push time in a month

PR merged apple/swift

Reviewers
[AutoDiff] NFC: Rename AutoDiff-specific method and improve documentation

findAbstractFunctionDecl in TypeCheckAttr.cpp has a very general name but the functionality is specific to AutoDiff. This patch renames it to findAutoDiffOriginalFunctionDecl and introduces minor improvements to the documentation.

+10 -5

1 comment

1 changed file

rxwei

pr closed time in a month

PullRequestReviewEvent

pull request commentapple/swift

[AutoDiff] NFC: Rename AutoDiff-specific method and improve documentation

@swift-ci please smoke test

rxwei

comment created time in a month

PR opened apple/swift

Reviewers
[AutoDiff] NFC: Rename AutoDiff-specific method and improve documentation

findAbstractFunctionDecl in TypeCheckAttr.cpp has a very general name but the functionality is specific to AutoDiff. This patch renames it to findAutoDiffOriginalFunctionDecl and introduces minor improvements to the documentation.

+10 -5

0 comment

1 changed file

pr created time in a month

create barnchrxwei/swift

branch : autodiff-find-original

created branch time in a month

startedapple/swift-system

started time in a month

PullRequestReviewEvent
PullRequestReviewEvent

delete branch rxwei/swift

delete branch : clarify-derived-conformances-conditions

delete time in a month

pull request commentapple/swift

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions.

@swift-ci please smoke test and merge

rxwei

comment created time in a month

PullRequestReviewEvent

pull request commentapple/swift

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions.

@swift-ci please smoke test and merge

rxwei

comment created time in a month

Pull request review commentapple/swift

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions.

 struct Foo<T: Differentiable, U: Differentiable>: @memberwise Differentiable { } ``` +###### Opt out of synthesis for a stored property++The synthesized implementation of `Differentiable` protocol requirements already+excludes stored properties that are not differentiable variables, such as stored+properties that do not conform to `Differentiable` and `let`+properties that do not have a non-mutating `move(along:)`. In addition to this+behavior, we also introduce a `@noDerivative` declaration attribute, which can+be attached to properties that the programmer does not wish to include in the+synthesized `Differentiable` protocol requirement implementation.++When a stored property is marked with `@noDerivative` in a type that declares a+conformance to `Differentiable`, it will not be treated as a differentiable+variable regardless of whether it conforms to `Differentiable`. That is, the+synthesized implementation of protocol requirements will not include this+property.++```swift+struct Foo<T: Differentiable, U: Differentiable>: @memberwise Differentiable {+    // `x` and `y` are the "differentiable variables".+    var x: T+    var y: U+    @noDerivative var customFlag: Bool+    @noDerivative let helperVariable: T+}+```++For clarity as to which stored properties are to be included for+differentiation, the compiler will recommend that all stored properties that+cannot be included as differentiable variables (due to either lacking a+conformance to `Differentiable` or being a non-`class`-bound `let` property) be+marked with `@noDerivative`. When a property is not included as a differentiable+variable and is not marked with `@noDerivative`, the compiler produces a warning+as asking the user to make the exclusion explicit along with fix-it suggestions

Thanks for the catch. Fixed.

rxwei

comment created time in a month

PullRequestReviewEvent

Pull request review commentapple/swift

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions.

 Method `move(along:)` will not be synthesized because a default implementation already exists.  ```swift-struct Point<T: Real>: @memberwise Differentiable, @memberwise AdditiveArithmetic {+struct Point<T: Real>: @memberwise Differentiable, AdditiveArithmetic {

This change was unintended. Fixed.

rxwei

comment created time in a month

PullRequestReviewEvent

push eventrxwei/swift

Richard Wei

commit sha 8df2d34603b737b00af323118c1017789db1c8e5

Address review comment.

view details

push time in a month

PullRequestReviewEvent

pull request commentapple/swift

Add a different implementation of Differentiation

Note for reviewers: Since differentiation is platform-independent, it shouldn't need to depend on platform runtime libraries like Darwin and Glibc. One possible future direction is to move tgmath derivatives to a cross-import overlay for Darwin/Glibc and _Differentiation.

edymtt

comment created time in a month

pull request commentapple/swift

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions.

@swift-ci please smoke test and merge

rxwei

comment created time in a month

pull request commentapple/swift

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions.

@swift-ci please smoke test and merge

rxwei

comment created time in a month

pull request commentapple/swift

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions.

@swift-ci please smoke test and merge

rxwei

comment created time in a month

pull request commentapple/swift

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions.

@swift-ci please smoke test and merge

rxwei

comment created time in a month

create barnchrxwei/swift

branch : foreachkeypath

created branch time in a month

pull request commentapple/swift

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions.

@swift-ci please smoke test and merge

rxwei

comment created time in a month

push eventrxwei/swift

Richard Wei

commit sha 4fe8dda893fd8ad5daedb9da1cf81ff61f6a2add

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions. Clarify `Differentiable` derived conformances synthesis conditions.

view details

push time in a month

push eventrxwei/swift

Richard Wei

commit sha 8804a0460dbfd435dd6fe7c6df927f133dec7e23

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions. Clarify `Differentiable` derived conformances synthesis conditions.

view details

push time in a month

create barnchrxwei/swift

branch : clarify-derived-conformances-conditions

created branch time in a month

push eventrxwei/swift

Richard Wei

commit sha a73023a6fbe64787f5e44f991b2f73d2720af62f

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions. Clarify `Differentiable` derived conformances synthesis conditions.

view details

push time in a month

PR opened apple/swift

Reviewers
[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions.

Clarify Differentiable derived conformances synthesis conditions.

+120 -32

0 comment

1 changed file

pr created time in a month

push eventrxwei/swift

Richard Wei

commit sha 58b3141cc45dad643340f01fd715b42bb08fba65

[AutoDiff] [Docs] Clarify 'Differentiable' derived conformances conditions. Clarify `Differentiable` derived conformances synthesis conditions.

view details

push time in a month

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentapple/swift

[AutoDiff] Generate transparent ossa reabstraction thunks.

 CanSILFunctionType buildThunkType(SILFunction *fn,       fn->getASTContext()); } +enum class CleanupOperation { Destroy, EndBorrow };++/// Forward function arguments, handling ownership convention mismatches.+/// Adapted from `forwardFunctionArguments` in SILGenPoly.cpp.+///+/// Forwarded arguments are appended to `forwardedArgs`.+///+/// Local allocations are appended to `localAllocations`. They need to be+/// deallocated via `dealloc_stack`.+///+/// Local values requiring cleanup are appended to `valuesToCleanup`.+static void forwardFunctionArgumentsConvertingOwnership(+    SILBuilder &builder, SILLocation loc, CanSILFunctionType fromTy,+    CanSILFunctionType toTy, ArrayRef<SILArgument *> originalArgs,+    SmallVectorImpl<SILValue> &forwardedArgs,+    SmallVectorImpl<AllocStackInst *> &localAllocations,+    SmallVectorImpl<std::pair<SILValue, CleanupOperation>>

Instead of using a newly defined CleanupOperation enum class, have you considered just using ValueOwnershipKind?

dan-zheng

comment created time in 2 months

PullRequestReviewEvent

delete branch rxwei/swift

delete branch : SR-13500

delete time in 2 months

push eventapple/swift

Richard Wei

commit sha aa20633f4e2b1f440f2d9ae11cacfa7e4993cdca

[AutoDiff] [Sema] Fix missing access-level mismatch diagnostic in `-enable-testing` builds. (#33800) When a derivative function is internal and its original function is public, there should be an error when the derivative is not `@usableFromInline`. This patch fixes a bug where the error does not appear in SwiftPM debug build configurations (or any configurations with `-enable-testing`). `ValueDecl::getEffectiveAccess()` should not be used when we perform formal access checking for AutoDiff-related declaration attributes because it will treat all internal declarations as public when `-enable-testing` is enabled. Instea we should use a combination of `ValueDecl::getFormalAccess()` and `ValueDecl::isUsableFromInline()`. Also, add a `RUN` line with `-enable-testing` to all AutoDiff declaration attribute test files. Resolves SR-13500 (rdar://68336300).

view details

push time in 2 months

PR merged apple/swift

Reviewers
[AutoDiff] [Sema] Fix missing access-level mismatch diagnostic in `-enable-testing` builds.

When a derivative function is internal and its original function is public, there should be an error when the derivative is not @usableFromInline. This patch fixes a bug where the error does not appear in SwiftPM debug build configurations (or any configurations with -enable-testing).

ValueDecl::getEffectiveAccess() should not be used when we perform formal access checking for AutoDiff-related declaration attributes because it will treat all internal declarations as public when -enable-testing is enabled. Instead, we should use a combination of ValueDecl::getFormalAccess() and ValueDecl::isUsableFromInline().

Also, add a RUN line with -enable-testing to all AutoDiff declaration attribute test files.

Resolves SR-13500 (rdar://68336300).

+5 -1

1 comment

4 changed files

rxwei

pr closed time in 2 months

pull request commentapple/swift

[AutoDiff] [Sema] Fix missing access-level mismatch diagnostic in `-enable-testing` builds.

@swift-ci please test

rxwei

comment created time in 2 months

PR opened apple/swift

Reviewers
[AutoDiff] [Sema] Fix missing access-level mismatch diagnostic in `-enable-testing` builds.

When a derivative function is internal and its original function is public, there should be an error when the derivative is not @usableFromInline. This patch fixes a bug where the error does not appear in SwiftPM debug build configurations (or any configurations with -enable-testing).

ValueDecl::getEffectiveAccess() should not be used when we perform formal access checking for AutoDiff-related declaration attributes because it will treat all internal declarations as public when -enable-testing is enabled. Instea we should use a combination of ValueDecl::getFormalAccess() and ValueDecl::isUsableFromInline().

Also, add a RUN line with -enable-testing to all AutoDiff declaration attribute test files.

Resolves SR-13500 (rdar://68336300).

+5 -1

0 comment

4 changed files

pr created time in 2 months

create barnchrxwei/swift

branch : SR-13500

created branch time in 2 months

PullRequestReviewEvent
PullRequestReviewEvent

pull request commentapple/swift

[AutoDiff] [Sema] Limit implicit `@differentiable` attribute creation.

We discussed a bit offline, but I feel we've missed an important issue in today's semantics as well as the semantics proposed in this PR. I think there are multiple aspects to the issue that this PR is trying to mitigate, but I think the root problem is the fact that conformances are implicitly changing declarations in protocol extensions. Let me dive into this with examples.

Case 1: Candidates in protocol extensions that do not match

First, should declarations in a protocol extension that almost matches a protocol requirement be modified so that it can be treat as a requirement? Similar to what you posted above, such an example would be:

protocol P1 {
  func method()
}

protocol P2: P1 {}
extension P2 {
  mutating func method() {}
}

struct S: P2 {}

In this example, the protocol extension method does not (but almost!) matches the protocol requirement. Note that it has nothing to do with whether the conformance is declared in the same file or not. The example above results in compilation errors similar to the ones you've shown. Therefore, a similar AD example should fail as well, namely:

protocol P1: Differentiable {
  @differentiable(wrt: x)
  func method(x: Float) -> Float
}

protocol P2: P1 {}
extension P2 {
  @differentiable(wrt: (self, x))
  func method(x: Float) -> Float { return 0 }
}

struct S: P2 {}

However this succeeds to compile today, because Sema is adding an implicit @differentiable attribute to the protocol extension method when checking the conformance. Now this will leads to very surprising behavior, consider the following example:

// File 1
protocol P1: Differentiable {
  @differentiable(wrt: x)
  func method(x: Float) -> Float
}

protocol P2: P1 {}
extension P2 {
  public func method(x: Float) -> Float { return 0 }
}

struct S1: P2 {} // Removing this will non-obviously cause file 2 to fail to compile
// File 2
struct S2: P2 {}

File 1 defines the S1: P2 conformance. It generates an implicit @differentiable(wrt: x) attribute on P2.method(x:) so that it is now part of the file's interface. File 2 compiles successfully as well, because when the compiler type-checks S2: P2 it finds a @differentiable(wrt: x) attribute.

Now, we modify file 1 by doing something that's seemingly completely harmless to file 2, that is, removing struct S1: P2 {}. Then file 2 will fail to compile! Because the implicit @differentiable(wrt: x) attribute is no longer generated. This is an API breakage that happened completely implicitly and is not at all obvious to the file 1's developer what broke it.

To sum up, IMO the root problem is not at all related to whether conformances are declared in the same file as the witness candidate in a protocol extension, it is the fact that protocol conformance checking is modifying arbitrarily other declarations implicitly, causing their interface to change in an entirely non-obvious way. To solve this problem, for this example:

public protocol P1: Differentiable {
  @differentiable(wrt: x)
  func method(x: Float) -> Float
}

public protocol P2: P1 {}
extension P2 {
  public func method(x: Float) -> Float { return 0 }
}

struct S1: P2 {}

IMO the right behavior we should go with is to not emit any implicit @differentiable attributes on anything but candidates declared inside the conforming type. So the example above should fail to compile regardless of whether S1 is defined in the current file or not. When a candidate declaration doesn't have an attribute that satisfies the protocol requirement, it is not a valid candidate.

Case 2: Candidates in protocol extensions that effectively match

Now, back to the original example in your PR description:

import _Differentiation

protocol P1: Differentiable {
  @differentiable(wrt: self)
  func callAsFunction(_ input: Float) -> Float
}

protocol P2: P1 {}

extension P2 {
  @differentiable(wrt: (self, input))
  public func callAsFunction(_ input: Float) -> Float {
    return input
  }
}
struct S: P2 {}

Semantically, P2.callAsFunction(_:) declared in the protocol extension effectively matches the protocol requirement, so ideally this case should be supported without errors. When type checking S: P2, maybe we can just generate an implicit function declaration in S whose body calls the protocol extension method. This will then inherit the @differentiable attribute and AD will differentiate it correctly just like how we generate a subset parameters thunk. This should be regardless of whether the conformance is in the same file or not, once we've resolved Case 1.


To summarize, today's protocol conformance checking is adding implicit attributes to other declarations. IMO this attribute-adding behavior should be limited, in a way that is more limiting than what you've done for this patch, to only declarations declared in the conforming type's declaration context. Does that make sense?

dan-zheng

comment created time in 2 months

pull request commentapple/swift

[AutoDiff] [Sema] Limit implicit `@differentiable` attribute creation.

The error message also seems non-standard:

file1.swift:14:15: note: instance method 'callAsFunction' must have explicit '@differentiable(wrt: self)' attribute to satisfy requirement instance method 'callAsFunction' (in protocol 'P1') because it is declared in a different file than the conformance of 'S' to 'P1'
  public func callAsFunction(_ input: Float) -> Float {
              ^
         @differentiable(wrt: self)

This seems to be suggesting that it's the other file's responsibility to fix, while providing no ways to actually fix it within the current file. What if this is imported from a different module?

dan-zheng

comment created time in 2 months

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentapple/swift

[AutoDiff] Fix forward-mode crashes related to tangent buffers

 void JVPCloner::Implementation::prepareForDifferentialGeneration() {   // Initialize tangent mapping for indirect results.   auto origIndResults = original->getIndirectResults();   auto diffIndResults = differential.getIndirectResults();+  auto isNonWrtInoutParameter = [&](unsigned i) {+    auto &paramInfo = original->getLoweredFunctionType()->getParameters()[i];+    return paramInfo.isIndirectInOut() && !getIndices().parameters->contains(i);+  };+  auto origNumParams = original->getLoweredFunctionType()->getNumParameters(); #ifndef NDEBUG-  unsigned numNonWrtInoutParameters = llvm::count_if(-    range(original->getLoweredFunctionType()->getNumParameters()),-    [&] (unsigned i) {-      auto &paramInfo = original->getLoweredFunctionType()->getParameters()[i];-      return paramInfo.isIndirectInOut() && !getIndices().parameters->contains(i);-    });+  unsigned numNonWrtInoutParameters =+      llvm::count_if(range(origNumParams), isNonWrtInoutParameter);+  assert(origIndResults.size() + numNonWrtInoutParameters ==+         diffIndResults.size()); #endif-  assert(origIndResults.size() + numNonWrtInoutParameters == diffIndResults.size());+  differentialBuilder.setInsertionPoint(differential.getEntryBlock());+  for (auto i : range(origNumParams)) {+    if (isNonWrtInoutParameter(i)) {+      auto &origParam = original->getArgumentsWithoutIndirectResults()[i];+      if (!activityInfo.isVaried(origParam, getIndices().parameters))+        continue;+      auto origSILType = origParam->getType();+      auto tangentSILType = getRemappedTangentType(origSILType);+      auto *tanBuf = differentialBuilder.createAllocStack(+          differential.getLocation(), tangentSILType);+      setTangentBuffer(origEntry, origParam, tanBuf);

IMHO there isn't a great reason to allow initialization info to be Optional in setTangentBuffer, because the programming interface should in principle enforce the invariants which all call sites make assumptions about. All VJPCloner visitor methods plus other places currently assume that every tangent buffer has an associated initialization info. Allowing initialization info to be Optional feels like an ad-hoc solution which on one hand specifically caters to prepareForDifferentialGeneration while on the other hand making all other call sites error-prone. One can easily forget to pass initialization info to setTangentBuffer when adding a new visitor in the future.

Since JVPCloner is emitting all tangent buffers (including in prepareForDifferentialGeneration), maybe we can delay the call to setTangentBuffer to the place where we actually know whether it is initialized or not?

efremale

comment created time in 2 months

PullRequestReviewEvent

Pull request review commentapple/swift

[AutoDiff] Fix forward-mode crashes related to tangent buffers

 class JVPCloner::Implementation final     auto &tanValDest = getTangentBuffer(si->getParent(), si->getDest());     diffBuilder.emitStoreValueOperation(loc, tanValSrc, tanValDest,                                         si->getOwnershipQualifier());+    if (si->getOwnershipQualifier() == StoreOwnershipQualifier::Init)+      tangentBufferInitializationInfo[tanValSrc] = NotInitialized;

I see where you are coming from, but the point of the suggested change is mainly about enforcing the invariant that all tangent buffers have an initialization info, which will help ensure the correctness of the algorithm and IMHO outweighs potential redundant lookups. Moreover, the redundant lookup can be avoided entirely if we make getTangentBufferInitializationInfo return a reference so that you can modify initInfo directly. Given that the redundant lookup can be avoided entirely, this enforcement to prevent algorithm errors seems strictly positive to me.

efremale

comment created time in 2 months

PullRequestReviewEvent

delete branch rxwei/swift

delete branch : SR-13474-differentiable-class-let

delete time in 2 months

push eventapple/swift

Richard Wei

commit sha 76d0648e60e79153b2fe79699397f1788b3bb5eb

[AutoDiff] [Sema] Include certain 'let' properties in 'Differentiable' derived conformances. (#33700) In `Differentiable` derived conformances, `let` properties are currently treated as if they had `@noDerivative` and excluded from the derived `Differentiable` conformance implementation. This is limiting to properties that have a non-mutating `move(along:)` (e.g. class properties), which can be mathematically treated as differentiable variables. This patch changes the derived conformances behavior such that `let` properties will be included as differentiable variables if they have a non-mutating `move(along:)`. This unblocks the following code: ```swift final class Foo: Differentiable { let x: ClassStuff // Class type with a non-mutating 'move(along:)' // Synthesized code: // struct TangentVector { // var x: ClassStuff.TangentVector // } // ... // func move(along direction: TangentVector) { // x.move(along: direction.x) // } } ``` Resolves SR-13474 (rdar://67982207).

view details

push time in 2 months

PR merged apple/swift

Reviewers
[AutoDiff] [Sema] Include certain 'let' properties in 'Differentiable' derived conformances.

Today, let properties are treated as if they had @noDerivative and excluded from the derived Differentiable conformance implementation. This is limiting to properties that have a non-mutating move(along:) (e.g. class properties), which can be mathematically treated as differentiable variables.

This patch changes the derived conformances behavior such that let properties will be included as differentiable variables if they have a non-mutating move(along:). This unblocks the following code:

final class Foo: Differentiable {
   let x: ClassStuff // Class type with a non-mutating 'move(along:)'

   // Synthesized code:
   //   struct TangentVector {
   //     var x: ClassStuff.TangentVector
   //   }
   //   ...
   //   func move(along direction: TangentVector) {
   //     x.move(along: direction.x)
   //   }
}

Resolves SR-13474 (rdar://67982207).

+147 -28

1 comment

5 changed files

rxwei

pr closed time in 2 months

pull request commentapple/swift

[AutoDiff] [Sema] Include certain 'let' properties in 'Differentiable' derived conformances.

@swift-ci please smoke test

rxwei

comment created time in 2 months

push eventrxwei/swift

Richard Wei

commit sha 702f738ebb0a2b2f115eba0b1aabbc33a2ffc0b0

[AutoDiff] [Sema] Include certain 'let' properties in 'Differentiable' derived conformances. Today, `let` properties are treated as if they had `@noDerivative` and excluded from the derived `Differentaible` conformance implementation. This is limiting to properties that have a non-mutating `move(along:)` (e.g. class properties), which can be mathematically treated as differentiable variables. This patch changes the derived conformances behavior such that `let` properties will be included as differentiable variables if they have a non-mutating 'move(along:)'. This unblocks the following code: ```swift class Foo: Differentiable { let x: ClassStuff // Class type // Synthesized code: // struct TangentVector { // var x: ClassStuff.TangentVector // } // ... // func move(along direction: TangentVector) { // x.move(along: direction.x) // } } ``` Resolves SR-13474 (rdar://67982207).

view details

push time in 2 months

push eventrxwei/swift

Richard Wei

commit sha 694e6915262113d393d28149363bf1acf89ca6e5

[AutoDiff] [Sema] Include certain 'let' properties in 'Differentiable' derived conformances. Today, `let` properties are treated as if they had `@noDerivative` and excluded from the derived `Differentaible` conformance implementation. This is limiting to properties that have a non-mutating `move(along:)` (e.g. class properties), which can be mathematically treated as differentiable variables. This patch changes the derived conformances behavior such that `let` properties will be included as differentiable variables if they have a non-mutating 'move(along:)'. This unblocks the following code: ```swift class Foo: Differentiable { let x: ClassStuff // Class type // Synthesized code: // struct TangentVector { // var x: ClassStuff.TangentVector // } // ... // func move(along direction: TangentVector) { // x.move(along: direction.x) // } } ``` Resolves SR-13474 (rdar://67982207).

view details

push time in 2 months

PR opened apple/swift

Reviewers
[AutoDiff] [Sema] Allow certain let properties in 'Differentiable' derived conformances.

Today, let properties are treated as if they had @noDerivative and excluded from the derived Differentaible conformance implementation. This is limiting to properties that have a non-mutating move(along:) (e.g. class properties), which can be mathematically treated as differentiable variables.

This patch changes the derived conformances behavior such that let properties will be included as differentiable variables if they have a non-mutating 'move(along:)'. This unblocks the following code:

class Foo: Differentiable {
   let x: ClassStuff // Class type with a non-mutating 'move(along:)'

   // Synthesized code:
   //   struct TangentVector {
   //     var x: ClassStuff.TangentVector
   //   }
   //   ...
   //   func move(along direction: TangentVector) {
   //     x.move(along: direction.x)
   //   }
}

Resolves SR-13474 (rdar://67982207).

+148 -28

0 comment

5 changed files

pr created time in 2 months

push eventrxwei/swift

Richard Wei

commit sha 4842e0e130ff10b77dbc03b5441bd8e73dfb89d2

[AutoDiff] [Sema] Allow certain let properties in 'Differentiable' derived conformances. Today, `let` properties are treated as if they had `@noDerivative` and excluded from the derived `Differentaible` conformance implementation. This is limiting to properties that have a non-mutating `move(along:)` (e.g. class properties), which can be mathematically treated as differentiable variables. This patch changes the derived conformances behavior such that `let` properties will be included as differentiable variables if they have a non-mutating 'move(along:)'. This unblocks the following code: ```swift class Foo: Differentiable { let x: ClassStuff // Class type // Synthesized code: // struct TangentVector { // var x: ClassStuff.TangentVector // } // ... // func move(along direction: TangentVector) { // x.move(along: direction.x) // } } ``` Resolves SR-13474 (rdar://67982207).

view details

push time in 2 months

create barnchrxwei/swift

branch : SR-13474-differentiable-class-let

created branch time in 2 months

Pull request review commentapple/swift

[AutoDiff] Fix forward-mode crashes related to tangent buffers

 class JVPCloner::Implementation final     auto diffBuilder = getDifferentialBuilder();     auto loc = cai->getLoc();     auto *bb = cai->getParent();-    auto &tanSrc = getTangentBuffer(bb, cai->getSrc());-    auto tanDest = getTangentBuffer(bb, cai->getDest());+    auto &tanSrc = getTangentBuffer(bb, cai->getSrc(), InitializationRequired);+    auto isInitialization = cai->isInitializationOfDest();+    auto tanDest = getTangentBuffer(bb, cai->getDest(),+                                    isInitialization ? InitializationNotRequired+                                                     : InitializationRequired);

Since the destination buffer is not being read by copy_addr, passing InitializationRequired here can cause the buffer to be redundantly initialized, regardless of isInitialization. How about optimizing this by:

  • Always passing InitializationNotRequired to getTangentBuffer.
  • Before we call createCopyAddr, we use the tangent buffer initialization info to determine whether we pass IsInitialization or IsNotInitialization to createCopyAddr.
efremale

comment created time in 2 months

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentapple/swift

[AutoDiff] Fix forward-mode crashes related to tangent buffers

 class JVPCloner::Implementation final     auto &tanValDest = getTangentBuffer(si->getParent(), si->getDest());     diffBuilder.emitStoreValueOperation(loc, tanValSrc, tanValDest,                                         si->getOwnershipQualifier());+    if (si->getOwnershipQualifier() == StoreOwnershipQualifier::Init)+      tangentBufferInitializationInfo[tanValSrc] = NotInitialized;

Since tanValSrc is not a buffer, are these two lines redundant? Should we define a method for setting buffer initialization info such that it asserts that the argument's value category is always an address? That could prevent erroneous additions to the dictionary.

void setTangentBufferInitializationInfo(SILValue buffer)
efremale

comment created time in 2 months

Pull request review commentapple/swift

[AutoDiff] Fix forward-mode crashes related to tangent buffers

 class JVPCloner::Implementation final     auto &diffBuilder = getDifferentialBuilder();     auto loc = dvi->getLoc();     auto tanVal = materializeTangent(getTangentValue(dvi->getOperand()), loc);-    diffBuilder.emitDestroyValue(loc, tanVal);+    diffBuilder.emitDestroyOperation(loc, tanVal);

Should we use emitDestroyValueOperation? IIRC emitDestroyOperation is for both buffers and objects. Here we are only dealing with objects.

efremale

comment created time in 2 months

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentapple/swift

[AutoDiff] Fix forward-mode crashes related to tangent buffers

 class JVPCloner::Implementation final     } else {       tanBuf =

Thanks for fixing this for me! Could you add a test case? Maybe something like the following will trigger this code:

differential(at: 0.0) { x in
    let tuple = (x, 0.0)
    return tuple.x
}
efremale

comment created time in 2 months

PullRequestReviewEvent

Pull request review commentapple/swift

[AutoDiff] Fix forward-mode crashes related to tangent buffers

 void JVPCloner::Implementation::prepareForDifferentialGeneration() {   // Initialize tangent mapping for indirect results.   auto origIndResults = original->getIndirectResults();   auto diffIndResults = differential.getIndirectResults();+  auto isNonWrtInoutParameter = [&](unsigned i) {+    auto &paramInfo = original->getLoweredFunctionType()->getParameters()[i];+    return paramInfo.isIndirectInOut() && !getIndices().parameters->contains(i);+  };+  auto origNumParams = original->getLoweredFunctionType()->getNumParameters(); #ifndef NDEBUG-  unsigned numNonWrtInoutParameters = llvm::count_if(-    range(original->getLoweredFunctionType()->getNumParameters()),-    [&] (unsigned i) {-      auto &paramInfo = original->getLoweredFunctionType()->getParameters()[i];-      return paramInfo.isIndirectInOut() && !getIndices().parameters->contains(i);-    });+  unsigned numNonWrtInoutParameters =+      llvm::count_if(range(origNumParams), isNonWrtInoutParameter);+  assert(origIndResults.size() + numNonWrtInoutParameters ==+         diffIndResults.size()); #endif-  assert(origIndResults.size() + numNonWrtInoutParameters == diffIndResults.size());+  differentialBuilder.setInsertionPoint(differential.getEntryBlock());+  for (auto i : range(origNumParams)) {+    if (isNonWrtInoutParameter(i)) {+      auto &origParam = original->getArgumentsWithoutIndirectResults()[i];+      if (!activityInfo.isVaried(origParam, getIndices().parameters))+        continue;+      auto origSILType = origParam->getType();+      auto tangentSILType = getRemappedTangentType(origSILType);+      auto *tanBuf = differentialBuilder.createAllocStack(+          differential.getLocation(), tangentSILType);+      setTangentBuffer(origEntry, origParam, tanBuf);

When would the caller let it default to None?

efremale

comment created time in 2 months

PullRequestReviewEvent

Pull request review commentapple/swift

[AutoDiff] Fix forward-mode crashes related to tangent buffers

 class JVPCloner::Implementation final     auto &tanValDest = getTangentBuffer(si->getParent(), si->getDest());     diffBuilder.emitStoreValueOperation(loc, tanValSrc, tanValDest,                                         si->getOwnershipQualifier());+    tangentBufferInitializationInfo[tanValDest] = Initialized;

Actually nvm, I mistook it for copy_addr. Since the source is an object (not an address), it needs no further handling.

efremale

comment created time in 2 months

PullRequestReviewEvent

Pull request review commentapple/swift

[AutoDiff] Fix forward-mode crashes related to tangent buffers

 void JVPCloner::Implementation::prepareForDifferentialGeneration() {   // Initialize tangent mapping for indirect results.   auto origIndResults = original->getIndirectResults();   auto diffIndResults = differential.getIndirectResults();+  auto isNonWrtInoutParameter = [&](unsigned i) {+    auto &paramInfo = original->getLoweredFunctionType()->getParameters()[i];+    return paramInfo.isIndirectInOut() && !getIndices().parameters->contains(i);+  };+  auto origNumParams = original->getLoweredFunctionType()->getNumParameters(); #ifndef NDEBUG-  unsigned numNonWrtInoutParameters = llvm::count_if(-    range(original->getLoweredFunctionType()->getNumParameters()),-    [&] (unsigned i) {-      auto &paramInfo = original->getLoweredFunctionType()->getParameters()[i];-      return paramInfo.isIndirectInOut() && !getIndices().parameters->contains(i);-    });+  unsigned numNonWrtInoutParameters =+      llvm::count_if(range(origNumParams), isNonWrtInoutParameter);+  assert(origIndResults.size() + numNonWrtInoutParameters ==+         diffIndResults.size()); #endif-  assert(origIndResults.size() + numNonWrtInoutParameters == diffIndResults.size());+  differentialBuilder.setInsertionPoint(differential.getEntryBlock());+  for (auto i : range(origNumParams)) {+    if (isNonWrtInoutParameter(i)) {+      auto &origParam = original->getArgumentsWithoutIndirectResults()[i];+      if (!activityInfo.isVaried(origParam, getIndices().parameters))+        continue;+      auto origSILType = origParam->getType();+      auto tangentSILType = getRemappedTangentType(origSILType);+      auto *tanBuf = differentialBuilder.createAllocStack(+          differential.getLocation(), tangentSILType);+      setTangentBuffer(origEntry, origParam, tanBuf);

Random idea: since setTangentBuffer is always called along with setting tangentBufferInitializationInfo, how about adding a parameter to setTangentBuffer? That way we could avoid potential invariant errors caused by not setting tangentBufferInitializationInfo.

setTangentBuffer(origEntry, origParam, tanBuf, NotInitialized)
efremale

comment created time in 2 months

PullRequestReviewEvent

Pull request review commentapple/swift

[AutoDiff] Fix forward-mode crashes related to tangent buffers

 class JVPCloner::Implementation final     } else {       tanBuf =

I feel this could be my code. This should be calling setTangentValue on the extracted value instead, since the result has an object type, not an address. Maybe it's not tested.

efremale

comment created time in 2 months

PullRequestReviewEvent

Pull request review commentapple/swift

[AutoDiff] Fix forward-mode crashes related to tangent buffers

 class JVPCloner::Implementation final     } else {       tanBuf =

This code (not your changes) feels very wrong in the first place... How is it a buffer? tuple_extract would never produce a buffer...

efremale

comment created time in 2 months

Pull request review commentapple/swift

[AutoDiff] Fix forward-mode crashes related to tangent buffers

 class JVPCloner::Implementation final       return;     }     // Emit tangent `struct_element_addr`.-    auto tanOperand = getTangentBuffer(bb, seai->getOperand());+    auto tanOperand =+        getTangentBuffer(bb, seai->getOperand(), InitializationRequired);     auto tangentInst =         diffBuilder.createStructElementAddr(loc, tanOperand, tanField);     // Update tangent buffer map for `struct_element_addr`.     setTangentBuffer(bb, seai, tangentInst);+    tangentBufferInitializationInfo[tangentInst] = Initialized;

struct_element_addr is valid on an uninitialized buffer. So I would probably let tangentBufferInitializationInfo ]tangentInst] inherit from tangentBufferInitializationInfo[tanOperand].

efremale

comment created time in 2 months

more