Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

fix: detach the torch tensors - #1526

#1526
Merged
JoanFM merged 6 commits into
docarray:maindocarray/docarray:mainfrom
makram93:fix-serializing-torch-tensorsmakram93/docarray:fix-serializing-torch-tensorsCopy head branch name to clipboard
May 11, 2023
Merged

fix: detach the torch tensors#1526
JoanFM merged 6 commits into
docarray:maindocarray/docarray:mainfrom
makram93:fix-serializing-torch-tensorsmakram93/docarray:fix-serializing-torch-tensorsCopy head branch name to clipboard

Conversation

@makram93

@makram93 makram93 commented May 10, 2023

Copy link
Copy Markdown
Contributor

Detach TorchTensors before converting to numpy for serialization.

Change is only needed for PyTorch as Tensorflow operates differently afaik. I have tried different initialization for tensorflow and the below serializing script worked without any error.

import time

from docarray import BaseDoc
from docarray.typing import TensorFlowTensor
import tensorflow as tf


class MyDoc(BaseDoc):
    x: TensorFlowTensor
    y: TensorFlowTensor


# 1. Eager execution with GradientTape
x = tf.Variable(3.0)  # trainable: Default: `True`, GradientTapes automatically watch uses of this variable.

with tf.GradientTape() as tape:
    y = x**2

doc = MyDoc(x=x, y=y)
doc.json()


# 2. GradientTape with tf.function
@tf.function
def square(a):
    return a**2


x = tf.Variable(tf.random.normal((3, 2)))

with tf.GradientTape() as tape:
    y = square(x)

doc = MyDoc(x=x, y=y)
doc.json()

# 3. TF constants
x = tf.Variable(tf.random.normal((3, 2)))
y = tf.constant(2.0)

doc = MyDoc(x=x, y=y)
doc.json()

# 4. Full example
w = tf.Variable(tf.random.normal((3, 2)), name="w")
b = tf.Variable(tf.zeros(2, dtype=tf.float32), name="b")
x = [[1.0, 2.0, 3.0]]

with tf.GradientTape(persistent=True) as tape:
    y = x @ w + b
    loss = tf.reduce_mean(y**2)

[dl_dw, dl_db] = tape.gradient(loss, [w, b])

doc = MyDoc(x=dl_dw, y=dl_db)
doc.json()

Signed-off-by: Mohammad Kalim Akram <kalim.akram@jina.ai>

@JoanFM JoanFM left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please add a test for the case reported in the issue?

Signed-off-by: Mohammad Kalim Akram <kalim.akram@jina.ai>
@makram93
makram93 requested a review from JoanFM May 10, 2023 13:56
Comment thread docarray/typing/tensor/torch_tensor.py
Comment thread tests/units/typing/tensor/test_torch_tensor.py Outdated
makram93 added 4 commits May 10, 2023 17:41
Signed-off-by: Mohammad Kalim Akram <kalim.akram@jina.ai>
Signed-off-by: Mohammad Kalim Akram <kalim.akram@jina.ai>
Signed-off-by: Mohammad Kalim Akram <kalim.akram@jina.ai>
@JoanFM
JoanFM merged commit 8dd050f into docarray:main May 11, 2023
@makram93
makram93 deleted the fix-serializing-torch-tensors branch May 11, 2023 07:15
@JoanFM JoanFM linked an issue May 11, 2023 that may be closed by this pull request
@samsja samsja mentioned this pull request May 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Serializing TorchTensor with grad fails

3 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.